Data utils parallelproj.data

Small data utilities used by the examples and for distributed / out-of-core workflows – for instance helpers to split sinograms into ordered subsets and memory-map them to disk. Most users only need these for large-scale or multi-process reconstructions.

Data utilities for memory-efficient iterative reconstruction.

class parallelproj.data.SubsetArrayMmap(path: str | ~os.PathLike, num_subsets: int, subset_shape: tuple[int, ...], *, dtype: ~numpy.dtype = <class 'numpy.float32'>, mode: str = 'r')[source]

Bases: object

Memory-mapped array split into equal subsets on disk.

The file on disk has shape (num_subsets, *subset_shape) in C-contiguous order, so each subset occupies a single contiguous block. The OS memory-maps the file and loads pages on demand; pages are evicted automatically when RAM is scarce.

Parameters:

  • path (str or Path) – Path to a binary file produced by to_subset_mmap() (or any code that writes data in the expected on-disk layout).

  • num_subsets (int) – Number of subsets stored in the file.

  • subset_shape (tuple of int) – Shape of a single subset’s data array, e.g. (num_rad, views_per_subset, num_planes, num_tofbins) for a TOF sinogram in RVP order.

  • dtype (numpy dtype, optional) – Element type of the stored data. Default float32.

  • mode (str, optional) – numpy.memmap() open mode. Use 'r' (default) for read-only access. Pass 'r+' to allow in-place writes, or 'w+' to create / overwrite the file.

Examples

>>> mmap = SubsetArrayMmap("y_subsets.bin", num_subsets=24,
...                        subset_shape=(171, 4, 19, 13))
>>> y_k = mmap[3]   # owned float32 ndarray of shape (171, 4, 19, 13)
>>> del y_k          # RAM freed immediately

Examples

RAM-efficient OSEM with disk-backed TOF sinograms

RAM-efficient OSEM with disk-backed TOF sinograms
__getitem__(k: int) → ndarray[source]

Return an owned NumPy copy of subset k.

The returned array is a plain ndarray (not a memmap), so its lifetime is fully controlled by Python’s reference counter. The underlying OS pages become eviction-eligible as soon as the array is deleted.

Examples

Parameters:

k (int)

Return type:

ndarray

nbytes_per_subset() → int[source]

Bytes occupied in RAM when one subset is loaded.

Examples

RAM-efficient OSEM with disk-backed TOF sinograms

RAM-efficient OSEM with disk-backed TOF sinograms
Return type:

int

nbytes_total() → int[source]

Total size of the on-disk file in bytes.

Examples

Return type:

int

property num_subsets: int

Number of subsets stored in the file.

property path: Path

Path to the binary file on disk.

property shape: tuple[int, ...]

(num_subsets, *subset_shape).

Type:

Full shape of the on-disk array

property subset_shape: tuple[int, ...]

Shape of a single subset’s data.

parallelproj.data.count_event_multiplicity(events: Array) → Array[source]

Count how many times each row appears in a 2-D event array.

Parameters:

events (Array) – 2-D integer array of shape (N, M) where each row represents one event and the columns are event attributes (e.g. crystal indices).

Returns:

1-D integer array of length N. Element i is the number of rows in events that are identical to row i.

Return type:

Array

Raises:

ValueError – If events is not a 2-D array.

Examples

PDHG and LM-SPDHG to optimize the Poisson logL and total variation

PDHG and LM-SPDHG to optimize the Poisson logL and total variation
parallelproj.data.to_subset_mmap(full_array: ~numpy.ndarray, subset_slices: list[tuple], path: str | ~os.PathLike, dtype: ~numpy.dtype = <class 'numpy.float32'>) → SubsetArrayMmap[source]

Write a full array to a subset-contiguous binary file.

Each non-contiguous subset slice is gathered from full_array and written as a contiguous block so that a subsequent mmap[k] read is a single sequential I/O operation. The resulting access pattern lets the OS prefetch the next subset from disk while the algorithm computes on the current one.

Parameters:

  • full_array (numpy ndarray) – Full data array. Any axis order is accepted; the subset shape is inferred from full_array[subset_slices[0]].

  • subset_slices (list of tuple) – Index tuples that select each subset from full_array. For PET sinograms these are returned by RegularPolygonPETLORDescriptor.get_distributed_views_and_slices(). All slices must select the same number of elements.

  • path (str or Path) – Destination binary file (created or overwritten).

  • dtype (numpy dtype, optional) – On-disk element type. Default float32.

Returns:

Read-mode wrapper around the newly written file.

Return type:

SubsetArrayMmap

Notes

full_array must be a CPU NumPy array. When working with a GPU or PyTorch backend, convert first with parallelproj.to_numpy_array():

from parallelproj import to_numpy_array
from parallelproj.data import to_subset_mmap

y_mmap = to_subset_mmap(to_numpy_array(y), subset_slices, "y.bin")

Examples

RAM-efficient OSEM with disk-backed TOF sinograms

RAM-efficient OSEM with disk-backed TOF sinograms