webknossos.dataset.view ¶

Classes:

View –

A View is essentially a bounding box to a region of a specific StorageBackend that also provides functionality.

View ¶

View(path_to_mag_view: Path, array_info: ArrayInfo, bounding_box: Optional[NDBoundingBox], mag: Mag, read_only: bool = False)

A View is essentially a bounding box to a region of a specific StorageBackend that also provides functionality. Write-operations are restricted to the bounding box. Views are designed to be easily passed around as parameters. A View, in its most basic form, does not have a reference to its StorageBackend.

Do not use this constructor manually. Instead use View.get_view() (also available on a MagView) to get a View.

Methods:

chunk –

This method chunks the view into multiple sub-views of size chunk_shape (in Mag(1)).
content_is_equal –
for_each_chunk –

The view is chunked into multiple sub-views of size chunk_shape (in Mag(1)),
for_zipped_chunks –

This method is similar to for_each_chunk in the sense that it delegates work to smaller chunks,
get_buffered_slice_reader –

The returned reader yields slices of data along a specified axis.
get_buffered_slice_writer –

The returned writer buffers multiple slices before they are written to disk.
get_dtype –

Returns the dtype per channel of the data. For example uint8.
get_view –

Returns a view that is limited to the specified bounding box.
map_chunk –

The view is chunked into multiple sub-views of size chunk_shape (in Mag(1)),
read –

The user can specify which data should be read.
read_bbox –

⚠️ Deprecated. Please use read() with relative_bounding_box or absolute_bounding_box in Mag(1) instead.
read_xyz –

The user can specify the bounding box in the dataset's coordinate system.
write –

The user can specify where the data should be written.

Attributes:

bounding_box (NDBoundingBox) –
global_offset (VecInt) –

⚠️ Deprecated, use view.bounding_box.in_mag(view.mag).topleft instead.
header (Header) –

⚠️ Deprecated, use info instead.
info (ArrayInfo) –
mag (Mag) –
read_only (bool) –
size (VecInt) –

⚠️ Deprecated, use view.bounding_box.in_mag(view.mag).size instead.

bounding_box `property` ¶

bounding_box: NDBoundingBox

global_offset `property` ¶

global_offset: VecInt

⚠️ Deprecated, use view.bounding_box.in_mag(view.mag).topleft instead.

header `property` ¶

header: Header

⚠️ Deprecated, use info instead.

info `property` ¶

info: ArrayInfo

mag `property` ¶

mag: Mag

read_only `property` ¶

read_only: bool

size `property` ¶

size: VecInt

⚠️ Deprecated, use view.bounding_box.in_mag(view.mag).size instead.

chunk ¶

chunk(chunk_shape: VecIntLike, chunk_border_alignments: Optional[VecIntLike] = None, read_only: bool = False) -> Generator[View, None, None]

This method chunks the view into multiple sub-views of size chunk_shape (in Mag(1)). The chunk_border_alignments parameter specifies the alignment of the chunks. The default is to align the chunks to the origin (0, 0, 0).

Example:

# ...
# let 'mag1' be a `MagView`
chunks = mag1.chunk(chunk_shape=(100, 100, 100), chunk_border_alignments=(50, 50, 50))

content_is_equal ¶

content_is_equal(other: View, args: Optional[Namespace] = None, executor: Optional[Executor] = None) -> bool

for_each_chunk ¶

for_each_chunk(func_per_chunk: Callable[[Tuple[View, int]], None], chunk_shape: Optional[Vec3IntLike] = None, executor: Optional[Executor] = None, progress_desc: Optional[str] = None, *, chunk_size: Optional[Vec3IntLike] = None) -> None

The view is chunked into multiple sub-views of size chunk_shape (in Mag(1)), by default one chunk per file. Then, func_per_chunk is performed on each sub-view. Besides the view, the counter i is passed to the func_per_chunk, which can be used for logging. Additional parameters for func_per_chunk can be specified using functools.partial. The computation of each chunk has to be independent of each other. Therefore, the work can be parallelized with executor.

If the View is of type MagView only the bounding box from the properties is chunked.

Example:

from webknossos.utils import named_partial

def some_work(args: Tuple[View, int], some_parameter: int) -> None:
    view_of_single_chunk, i = args
    # perform operations on the view
    ...

# ...
# let 'mag1' be a `MagView`
func = named_partial(some_work, some_parameter=42)
mag1.for_each_chunk(
    func,
)

for_zipped_chunks ¶

for_zipped_chunks(func_per_chunk: Callable[[Tuple[View, View, int]], None], target_view: View, source_chunk_shape: Optional[Vec3IntLike] = None, target_chunk_shape: Optional[Vec3IntLike] = None, executor: Optional[Executor] = None, progress_desc: Optional[str] = None, *, source_chunk_size: Optional[Vec3IntLike] = None, target_chunk_size: Optional[Vec3IntLike] = None) -> None

This method is similar to for_each_chunk in the sense that it delegates work to smaller chunks, given by source_chunk_shape and target_chunk_shape (both in Mag(1), by default using the larger of the source_views and the target_views file-sizes). However, this method also takes another view as a parameter. Both views are chunked simultaneously and a matching pair of chunks is then passed to the function that shall be executed. This is useful if data from one view should be (transformed and) written to a different view, assuming that the transformation of the data can be handled on chunk-level. Additionally to the two views, the counter i is passed to the func_per_chunk, which can be used for logging.

The mapping of chunks from the source view to the target is bijective. The ratio between the size of the source_view (self) and the source_chunk_shape must be equal to the ratio between the target_view and the target_chunk_shape. This guarantees that the number of chunks in the source_view is equal to the number of chunks in the target_view. The target_chunk_shape must be a multiple of the file size on disk to avoid concurrent writes.

Example use case: downsampling from Mag(1) to Mag(2) - size of the views: 16384³ (8192³ in Mag(2) for target_view) - automatic chunk sizes: 2048³, assuming default file-lengths (1024³ in Mag(2), which fits the default file-length of 32*32)

get_buffered_slice_reader ¶

get_buffered_slice_reader(offset: Optional[Vec3IntLike] = None, size: Optional[Vec3IntLike] = None, buffer_size: int = 32, dimension: int = 2, *, relative_bounding_box: Optional[NDBoundingBox] = None, absolute_bounding_box: Optional[NDBoundingBox] = None, use_logging: bool = False) -> BufferedSliceReader

The returned reader yields slices of data along a specified axis. Internally, it reads multiple slices from disk at once and buffers the data.

Arguments:

The user can specify where the writer should start:
- relative_bounding_box in Mag(1)
- absolute_bounding_box in Mag(1)
- ⚠️ deprecated: offset and size in the current Mag, offset used to be relative for View and absolute for MagView
buffer_size: amount of slices that get buffered
dimension: dimension along which the data is sliced (x: 0, y: 1, z: 2; default is 2)).

The reader must be used as a context manager using the with syntax (see example below). Entering the context returns an iterator yielding slices (np.ndarray).

Usage:

view = ...
with view.get_buffered_slice_reader() as reader:
    for slice_data in reader:
        ...

get_buffered_slice_writer ¶

get_buffered_slice_writer(offset: Optional[Vec3IntLike] = None, buffer_size: int = 32, dimension: int = 2, json_update_allowed: bool = True, *, relative_offset: Optional[Vec3IntLike] = None, absolute_offset: Optional[Vec3IntLike] = None, relative_bounding_box: Optional[NDBoundingBox] = None, absolute_bounding_box: Optional[NDBoundingBox] = None, use_logging: bool = False) -> BufferedSliceWriter

The returned writer buffers multiple slices before they are written to disk. As soon as the buffer is full, the data gets written to disk.

Arguments:

The user can specify where the writer should start:
- relative_offset in Mag(1)
- absolute_offset in Mag(1)
- relative_bounding_box in Mag(1)
- absolute_bounding_box in Mag(1)
- ⚠️ deprecated: offset in the current Mag, used to be relative for View and absolute for MagView
buffer_size: amount of slices that get buffered
dimension: dimension along which the data is sliced (x: 0, y: 1, z: 2; default is 2)).

The writer must be used as context manager using the with syntax (see example below), which results in a generator consuming np.ndarray-slices via writer.send(slice). Exiting the context will automatically flush any remaining buffered data to disk.

Usage:

data_cube = ...
view = ...
with view.get_buffered_slice_writer() as writer:
    for data_slice in data_cube:
        writer.send(data_slice)

get_dtype ¶

get_dtype() -> dtype

Returns the dtype per channel of the data. For example uint8.

get_view ¶

get_view(offset: Optional[Vec3IntLike] = None, size: Optional[Vec3IntLike] = None, *, relative_offset: Optional[Vec3IntLike] = None, absolute_offset: Optional[Vec3IntLike] = None, relative_bounding_box: Optional[NDBoundingBox] = None, absolute_bounding_box: Optional[NDBoundingBox] = None, read_only: Optional[bool] = None) -> View

Returns a view that is limited to the specified bounding box. The new view may exceed the bounding box of the current view only if read_only is set to True.

The default is to return the same view as the current bounding box, in case of a MagView that's the layer bounding box. One can supply one of the following keyword argument combinations:

relative_bounding_boxin Mag(1)
absolute_bounding_box in Mag(1)
relative_offset and size, both in Mag(1)
absolute_offset and size, both in Mag(1)
⚠️ deprecated: offset and size, both in the current Mag. offset used to be relative for View and absolute for MagView

Example:

# ...
# let mag1 be a MagView
view = mag1.get_view(absolute_offset=(10, 20, 30), size=(100, 200, 300))

# works because the specified sub-view is completely in the bounding box of the view
sub_view = view.get_view(relative_offset=(50, 60, 70), size=(10, 120, 230))

# fails because the specified sub-view is not completely in the bounding box of the view
invalid_sub_view = view.get_view(relative_offset=(50, 60, 70), size=(999, 120, 230))

# works because `read_only=True`
valid_sub_view = view.get_view(relative_offset=(50, 60, 70), size=(999, 120, 230), read_only=True)

map_chunk ¶

map_chunk(func_per_chunk: Callable[[View], Any], chunk_shape: Optional[Vec3IntLike] = None, executor: Optional[Executor] = None, progress_desc: Optional[str] = None) -> List[Any]

The view is chunked into multiple sub-views of size chunk_shape (in Mag(1)), by default one chunk per file. Then, func_per_chunk is performed on each sub-view and the results are collected in a list. Additional parameters for func_per_chunk can be specified using functools.partial. The computation of each chunk has to be independent of each other. Therefore, the work can be parallelized with executor.

If the View is of type MagView only the bounding box from the properties is chunked.

Example:

from webknossos.utils import named_partial

def some_work(view: View, some_parameter: int) -> None:
    # perform operations on the view
    ...

# ...
# let 'mag1' be a `MagView`
func = named_partial(some_work, some_parameter=42)
results = mag1.map_chunk(
    func,
)

read ¶

read(offset: Optional[Vec3IntLike] = None, size: Optional[Vec3IntLike] = None, *, relative_offset: Optional[Vec3IntLike] = None, absolute_offset: Optional[Vec3IntLike] = None, relative_bounding_box: Optional[NDBoundingBox] = None, absolute_bounding_box: Optional[NDBoundingBox] = None) -> ndarray

The user can specify which data should be read. The default is to read all data of the view's bounding box. Alternatively, one can supply one of the following keyword argument combinations: * relative_offset and size, both in Mag(1) * absolute_offset and size, both in Mag(1) * relative_bounding_box in Mag(1) * absolute_bounding_box in Mag(1) * ⚠️ deprecated: offset and size, both in the current Mag. offset used to be relative for View and absolute for MagView

If the specified bounding box exceeds the data on disk, the rest is padded with 0.

Returns the specified data as a np.array.

Example:

import numpy as np

# ...
# let mag1 be a MagView
view = mag1.get_view(absolute_offset=(10, 20, 30), size=(100, 200, 300))

assert np.array_equal(
    view.read(absolute_offset=(0, 0, 0), size=(100, 200, 300)),
    view.read(),
)

read_bbox ¶

read_bbox(bounding_box: Optional[BoundingBox] = None) -> ndarray

⚠️ Deprecated. Please use read() with relative_bounding_box or absolute_bounding_box in Mag(1) instead. The user can specify the bounding_box in the current mag of the requested data. See read() for more details.

read_xyz ¶

read_xyz(relative_bounding_box: Optional[NDBoundingBox] = None, absolute_bounding_box: Optional[NDBoundingBox] = None) -> ndarray

The user can specify the bounding box in the dataset's coordinate system. The default is to read all data of the view's bounding box. Alternatively, one can supply one of the following keyword arguments: * relative_bounding_box in Mag(1) * absolute_bounding_box in Mag(1)

Returns the specified data as a np.array.

write ¶

write(data: ndarray, offset: Optional[Vec3IntLike] = None, json_update_allowed: bool = True, *, relative_offset: Optional[Vec3IntLike] = None, absolute_offset: Optional[Vec3IntLike] = None, relative_bounding_box: Optional[NDBoundingBox] = None, absolute_bounding_box: Optional[NDBoundingBox] = None) -> None

The user can specify where the data should be written. The default is to write the data to the view's bounding box. Alternatively, one can supply one of the following keywords:

relative_offset in Mag(1) -> only usable for 3D datasets
absolute_offset in Mag(1) -> only usable for 3D datasets
relative_bounding_box in Mag(1)
absolute_bounding_box in Mag(1)

⚠️ The offset parameter is deprecated. This parameter used to be relative for View and absolute for MagView, and specified in the mag of the respective view.

Writing data to a segmentation layer manually does not automatically update the largest_segment_id. To set the largest segment id properly run the refresh_largest_segment_id method on your layer or set the largest_segment_id property manually..

Example:

ds = Dataset(DS_PATH, voxel_size=(1, 1, 1))

segmentation_layer = cast(
    SegmentationLayer,
    ds.add_layer("segmentation", SEGMENTATION_CATEGORY),
)
mag = segmentation_layer.add_mag(Mag(1))

mag.write(data=MY_NP_ARRAY)

segmentation_layer.refresh_largest_segment_id()

Note that writing compressed data which is not aligned with the blocks on disk may result in diminished performance, as full blocks will automatically be read to pad the write actions.

webknossos.dataset.view ¶

View ¶

bounding_box property ¶

global_offset property ¶

header property ¶

info property ¶

mag property ¶

read_only property ¶

size property ¶

chunk ¶

content_is_equal ¶

for_each_chunk ¶

for_zipped_chunks ¶

get_buffered_slice_reader ¶

get_buffered_slice_writer ¶

get_dtype ¶

get_view ¶

map_chunk ¶

read ¶

read_bbox ¶

read_xyz ¶

write ¶

bounding_box `property` ¶

global_offset `property` ¶

header `property` ¶

info `property` ¶

mag `property` ¶

read_only `property` ¶

size `property` ¶