Skip to content

webknossos.dataset.dataset

#   class Dataset:

A dataset is the entry point of the Dataset API. An existing dataset on disk can be opened or new datasets can be created.

A dataset stores the data in .wkw files on disk with metadata in datasource-properties.json. The information in those files are kept in sync with the object.

Each dataset consists of one or more layers (webknossos.dataset.layer.Layer), which themselves can comprise multiple magnifications (webknossos.dataset.mag_view.MagView).

When using Dataset.open_remote() an instance of the RemoteDataset subclass is returned.

#   Dataset( dataset_path: Union[str, os.PathLike], voxel_size: Union[Tuple[float, float, float], NoneType] = None, name: Union[str, NoneType] = None, exist_ok: bool = _UNSET, *, scale: Union[Tuple[float, float, float], NoneType] = None, read_only: bool = False )

Creates a new dataset and the associated datasource-properties.json. If the dataset already exists and exist_ok is set to True, it is opened (the provided voxel_size and name are asserted to match the existing dataset). Currently exist_ok=True is the deprecated default and will change in future releases. Please use Dataset.open if you intend to open an existing dataset and don't want/need the creation behavior. scale is deprecated, please use voxel_size instead.

#  
@classmethod
def open( cls, dataset_path: Union[str, os.PathLike] ) -> webknossos.dataset.dataset.Dataset:

To open an existing dataset on disk, simply call Dataset.open("your_path"). This requires datasource-properties.json to exist in this folder. Based on the datasource-properties.json, a dataset object is constructed. Only layers and magnifications that are listed in the properties are loaded (even though there might exist more layers or magnifications on disk).

The dataset_path refers to the top level directory of the dataset (excluding layer or magnification names).

#  
@classmethod
def open_remote( cls, dataset_name_or_url: str, organization_id: Union[str, NoneType] = None, sharing_token: Union[str, NoneType] = None, webknossos_url: Union[str, NoneType] = None ) -> webknossos.dataset.dataset.RemoteDataset:

Opens a remote webknossos dataset. Image data is accessed via network requests. Dataset metadata such as allowed teams or the sharing token can be read and set via the respective RemoteDataset properties.

  • dataset_name_or_url may be a dataset name or a full URL to a dataset view, e.g. https://webknossos.org/datasets/scalable_minds/l4_sample_dev/view If a URL is used, organization_id, webknossos_url and sharing_token must not be set.
  • organization_id may be supplied if a dataset name was used in the previous argument, it defaults to your current organization from the webknossos_context. You can find your organization_id here.
  • sharing_token may be supplied if a dataset name was used and can specify a sharing token.
  • webknossos_url may be supplied if a dataset name was used, and allows to specifiy in which webknossos instance to search for the dataset. It defaults to the url from your current webknossos_context, using https://webknossos.org as a fallback.
#  
@classmethod
def download( cls, dataset_name_or_url: str, organization_id: Union[str, NoneType] = None, sharing_token: Union[str, NoneType] = None, webknossos_url: Union[str, NoneType] = None, bbox: Union[webknossos.geometry.bounding_box.BoundingBox, NoneType] = None, layers: Union[List[str], str, NoneType] = None, mags: Union[List[webknossos.geometry.mag.Mag], NoneType] = None, path: Union[str, os.PathLike, NoneType] = None, exist_ok: bool = False ) -> webknossos.dataset.dataset.Dataset:

Downloads a dataset and returns the Dataset instance.

  • dataset_name_or_url may be a dataset name or a full URL to a dataset view, e.g. https://webknossos.org/datasets/scalable_minds/l4_sample_dev/view If a URL is used, organization_id, webknossos_url and sharing_token must not be set.
  • organization_id may be supplied if a dataset name was used in the previous argument, it defaults to your current organization from the webknossos_context. You can find your organization_id here.
  • sharing_token may be supplied if a dataset name was used and can specify a sharing token.
  • webknossos_url may be supplied if a dataset name was used, and allows to specifiy in which webknossos instance to search for the dataset. It defaults to the url from your current webknossos_context, using https://webknossos.org as a fallback.
  • bbox, layers, and mags specify which parts of the dataset to download. If nothing is specified the whole image, all layers, and all mags are downloaded respectively.
  • path and exist_ok specify where to save the downloaded dataset and whether to overwrite if the path exists.

Getter for dictionary containing all layers.

#   voxel_size: Tuple[float, float, float]
#   scale: Tuple[float, float, float]

Deprecated, use voxel_size instead.

#   name: str
#   default_view_configuration: Union[webknossos.dataset.properties.DatasetViewConfiguration, NoneType]
#   read_only: bool
#   def upload( self, new_dataset_name: Union[str, NoneType] = None, layers_to_link: Union[list[webknossos.client._upload_dataset.LayerToLink], NoneType] = None, jobs: Union[int, NoneType] = None ) -> webknossos.dataset.dataset.RemoteDataset:

Uploads this dataset to webKnossos.

The new_dataset_name parameter allows to assign a specific name for the dataset. layers_to_link allows to add (or override) a layer in the uploaded dataset, so that it links to a layer of an existing dataset in webKnossos. That way, already existing layers don't need to be uploaded again.

If supplied, the jobs parameter will determine the number of simultaneous chunk uploads. Defaults to 5.

Returns the RemoteDataset upon successful upload.

#   def get_layer(self, layer_name: str) -> webknossos.dataset.layer.Layer:

Returns the layer called layer_name of this dataset. The return type is webknossos.dataset.layer.Layer.

This function raises an IndexError if the specified layer_name does not exist.

#   def add_layer( self, layer_name: str, category: Literal['color', 'segmentation'], dtype_per_layer: Union[str, numpy.dtype, type, NoneType] = None, dtype_per_channel: Union[str, numpy.dtype, type, NoneType] = None, num_channels: Union[int, NoneType] = None, data_format: Union[str, webknossos.dataset._array.DataFormat] = <DataFormat.WKW: 'wkw'>, **kwargs: Any ) -> webknossos.dataset.layer.Layer:

Creates a new layer called layer_name and adds it to the dataset. The dtype can either be specified per layer or per channel. If neither of them are specified, uint8 per channel is used as default. When creating a "Segmentation Layer" (category="segmentation"), the parameter largest_segment_id also has to be specified.

Creates the folder layer_name in the directory of self.path.

WKW layers can only be added to datasets on local file systems.

The return type is webknossos.dataset.layer.Layer.

This function raises an IndexError if the specified layer_name already exists.

#   def get_or_add_layer( self, layer_name: str, category: Literal['color', 'segmentation'], dtype_per_layer: Union[str, numpy.dtype, type, NoneType] = None, dtype_per_channel: Union[str, numpy.dtype, type, NoneType] = None, num_channels: Union[int, NoneType] = None, data_format: Union[str, webknossos.dataset._array.DataFormat] = <DataFormat.WKW: 'wkw'>, **kwargs: Any ) -> webknossos.dataset.layer.Layer:

Creates a new layer called layer_name and adds it to the dataset, in case it did not exist before. Then, returns the layer.

For more information see add_layer.

#   def add_layer_like( self, other_layer: webknossos.dataset.layer.Layer, layer_name: str ) -> webknossos.dataset.layer.Layer:
#   def add_layer_for_existing_files( self, layer_name: str, category: Literal['color', 'segmentation'], **kwargs: Any ) -> webknossos.dataset.layer.Layer:
#   def get_segmentation_layer(self) -> webknossos.dataset.layer.SegmentationLayer:

Deprecated, please use get_segmentation_layers().

Returns the only segmentation layer. Fails with a IndexError if there are multiple segmentation layers or none.

#   def get_segmentation_layers(self) -> List[webknossos.dataset.layer.SegmentationLayer]:

Returns all segmentation layers.

#   def get_color_layer(self) -> webknossos.dataset.layer.Layer:

Deprecated, please use get_color_layers().

Returns the only color layer. Fails with a RuntimeError if there are multiple color layers or none.

#   def get_color_layers(self) -> List[webknossos.dataset.layer.Layer]:

Returns all color layers.

#   def delete_layer(self, layer_name: str) -> None:

Deletes the layer from the datasource-properties.json and the data from disk.

#   def add_copy_layer( self, foreign_layer: Union[str, pathlib.Path, webknossos.dataset.layer.Layer], new_layer_name: Union[str, NoneType] = None ) -> webknossos.dataset.layer.Layer:

Copies the data at foreign_layer which belongs to another dataset to the current dataset. Additionally, the relevant information from the datasource-properties.json of the other dataset are copied too. If new_layer_name is None, the name of the foreign layer is used.

#   def copy_dataset( self, new_dataset_path: Union[str, pathlib.Path], voxel_size: Union[Tuple[float, float, float], NoneType] = None, chunk_size: Union[webknossos.geometry.vec3_int.Vec3Int, Tuple[int, int, int], numpy.ndarray, Iterable[int], int, NoneType] = None, chunks_per_shard: Union[webknossos.geometry.vec3_int.Vec3Int, Tuple[int, int, int], numpy.ndarray, Iterable[int], int, NoneType] = None, data_format: Union[str, webknossos.dataset._array.DataFormat, NoneType] = None, compress: Union[bool, NoneType] = None, block_len: Union[int, NoneType] = None, file_len: Union[int, NoneType] = None, args: Union[argparse.Namespace, NoneType] = None ) -> webknossos.dataset.dataset.Dataset:

Creates a new dataset at new_dataset_path and copies the data from the current dataset to empty_target_ds. If not specified otherwise, the voxel_size, chunk_size, chunks_per_shard and compress of the current dataset are also used for the new dataset. The method also accepts the parameters block_len and file_size, which were deprecated by chunk_size and chunks_per_shard. WKW layers can only be copied to datasets on local file systems.

#   def shallow_copy_dataset( self, new_dataset_path: Union[str, os.PathLike], name: Union[str, NoneType] = None, make_relative: bool = False, layers_to_ignore: Union[Iterable[str], NoneType] = None ) -> webknossos.dataset.dataset.Dataset:

Create a new dataset at the given path. Link all mags of all existing layers. In addition, link all other directories in all layer directories to make this method robust against additional files e.g. layer/mappings/agglomerate_view.hdf5. This method becomes useful when exposing a dataset to webknossos. Only datasets on local filesystems can be shallow copied.

#  
@classmethod
def create( cls, dataset_path: Union[str, os.PathLike], voxel_size: Tuple[float, float, float], name: Union[str, NoneType] = None ) -> webknossos.dataset.dataset.Dataset:

Deprecated, please use the constructor Dataset() instead.

#  
@classmethod
def get_or_create( cls, dataset_path: Union[str, pathlib.Path], voxel_size: Tuple[float, float, float], name: Union[str, NoneType] = None ) -> webknossos.dataset.dataset.Dataset:

Deprecated, please use the constructor Dataset() instead.

#   class RemoteDataset(Dataset):

Representation of a dataset on the webknossos server, returned from Dataset.open_remote(). Read-only image data is streamed from the webknossos server using the same interface as Dataset. Additionally, metadata can be set via the additional properties below.

#   RemoteDataset( dataset_path: upath.core.UPath, dataset_name: str, organization_id: str, sharing_token: Union[str, NoneType], context: AbstractContextManager )

Do not call manually, please use Dataset.remote_open() instead.

#  
@classmethod
def open( cls, dataset_path: Union[str, os.PathLike] ) -> webknossos.dataset.dataset.Dataset:

Do not call manually, please use Dataset.remote_open() instead.

#   url: str
#   display_name: Union[str, NoneType]
#   description: Union[str, NoneType]
#   tags: Tuple[str, ...]
#   is_public: bool
#   sharing_token: str
#   allowed_teams: tuple[webknossos.administration.user.Team, ...]
Back to top