Skip to content

Contributions & Development

How to contribute

We welcome community feedback and contributions! We are happy to have

Development

The webknossos-libs repository is structured as a mono-repo, containing multiple packages:

  • cluster_tools
  • webknossos
  • wkcuber
  • (docs, see below for Documentation)

See below for specifics of the different packages. Let's have a look at the common tooling first:

  • poetry is used for dependency management and publishing. By default, it creates a virtual environment for each package. To run commands inside this package, prefix them with poetry run, e.g. poetry run python myscript.py, or enter the virtual environment with poetry shell. The creation of a separate environment can be disabled (e.g. if you want to manage this manually), see here for details. To install the preferred version for this repository, run pip install -f requirements.txt

To install the dependencies for all sub-projects, run make install. * Tooling we use across the sub-projects: * format.sh: black and isort * lint.sh: pylint * typecheck.sh: mypy * test.sh: pytest and custom scripts

Those are also accessible via make commands from the top-level directory, running the respective scripts for each sub-project , e.g. make format, make lint, …

Internal workflows for scalable minds:

  • CI: We use continuous integration with github actions, please see the CI workflow for details.
  • Releases are triggered via the Automatic release github action, using the Run workflow button in the topright corner. This updates the changelog and pushes a new tag, which triggers another CI run building and publishing the package.

webknossos package

The webknossos folder contains examples, which are not part of the package, but are tested via tests/test_examples.py and added to the documentation (see docs/src/webknossos-py/examples).

The tests also contain functionality for the webKnossos client. There a two modes to run the tests:

  1. test.sh --refresh-snapshots, sending network requests to a webKnossos instance: This expects a local webKnossos setup with specific test data, which is shipped with webKnossos. If you're starting and running webKnossos manually, please use port 9000 (the default) and run the tools/postgres/prepareTestDb.sh script in the webKnossos repository (⚠️ this overwrites your local webKnossos database). Alternatively, a docker-compose setup is started automatically for the tests, see test.sh and tests/docker-compose.yml for details. The network requests & response are recorded as "cassettes" by vcr.py, see next point:
  2. test.sh replays responses from previous network snapshots using vcr.py via pytest-recording. No additional network requests are allowed in this mode.

The code under webknossos/client/_generated is auto-generated! Please don't adapt anything in the generated folder manually, but re-run the code generation.

The client code is generated using openapi-python-client and InducOapi.

To re-generate the code, run 

./generate_client.sh

wkcuber package

Currently the test setup consists of different scripts as well as pytest tests. The following commands are run in CI: 

tar -xzvf testdata/WT1_wkw.tar.gz
poetry run pytest tests
poetry run tests/scripts/all_tests.sh

There is also a test.sh which is currently outdated, see issue #580.

cluster_tools package

For testing the slurm setup a docker-compose setup is available. Please see the respective Readme for details.

For testing the kubernetes setup, we recommend a Kubernetes-in-Docker setup.

Documentation

We render a common documentation for webKnossos itself and webknossos-libs from this repository using mkdocs. Source-files for the documentation are stored at docs/src:

  • docs/src/webknossos: Server & Website documentation, linked from the webknossos repository (must be available under docs/wk-repo, see below).
  • docs/src/api: Generated using pdoc from Python docstrings.
  • docs/src/webknossos-py & docs/src/wkcuber: Documentation for the respective Python Packages

The structure of the documentation page is given by docs/mkdocs.yml.

To generate the documentation locally, clone or link the webKnossos repository to docs/wk-repo first and then start the documentation server 

git clone --depth 1 git@github.com:scalableminds/webknossos.git docs/wk-repo
docs/generate.sh

You can use

  • docs/generate.sh for hot-reloading markdown docs,
  • docs/generate.sh --api to get the pure pdoc documentation, hot-reloading docstrings,
  • docs/generate.sh --persist to persist the docs under docs/out.
Back to top