Skip to content

Developer notes

A few random notes for developers and contributors. This page needs to be reorganized a bit.

Manage script

A bash script manage.sh script at the repository root provides commands for build, test, doc generation, and so on.

Command Description
./manage.sh build recompile the library
./manage.sh doc rebuild the doc website in site/
./manage.sh docs serve the website on localhost:8000
./manage.sh cython update the Cython binding definitions and recompile the Python module
./manage.sh test test_array_ run all tests starting with the given string

Documentation building

We use mkdocs, with material theme, and several markdown, theme, and mkdocs plugins. See mkdocs.yml. The site is generated in the site/ subfolder. We use GitHub Pages to serve the website.

Several parts of the documentation are auto-generated, via mkdocs hooks implemented in utils/hooks.py. Building the documentation requires Python dependencies found in utils/requirements-build.txt. In particular, we use the mkdocs-simple-hooks package to make it possible to use custom Python functions as mkdocs plugin hooks.

  • API documentation: the list of functions to document is found in the docs/api/*.md files. At documentation build time, the API doc generation script (utils/gendoc.py) parses the library header files, extracts the doxygen docstrings, and inserts them at the right places in the API documentation pages.
  • Enumerations: the documentation file api/enums.md contains a list of headers of enumerations. A script parses the enums in the library header files and inserts them in this file, at documentation build time.
  • Colormaps: colormaps definitions are saved in a CSV file in data/textures/color_texture.csv. This file is parsed by utils/export_colormap.py and the table of all colormaps is automatically generated, using NumPy and Pillow to generate base64-encoded individual colormap images. The table is inserted at the end of docs/user/colormaps.md.
  • Visual documentation: visuals are documented manually. The screenshots are generated by the builtin_visual.c unit tests.
  • Graphics documentation: the item, vertex, params structure fields are automatically generated by a script that parses the relevant struct definitions in the library header files.
  • Code snippets and screenshots: the documentation build script parses <!-- CODE_PYTHON path/to/file.py --> and <!-- IMAGE path/to/image.png --> in documentation sources and inserts the code file contents, or the image.

Note

The API doc generation uses joblib to save time when live-regenerating the documentation. However the cache in utils/.joblib must be deleted (so that it's automatically recreated) whenever the Datoviz code/API changes. Otherwise, the API doc generation script may fail.

Shaders

All shaders include common GLSL files found in include/datoviz/glsl/. This path must be passed to the glslc command with the -I flag. This is what the CMake script is using.

Compiled shaders of the builtin graphics are bundled into the library, using a special CMake command. The binary contents of the SPIR-V-compiled shaders are integrated in build/_shaders.c, which is compiled along with the other C source files of the library.

Dependencies

Dear ImGUI

Datoviz integrates Dear ImGUI via a git submodule (fork in the Datoviz GitHub organization). There's a custom branch based on master, but which an additional patch applied to it in order to support creating GUIs with integrated Datoviz canvases (not yet implemented).

C formatting

We use clang format to automatically format all C source files. The rules are defined in .clang-format.

We follow loosely this coding guide.

Command-line tool

Datoviz includes an executable that implements test and examples, implemented in the cli/ subfolder.

Shaders and binary resource embedding

Important binary resources such as SPIR-V compiled shaders of all included graphics, and the colormap texture, are built directly into the compiled library object. A cmake script loads these files and generates big build/_colortex.c and build/_shaders.c files, which are then compiled and linked into the library.

Environment variables

Environment variable Description
DVZ_FPS=1 Show the number of frames per second
DVZ_LOG_LEVEL=0 Logging level
  • Vertical synchronization is activated by default. The refresh rate is typically limited to 60 FPS. Deactivating it (which is automatic when using DVZ_FPS=1) leads to the event loop running as fast as possible, which is useful for benchmarking. It may lead to high CPU and GPU utilization, whereas vertical synchronization is typically light on CPU cycles. Note also that user interaction seems laggy when vertical synchronization is active (the default). When it comes to GUI interaction (mouse movements, drag and drop, and so on), we're used to lags lower than 10 milliseconds, which a frame rate of 60 FPS cannot achieve.
  • Logging levels: 0=trace, 1=debug, 2=info, 3=warning, 4=error
  • DPI scaling factor: Datoviz natively supports DPI scaling for linewidths, font size, axes, etc. Since automatic cross-platform DPI detection does not seem reliable, Datoviz simply uses sensible defaults but provides an easy way for the user to increase or decrease the DPI via this environment variable. This is useful on high-DPI/Retina monitors.