Development

This document covers the development tooling used in this project: dependency management with uv, linting and formatting with Ruff, documentation generation with pdoc, Jupyter notebook tooling, Git workflow, and automated versioning with python-semantic-release.

Logging Details

The package uses Python's logging module with a Rich formatting handler for structured, color-coded output. See the README for the quick start.

`setup_logging()` Parameters

Parameter	Type	Default	Description
`log_dir`	`str \| None`	`None`	Directory for auto-generated log files (`<timestamp>_import.log`)
`log_file`	`str \| None`	`None`	Full path to a log file (takes precedence over `log_dir`)
`level`	`int`	`logging.DEBUG`	Logging level for file handler
`enable_markup`	`bool`	`True`	Enable Rich markup in log messages

from opensilex_python_client._logging import setup_logging

# Auto-generate a timestamped log file in the given directory
setup_logging(log_dir="./logs")
# → Creates logs/20260721_143022_import.log

# Or specify a full path
setup_logging(log_file="/var/log/myapp.log", level=logging.INFO)

Rich Console vs Logger

Feature	`_console` (Rich Console)	`logger` (logging)
Input	User-facing CLI output	Structured, programmable logging
Format	Colored, styled, emojis	Color-coded via RichHandler with source file
Use case	Progress bars, status, prompts	Debug traces, warnings, errors
Example	`_console.print("[green]✓[/green] Done")`	`logger.info("Variable created: %s", uri)`

These are complementary — logger for debugging and tracing, _console for user-facing CLI output.

Log Output Format

Each log line includes:

Timestamp: YYYY-MM-DD HH:MM:SS
Level badge: Color-coded [DEBUG], [INFO], [WARNING], [ERROR], [CRITICAL]
Source file: filename.py
Module name: opensilex_python_client.module
Message: Log message

Example output:

2026-07-21 14:30:22,123  [INFO]  import_variables_from_csv  opensilex_python_client.variables.import_variables_from_csv  Loading CSV and configuration: csv=test.csv, config=config.yaml

uv Integration

This project uses uv for dependency management and packaging. uv provides a modern and extremely fast way to manage Python dependencies and build packages.

Features

Dependency Management: uv manages project dependencies through pyproject.toml
Virtual Environment: Automatically creates and manages a virtual environment
Build System: Integrated build system for creating Python packages
Lock File: Generates a uv.lock file for reproducible installations

Common Commands

# Install dependencies
uv sync

# Add a new dependency
uv add package-name

# Add a development dependency
uv add --dev package-name

# Update dependencies
uv sync --upgrade

# Run a command within the virtual environment
uv run python script.py

# Activate the virtual environment
uv run shell

Project Structure

The project follows a standard Python package structure:

pyproject.toml: Project configuration and dependencies
uv.lock: Locked dependencies for reproducible builds
src/: Source code directory
tests/: Test files directory

Development

For more information, visit uv's official documentation.

Ruff Integration

Ruff is an extremely fast Python linter and code formatter, written in Rust. It can replace Flake8, Black, isort, pyupgrade, and more, while being much faster than any individual tool.

VSCode Default Configuration

The file .vscode/default_settings.json provides a recommended configuration for using Ruff in VSCode:

{
    "[python]": {
      "editor.formatOnSave": true,
      "editor.codeActionsOnSave": {
        "source.fixAll": "explicit",
        "source.organizeImports": "explicit"
      },
      "editor.defaultFormatter": "charliermarsh.ruff"
    },
    "notebook.formatOnSave.enabled": true,
    "notebook.codeActionsOnSave": {
      "notebook.source.fixAll": "explicit",
      "notebook.source.organizeImports": "explicit"
    },
    "ruff.lineLength": 88
}

Explanation of each line:

editor.formatOnSave: Enables automatic formatting on save for all files.
[python].editor.defaultFormatter: Sets Ruff as the default formatter for Python files.
[python].editor.codeActionsOnSave.source.organizeImports: Organizes Python imports automatically on save.
[python].editor.codeActionsOnSave.source.fixAll: Applies all available code fixes (including linting) on save.
ruff.lineLength: Line length for your Python files

Useful Ruff Commands

You can run the following commands directly in the shell:

# Lint all Python files in the current directory
ruff check .

# Format all Python files in the current directory
ruff format .

# Automatically fix all auto-fixable problems
ruff check . --fix

For more information, see the official Ruff VSCode extension documentation and the Ruff documentation.

You can enable specific rules over a catalog of over 800+ rules, depending on your needs or framework of choice. Check it out at the Ruff rules documentation.

Documentation (pdoc)

This project uses pdoc to generate and serve documentation.

Configuration

The documentation configuration is managed in the pyproject.toml file:

[tool.pyscaf.documentation]
output_path = "docs"

[tool.pyscaf.documentation.pdoc]
# pdoc arguments are automatically converted to CLI arguments
# Boolean values: true -> --flag, false -> --no-flag
# Lists: ["value1", "value2"] -> --flag value1 --flag value2
# Strings: "value" -> --flag value

Scripts

Two scripts are available to manage documentation. Both scripts use the configuration defined in the [tool.pyscaf.documentation.pdoc] section.

These scripts are dev-only: they live in src/pyscaf/ and are not installed with the package (e.g. via uv tool install). Run them from the project root with uv run.

Generate documentation

Generates static documentation files to the directory specified in tool.pyscaf.documentation.output_path.

uv run python -m pyscaf.documentation.scripts.parse_doc generate

Serve documentation

Starts a local documentation server for interactive browsing.

uv run python -m pyscaf.documentation.scripts.parse_doc serve

pdoc Arguments

All arguments in the [tool.pyscaf.documentation.pdoc] section are automatically converted to pdoc CLI arguments:

Boolean values: true becomes --flag, false becomes --no-flag
Lists: ["value1", "value2"] becomes --flag value1 --flag value2
Strings: "value" becomes --flag value

The output argument is dropped, as the behaviour to write instead of serve depends on the script use.

For example:

[tool.pyscaf.documentation.pdoc]
html = true
show_source = false
template_directory = "custom_templates"
external_links = ["https://example.com"]

Becomes:

pdoc --html --no-show-source --template-directory custom_templates --external-links https://example.com

Jupyter Tools

This project includes a set of tools for manipulating Jupyter notebooks.

Features

The Jupyter Tools action adds the following capabilities:

py_to_notebook.py — Convert Python files to Jupyter notebooks
- Handles cell markers and tags
- Preserves markdown links
- Supports jupytext format
execute_notebook.py — Execute Jupyter notebooks
- Runs notebooks with proper error handling
- Updates execution metadata
- Supports parameterized execution
notebook_to_html.py — Convert notebooks to HTML
- Creates standalone HTML files
- Includes CSS styling
- Preserves code highlighting
notebook_to_pdf.py — Convert notebooks to PDF
- Uses nbconvert with LaTeX backend
- Supports custom templates
- Handles complex formatting
main.py — Main CLI interface
- Process entire directories
- Complete pipeline from Python to HTML
- Batch processing capabilities

Dependencies

The following dependencies are added to the project's dev group:

jupytext — For Python to notebook conversion
nbconvert — For notebook format conversion
weasyprint — For PDF generation
pandoc — For document conversion

Usage

The scripts live in src/pyscaf/jupyter_tools/scripts/ and are dev-only: they are not installed with the package (e.g. via uv tool install). Run them from the project root with uv run:

# Convert a Python file to notebook
uv run python -m pyscaf.jupyter_tools.scripts.py_to_notebook input.py output.ipynb

# Execute a notebook
uv run python -m pyscaf.jupyter_tools.scripts.execute_notebook notebook.ipynb

# Convert notebook to HTML
uv run python -m pyscaf.jupyter_tools.scripts.notebook_to_html notebook.ipynb output.html

# Convert notebook to PDF
uv run python -m pyscaf.jupyter_tools.scripts.notebook_to_pdf notebook.ipynb output.pdf

# Convert all Python files to notebooks (dirs from [tool.pyscaf.jupyter_tools])
uv run python -m pyscaf.jupyter_tools.scripts.main py-to-nb-all

# Execute all notebooks
uv run python -m pyscaf.jupyter_tools.scripts.main exec-nb-all

# Convert all notebooks to HTML
uv run python -m pyscaf.jupyter_tools.scripts.main nb-to-html-all

Requirements

Python 3.14+

Git

This project is hosted on GitLab. Branches are merged to main via merge requests, and commit messages must follow Conventional Commits (see Semantic Release & Changelog below).

Semantic Release & Changelog

This project uses python-semantic-release for automated versioning, changelog generation, and package publishing.

Semantic release automates:

Version management: Automatically bump version numbers based on commit messages
Changelog generation: Create detailed changelogs from conventional commits
Package publishing: Deploy to PyPI (TestPyPI automatically, Production PyPI manually)
GitHub/GitLab releases: Create releases with assets

Conventional Commits

All commit messages must follow the Conventional Commits format:

Prefix	Meaning	Version Bump
`feat:`	New feature	minor
`fix:`	Bug fix	patch
`docs:`	Documentation only	none
`refactor:`	Code refactoring	none
`test:`	Test changes	none
`chore:`	Maintenance tasks	none
`BREAKING CHANGE:`	Breaking change	major

Examples:

git commit -m "feat: add Rich progress bars for variable creation"
git commit -m "fix: remove non-existent OntologyApi patches from tests"
git commit -m "docs: update logging section in README"

Configuration

The action configures pyproject.toml with default semantic-release settings:

[tool.semantic_release]
version_variables = ["src/opensilex_python_client/__init__.py:__version__"]
upload_to_pypi = true
upload_to_release = true
branch = "main"

[tool.semantic_release.remote]
type = "gitlab"  # or "github"

Generating the Changelog

# Generate changelog from commits since last release
uv run semantic-release changelog

# Generate changelog AND bump version (creates git tag)
uv run semantic-release version

# Preview what would be released
uv run semantic-release changelog --dry-run

This will:

Update CHANGELOG.md with all commits since the last version
Bump the version in src/opensilex_python_client/__init__.py
Create a git tag (e.g., v2.1.0)
Push the tag to the remote repository

Workflow

# 1. Make changes with conventional commits
git commit -m "feat: add new feature"
git commit -m "fix: fix a bug"

# 2. Push to remote
git push origin main

# 3. Generate changelog and version
uv run semantic-release version

# 4. Push the new tag
git push origin --tags

PyPI Token Setup

TestPyPI (https://test.pypi.org):
- Go to Account Settings → API tokens
- Create a new token with "Entire account" scope
- Copy the token value
Production PyPI (https://pypi.org):
- Go to Account Settings → API tokens
- Create a new token with "Entire account" scope
- Copy the token value
Add to CI Secrets:
- Go to your repository → Settings → CI/CD → Variables (GitLab) or Settings → Secrets (GitHub)
- Add TEST_PYPI_PASSWORD with your TestPyPI token
- Add PYPI_PASSWORD with your production PyPI token

Navigation Menu

Repository navigation

development.md

development.md

Development

Logging Details

`setup_logging()` Parameters

Rich Console vs Logger

Log Output Format

uv Integration

Features

Common Commands

Project Structure

Development

Ruff Integration

VSCode Default Configuration

Explanation of each line:

Useful Ruff Commands

Documentation (pdoc)

Configuration

Scripts

Generate documentation

Serve documentation

pdoc Arguments

Jupyter Tools

Features

Dependencies

Usage

Requirements

Git

Semantic Release & Changelog

Conventional Commits

Configuration

Generating the Changelog

Workflow

PyPI Token Setup

Resources

Official Documentation

Related Tools

Best Practices

FilesExpand file tree

development.md

Latest commit

History

development.md

File metadata and controls

Development

Logging Details

setup_logging() Parameters

Rich Console vs Logger

Log Output Format

uv Integration

Features

Common Commands

Project Structure

Development

Ruff Integration

VSCode Default Configuration

Explanation of each line:

Useful Ruff Commands

Documentation (pdoc)

Configuration

Scripts

Generate documentation

Serve documentation

pdoc Arguments

Jupyter Tools

Features

Dependencies

Usage

Requirements

Git

Semantic Release & Changelog

Conventional Commits

Configuration

Generating the Changelog

Workflow

PyPI Token Setup

Resources

Official Documentation

Related Tools

Best Practices

`setup_logging()` Parameters