opensilex_python_client

A Python package for importing and managing variables and measurement data in OpenSILEX. Give it a CSV file and a YAML configuration, and the CLI:

• Auto-generates URIs from component names (no manual calculation needed)
• Automatically creates components (Entity, Characteristic, Method, Unit)
• Manages groups with support for up to 2 groups per variable
• Imports measurement data (long and wide CSV formats) with a full per-row report

Quick Start

Prerequisites: Python >= 3.14 and a running OpenSILEX instance (default: http://localhost:8666).

1. Install uv

Installation docs

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Install the package

Requires an SSH key associated with your GitLab account (see GitLab SSH keys).

uv tool install git+ssh://git@forge.inrae.fr/OpenSILEX/data-analysis-visualisation/opensilex-clients/opensilex-ws-python-client.git

Alternatively, install from the public GitHub mirror (no SSH key needed):

uv tool install https://github.com/OpenSILEX/opensilexClientPython.git

This installs five CLI commands, all prefixed with opensilex-py- (see Commands).

3. Download the example files

opensilex-py-download-variable-config-example

Copies test_variables.csv and test_config.yaml into the current directory.

4. Import variables

opensilex-py-run-variable-import \
    --host http://localhost:8666/rest \
    --identifier guest@opensilex.org \
    --password guest \
    --csv test_variables.csv \
    --config test_config.yaml \
    --verbose

During import you will see progress bars and summary lines: Components summary: created=N, existing=N, failed=N and Variable creation summary: created=N, existing=N, failed=N. All detailed logs are written to a timestamped file in the CSV directory.

Commands

Command	Description
opensilex-py-download-variable-config-example	Download the example CSV + YAML files for variable import
opensilex-py-run-variable-import	Import variables from a CSV (components + variables + groups)
opensilex-py-download-data-config-example	Download the example CSV + YAML files for data import
opensilex-py-run-data-import	Import measurement data from a CSV via the AddList API (POST /core/data)
opensilex-py-generate-data-config	Generate a starter YAML configuration from a CSV file

Every command supports --help for the full argument list. --host, --identifier and --password can also be supplied via the OPENSILEX_HOST, OPENSILEX_IDENTIFIER and OPENSILEX_PASSWORD environment variables — prefer the environment variable for the password rather than passing it in clear text on the command line.

Usage Examples

Import variables

# download the example files (test_variables.csv, test_config.yaml, ...)
opensilex-py-download-variable-config-example

# import variables (guest credentials example)
opensilex-py-run-variable-import \
    --host http://localhost:8666/rest \
    --identifier guest@opensilex.org \
    --password guest \
    --csv test_variables.csv \
    --config test_config.yaml \
    --verbose

# quick import without group attachment
opensilex-py-run-variable-import --csv variables.csv --config config.yaml --skip-groups

Full argument reference, column mapping, YAML configuration and CSV examples: docs/variables.md.

Import data

# download the example files (config + CSV, long and wide formats)
opensilex-py-download-data-config-example --dest ./data_examples

# import data
opensilex-py-run-data-import \
    --host http://localhost:8666/rest \
    --identifier admin@opensilex.org \
    --password secret \
    --csv data.csv \
    --config config.yaml \
    --verbose

A report CSV <csv>_data_report.csv (plus <csv>_data_errors.csv when needed) is written next to the input file. Full argument reference, YAML configuration (long/wide formats, mappings, provenance) and report details: docs/data-import.md.

Generate a data config

Generate a starter YAML configuration from a CSV file (header row and column roles are auto-detected, distinct values pre-filled as mapping keys):

# long format (default) – 1 row = 1 measurement
opensilex-py-generate-data-config --csv data.csv

# wide format – 1 row = N measurements (one column per measurement)
opensilex-py-generate-data-config --csv data.csv --format wide

Download example files

# variable import examples (default: current directory)
opensilex-py-download-variable-config-example

# data import examples into a custom directory
opensilex-py-download-data-config-example --dest ./data_examples

Configuration

Each import is driven by a CSV file (the data) and a YAML configuration file (column mapping, groups, mappings, batch size…).

Since v2.0, CSV column names can be customized via the YAML config. Each semantic role (entity, characteristic, method, unit, variable, datatype, groups…) can be mapped to either a column index (1-based integer) or a column header name (string):

csv:
  column_mappings:
    entity_name: 1                  # 1st CSV column
    characteristic_name: "Caract"   # column header name
    method_name: 3
    unit_name: 4
    variable_name: "NomVar"
    datatype_uri: "TypeDonnee"
    variable_description: "Description"
    group1: 9
    group2: 10

• Required roles (entity_name, characteristic_name, method_name, unit_name, variable_name, datatype_uri): if the resolved column is not found, the import aborts.
• Optional roles (variable_description, variable_alternative_name, time_interval, group1, group2, …): if the column is not found, a warning is printed and the import continues.

See docs/variables.md for the full reference and examples.

Logging

The package uses Python's logging module with a Rich formatting handler for structured, color-coded output.

from opensilex_python_client import logger

logger.info("Starting import...")
logger.debug("API response: %s", response)
logger.warning("Variable already exists: %s", name)
logger.error("Failed to connect: %s", exc_info=True)

Set the log level at runtime with logger.setLevel(logging.DEBUG). Each module exposes a logger via the centralized get_logger() function (from opensilex_python_client._logging import get_logger).

For file logging, setup_logging() configures both console and file logging in one call:

from opensilex_python_client._logging import setup_logging

setup_logging(log_dir="./logs")  # → logs/<timestamp>_import.log

• Console: only WARNING and above are displayed (keeps terminal output clean)
• File: all levels are recorded (full details for debugging)

See docs/development.md — Logging Details for the setup_logging() parameters, the console vs logger comparison, and the log output format.

Documentation

• Variable import reference — CLI arguments, column mapping, YAML configuration, execution flow, API functions
• Data import reference — CLI arguments, YAML configuration (long/wide), report and failure isolation
• Migration guide v1 → v2 — changes from standalone scripts to the package
• Development guide — uv, Ruff, pdoc, Jupyter tools, Git, semantic-release
• OpenSILEX website
• OpenSILEX technical documentation
• uv documentation

API documentation is generated with pdoc — see the Development guide for the generate/serve commands.

Development

uv sync                  # install all dependencies (dev group included)
ruff format src tests    # format
ruff check --fix src tests  # lint + auto-fix
pytest                   # run the test suite

Versioning uses python-semantic-release with Conventional Commits (feat:, fix:, docs:, …). See the Development guide for the full workflow.