Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,52 @@ Format based on [Keep a Changelog](https://keepachangelog.com/pl/1.1.0/);
semantic versioning. The version of the data schema is separate from the version of the package
and is reported as `SCHEMA_VERSION`.

## 1.6.0 (2026-10-24)

### Added
- **Charts** (`gridqueue.plots`, optional extra `pip install "gridqueue[plot]"`): field coverage,
quality rules, column-shift audit, stage of the proceeding, requested capacity by class and
voltage, Kaplan-Meier processing-time curves with censoring and a 95 % band, node loading,
flows / transitions / linkage audit of two editions, location resolution, a four-panel
overview, and `save_all`, which draws every chart the data supports and lists the skipped ones
with the reason. A chart is a view of a result, never a second computation of it. Figures are
created without `pyplot`, so the module runs unchanged in scripts, notebooks and the browser.
- **CLI**: `gridqueue compare` (flows, transitions, linkage precision and sensitivity of two
editions) and `gridqueue plot` (every chart, PNG/SVG/PDF).
- `parse_sources`: any number of files of one or several publishers into one panel; every file is
parsed by a fresh adapter instance, so each document keeps the as-at date it states.
`split_editions` separates two or more editions of one register dropped together.
- `kaplan_meier_by_group`: Kaplan-Meier summary per group (or per combination of columns) with the
inclusion criterion of the article.
- `geocode_frame`: writes the resolver's result back to the panel (`wspolrzedne` as `"lat,lon"`,
`_geo_pewnosc`, `_geo_dokladnosc`, `_geo_powod`); source coordinates are kept.
- `node_loading(..., napiecie=...)`: voltage filter of the queue. `napiecie="110"` is the base
variant of the article (median 6.500 / 4.688, 29 of 36 / 33 of 37 for ENERGA-OPERATOR,
31 August 2026); the default is the all-voltage upper variant (8.611 / 7.638).
- `gridqueue.examples`: a synthetic panel with node rows, two synthetic editions and a small
gazetteer, labelled as synthetic, for trying every function without a real disclosure.
- **Browser page, simple view** (the default): drop the documents and get the main charts, a few
sentences with the key numbers, and one ZIP with charts, data and report. Two editions of one
register dropped together are recognised by their as-at dates and compared.
- **Browser page, full view**: several files at once, manual choice of the adapter, a later edition
(flows, transitions, linkage audit), an optional gazetteer, schema validation and the column-shift
audit, processing times and node loading, all charts (PNG 300 dpi and SVG), and export to CSV,
XLSX, JSON and one ZIP archive; two synthetic demonstration modes; a table of what runs where.
- **Colab notebook**: installs 1.6.0 with charts; adds schema validation, the column-shift audit,
key sensitivity (checked against the article), ENERGA-OPERATOR processing times and node loading
(both checked against the Supplementary Material), the resolver on a synthetic gazetteer, all
charts and a downloadable results archive.

### Fixed
- **Censoring date of `processing_time`.** The default censored every open application at the
single latest publication date of the frame, node rows included. ENERGA-OPERATOR publishes its
applications as at 30 June and its node capacities as at 31 August 2026, so every censored time
was two months too long (median for consumption at 110 kV 256 d instead of 240 d). Each row is
now censored at its own publication date, which is also correct for panels of several publishers.
- **Interval of the Kaplan-Meier median**: the lower and upper bounds were swapped
(`mediana_dolna > mediana > mediana_gorna`).
- `kaplan_meier` raised `KeyError` for a group in which every observation is censored.

## 1.5.2 (2026-10-17)

### Added
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ abstract: >-
and the corresponding registers from other jurisdictions into a single set of rows in a 21-field
schema derived from the structure of the legal obligation.
type: software
version: "1.2.1"
date-released: "2026-10-04"
version: "1.6.0"
date-released: "2026-10-24"
license: Apache-2.0
repository-code: "https://github.com/s-matysik/gridqueue"
url: "https://s-matysik.github.io/gridqueue/"
Expand Down
14 changes: 7 additions & 7 deletions CODE_METADATA.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,22 @@ listed again in the final section.

| Nr | Item | Value |
|----|------|-------|
| C1 | Current code version | `1.2.1` (declared identically in `pyproject.toml` and in `gridqueue.__version__`; CI asserts the two agree on every push) |
| C2 | Permanent link to code / repository used for this code version | https://github.com/s-matysik/gridqueue (tag `v1.2.1`) |
| C3 | Permanent link to Reproducible Capsule | ⟦TEAM DECISION⟧, archive DOI. The repository is ready for a Zenodo deposit of tag `v1.2.1`; enabling the integration requires the team's account |
| C1 | Current code version | `1.6.0` (declared identically in `pyproject.toml` and in `gridqueue.__version__`; CI asserts the two agree on every push) |
| C2 | Permanent link to code / repository used for this code version | https://github.com/s-matysik/gridqueue (tag `v1.6.0`) |
| C3 | Permanent link to Reproducible Capsule | ⟦TEAM DECISION⟧, archive DOI. The repository is ready for a Zenodo deposit of tag `v1.6.0`; enabling the integration requires the team's account |
| C4 | Legal Code License | Apache-2.0, `LICENSE.txt` at the distribution root. Input data licences are fixed and separate: OpenStreetMap gazetteer under ODbL 1.0; operators' statutory publications under the terms each publisher applies to its own site |
| C5 | Code versioning system used | git |
| C6 | Software code languages, tools and services used | Python, `requires-python >= 3.10`; build backend `setuptools.build_meta` (`setuptools>=68`); test runner `pytest` |
| C7 | Compilation requirements, operating environments and dependencies | No compilation (pure Python). Runtime: `pandas>=2.0`, `pdfplumber>=0.11`, `openpyxl>=3.1`, `requests>=2.31`, `pyarrow>=14`. Test extra: `pytest>=8.0`. Build: `setuptools>=68`. The location resolver needs no third-party string-matching library; matching is implemented in `geoloc.py` |
| C7 | Compilation requirements, operating environments and dependencies | No compilation (pure Python). Runtime: `pandas>=2.0`, `pdfplumber>=0.11`, `openpyxl>=3.1`, `requests>=2.31`, `pyarrow>=14`. Optional extra `plot`: `matplotlib>=3.7` (charts). Test extra: `pytest>=8.0`, `matplotlib>=3.7`. Build: `setuptools>=68`. The location resolver needs no third-party string-matching library; matching is implemented in `geoloc.py` |
| C8 | If available, link to developer documentation / manual | https://s-matysik.github.io/gridqueue/, eight generated pages (schema with legal basis per field, adapters, extraction method, quality rules, measured validation, CLI, limitations). `README.md` at the distribution root duplicates the quick path |
| C9 | Support email for questions | ⟦TEAM DECISION⟧, corresponding author's institutional address |

## Required Metadata, Software

| Nr | Item | Value |
|----|------|-------|
| S1 | Current software version | `1.2.1`, matching C1 |
| S2 | Permanent link to executables of this version | Source distribution only, `gridqueue-1.2.1.tar.gz`, pure Python, no binaries, no compiled extensions |
| S1 | Current software version | `1.6.0`, matching C1 |
| S2 | Permanent link to executables of this version | Source distribution only, `gridqueue-1.6.0.tar.gz`, pure Python, no binaries, no compiled extensions |
| S3 | Legal Software License | Apache-2.0, same licence as C4 |
| S4 | Computing platforms / Operating Systems | Linux, macOS, Windows × Python 3.10, 3.11, 3.12, 3.13, a twelve-cell matrix exercised by GitHub Actions (`.github/workflows/ci.yml`) on every push and pull request. The measured figures reported in the paper were produced on macOS/arm64 with Python 3.11.15 |
| S5 | Installation requirements & dependencies | As C7. No GPU, no network access required at run time once the source documents and the gazetteer are local |
Expand Down Expand Up @@ -113,7 +113,7 @@ blocker is server-side client filtering, the site returns a rejection page to th
control the operator put in place deliberately, so it was not attempted. Electricity North West:
rows behind the same access control as UK Power Networks; metadata only.

**Size.** Read from the built source distribution `gridqueue-1.2.1.tar.gz`: 7 core modules under `src/gridqueue`, 2,125 lines, `geoloc.py` 672, `schema.py` 480, `quality.py` 370, `layout.py` 353, `cli.py` 148, `registry.py` 64, `__init__.py` 38, and 9 adapter modules, 1,777 lines (`pl_pse.py` 303, `pl_energa.py` 282, `_ptpiree.py` 278, `pl_boryszew.py` 232, `pl_stoen.py` 194, `uk_nationalgrid.py` 166, `pl_tauron.py` 161, `base.py` 156, `__init__.py` 5); 3,902 lines in total under `src`. `_ptpiree.py` is shared by the PSE and ENERGA adapters because both publishers fill the same agreed industry template, one as a spreadsheet and one as a print document. Tests: 7 files, 1,040 lines (`test_geoloc_wspolrzedne.py` 280, `test_schema.py` 164, `test_pse_energa.py` 147, `test_adapters.py` 144, `test_quality.py` 124, `test_geoloc.py` 95, `test_layout.py` 86). Runnable workflows shipped alongside the package: `examples/research_use_case.py` 168 lines, and `reproducibility/` 382 lines (`run_figures.py` 248, `run_validation.py` 134). `python -m pytest -q`: **153 passed, 2 skipped (155 collected), 0 failed, 0 errors**, the two skips are the PSE and ENERGA integration tests, which require the source documents and are skipped when those are absent from the working tree. The twelve-cell OS × Python matrix in CI runs the same suite.
**Size.** Read from the built source distribution `gridqueue-1.6.0.tar.gz`: 7 core modules under `src/gridqueue`, 2,125 lines, `geoloc.py` 672, `schema.py` 480, `quality.py` 370, `layout.py` 353, `cli.py` 148, `registry.py` 64, `__init__.py` 38, and 9 adapter modules, 1,777 lines (`pl_pse.py` 303, `pl_energa.py` 282, `_ptpiree.py` 278, `pl_boryszew.py` 232, `pl_stoen.py` 194, `uk_nationalgrid.py` 166, `pl_tauron.py` 161, `base.py` 156, `__init__.py` 5); 3,902 lines in total under `src`. `_ptpiree.py` is shared by the PSE and ENERGA adapters because both publishers fill the same agreed industry template, one as a spreadsheet and one as a print document. Tests: 7 files, 1,040 lines (`test_geoloc_wspolrzedne.py` 280, `test_schema.py` 164, `test_pse_energa.py` 147, `test_adapters.py` 144, `test_quality.py` 124, `test_geoloc.py` 95, `test_layout.py` 86). Runnable workflows shipped alongside the package: `examples/research_use_case.py` 168 lines, and `reproducibility/` 382 lines (`run_figures.py` 248, `run_validation.py` 134). `python -m pytest -q`: **153 passed, 2 skipped (155 collected), 0 failed, 0 errors**, the two skips are the PSE and ENERGA integration tests, which require the source documents and are skipped when those are absent from the working tree. The twelve-cell OS × Python matrix in CI runs the same suite.

**Functionalities.** Document ingest with per-row provenance; layout-based extraction; unit
normalisation to the canonical kW (MW→kW, with the source unit retained in an extension column); resource-class and process-status vocabulary mapping; composite-date
Expand Down
Loading
Loading