8191: Data adapter

.env

@@ -32,4 +32,18 @@ DEFAULT_URI=http://localhost
 
 ###> app ###
 APP_BROKER_BASE_URI=http://scorpio.local:9090/
+
+# JSON-LD contexts attached to every entity, outermost last so the ETSI core
+# context resolves the NGSI-LD terms and the domain context resolves the
+# Smart Data Models ones.
+ENTER_NGSI_CONTEXT_URLS='https://raw.githubusercontent.com/smart-data-models/dataModel.Parking/master/context.jsonld,https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld'
+
+# A single domain context, for the Link header that read requests need. Only
+# used by `task broker:entities`, and it must be the context defining the type
+# being read — Parking while that is the only model published.
+ENTER_NGSI_DOMAIN_CONTEXT=https://raw.githubusercontent.com/smart-data-models/dataModel.Parking/master/context.jsonld
+
+# Where each feed is read from is not configured here. It belongs to the data
+# set rather than to the environment, and lives in config/sources.yaml — see
+# docs/adr/007-source-manifest.md.
 ###< app ###

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,19 @@
+<?php
+#### Link to ticket
+
+Please add a link to the ticket being addressed by this change.
+
+#### Description
+
+Please include a short description of the suggested change and the reasoning behind the approach you have chosen.
+
+#### Screenshot of the result
+
+If your change affects the user interface you should include a screenshot of the result with the pull request.
+
+#### Checklist
+
+- [ ] My code is covered by test cases.
+- [ ] My code passes our test (all our tests).
+- [ ] My code passes our static analysis suite.
+- [ ] My code passes our continuous integration process.

CHANGELOG.md

@@ -7,4 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- `app:import` command with `--dry-run` and `--limit`, exposed as `task import`.
+- `SourceInterface`: extension point for further data sets, discovered through
+  `#[AutoconfigureTag('app.source')]`.
+- `Wgs84Transformer`: reprojects coordinates from any registered CRS to WGS84,
+  for a single position or a whole GeoJSON geometry of any type.
+- `FeedReader`: reads a feed from a filesystem path or an http(s) URL and decodes
+  it, without interpreting its shape.
+- `NgsiEntity`: builds normalized NGSI-LD entities.
+- `NgsiLdBroker`: idempotent batch upsert to an NGSI-LD context broker.
+- `task broker:entities` for reading entities back out of the broker.
+- Architecture Decision Records under `docs/adr`.
+- Added test suite
+- `config/sources.yaml`: one record per data set, holding the feed URL, its CRS
+  and the model it is published as alongside the metadata no code can state —
+  owner, contact, licence, update frequency, and the source fields deliberately
+  left unpublished with the reason for each. Field names follow DCAT-AP.
+- `SourceCatalog` and `SourceDescriptor`: read and validate the manifest,
+  failing loudly on a missing or malformed record rather than defaulting.
+
+### Changed
+
+- Sources read their feed URL, CRS and model from `config/sources.yaml` instead
+  of holding them in an environment variable and a class constant.
+
+### Removed
+
+- `ENTER_MTM_SPATIALMAPS_HANDICAP_PARKING_SOURCE`, and with it the pattern of
+  one environment variable per data set. Where a feed is read from is a fact
+  about the data set, not about the environment.
+
 [Unreleased]: https://github.com/itk-dev/enter

README.md

@@ -4,6 +4,7 @@ We use [DDEV](https://ddev.com/) and [Task](https://taskfile.dev/) for developme
 
 ``` shell
 task site:install
+```
 
 ``` shell
 task site:update
@@ -12,6 +13,63 @@ ddev launch
 
 Run `task` to see what cool task are available. Running `ddev` can help with other stuff.
 
+## Adapter
+
+Takes an Aarhus open-data set, converts it to [NGSI-LD], and upserts it into the
+context broker.
+
+``` text
+source feed (JSON)
+  → SourceInterface implementation   maps fields, fixes quirks, picks the data model
+    → NgsiEntity                     normalized NGSI-LD: Property / GeoProperty / Relationship
+      → NgsiLdBroker                 POST /ngsi-ld/v1/entityOperations/upsert
+        → context broker
+```
+
+| Class                                       | Responsibility                                    |
+| ------------------------------------------- | ------------------------------------------------- |
+| `App\Source\SourceInterface`                | Contract for one input data set                   |
+| `App\Source\SourceCatalog`                  | Reads the source manifest                         |
+| `App\Source\SourceDescriptor`               | One manifest entry: what a data set is and where  |
+| `App\Source\FeedReader`                     | Feed URL → decoded JSON                           |
+| `App\Source\MtmSpatialMaps\HandicapParking` | Disabled parking bays → `OnStreetParking`         |
+| `App\Geo\Wgs84Transformer`                  | Any registered CRS → WGS84, any GeoJSON geometry  |
+| `App\Ngsi\NgsiEntity`                       | Builds normalized NGSI-LD entities                |
+| `App\Broker\NgsiLdBroker`                   | Batch upsert to the broker                        |
+| `App\Command\ImportCommand`                 | `app:import`                                      |
+
+``` shell
+task import                                                # list the available sources
+task import -- mtm_spatialmaps-handicap-parking                        # import one
+task import -- mtm_spatialmaps-handicap-parking --dry-run --limit 5    # print the payload instead
+task broker:entities -- OnStreetParking 10                 # read back what landed
+```
+
+### Source manifest
+
+Every data set is recorded in [config/sources.yaml](config/sources.yaml), keyed
+by the identifier `app:import` takes as its argument. The feed URL, the
+coordinate reference system it publishes and the Smart Data Model it is
+published as are read from there, so each exists in one place only. The rest of
+an entry is what no code can state: owner, contact, licence, update frequency,
+and the source fields deliberately left unpublished with the reason for each.
+The field mapping stays in the source class, which is the only place that knows
+the feed's shape.
+
+Field names follow [DCAT-AP], the metadata profile European data portals
+harvest, so registering a data set is a translation of its entry rather than a
+new survey. See [ADR 007](docs/adr/007-source-manifest.md).
+
+Adding a data set means adding one `SourceInterface` implementation and one
+manifest entry. The class is discovered through
+`#[AutoconfigureTag('app.source')]` and shows up as an `app:import` argument
+with no further wiring.
+
+Design decisions are recorded in [docs/adr](docs/adr/README.md).
+
+[NGSI-LD]: https://www.etsi.org/committee/cim
+[DCAT-AP]: https://semiceu.github.io/DCAT-AP/
+
 ## Broker
 
 A [Scorpio Broker](https://scorpio.readthedocs.io/) is part of the development setup.

Taskfile.yml

@@ -36,6 +36,23 @@ tasks:
   test:integration: *test_task
   test:application: *test_task
 
+  import:
+    desc: 'Import a source into the broker, e.g. task import -- mtm_spatialmaps-handicap-parking'
+    cmd: ddev console app:import {{.CLI_ARGS}}
+
+  # The broker itself needs no start task: Scorpio is a DDEV custom service
+  # (.ddev/docker-compose.scorpio.yaml), so it comes up with the rest of the
+  # site. Read requests need the domain context in a Link header, which the
+  # script supplies; the env values come from the dotenv block above, and are
+  # passed in explicitly because the container shell does not read .env.
+  broker:entities:
+    desc: 'List broker entities of a type, e.g. task broker:entities -- OnStreetParking 10'
+    cmd: >-
+      ddev exec sh -c
+      "APP_BROKER_BASE_URI='$APP_BROKER_BASE_URI'
+      ENTER_NGSI_DOMAIN_CONTEXT='$ENTER_NGSI_DOMAIN_CONTEXT'
+      sh task/scripts/broker-entities {{.CLI_ARGS}}"
+
   coding-standards:apply:
     desc: 'Apply coding standards'
     cmds:

config/sources.yaml

@@ -0,0 +1,46 @@
+# The data sets this application publishes, one entry per source key.
+#
+# The facts the import needs — where the feed is read from, the coordinate
+# reference system it publishes, the Smart Data Model it is published as — are
+# read from here, so they exist in exactly one place.
+#
+# The remaining fields record what no code can state: who owns the data, on
+# what terms, how often it changes, and which source fields are deliberately
+# not published. The rule is to record only what the code cannot tell you, so
+# the field mapping itself stays in the source class, which is the only place
+# that knows the feed's shape.
+#
+# Field names follow DCAT-AP, the metadata profile European data portals
+# harvest, so registering a data set is a translation of its entry rather than
+# a new survey. See docs/adr/007-source-manifest.md.
+
+sources:
+    mtm_spatialmaps-handicap-parking:
+        title: 'Handicapparkering, Aarhus Kommune'
+        description: >-
+            Disabled parking bays in Aarhus Municipality, with the number of
+            reserved bays per location.
+        publisher: 'Aarhus Kommune'
+        contact: ppg@aarhus.dk
+        landing_page: 'https://www.opendata.dk/city-of-aarhus/parkering-i-aarhus-kommune'
+        access_url: 'https://webkort.aarhuskommune.dk/spatialmap?page=get_geojson_opendata&datasource=invap'
+        media_type: application/geo+json
+        crs: 'EPSG:25832'
+        model: OnStreetParking
+        update_frequency: continuous
+
+        # The portal states no licence for this data set. DCAT-AP requires
+        # one, so it has to be settled with the data owner before the
+        # catalogue can be registered anywhere.
+        licence: ~
+
+        # Fields the feed carries that are not published. Recorded here
+        # because the source class shows what is mapped but cannot show what
+        # was left out, or why.
+        omitted_fields:
+            ident: 'Single-letter code; its meaning is not documented and not confirmed by the data owner.'
+            oprettet_af: 'Directory username of the municipal employee who created the record — personal data.'
+            rettet_af: 'Directory username of the municipal employee who last edited the record — personal data.'
+            oprettet_dato: 'Describes the register record rather than the parking bay.'
+            rettet_dato: 'Describes the register record rather than the parking bay. A candidate for observedAt, not yet mapped.'
+            mi_style: 'MapInfo rendering style, empty throughout the export.'

docs/adr/001-architecture-symfony-docker.md

@@ -0,0 +1,87 @@
+# 001: Architecture — Symfony 8 on the ITK Dev Docker template
+
+| Field              | Value                                  |
+|--------------------|----------------------------------------|
+| **Created By**     | Jeppe Krogh                            |
+| **Date**           | 2026-08-24                             |
+| **Decision Maker** | ITK Dev team                           |
+| **Stakeholders**   | ITK Dev developers, future maintainers |
+| **Status**         | Draft                                  |
+
+## Context
+
+The adapter reads open data sets, converts them to a standard smart-city
+representation, and publishes them to a context broker. It needs a runtime, an
+HTTP client, a console for running imports, and a local development environment
+including a broker to import into. It has no web UI and no domain data of its
+own.
+
+ITK Dev maintains a fleet of PHP services with an established Docker-based
+development convention, expressed as versioned project templates with shared CI
+and coding-standards configuration. A new application either adopts that or
+diverges from it.
+
+This ADR serves to decide the runtime, framework and development
+environment the application is built on.
+
+### Drivers
+
+- **Functional:** scheduled console commands; outbound HTTP; a local broker.
+  No database and no HTTP surface of its own.
+- **Non-functional:** minimal onboarding cost; shared tooling rather than
+  reimplemented tooling; reproducible across developers and CI; long-term
+  vendor support.
+
+### Options Considered
+
+1. **PHP 8.4 / Symfony 8 on the ITK Dev `symfony-8` template.** Matches the
+   organisation's existing stack, so CI, coding standards and task runner come
+   for free; the console component suits scheduled imports. Provisions a web
+   server, database and mail catcher this application never uses, and its PHP
+   version runs ahead of developer hosts, making containers mandatory.
+2. **Minimal framework project without the template, run on the host.** No
+   unused services, no container requirement for the application — but shared
+   CI and coding-standards config would be reimplemented by hand, and a local
+   broker needs containers anyway, so the dependency is moved rather than
+   removed.
+3. **A second entry point in an existing internal application.** One
+   deployment to operate, but couples a batch importer's release cycle to a
+   user-facing application and inherits dependencies it has no use for.
+4. **A different language ecosystem on a bespoke setup.** Richer geospatial
+   libraries in some ecosystems, but no internal expertise and no shared
+   tooling. The transformations needed are available as mature libraries in the
+   established stack too.
+
+## Decision
+
+**PHP 8.4 + Symfony 8** on the ITK Dev `symfony-8` template, as its **own
+deployable service**, with a containerised broker overlay for local development.
+
+- Standardising costs less over the application's lifetime than trimming unused
+  services. A second toolchain must be learned and patched; idle containers
+  cost only disk.
+- A batch importer and a user-facing application have different lifecycles and
+  failure modes, so they stay separate services.
+- No domain persistence is needed — the broker is the system of record — so the
+  template's database service is left unused rather than removed, keeping
+  template updates a clean diff.
+- Local development includes a real broker, so imports are verified end to end
+  rather than only as serialised output.
+
+## Consequences
+
+### Positive
+
+- Onboarding cost close to zero; CI and coding standards work from the first
+  commit.
+- No schema, no migrations, no state to keep consistent with the broker.
+
+### Negative / Trade-offs
+
+- **Containers are mandatory.** The template's PHP runs ahead of developer
+  hosts, so dependency management, console commands and tests cannot run
+  natively. Most likely source of first-run confusion.
+- A web server, database and mail catcher are provisioned and never used.
+- Broker images are not published for every CPU architecture, so local start-up
+  may be slow under emulation.
+- The application follows the template's choices; deviating later has a cost.