Repository navigation
Conversation
The Public API listed its countries in several places: the v2 routes' `CountryId` type (`Literal["us", "uk"]`), an unused `SUPPORTED_COUNTRY_IDS` set in the query models, catalog extraction's `SUPPORTED_COUNTRIES` and its required packages and installed models, the set of countries the v1 routes copy into v2, and the household copy command's options. `policyengine_api/supported_countries.py` now holds the one list, `SupportedCountryId`, and `SUPPORTED_COUNTRY_IDS` derived from it; each of those places derives from it. The list lives in its own module, not in the query models, because the query models import the catalog package, which must import the list too. Tests: in a fresh interpreter, adding "zz" to the one list adds it to every derived list (this failed before the change: `CountryId` rejected "zz"); the catalog's per-country settings must cover exactly the listed countries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reference
Adds the v2 `countries` table: one row per country the Public API serves,
with `household_enabled` and `society_wide_enabled` saying which
calculations it supports (a check requires society-wide countries to be
household capable). `users.primary_country` and the `country_id` of
households, policies and the four user association tables reference it,
with `ON DELETE RESTRICT`, instead of each checking `IN ('us', 'uk')`. The
tables that pair v1 records with their v2 copies keep their checks.
Revision `fb1a3e40d3d6` was autogenerated. Its one hand-written operation
inserts the US and UK rows, both capable of both, between the table and the
foreign keys, which otherwise fail on every database whose rows already
name either country. This insert is an exception to the rule that revisions
contain only generated operations, approved for this revision alone; the
revision's comment and the Alembic skill say so. The generated downgrade
restores the checks and drops the table with its rows.
The privilege manifest gives the countries table no role access yet.
Tests (failed first because the `countries` table and `Country` model did
not exist): model structure and foreign keys; SQLite persistence (an
unlisted country is refused, a listed country in use cannot be deleted, a
household-only row lets records name that country); the revision's
operations; and the Postgres lifecycle (upgrade keeps existing US and UK
rows, unlisted countries are refused, downgrade and re-upgrade, a downgrade
is refused while records name another country, and the previous release's
table usage still works after the upgrade).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`GET /v2/countries` lists every row of the countries table, ordered by ID: each country's `id`, `household_enabled` and `society_wide_enabled`. It takes no query parameters (any is refused with a 422), needs no user credential, and answers 503 when the v2 database is not configured or cannot be reached, and a secret-safe 500 for anything else, through the shared v2 error mapping. Requests are logged under the `metadata` route group. The route follows the v2 layout: a `countries` route package, and a service package with a read connector, a session container, a transformation and a read type. The Public API's runtime role gets SELECT on `countries` in the privilege manifest, which the deploy grants after the schema upgrade. The migration contract registry gains the `country_selection_v2` workflow, and the cross-database integration workflow runs the new integration test. Tests (failed first: the route and service tests could not import `policyengine_api.services.v2.countries`, the OpenAPI test did not find `/v2/countries`, the generated contract files were stale, and the integration workflow did not name the new test): route responses, the query refusal, 405 for other methods, 503 and 500 mapping and the OpenAPI document; the service against SQLite; and, against migrated disposable Postgres as the runtime role, the route returning the two rows the revision inserted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`policyengine_api.asgi` now checks, while the application loads, that the v2 `countries` table holds one row for each country in `policyengine_api/supported_countries.py` and no other. A worker that finds a difference raises `CountryListMismatchError`, naming the listed countries without a row and the rows not listed, so it never serves a request; on Cloud Run the revision then fails its health checks. The check runs on Cloud Run (`K_SERVICE` is set) and wherever a v2 runtime database URL or its secret resource is configured. A local or test startup without v2 configuration opens no connection. A missing table, a missing privilege and unusable v2 configuration also stop startup. A database that cannot be reached is logged, by error type only, and startup continues: the v1 routes do not read the v2 database, and their copies into it already continue without it. The countries service gains `verify_countries_match_code_list`, which reads the row IDs through a new read connector and compares them in a validator. Tests (failed first: `policyengine_api.services.v2.countries.validators` did not exist): the comparison in every direction; the service against SQLite; the startup function's selection, each failure that stops startup, and the logged skip without the error's detail; importing the ASGI module with an unusable v2 runtime URL failing without printing the password; and, against migrated disposable Postgres as the runtime role, the check passing on the revision's rows and failing once an extra row exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review found that the startup check sorted its failures by exception type in a way that stopped startup for a brief Secret Manager outage (reported as unusable v2 configuration) while only logging rejected credentials (a connection error). The check now stops startup only when the database answers and the answer disagrees with the country list: different countries, a missing table, or no privilege to read it. When it cannot get an answer, because the v2 configuration cannot be used, the secret cannot be resolved, or the database cannot be reached or queried, it logs why and startup continues; the readiness check already reports unusable v2 configuration as not ready and recovers once the secret resolves. The rule `api.startup.no-import-side-effects` now says what deployed startup does: it opens no network connections unless the process runs on Cloud Run or has a v2 runtime database configured. The country list and the startup check join the mypy file list and rule `api.quality.mypy-v2`. The country list's docstring and the Alembic skill now say that a deploy runs a country-adding revision before the new release starts, so the previous release refuses to start afterwards, and list the US-and-UK checks a new country must also widen. Tests (failed first: configuration errors and `InterfaceError` were raised, and importing the ASGI module with an unusable v2 URL exited with an error): configuration and secret failures are logged with their message and startup continues, connection and interface errors are logged by type only, and importing the ASGI module with an unusable v2 URL logs the skipped check without the password. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #4012 +/- ##
===========================================
+ Coverage 50.12% 87.80% +37.67%
===========================================
Files 243 270 +27
Lines 14726 16333 +1607
Branches 2343 2653 +310
===========================================
+ Hits 7382 14341 +6959
+ Misses 6721 1306 -5415
- Partials 623 686 +63 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
In a full test run, the tests that run Alembic commands load Alembic's logging configuration, which disables every logger that already exists, including the startup check's. Its four log tests then captured nothing and failed, though they passed on their own. An autouse fixture now re-enables the check's logger for each of its tests. Tests (failed first: running the Alembic unit tests before the startup check tests failed all four log tests); with the fixture, both files pass together (46 passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4009
Summary
policyengine_api/supported_countries.pyholdsSupportedCountryId("us","uk") andSUPPORTED_COUNTRY_IDS. The v2 routes'CountryId, catalog extraction's country list, required packages and installed models, the catalog's supported-country check, and the household copy command's--country-idchoices all derive from it. It lives in its own module rather than inquery_parameters.py, because the query models import the catalog package, which must import the list too.countriestable. One row per served country, withhousehold_enabledandsociety_wide_enabled(a check requires a society-wide country to be household capable).users.primary_countryand thecountry_idof households, policies and the four user association tables referencecountries.idwithON DELETE RESTRICT, replacing theirIN ('us', 'uk')checks. The tables that pair v1 records with their v2 copies keep their checks.fb1a3e40d3d6was autogenerated. Its one hand-written operation inserts theukandusrows, both capable of both, between the table and the foreign keys, which otherwise fail on any database whose rows already name either country. This is an approved one-off exception to the rule that revisions contain only generated operations; the revision's comment and the Alembic skill say so, and upgrade and downgrade tests cover it. The generated downgrade restores the checks and drops the table.GET /v2/countrieslists every row, ordered by ID. It takes no query parameters (422 otherwise) and no credential, answers 503 when the v2 database is not configured or reachable, and a fixed 500 for anything else. The runtime role gets SELECT oncountriesin the privilege manifest.policyengine_api.asginow checks that the table names exactly the countries in the code list. On Cloud Run (K_SERVICEset), or wherever a v2 runtime database is configured, a worker whose check finds different countries, a missing table or no SELECT privilege raises, so it never serves and the tagged candidate fails its health checks. When the check cannot query the table (unusable v2 configuration, a secret that cannot be resolved, a database that cannot be reached or refuses the credentials), it logs why and startup continues: the v1 routes do not read the v2 database, and readiness already reports unusable v2 configuration as not ready. Local and test startups without v2 configuration open no connection.Consequence for adding a country later
A deploy runs migrations before it starts the new release. After a revision that adds a country's row, instances of the previous release that start (scale-out during the deploy, or a rollback) refuse to start until that revision is downgraded. The country list's docstring and the Alembic skill say this, and list the US-and-UK checks a new country must also widen (the v1-to-v2 pairing tables and v1
household_mirror_events).Migration notes
metadatagains/v2/countries(new workflowcountry_selection_v2in the contract registry; generated contract documents refreshed).countriestable and its foreign keys in Supabase, and the startup check on Cloud Run.Rules
This PR changes
rules/registry.toml, so it needs a review from a core developer other than the author:api.startup.no-import-side-effectsnow says that starting up opens no network connections unless the process runs on Cloud Run or has a v2 runtime database configured, which is what the startup check makes true. Its tests are unchanged and still cover startup without v2 configuration.api.quality.mypy-v2adds the country list and the startup check module (also added to[tool.mypy].files).docs/generated/rules.mdis regenerated.Inline suppressions
One added:
# noqa: E402on theCountryimport inpolicyengine_api/data/v2/models/__init__.py. Every model import in that module carries it, because the module must set the metadata naming convention before any table is declared, so the imports cannot move to the top.Environment variables
The startup check reads:
K_SERVICE: set by Cloud Run on every revision; the gunicorn workers inherit it. Not set on GitHub runners.V2_RUNTIME_DATABASE_URL_SECRET_RESOURCE: GitHub environment secret (staging,production). Jobenv:of the staging and production deploy jobs inpush.yml→ exported bycloud_run_env.sh→ required byvalidate_cloud_run_deploy_env.sh→ set on the revision bydeploy_cloud_run_candidate.sh(--set-env-vars). Existing wiring;run_release_tests.shunsets it, somake testskips the check.V2_SUPABASE_PROJECT_REF,V2_SUPABASE_ENVIRONMENT: GitHub environment variables, same path. Read through the existingload_v2_runtime_database_settings.V2_RUNTIME_DATABASE_URL: read only to decide whether to run the check. Not set on Cloud Run; the one workflow that sets it (process-v1-household-mirror-event.yml, from theV2_DATA_WRITE_DATABASE_URLsecret) runs a script that does not import the ASGI module.The workflow change adds one test file to an existing step of
v2-integration-check.yml; that job'sV2_MIGRATION_DATABASE_URL,V2_ALEMBIC_DISPOSABLE_TESTandDEPLOYMENT_ENVIRONMENTare literals for its disposable service containers.Not verified:
countriesin staging and production. It is declared inprivileges.pyand granted by the deploy'sreconcile_v2_privileges.py applyafter the schema upgrade and before the candidate starts; nobody has checked it in a deployed database yet. If it is missing, the candidate refuses to start and the release stops before taking traffic.What this PR waits for
Review only, including a core developer for the rule change. It is the bottom of a stack; #4017 above it removes the unused routes and three countries. Its migration is in the v2 Alembic chain, so if another open PR also adds a v2 revision, one merges first and the other rebases onto the new head.
After it deploys, an operator check is worth doing:
GET /v2/countriesin staging, then production, should listukandus.Testing
Each commit was written test-first:
14a662414(one country list): failed first becauseCountryIdrejected a country added to the list; then a fresh-interpreter test shows every derived list picks it up, and the catalog's per-country settings cover exactly the listed countries.2729c37f4(table and revision): failed first because thecountriestable andCountrymodel did not exist. Then: model and foreign-key tests, SQLite persistence (an unlisted country is refused, a country in use cannot be deleted, a household-only row lets records name that country), the revision's operations, and the v2 Postgres lifecycle against disposable PostgreSQL 17 (upgrade keeps existing US and UK rows, unlisted countries refused, downgrade and re-upgrade, a downgrade refused while records name another country, the previous release's table usage against the new schema): 10 passed.0d30696eb(GET /v2/countries): failed first (the countries service package did not exist, the OpenAPI document lacked the path, the generated contract files were stale, the integration workflow did not name the test). Then 306 passed across the route, service, metadata routes, contract, workflow, privilege, CRUD-boundary, migration-flag, import and query-parameter tests; the integration test as the runtime role on disposable PostgreSQL passed.f9a19c11e(startup check): failed first (no validator module). Then 21 unit tests and the runtime-role integration test (the check passes on the revision's rows and fails once an extra row exists) passed, with 203 passing in the related startup, readiness, ASGI and privilege suites.6b8cbd2bd(review follow-up): failed first (configuration and interface errors raised; the ASGI import exited with an error); then 13 startup tests and 77 in the startup, import, ASGI and readiness suites passed.975c80fc1(test isolation): in a full run, the Alembic unit tests load Alembic's logging configuration, which disables the startup check's logger, so its four log tests failed after them (reproduced by running the two files together); an autouse fixture re-enables the logger, and the unit, contract and to-refactor suites pass (3,425).ruff format --check .,ruff check .,uv run --frozen --extra dev mypy,python scripts/run_quality_guards.py;python -m scripts.rules guard --base-ref origin/masteron the last.🤖 Generated with Claude Code