Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
git diff --exit-code src/pdc_api_client/_generated.py

test:
name: Unit tests (Python ${{ matrix.python }}${{ matrix.lowest_direct && ', lowest direct dependencies' || '' }})
name: Unit tests and type checks (Python ${{ matrix.python }}${{ matrix.lowest_direct && ', lowest direct dependencies' || '' }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
Expand All @@ -68,6 +68,11 @@ jobs:
PYTHON: ${{ matrix.python }}
LOWEST_DIRECT: ${{ matrix.lowest_direct && 'true' || '' }}
UPGRADE: ${{ (github.event_name == 'schedule' || inputs.upgrade) && 'true' || '' }}
- run: just typecheck --python="$PYTHON" ${LOWEST_DIRECT:+--lowest-direct}
if: ${{ !cancelled() }}
env:
PYTHON: ${{ matrix.python }}
LOWEST_DIRECT: ${{ matrix.lowest_direct && 'true' || '' }}

integration:
name: Integration tests
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ A framework-free Python client for the Philanthropy Data Commons (PDC) API. The

- `just test [--python VERSION] [--lowest-direct] [--upgrade] [-- ARGS]` runs the unit tests, which mock all HTTP with `responses`. `just integration` takes the same options. Both run against the installed package in an isolated environment, with the versions in `uv.lock` unless `--lowest-direct` or `--upgrade` resolves them fresh. Arguments after `--` go to pytest.
- `just pdc up` starts PDC's CI stack in Docker; `just integration` runs the live suite against it.
- `just typecheck` runs mypy (with the pydantic plugin) over `src` and both test suites, mypy in strict mode over `src`, then pyrefly and ty over `src` and both test suites, and fails at the end if any of them failed. `just typecheck-mypy`, `just typecheck-pyrefly`, and `just typecheck-ty` run one checker each. All take `--python` to check against a Python version other than `.python-version`.
- Verify with `just check` (lint plus both suites). The unit tests can't catch a change in PDC's behavior; if the stack isn't available, run `just test` and `just lint` and say that integration verification is pending.
- `just typecheck` runs mypy (with the pydantic plugin) over `src` and both test suites, mypy in strict mode over `src`, then pyrefly and ty over `src` and both test suites, and fails at the end if any of them failed. `just typecheck-mypy`, `just typecheck-pyrefly`, and `just typecheck-ty` run one checker each. All take `--python` and `--lowest-direct`, which work as they do for `just test`.
- Verify with `just check` (lint, type checks, and both suites). The unit tests can't catch a change in PDC's behavior; if the stack isn't available, run `just lint`, `just typecheck`, and `just test` and say that integration verification is pending.

## Standards

Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ This project follows [Semantic Versioning](https://semver.org). Until 1.0, a min

PDC does not bump its spec version for every API change, so each entry names the spec version and the date the spec was downloaded.

## Unreleased

- The package is now type checked with mypy (including strict mode), pyrefly, and ty, so its `py.typed` marker is accurate. The testing helpers and seeding modules are fully annotated.
- The minimum versions are now `requests>=2.34`, `urllib3>=2`, and, for the `seeding` extra, `faker>=26`, the first releases that ship type information.
- `APISession(timeout=...)` accepts a float.
- `KeycloakAdmin.ensure_organization` raises `LookupError` if Keycloak creates an organization without returning its id.

## 0.2.1 (2026-11-12)

- `create_initiative` and `create_initiative_field_value` create an initiative and add a field value to it. `seed_initiative` and `seed_initiative_field_value` now call them.
Expand Down
4 changes: 3 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ just test # unit tests, no network
just typecheck # mypy, pyrefly, and ty
just pdc up # start PDC's CI stack in Docker
just integration # integration tests against that stack
just check # lint, unit tests, and integration tests
just check # lint, type checks, unit tests, and integration tests
```

`just test` and `just integration` install the package into an isolated environment and run the tests against it, using the Python version in `.python-version` and the versions in `uv.lock`. Both take the same options, and pass arguments after `--` to pytest:
Expand All @@ -21,6 +21,8 @@ just check # lint, unit tests, and integration tests
just test --python 3.12 --lowest-direct -- -k token
```

`just typecheck` takes `--python` and `--lowest-direct` too. CI type checks Python 3.12 with the lowest versions and Python 3.14 with `uv.lock`.

`uv.lock` pins the versions used for development and CI. Update it with `uv lock --upgrade`. CI also runs both suites weekly with `--upgrade`, and opens an issue when that run fails.

`just pdc up` downloads PDC's [`compose-ci.yml`](https://github.com/PhilanthropyDataCommons/service/blob/main/compose-ci.yml) and Keycloak realm from the service repo's main branch and starts the stack: the PDC API, Keycloak, Postgres, and S3. The integration tests reach Keycloak at `pdc-auth`, so add this line to `/etc/hosts`:
Expand Down
1 change: 1 addition & 0 deletions integration_tests/test_pdc_group_membership.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ def create(org_id: str) -> int:
scope=[PermissionGrantEntityType.ANY],
),
)
assert grant.id is not None
grant_ids.append(grant.id)
return changemaker.id

Expand Down
1 change: 1 addition & 0 deletions integration_tests/test_permission_grants.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ def test_authenticated_users_changemaker_grant_roundtrips(self, api_session, see
)

created = create_permission_grant(api_session, spec=spec)
assert created.id is not None
created_grant_ids.append(created.id)

assert isinstance(created, ChangemakerPermissionGrant)
Expand Down
2 changes: 2 additions & 0 deletions integration_tests/test_seed.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ class TestSeedProposalBundle:
def test_creates_complete_bundle(self, api_session):
bundle = seed_proposal_bundle(api_session)
assert isinstance(bundle, SeededProposalBundle)
assert bundle.changemaker is not None
assert bundle.changemaker.id > 0
assert bundle.funder.short_code
assert bundle.opportunity.id > 0
Expand All @@ -91,6 +92,7 @@ def test_bundle_seeds_standard_proposal_fields(self, api_session):

def test_bundle_is_retrievable(self, api_session):
bundle = seed_proposal_bundle(api_session)
assert bundle.changemaker is not None
results = list(
get_changemaker_proposals(
api_session,
Expand Down
25 changes: 15 additions & 10 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,37 +23,42 @@ format:
# Run all three type checkers, failing at the end if any failed
[group('dev')]
[arg("python", long)]
typecheck python=python:
[arg("lowest_direct", long="lowest-direct", value="true")]
typecheck python=python lowest_direct="false":
#!/usr/bin/env bash
status=0
for checker in mypy pyrefly ty; do
{{just_executable()}} typecheck-$checker --python={{python}} || status=1
{{just_executable()}} typecheck-$checker --python={{python}} \
{{ if lowest_direct == "true" { "--lowest-direct" } else { "" } }} || status=1
done
exit $status

# Type check the package and tests with mypy, then the package in strict mode
[group('dev')]
[arg("python", long)]
typecheck-mypy python=python:
[arg("lowest_direct", long="lowest-direct", value="true")]
typecheck-mypy python=python lowest_direct="false":
#!/usr/bin/env bash
status=0
run="uv run --python={{python}} --isolated --no-dev --group tests --group typecheck --"
run="uv run --python={{python}} {{ if lowest_direct == "true" { "--resolution=lowest-direct" } else { "" } }} --isolated --no-dev --group tests --group typecheck --"
$run mypy --python-version={{python}} || status=1
$run mypy --python-version={{python}} --strict src || status=1
exit $status

# Type check the package and tests with pyrefly (passes through args)
[group('dev')]
[arg("python", long)]
typecheck-pyrefly python=python *args:
uv run --python={{python}} --isolated --no-dev --group tests --group typecheck -- \
[arg("lowest_direct", long="lowest-direct", value="true")]
typecheck-pyrefly python=python lowest_direct="false" *args:
uv run --python={{python}} {{ if lowest_direct == "true" { "--resolution=lowest-direct" } else { "" } }} --isolated --no-dev --group tests --group typecheck -- \
pyrefly check --python-version={{python}} {{args}}

# Type check the package and tests with ty (passes through args)
[group('dev')]
[arg("python", long)]
typecheck-ty python=python *args:
uv run --python={{python}} --isolated --no-dev --group tests --group typecheck -- \
[arg("lowest_direct", long="lowest-direct", value="true")]
typecheck-ty python=python lowest_direct="false" *args:
uv run --python={{python}} {{ if lowest_direct == "true" { "--resolution=lowest-direct" } else { "" } }} --isolated --no-dev --group tests --group typecheck -- \
ty check --python-version={{python}} {{args}}

# Run the unit tests against the installed package
Expand Down Expand Up @@ -82,9 +87,9 @@ integration python=python lowest_direct="false" upgrade="false" *args:
--isolated --no-editable --no-dev --group tests --reinstall -- \
python -I -m pytest -c integration_tests/pytest.ini integration_tests/ {{args}}

# Run lint, unit tests, and integration tests (run `just pdc up` first)
# Run lint, type checks, unit tests, and integration tests (run `just pdc up` first)
[group('test')]
check: lint test integration
check: lint typecheck test integration

# Download the current PDC OpenAPI spec
[group('codegen')]
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ classifiers = [
]
dependencies = [
"pydantic>=2.7",
"requests>=2.30",
"urllib3>=1.26.5",
"requests>=2.34",
"urllib3>=2",
]

[project.optional-dependencies]
testing = ["responses>=0.23"]
seeding = ["faker>=20", "python-keycloak>=6"]
seeding = ["faker>=26", "python-keycloak>=6"]

[project.urls]
Repository = "https://github.com/PhilanthropyDataCommons/pdc-python-client"
Expand Down
19 changes: 11 additions & 8 deletions src/pdc_api_client/client.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import json
from collections.abc import Generator
from datetime import datetime
from typing import ClassVar, Protocol
from typing import Any, ClassVar, Protocol, Self

import requests

Expand Down Expand Up @@ -32,11 +32,11 @@ def _iterate_pages(
session: APISession,
*,
endpoint: str,
params: dict | None = None,
params: dict[str, Any] | None = None,
count_per_page: int = 50,
max_requests: int | None = None,
**kwargs,
) -> Generator[dict, None, None]:
**kwargs: Any,
) -> Generator[dict[str, Any], None, None]:
params = params or {}
page = 1
n_pages = None
Expand Down Expand Up @@ -180,7 +180,7 @@ def get_application_form_by_name(session: APISession, *, name: str) -> Changemak
raise errors.ApplicationFormNotFoundError(detail=f"No application form named {name!r}")


def create_changemaker_proposal(session: APISession, *, data: dict) -> Proposal:
def create_changemaker_proposal(session: APISession, *, data: dict[str, Any]) -> Proposal:
response = session.post("proposals", json=data)
return Proposal.model_validate(response.json())

Expand Down Expand Up @@ -291,6 +291,9 @@ class PageableBundle(Protocol):
pdc_endpoint: ClassVar[str]
default_page_size: ClassVar[int]

@classmethod
def model_validate(cls, obj: Any) -> Self: ...


def get_bundle_page[BundleT: PageableBundle](
session: APISession,
Expand All @@ -299,10 +302,10 @@ def get_bundle_page[BundleT: PageableBundle](
page: int = 1,
count_per_page: int | None = None,
content: str | None = None,
extra_params: dict | None = None,
path_params: dict | None = None,
extra_params: dict[str, Any] | None = None,
path_params: dict[str, Any] | None = None,
) -> BundleT:
params: dict = {"_page": page, "_count": count_per_page or bundle_type.default_page_size}
params: dict[str, Any] = {"_page": page, "_count": count_per_page or bundle_type.default_page_size}
if content:
params["_content"] = content
if extra_params:
Expand Down
5 changes: 4 additions & 1 deletion src/pdc_api_client/errors.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
from typing import Any


class PDCException(Exception):
def __init__(self, *, detail, code=None):
def __init__(self, *, detail: Any, code: int | None = None) -> None:
super().__init__(str(detail))
self.detail = detail
self.code = code
Expand Down
19 changes: 8 additions & 11 deletions src/pdc_api_client/permission_grants.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from datetime import datetime
from typing import Annotated, ClassVar, Literal
from typing import Annotated, Any, ClassVar, Literal, Self
from uuid import UUID

from pydantic import Field, TypeAdapter, field_validator, model_validator
Expand Down Expand Up @@ -43,7 +43,7 @@ class _PermissionGrantBase(BaseSchema):
created_at: datetime | None = Field(default=None, alias="createdAt")

@model_validator(mode="after")
def _check_grantee_keys_match_type(self):
def _check_grantee_keys_match_type(self) -> Self:
if self.grantee_type == PermissionGrantGranteeType.USER:
if self.grantee_user_keycloak_user_id is None:
raise ValueError("grantee_user_keycloak_user_id is required when grantee_type is 'user'")
Expand Down Expand Up @@ -194,9 +194,6 @@ class InitiativeFieldValuePermissionGrant(_PermissionGrantBase):

_PERMISSION_GRANT_ADAPTER: TypeAdapter[PermissionGrant] = TypeAdapter(PermissionGrant)

# PDC rejects write bodies that include server-managed fields.
_READONLY_FIELDS = frozenset({"id", "created_by", "created_at"})


class PermissionGrantBundle(BaseSchema):
pdc_endpoint: ClassVar[str] = _ENDPOINT
Expand All @@ -210,7 +207,7 @@ class DefaultPermissionGrant(_PermissionGrantBase):

@field_validator("context_entity_type")
@classmethod
def _reject_any_context(cls, value):
def _reject_any_context(cls, value: PermissionGrantEntityType) -> PermissionGrantEntityType:
# PDC accepts "any" in scope but not as a default's context.
if value == PermissionGrantEntityType.ANY:
raise ValueError("context_entity_type cannot be 'any'")
Expand All @@ -224,13 +221,13 @@ class DefaultPermissionGrantBundle(BaseSchema):
total: int


def _validate_grant(payload: dict) -> PermissionGrant:
def _validate_grant(payload: dict[str, Any]) -> PermissionGrant:
return _PERMISSION_GRANT_ADAPTER.validate_python(payload)


def _grant_to_payload(spec: PermissionGrant | DefaultPermissionGrant) -> dict:
# PDC rejects the unset grantee key, so drop None values.
return spec.model_dump(mode="json", by_alias=True, exclude=_READONLY_FIELDS, exclude_none=True)
def _grant_to_payload(spec: PermissionGrant | DefaultPermissionGrant) -> dict[str, Any]:
# PDC rejects server-managed fields and the unset grantee key in write bodies.
return spec.model_dump(mode="json", by_alias=True, exclude={"id", "created_by", "created_at"}, exclude_none=True)


def list_permission_grants(
Expand All @@ -244,7 +241,7 @@ def list_permission_grants(
grantee_type: PermissionGrantGranteeType | None = None,
verb: PermissionGrantVerb | None = None,
) -> PermissionGrantBundle:
extra_params: dict = {}
extra_params: dict[str, Any] = {}
if changemaker is not None:
extra_params["changemaker"] = changemaker
if funder is not None:
Expand Down
22 changes: 14 additions & 8 deletions src/pdc_api_client/seeding/keycloak_admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ def ensure_organization(self, *, name: str, alias: str, domain: str) -> Keycloak
org_id = self._admin.create_organization(
{"name": name, "alias": alias, "domains": [{"name": domain, "verified": False}]}
)
if org_id is None:
raise LookupError(f"Keycloak did not return an id for new organization {alias!r}")
return KeycloakOrganization(id=org_id, alias=alias, name=name)

def get_organization(self, *, alias: str) -> KeycloakOrganization | None:
Expand All @@ -54,9 +56,10 @@ def get_organization(self, *, alias: str) -> KeycloakOrganization | None:
return None
return KeycloakOrganization(id=existing["id"], alias=existing["alias"], name=existing["name"])

def _find_org_by_alias(self, alias: str) -> dict | None:
def _find_org_by_alias(self, alias: str) -> dict[str, Any] | None:
# Keycloak's org search matches names, not aliases, so filter the full listing.
for org in self._admin.get_organizations():
organizations: list[dict[str, Any]] = self._admin.get_organizations()
for org in organizations:
if org.get("alias") == alias:
return org
return None
Expand Down Expand Up @@ -93,14 +96,16 @@ def ensure_user(
)
return KeycloakUser(id=user_id, username=username, email=email)

def _find_user_by_username(self, username: str) -> dict | None:
for user in self._admin.get_users({"username": username, "exact": True}):
def _find_user_by_username(self, username: str) -> dict[str, Any] | None:
users: list[dict[str, Any]] = self._admin.get_users({"username": username, "exact": True})
for user in users:
if user.get("username") == username:
return user
return None

def _find_user_by_email(self, email: str) -> dict | None:
for user in self._admin.get_users({"email": email, "exact": True}):
def _find_user_by_email(self, email: str) -> dict[str, Any] | None:
users: list[dict[str, Any]] = self._admin.get_users({"email": email, "exact": True})
for user in users:
if (user.get("email") or "").lower() == email.lower():
return user
return None
Expand Down Expand Up @@ -166,8 +171,9 @@ def set_direct_grants_enabled(self, *, client_id: str, enabled: bool) -> None:
target = self._find_client(client_id)
self._admin.update_client(target["id"], {"directAccessGrantsEnabled": enabled})

def _find_client(self, client_id: str) -> dict:
for client in self._admin.get_clients():
def _find_client(self, client_id: str) -> dict[str, Any]:
clients: list[dict[str, Any]] = self._admin.get_clients()
for client in clients:
if client.get("clientId") == client_id:
return client
raise LookupError(f"Keycloak client {client_id!r} not found in realm {self.realm!r}")
Loading
Loading