Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ name: CI
# moves through a pull request whose own run tested the merge, so main has no
# push trigger.
#
# The contract is the root README's conventions: pnpm with a committed
# The contract is the conventions in docs/development.md: pnpm with a committed
# lockfile, and `lint`, `typecheck`, `test`, `build` scripts in package.json,
# plus `lint:css` (stylelint).

Expand Down
43 changes: 43 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Contributing

Issues and pull requests are welcome. The ONE Data team maintains this repository. For a large change, open an issue first so we can agree on the approach before you write it.

Participation is governed by the [Code of Conduct](https://github.com/DevelopmentFinanceObservatory/.github/blob/main/CODE_OF_CONDUCT.md). Report a security problem through GitHub private vulnerability reporting (the "Report a vulnerability" button under the Security tab), as set out in the [security policy](https://github.com/DevelopmentFinanceObservatory/.github/blob/main/SECURITY.md), and not in a public issue. For other questions see [SUPPORT.md](https://github.com/DevelopmentFinanceObservatory/.github/blob/main/SUPPORT.md).

## Set up

You need Node 22.18 or newer, pnpm, [just](https://github.com/casey/just), and Docker for the database tests.

```bash
just setup # pnpm install and .env.local from .env.example
just doctor # checks node, pnpm and the pnpm version in package.json
just dev # http://localhost:3000
```

The README says which parts of the site need a reachable `apis` or `ai` service. No contribution needs access to ONE's cloud project.

## Before you open a pull request

```bash
just db-up # local Postgres 18 on port 5433, needed by the tests
just check # lint, typecheck, test, build
```

`just check` runs what the `ci` workflow runs (ESLint, Stylelint, `tsc`, the tests and the build), apart from the migration bundle build. Run `just format` to apply Prettier and Stylelint fixes.

A pull request needs:

- a passing `ci` check (draft pull requests run no CI until marked ready)
- an approving review from a code owner
- tests for changed behaviour in the `test` directories under `db/` and `src/lib/`
- updated documentation when you change a command, an environment variable or an endpoint

If you change `db/schema.ts`, generate the migration with `just db-generate`, review the SQL, and commit it with the schema change. Never edit a migration that has already been applied. Write a new one. `docs/development.md` has the full workflow.

## Branches

`dev` is the default branch. Branch from it and open the pull request against it. `main` is the production branch and moves only by a release pull request from `dev`.

## Maintainer notes

`docs/development.md` covers code conventions, releases, the deployed services and the deploy configuration. `deploy/README.md` covers the image, migrations on deploy and the environment contract.
22 changes: 22 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

MIT License

Copyright (c) 2026 ONE Campaign

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
147 changes: 49 additions & 98 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,103 +1,54 @@
# dfo-website

The public site of the Development Finance Observatory (DFO). A single
Next.js application serves the frontend (chat and canvas workspace,
reference pages) and acts as the backend proxy for **all** API traffic to
the platform, including MCP. This is not the DFO brochure website, the small
live site that gets archived at launch.

## Status: in progress

The application code lands in slices as Ode builds it; the homepage, the
data search, the Studio and user accounts are in.
Around it:

- `deploy/` holds the Dockerfile pattern and the division of responsibility
with `infra`. They define the Cloud Run service, and this repo's CI
only swaps images onto it.
- `db/` holds the schema migration runner: a Cloud Run Job connects to the
website databases as an IAM service account through the Cloud SQL
connector and applies forward-only migrations before each deploy moves
the service to a new image.
- `.github/workflows/` holds the CI and deploy workflows. CI runs `lint`,
`typecheck`, `test`, and `build` on every PR that is not a draft, and on
each push to `dev`. The deploy workflow runs on pushes to `dev`
that touch the image's inputs.
- `docs/development.md` — how to run the app locally and the conventions
the code follows (styles, components, animation system).
- `docs/feedback.md` — the feedback endpoint (`POST /api/feedback`): the
frontend contract, the storage/Slack/email sinks and their config, and
the local dev loop.
- `docs/analytics.md` — usage events (the `event` table and what each
surface records), Sentry, and the Studio prompt allowance.

## Repo map

| Repo | What it is |
| ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`dcp`](https://github.com/DevelopmentFinanceObservatory/dcp) | The knowledge graph engine, ONE's fork of Data Commons, serving the graph over the mixer and SDMX APIs. |
| [`apis`](https://github.com/DevelopmentFinanceObservatory/apis) | The Search, Datasets, Graph, and Presentation APIs. The Search, Datasets, and Graph APIs read from the `dcp` graph, and the Presentation API persists to PostgreSQL. |
| [`ai`](https://github.com/DevelopmentFinanceObservatory/ai) | The agent harness and the DFO's MCP servers, deployed as one Cloud Run service. |
| [`spec-kit`](https://github.com/DevelopmentFinanceObservatory/spec-kit) | Shared spec models (spec-id hashing, materialisation, manifest and component-spec schemas) that publish the TS types this app renders from [forthcoming]. |
| [`website`](https://github.com/DevelopmentFinanceObservatory/website) | This repo, the public Next.js site and the proxy that fronts the platform for external traffic. |
| [`skills`](https://github.com/DevelopmentFinanceObservatory/skills) | The open repository of domain-knowledge packages agents load dynamically at runtime [forthcoming]. |
| [`infra`](https://github.com/DevelopmentFinanceObservatory/infra) | Infrastructure as code: the Cloud Run service definitions, service accounts, scaling, and secrets. |

## Deployed services

All services run in the `dev-fin-observatory` GCP project, region
`northamerica-northeast1`.

| Repo | Service |
| --------- | ---------------------------------------------------------------------------------------------------------------- |
| `website` | https://dfo-dev-website-staging-zms5lj4zhq-nn.a.run.app |
| `apis` | https://dfo-dev-apis-zms5lj4zhq-nn.a.run.app/docs (needs an invoker identity; reach it through the gcloud proxy) |
| `ai` | https://dfo-dev-ai-zms5lj4zhq-nn.a.run.app/docs |
| `dcp` | https://dfo-dev-dc-datacommons-service-zms5lj4zhq-nn.a.run.app |

`website`'s service is the one `dev` deploys to. `apis`, `ai`, and `dcp`
run their own dev services. `apis` and `ai` return 404 at the root and
serve Swagger UI at `/docs`. The website service is reachable by anyone
with the URL.

## Conventions

Changing any of these is fine — change the workflows and `deploy/Dockerfile`
in the same PR.

- **pnpm** as the package manager, with a committed `pnpm-lock.yaml` and a
`packageManager` field in `package.json`. `ci.yml`'s pnpm setup reads its
version from that field and fails without it.
- **TypeScript, strict**, Next.js App Router.
- `package.json` defines `lint`, `typecheck`, `test`, and `build` scripts.
That is the contract `ci.yml` runs.
- `next.config.*` sets `output: 'standalone'`, so `deploy/Dockerfile` can
copy the standalone server into the runtime image.

## Deploys

GitHub Actions builds the `dfo-website` image and updates the Cloud Run
dev service automatically on every push to `dev`. `main` is the
production release branch. A push to `main` builds and deploys production
through the `production` GitHub Environment, which needs approval from the
release reviewers.
`infra` owns the Cloud Run service _definition_ (service account,
scaling, secrets, env vars). This repo's CI only swaps images on the
service `infra` already defined. See `deploy/README.md` for the
vars/secrets and prerequisites.
# DFO website

The Next.js application behind [developmentfinance.org](https://developmentfinance.org): the data search, the Studio (a chat and canvas workspace for building charts and tables), reference pages, and user accounts. It also acts as the proxy that carries all outside traffic, including MCP, to the platform APIs.

It is part of the [Development Finance Observatory](https://github.com/DevelopmentFinanceObservatory), an initiative of [ONE Data](https://data.one.org) built on [Google Data Commons](https://datacommons.org).

This repository is under active development. Interfaces, schemas, and data may change without notice.

## What's inside

- `src/` holds the application: the App Router pages and API routes in `src/app`, atomic components in `src/components`, and the proxy, search, Studio, auth and feedback code in `src/lib`.
- `db/` holds the Drizzle schema, the forward-only migrations and the runner that applies them.
- `deploy/` holds the Dockerfile and the deploy notes.
- `.github/workflows/` holds CI and the deploy workflows.
- `docs/` holds the maintainer and endpoint documentation.

The site talks to three neighbouring repositories: [`apis`](https://github.com/DevelopmentFinanceObservatory/apis) for search, datasets and graph queries, [`ai`](https://github.com/DevelopmentFinanceObservatory/ai) for the Studio agent and MCP servers, and [`infra`](https://github.com/DevelopmentFinanceObservatory/infra) for the Cloud Run services that run it. The org profile has the [full repository map](https://github.com/DevelopmentFinanceObservatory).

## Getting started

You need Node 22.18 or newer and pnpm. `just doctor` checks both, and `just setup` installs dependencies and creates `.env.local` from `.env.example`.

```bash
pnpm install
cp .env.example .env.local
pnpm dev
```

Open [http://localhost:3000](http://localhost:3000).

Only the dev apis proxy below needs access to ONE's Google Cloud project (`dev-fin-observatory`). What runs depends on which back ends you can reach:

- The static pages, `pnpm lint` and `pnpm typecheck` run with no configuration.
- The data search and graph pages call the DFO apis service at `SEARCH_API_BASE`. `.env.example` points it at `http://localhost:8083`, where ONE maintainers run a `gcloud run services proxy` to the dev deployment; that proxy needs an account with invoker access in `dev-fin-observatory`, as `docs/development.md` describes. Without that access, point it at your own `apis` instance.
- The Studio needs an `ai` harness at `STUDIO_API_BASE` (default `http://localhost:8080`) and a `STUDIO_TOKEN`, which ONE issues for its own deployments. Run the `ai` repository locally to use it.
- Accounts and saved conversations need local Postgres (`just db-up && just db-migrate`, which requires Docker) and your own OAuth client credentials for at least one sign-in provider.
- `pnpm test` needs the local Postgres but nothing from the cloud.

Without `DATABASE_URL`, the dev server logs a warning and keeps serving the pages that do not use the database. `docs/development.md` lists every variable and the full command set.

## Documentation

- [`docs/development.md`](docs/development.md): running locally, the database, code conventions, branches, releases and the deployed services.
- [`docs/feedback.md`](docs/feedback.md): the feedback endpoint (`POST /api/feedback`), its sinks and the local development loop.
- [`docs/analytics.md`](docs/analytics.md): usage events, Sentry and the Studio prompt allowance.
- [`deploy/README.md`](deploy/README.md): the image build, migrations on deploy, GitHub variables and the runtime environment contract.

## Contributing

`dev` is the default branch. Branch off it and open a pull request
into it. The `ci` check has to pass before the PR merges, and force-pushes
are blocked. Each merge deploys to the dev service.
Issues and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).

`main` is production, the same model `apis` uses. To release:
## Licence

1. Confirm the `dev` commit you want is deployed and validated on the
dev service.
1. Open a pull request from `dev` into `main`. `main` requires one approval
and the `ci` check, and blocks force-pushes and deletion.
1. Merge it. `deploy-prod.yml` builds and deploys that commit once the
release reviewers approve the `production` environment.
1. Tag it: `git tag prod-YYYY-MM-DD <sha> && git push origin prod-YYYY-MM-DD`.
MIT. See [LICENSE](LICENSE).
4 changes: 2 additions & 2 deletions deploy/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# syntax=docker/dockerfile:1
#
# Cloud Run image for the DFO website: the Next.js app in standalone output
# mode (next.config.ts sets `output: 'standalone'`, per the root README's
# conventions).
# mode (next.config.ts sets `output: 'standalone'`, per the conventions in
# docs/development.md).
#
# Build from the REPO ROOT, not from deploy/:
#
Expand Down
Loading
Loading