Weir system tests

Independent blackbox integration tests and database throughput comparisons for Weir.

Overview

Applications do not need this repository. Use it to test Weir across components or to measure its performance. This module uses the published Go SDK and public protocol. It imports no server packages.

Tools

Tool	Purpose
tests/integration	Real multi-node integration tests against owned MongoDB and Elasticsearch fixtures
cmd/weir-lab	Paired native-versus-Weir throughput comparisons on owned fixtures
scripts/run_performance_matrix.py, scripts/summarize_performance.py	Run a read/mixed/write/Lua matrix and compare two matrix runs
cmd/weir-crawler	Crawler mutation benchmark on an owned Search fixture
cmd/weir-bench	Fixed and saturation comparisons against independently deployed endpoints
cmd/weir-soak, scripts/soak-faults.sh	Long-running load through deployed Weir replicas, with every outcome checked against the databases, under scheduled faults

Common Make targets

Target	Action
make prepare	Install the pinned Weir server into .tools/
make test	Run unit and race tests. Starts no databases or Weir processes.
make integration	Run integration tests against real databases and Weir replicas
make benchmark	Run the saturation benchmark
make diagnostic	Run a finite single-operation comparison

make benchmark and make diagnostic write to a fixed directory under results/local/ and refuse an existing one. Remove it before running them again.

Requirements

Item	Requirement
Go	1.27.4
Operating system	Linux or macOS
Python	Python 3
make, jq	Any version
C compiler	On Linux, for go test -race
Docker	A local daemon with a Unix socket
Saturation benchmark	Linux with Docker, to enforce database CPU quotas
Soak and rollout tools	Deployed Weir replicas. The fault and rollout scripts also use kubectl and helm.

Fixtures use ordinary endpoints and processes and work without Kubernetes.

Quick start

Step 1. Get the repository.

git clone https://github.com/batchstream/weir-tests.git
cd weir-tests

Step 2. Install the pinned Weir server.

make prepare

Step 3. Run the unit tests.

make test

Step 4. Pull the database images. This downloads about 1 GB once. The first integration test allows only 4 minutes for fixtures to start, so pull them first.

docker pull "$(jq -r .mongodb_image versions.json)"
docker pull "$(jq -r .elasticsearch_image versions.json)"

Step 5. Run the integration tests. The run has a 10-minute limit.

make integration

Both test targets print ok for each passing package. The integration tests log their fixture directory as fixture artifacts:. Benchmark output belongs under the ignored results/ directory.

Documentation

Guide	Contents
Integration tests	Preparation, the native MongoDB fixture and coverage
Benchmarks	Saturation benchmark, performance matrix and deployed endpoints
Crawler benchmark	Crawler mutation workload
Soak	Long load, outcome checks, fault injection and rollout scenarios
CI and output	Pinned versions, CI jobs and generated output

Development

go test -race ./...
go vet ./...
go vet -tags=integration ./...
python3 -m unittest discover -s scripts -p '*_test.py'

versions.json pins the Weir server and database images. go.mod pins the SDK and protocol. See CI and output for what CI runs.

• Contributing: CONTRIBUTING.md
• Security reports: SECURITY.md

Related repositories

Repository	Purpose
weir	Server
weir-go	Go client SDK
weir-protocol	Public gRPC schema and validation
weir-charts	Helm chart
weir-tests	System tests and benchmarks (this repository)

License

MIT. See LICENSE.