Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions .github/workflows/runtime-artifact-evidence-gate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
name: Runtime Artifact Evidence Gate

on:
workflow_dispatch:
schedule:
# Verify immutable inputs before the main daily operating window. The
# individual publisher workflows retain responsibility for generating them.
- cron: "10 12 * * 1-5"

permissions:
contents: read
id-token: write
issues: write

concurrency:
group: runtime-artifact-evidence-gate
cancel-in-progress: false

jobs:
verify-required-runtime-artifacts:
runs-on: ubuntu-latest
timeout-minutes: 15
env:
GCP_PROJECT_ID: longbridgequant
GCP_WORKLOAD_IDENTITY_PROVIDER: projects/252919773759/locations/global/workloadIdentityPools/github-actions/providers/github-main
GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: qsl-artifact-evidence@longbridgequant.iam.gserviceaccount.com
steps:
- uses: actions/checkout@v6

- uses: actions/setup-python@v6
with:
python-version: "3.12"

- name: Authenticate read-only evidence identity
uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3
with:
workload_identity_provider: ${{ env.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ env.GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT }}

- name: Set up Google Cloud CLI
uses: google-github-actions/setup-gcloud@v3
with:
project_id: ${{ env.GCP_PROJECT_ID }}
version: ">= 416.0.0"

- name: Validate control-plane registry
run: |
set -euo pipefail
mkdir -p data/output/runtime-artifact-evidence
python3 python/scripts/build_config.py --check
python3 python/scripts/build_config.py --runtime-artifact-evidence-registry \
> data/output/runtime-artifact-evidence/registry.json

- name: Verify remote objects, manifests, digests, and freshness
id: verify
continue-on-error: true
run: |
set -euo pipefail
python3 python/scripts/verify_runtime_artifact_evidence.py \
--registry data/output/runtime-artifact-evidence/registry.json \
--output data/output/runtime-artifact-evidence/receipt.json

- name: Summarize evidence receipt
if: always()
run: |
set -euo pipefail
python3 - <<'PY' >> "$GITHUB_STEP_SUMMARY"
import json
from pathlib import Path

receipt_path = Path("data/output/runtime-artifact-evidence/receipt.json")
print("## Runtime Artifact Evidence Gate")
print()
if not receipt_path.exists():
print("- receipt: `missing`")
raise SystemExit(0)
receipt = json.loads(receipt_path.read_text(encoding="utf-8"))
summary = receipt.get("summary", {})
print(f"- status: `{receipt.get('status')}`")
print(f"- required artifacts: `{summary.get('required_artifact_count')}`")
print(f"- verified artifacts: `{summary.get('verified_artifact_count')}`")
print(f"- failed artifacts: `{summary.get('failed_artifact_count')}`")
print("- boundary: read-only; no publishing, runtime change, or order submission")
PY

- name: Upload evidence receipt
if: always()
uses: actions/upload-artifact@v7
with:
name: runtime-artifact-evidence-receipt
path: data/output/runtime-artifact-evidence/
if-no-files-found: warn
retention-days: 30

- name: Create or update parked-evidence issue
if: always() && steps.verify.outcome == 'failure'
env:
GH_TOKEN: ${{ github.token }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
set -euo pipefail
gh label create runtime-artifact-evidence \
--description "Read-only required runtime artifact verification findings" \
--color d93f0b \
--force >/dev/null
body="$(mktemp)"
{
echo "## Runtime artifact evidence requires attention"
echo
echo "- Run: ${RUN_URL}"
echo "- This gate is read-only. It did not publish data, alter a runtime target, or submit an order."
echo "- Keep affected routes parked until the owning publisher emits a verified artifact and manifest."
echo
echo '```json'
if [ -s data/output/runtime-artifact-evidence/receipt.json ]; then
python3 -m json.tool data/output/runtime-artifact-evidence/receipt.json
else
echo '{"status":"receipt_missing"}'
fi
echo '```'
} > "$body"
existing="$(gh issue list --state open --label runtime-artifact-evidence --json number --jq '.[0].number // ""')"
if [ -n "$existing" ]; then
gh issue comment "$existing" --body-file "$body"
else
gh issue create \
--title "Runtime artifact evidence requires attention" \
--label runtime-artifact-evidence \
--body-file "$body"
fi

- name: Fail when external evidence is invalid
if: steps.verify.outcome == 'failure'
run: exit 1
12 changes: 10 additions & 2 deletions docs/qsl_universal_dry_run_coverage_v1.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,17 @@
| 层次 | 含义 | 当前结果 |
| --- | --- | --- |
| 已声明路径 | 策略域、策略许可和平台 `dry_run` 许可匹配 | 59 条 |
| 默认可构建路径 | 不需要临时人工补充制品,即可生成通过 runtime policy 校验的不下单目标 | 57 条 |
| 默认可构建路径 | 不需要临时人工补充制品,即可生成通过 runtime policy 校验的不下单目标 | 59 条 |

如果策略依赖 `runtime_artifacts.feature_snapshot.required=true`,默认可构建路径还必须配置成对的、`gs://` 开头的快照和 manifest URI。缺失时策略会被标成 **PARKED**,健康检查 `strategy_platform_dry_run_coverage` 以 critical failure 失败,执行保持关闭;这不是允许自动补一个猜测的 URI。

当前受此规则约束的是 `hk_low_vol_dividend_quality_snapshot` 的 IBKR 与 LongBridge 两条已声明路径。它们需要由 [HkEquitySnapshotPipelines](https://github.com/QuantStrategyLab/HkEquitySnapshotPipelines) 发布并验证的 `feature_snapshot` 与 manifest 后才能成为默认可构建路线。该流水线的制品、来源、回测和人工审阅是独立证据边界;完成后可通过配置默认 URI,或在一次已审阅的不下单运行中成对传入两个 URI。自动修复只能发现、暂停、提示和复验,绝不伪造数据制品或提高生命周期。
当前受此规则约束的是 `global_etf_rotation`、`russell_top50_leader_rotation` 与 `hk_low_vol_dividend_quality_snapshot`。前两者由 [UsEquitySnapshotPipelines](https://github.com/QuantStrategyLab/UsEquitySnapshotPipelines) 发布,港股由 [HkEquitySnapshotPipelines](https://github.com/QuantStrategyLab/HkEquitySnapshotPipelines) 发布;三者都必须先通过各自来源、回测和人工审阅边界,才可成为默认可构建路线。自动修复只能发现、暂停、提示和复验,绝不伪造数据制品或提高生命周期。

### 只读制品证据门

`Runtime Artifact Evidence Gate` 从同一份 `platform-config.json` 生成全部必需制品清单,并以专用只读身份逐项验证:对象可读、manifest 的策略身份、SHA-256 与快照文件一致、以及 `snapshot_as_of` 没有超过策略声明的时效预算。当前预算由发布节奏决定:日更全球 ETF 保留 5 个自然日以覆盖周末与一个市场假日;月更 Russell 与港股保留 40 个自然日。

该门只会生成回执和待处理事项。它不会发布数据、修改 URI、改变运行目标、提升生命周期或提交订单;异常路线必须继续保持 **PARKED**,直到制品所属流水线重新发布并通过验证。

## 插件边界

Expand All @@ -62,6 +68,8 @@
python3 python/scripts/build_config.py --check
python3 python/scripts/runtime_settings.py validate
python3 python/scripts/build_config.py --platform-health-report
python3 python/scripts/build_config.py --runtime-artifact-evidence-registry > /tmp/runtime-artifact-registry.json
python3 python/scripts/verify_runtime_artifact_evidence.py --registry /tmp/runtime-artifact-registry.json
node tests/strategy_switch_worker_validation.mjs
```

Expand Down
9 changes: 6 additions & 3 deletions platform-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -519,7 +519,8 @@
"feature_snapshot": {
"required": true,
"path": "gs://qsl-runtime-logs-shared/strategy-artifacts/us_equity/global_etf_rotation/global_etf_rotation_feature_snapshot_latest.csv",
"manifest_path": "gs://qsl-runtime-logs-shared/strategy-artifacts/us_equity/global_etf_rotation/global_etf_rotation_feature_snapshot_latest.csv.manifest.json"
"manifest_path": "gs://qsl-runtime-logs-shared/strategy-artifacts/us_equity/global_etf_rotation/global_etf_rotation_feature_snapshot_latest.csv.manifest.json",
"max_age_days": 5
}
},
"features": {
Expand Down Expand Up @@ -565,7 +566,8 @@
"feature_snapshot": {
"required": true,
"path": "gs://qsl-runtime-logs-shared/strategy-artifacts/us_equity/russell_top50_leader_rotation_staging/russell_top50_leader_rotation_feature_snapshot_latest.csv",
"manifest_path": "gs://qsl-runtime-logs-shared/strategy-artifacts/us_equity/russell_top50_leader_rotation_staging/russell_top50_leader_rotation_feature_snapshot_latest.csv.manifest.json"
"manifest_path": "gs://qsl-runtime-logs-shared/strategy-artifacts/us_equity/russell_top50_leader_rotation_staging/russell_top50_leader_rotation_feature_snapshot_latest.csv.manifest.json",
"max_age_days": 40
}
},
"features": {
Expand Down Expand Up @@ -726,7 +728,8 @@
"feature_snapshot": {
"required": true,
"path": "gs://qsl-runtime-logs-shared/strategy-artifacts/hk_equity/hk_low_vol_dividend_quality_snapshot_staging/hk_low_vol_dividend_quality_snapshot_factor_snapshot_latest.csv",
"manifest_path": "gs://qsl-runtime-logs-shared/strategy-artifacts/hk_equity/hk_low_vol_dividend_quality_snapshot_staging/hk_low_vol_dividend_quality_snapshot_factor_snapshot_latest.csv.manifest.json"
"manifest_path": "gs://qsl-runtime-logs-shared/strategy-artifacts/hk_equity/hk_low_vol_dividend_quality_snapshot_staging/hk_low_vol_dividend_quality_snapshot_factor_snapshot_latest.csv.manifest.json",
"max_age_days": 40
}
},
"features": {
Expand Down
91 changes: 90 additions & 1 deletion python/scripts/build_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
}
SCHEDULER_FIELDS = {"timezone", "main_time", "probe_time", "precheck_time"}
MARKET_FIELDS = {"market", "market_calendar", "market_timezone"}
FEATURE_SNAPSHOT_FIELDS = {"required", "path", "manifest_path"}
FEATURE_SNAPSHOT_FIELDS = {"required", "path", "manifest_path", "max_age_days"}
# These platforms have a runtime variable pair through which an immutable
# feature snapshot and its manifest can be supplied to a generated target.
# Keep this small, explicit set next to the config validation so coverage does
Expand Down Expand Up @@ -356,6 +356,17 @@ def validate(config: dict) -> list[str]:
errors.append(
f"strategy {sid}: live feature snapshot requires path and manifest_path"
)
max_age_days = feature_snapshot.get("max_age_days")
if required is True and not isinstance(max_age_days, int):
errors.append(
f"strategy {sid}: required feature snapshot max_age_days must be an integer"
)
elif isinstance(max_age_days, bool) or (
isinstance(max_age_days, int) and max_age_days < 1
):
errors.append(
f"strategy {sid}: feature snapshot max_age_days must be at least 1"
)
return errors


Expand Down Expand Up @@ -479,6 +490,69 @@ def build_strategy_platform_dry_run_coverage(config: dict | None = None) -> dict
}


def build_runtime_artifact_evidence_registry(config: dict | None = None) -> dict[str, object]:
"""Build the immutable, no-order verification plan for required snapshots.

This registry deliberately describes only controller-declared artifacts. It
never publishes data, rewrites a URI, or grants a strategy a higher
lifecycle stage. The verifier consumes it using a read-only cloud
identity and reports failures back to the operator.
"""
config = config if config is not None else load_config()
coverage_by_profile = {
str(row["profile"]): row
for row in build_strategy_platform_dry_run_coverage(config)["profiles"]
if isinstance(row, dict) and isinstance(row.get("profile"), str)
}
entries: list[dict[str, object]] = []
for profile, strategy in sorted(config.get("strategies", {}).items()):
if not isinstance(strategy, dict):
continue
runtime_artifacts = strategy.get("runtime_artifacts")
feature_snapshot = (
runtime_artifacts.get("feature_snapshot")
if isinstance(runtime_artifacts, dict)
else None
)
if not isinstance(feature_snapshot, dict) or feature_snapshot.get("required") is not True:
continue
snapshot_path = feature_snapshot.get("path")
manifest_path = feature_snapshot.get("manifest_path")
max_age_days = feature_snapshot.get("max_age_days")
if not (
isinstance(snapshot_path, str)
and snapshot_path.startswith("gs://")
and isinstance(manifest_path, str)
and manifest_path.startswith("gs://")
and isinstance(max_age_days, int)
and not isinstance(max_age_days, bool)
and max_age_days >= 1
):
continue
coverage = coverage_by_profile.get(str(profile), {})
entries.append(
{
"profile": str(profile),
"domain": str(strategy.get("domain") or ""),
"snapshot_path": snapshot_path,
"manifest_path": manifest_path,
"max_age_days": max_age_days,
"dry_run_platforms": list(coverage.get("buildable_dry_run_platforms", [])),
"boundary": "read_only_evidence_check_no_publish_no_execution",
}
)
return {
"schema_version": "runtime_artifact_evidence_registry.v1",
"entries": entries,
"summary": {"required_artifact_count": len(entries)},
"boundary": (
"Read-only validation plan. A passing entry proves the declared object, "
"manifest digest, and freshness contract; it does not authorize paper, "
"shadow, or live execution."
),
}


def validate_notification_references(config: dict, errors: list[str]) -> None:
"""Keep every Telegram notification route in runtime configuration."""
notifications = config.get("notifications")
Expand Down Expand Up @@ -1071,6 +1145,11 @@ def main() -> int:
parser.add_argument("--live-candidate-queue", action="store_true", help="Print live-candidate queue JSON and exit")
parser.add_argument("--platform-health-report", action="store_true", help="Print platform health report JSON and exit")
parser.add_argument("--automation-registry", action="store_true", help="Print strategy automation registry JSON and exit")
parser.add_argument(
"--runtime-artifact-evidence-registry",
action="store_true",
help="Print the read-only verification registry for required runtime artifacts and exit",
)
args = parser.parse_args()

config = load_config()
Expand All @@ -1081,6 +1160,16 @@ def main() -> int:
if args.automation_registry:
print(json.dumps(build_strategy_automation_registry(config), ensure_ascii=False, indent=2, sort_keys=True))
return 0
if args.runtime_artifact_evidence_registry:
print(
json.dumps(
build_runtime_artifact_evidence_registry(config),
ensure_ascii=False,
indent=2,
sort_keys=True,
)
)
return 0

errors = validate(config)
if errors:
Expand Down
Loading