Skip to content

[perf] nightly: project trash timing exceeds the previous-run threshold again #401

Open

Description

@hsliuustc0106

Observed failure

On main 58b6a96df89f92f240b2931bc55fdb3cd779d244, nightly 44425602809 / daily-benchmark measured storage.delete.trash at 232.9 ms, versus 130.0 ms in the previous reference run: +102.9 ms / +79.15%.

The benchmark completed and uploaded its report. The threshold step exited 1, naming only REGRESSION storage.delete.trash; no suite failure or incomplete sample was reported. The rule requires both current > 1.5× reference and a delta of at least 50 ms. Both were met; the fixed baseline is unseeded, so 130.0 ms is a previous-run sample, not a frozen baseline. Pan p95 did not trigger the gate (delta 33.4 ms).

Reproduction and limits

GitHub-hosted Ubuntu 22.04.5, image 20261111.360.2, Python 3.12.16, runner 2.338.2. On the tested revision with repository dependencies installed:

python scripts/bench_creation_projects.py --scenario delete --commits 600 --nodes 200 --projects 0 --json

This creates disposable synthetic projects. It separately measures client snapshot and times one execute(project.trash) operation. Earlier scheduled values were 104.4, 148.3 and 107.1 ms; #299 also recorded a separate 172.4 vs 107.2 ms signal while tracking an artifact-layout bug. These are different runner instances/revisions, not matched repeats. Changes since the previous nightly were chat-draft serialization/tests (#395) and documentation (#399), with no storage or benchmark file changes. This establishes a threshold failure, not a confirmed product regression; no independent latency reproduction has been run.

Acceptance

  • Preserve the two JSON samples and runner details. Repeat the same workload on matched environments, reporting individual values and a stable aggregate.
  • If reproducible, profile the trash operation and distinguish storage work from runner/filesystem variation. Keep the existing threshold and missing-sample checks unless evidence supports a change.

Related: #299 (historical threshold signal), #180/#182 (deletion performance), #213/#216 (benchmark). No active issue tracking this exact threshold failure was found.

Activity

  1. added
    area:ciContinuous integration and checks
    area:performancePerformance and responsiveness / 性能与响应速度
    on Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciContinuous integration and checksarea:performancePerformance and responsiveness / 性能与响应速度

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions