[andypalmi]

Description

Several Cypress specs only passed because of the order the suite runs them in. They relied on state an earlier spec had or had not created, so moving, sharding or running a spec on its own could make it fail. This makes every spec independent of run order, which is needed before the UI jobs can be sharded (#9006).

EE

1. applications/pipelines left a device group and pipeline behind, which broke applications/device-groups. It now deletes them at the end of the test.
2. instance-2-with-devices is never started, so its stored state stayed running until a page polled the driver and rewrote it to unknown. The environment now stores unknown up front, and applications/applications expects 1 running and 1 stopped instance.
3. rbacTeamFactory was not awaited, so the first spec could run before the RBAC fixtures existed. It is now awaited.
4. devices/developer-mode could start while a device was still in developer mode from the previous test. beforeEach now puts BTeam devices back into fleet mode.

OS
5. team/team-membership removed the member in the third row, which was dashboard-dave in some orders and broke brokers/clients and brokers/hierarchy. It now removes charlie by name.
6. devices/assignment left new-device-* devices in application-2, which broke applications/devices. They are now deleted in after().
7. applications/devices expected exactly 3 devices, but devices/bulk deletes one of them. It now compares against the API count.
8. The rollback test in instances/version-history/snapshots really rolled back instance-1-1 and removed its dashboard module, which broke team/instances. The rollback response is now stubbed, since the test only checks the request.
9. team/instances did not wait for the dashboard instances request before asserting on the table. It now does.
10. admin/stacks versioned and deactivated stack1, which instance-1-1 uses, and that broke instances. A new unused stack-to-replace fixture is versioned instead.
11. admin could lose the navigation click when it was made before the team overview had loaded. Both admin tests now wait for it.

Items 4, 9 and 11 also cover flakes reported in #8478.

Testing

Each suite was run with retries disabled, in a container limited to the same 4 CPUs as a GitHub runner:

1. In alphabetical order and in the current CI order: all OS and EE specs pass.
2. Split into shards, each on a fresh server (EE in 3, OS in 2): all specs pass.
3. Each spec that leaves state behind, run directly before the spec it used to break: all pass.

No workflow changes. Sharding follows in #9006.

Closes #9004
Closes #9005

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.46%. Comparing base (40b10a5) to head (192fb40).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9012   +/-   ##
=======================================
  Coverage   79.46%   79.46%           
=======================================
  Files         475      475           
  Lines       26018    26018           
  Branches     6934     6934           
=======================================
  Hits        20674    20674           
  Misses       5344     5344           

Flag	Coverage Δ
backend	79.46% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.