Make compaction planner configurable - #6722
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ab1dba9d87
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| /// Maximum number of splits fetched from the metastore per scan. | ||
| scan_page_size: usize, | ||
| /// Interval between compaction planner scan-and-plan cycles. | ||
| scan_and_plan_interval: HumanDuration, |
There was a problem hiding this comment.
When a standalone-compactor deployment sets scan_and_plan_interval: 0s, HumanDuration accepts it and the planner repeatedly schedules ScanAndPlan with no delay, continuously issuing metastore scans and consuming actor/runtime capacity. Reject a zero duration during node-config validation rather than starting this tight production loop.
AGENTS.md reference: AGENTS.md:L21-L22
Useful? React with 👍 / 👎.
| #[serde(deny_unknown_fields, default)] | ||
| pub struct CompactionPlannerConfig { | ||
| /// Maximum number of splits fetched from the metastore per scan. | ||
| scan_page_size: usize, |
There was a problem hiding this comment.
When scan_page_size: 0 is configured, the planner passes a zero limit to every metastore query, so each scan returns no splits and compaction never gets planned even though the actor remains healthy. Make this field nonzero or reject zero while loading the node configuration.
AGENTS.md reference: AGENTS.md:L21-L22
Useful? React with 👍 / 👎.
1 participant
Description
Makes the planner configurable. At high scale this could be desirable.