[feature](fe) Add meta service RPC rate limit dry-run mode by mymeiyi · Pull Request #66940 · apache/doris

mymeiyi · 2026-08-19T07:37:14Z

Enabling FE-side meta service RPC rate limiting immediately waits for or rejects requests, so operators cannot evaluate whether the configured limits fit production traffic first.
Add a dry-run mode that evaluates the shared rate limiter and reports would-wait and would-reject decisions without delaying or rejecting RPCs.

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Enabling FE-side meta service RPC rate limiting immediately waits for or rejects requests, so operators cannot evaluate whether the configured limits fit production traffic first. Add a dry-run mode that evaluates the shared rate limiter and reports would-wait and would-reject decisions without delaying or rejecting RPCs. Enable rate limit evaluation and dry-run mode by default so deployments collect observations before enforcement. ### Release note Add meta_service_rpc_rate_limit_dry_run and enable FE meta service RPC rate limit dry-run evaluation by default. ### Check List (For Author) - Test: Unit tests added but not run, as requested - Behavior changed: Yes, rate limit evaluation and dry-run reporting are enabled by default without enforcing limits - Does this need documentation: No

hello-stephen · 2026-08-19T07:37:19Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

mymeiyi · 2026-08-19T07:39:42Z

/review

Copilot

Pull request overview

Adds a “dry-run” mode to the FE meta service RPC rate limiter so operators can observe would-wait / would-reject outcomes (via logs/metrics) without actually delaying or rejecting RPC traffic. This fits into Doris FE cloud-mode meta service client behavior and its operational observability (CloudMetrics).

Changes:

Introduces meta_service_rpc_rate_limit_dry_run and uses it in MetaServiceRpcRateLimiter.acquire() to bypass waiting/rejecting while still reporting outcomes.
Adds new CloudMetrics for dry-run would-reject counts and would-wait latency histograms.
Extends existing FE tests to cover dry-run behavior and ensures test helpers explicitly disable dry-run when needed.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`fe/fe-core/src/main/java/org/apache/doris/cloud/rpc/MetaServiceRpcRateLimiter.java`	Adds dry-run branches for would-reject / would-wait reporting without enforcement.
`fe/fe-core/src/main/java/org/apache/doris/metric/CloudMetrics.java`	Registers new per-method + aggregate counters and per-method histogram for dry-run rate-limit observations.
`fe/fe-common/src/main/java/org/apache/doris/common/Config.java`	Adds the new dry-run config flag and adjusts rate-limit-related defaults.
`fe/fe-core/src/test/java/org/apache/doris/cloud/rpc/MetaServiceRpcRateLimiterTest.java`	Adds tests validating dry-run does not reject or wait.
`fe/fe-core/src/test/java/org/apache/doris/cloud/rpc/MetaServiceProxyTest.java`	Ensures tests save/restore the new dry-run config and keep rate limiting disabled by default within tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+    @ConfField(mutable = true, description = "Whether to only evaluate and report meta service RPC rate limits "
+            + "without waiting or rejecting requests. This takes effect only when meta service RPC rate limiting "
+            + "is enabled.")
+    public static boolean meta_service_rpc_rate_limit_dry_run = true;



    @ConfField(mutable = true, description = "Whether to enable QPS rate limit for RPC requests to meta service.")
-    public static boolean meta_service_rpc_rate_limit_enabled = false;
+    public static boolean meta_service_rpc_rate_limit_enabled = true;


github-actions · 2026-08-19T07:56:28Z

Codex automated review failed and did not complete.

Error: Review context preparation failed before Codex ran; inspect the 'Prepare authoritative PR context and required AGENTS guides' step.
Workflow run: https://github.com/apache/doris/actions/runs/32228859946

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

gavinchou · 2026-08-19T08:10:20Z

+        boolean dryRun = Config.meta_service_rpc_rate_limit_dry_run;
        if (nanosToWait < 0) {
+            if (dryRun) {
+                if (MetricRepo.isInit && Config.isCloudMode()) {


why should we check Config.isCloudMode() in a cloud context object MetaServiceRpcRateLimiter?
the cloud check is redundant?

gavinchou · 2026-08-19T08:12:05Z

+                    CloudMetrics.META_SERVICE_RPC_ALL_RATE_LIMIT_DRY_RUN_REJECTED.increase(1L);
+                    CloudMetrics.META_SERVICE_RPC_RATE_LIMIT_DRY_RUN_REJECTED.getOrAdd(methodName).increase(1L);
+                }
+                LOG.debug("meta service rpc rate limiter dry run would reject request, method: {}, permits: {}, "


better check if (debug enabled) first, because it may be a hot path.

gavinchou · 2026-08-19T08:13:52Z

+                    CloudMetrics.META_SERVICE_RPC_ALL_RATE_LIMIT_DRY_RUN_REJECTED.increase(1L);
+                    CloudMetrics.META_SERVICE_RPC_RATE_LIMIT_DRY_RUN_REJECTED.getOrAdd(methodName).increase(1L);


the metrics should always be recorded if it is throttled
the meta_service_rpc_rate_limit_dry_run is used to return 0 instead of throw error only.

gavinchou · 2026-08-19T08:14:04Z


        long waitMs = TimeUnit.NANOSECONDS.toMillis(nanosToWait);
+        if (dryRun) {
+            if (MetricRepo.isInit && Config.isCloudMode()) {


gavinchou · 2026-08-19T08:15:02Z

    public static AutoMappedMetric<LongCounterMetric> META_SERVICE_RPC_TOTAL;
    public static AutoMappedMetric<LongCounterMetric> META_SERVICE_RPC_FAILED;
    public static AutoMappedMetric<LongCounterMetric> META_SERVICE_RPC_RATE_LIMITED;
+    public static AutoMappedMetric<LongCounterMetric> META_SERVICE_RPC_RATE_LIMIT_DRY_RUN_REJECTED;


the name should not contains dryrun, because when we are not running in dry run mode, the throttled metrics should be recorded too.

if there are metrics record throttled request, reuse them, so there are no more new metrics should be added

Copilot AI lite review requested due to automatic review settings August 19, 2026 07:37

mymeiyi requested a review from gavinchou as a code owner August 19, 2026 07:37

Copilot started reviewing on behalf of mymeiyi August 19, 2026 07:48 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

gavinchou reviewed Aug 19, 2026

View reviewed changes

gavinchou added the dev/4.1.x label Aug 19, 2026

Navigation Menu

Uh oh!

[feature](fe) Add meta service RPC rate limit dry-run mode - #66940

[feature](fe) Add meta service RPC rate limit dry-run mode#66940
mymeiyi wants to merge 1 commit into
apache:masterfrom
mymeiyi:fe-enable-rpc-limiter

mymeiyi commented Aug 19, 2026

Uh oh!

hello-stephen commented Aug 19, 2026

Uh oh!

mymeiyi commented Aug 19, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		CloudMetrics.META_SERVICE_RPC_ALL_RATE_LIMIT_DRY_RUN_REJECTED.increase(1L);
		CloudMetrics.META_SERVICE_RPC_RATE_LIMIT_DRY_RUN_REJECTED.getOrAdd(methodName).increase(1L);

Uh oh!

Conversation

mymeiyi commented Aug 19, 2026

Uh oh!

hello-stephen commented Aug 19, 2026

Uh oh!

mymeiyi commented Aug 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions Bot commented Aug 19, 2026

Uh oh!

gavinchou Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

gavinchou Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

gavinchou Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

gavinchou Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

gavinchou Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

gavinchou Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants