Skip to content

Parse histogram buckets such as .005 and 1e-3 as numbers - #274

Merged
Watson1978 merged 2 commits into
fluent:masterfrom
RaphaelFakhri:fix-histogram-bucket-parsing
Nov 7, 2026
Merged

Watson1978 merged 2 commits into
fluent:masterfrom
RaphaelFakhri:fix-histogram-bucket-parsing

Conversation

@RaphaelFakhri

@RaphaelFakhri RaphaelFakhri commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

The buckets parameter of a histogram <metric> converted any value that does not match digits.digits with String#to_i:

  • buckets .005, .01, .025 became [0, 0, 0], so every observation fell into the +Inf bucket without an error.
  • buckets 1e-3, 1e-2, 0.1 became [1, 1, 0.1], and startup failed with Unsorted buckets, typo?.
  • A value that is not a number, such as abc, became 0.

This change parses each bucket with Float, and raises ConfigError for a value that is not a number. When the old conversion gives the same number, the old result is kept. This keeps the existing le labels. For example, 1 stays "1", 100 stays "100.0" and -5 stays "-5". The old regexp did not escape ., so integers of three or more digits were floats on master.

Tests

spec/fluent/plugin/prometheus/histogram_spec.rb covers integer and decimal buckets, buckets that keep the same values and types as master (10, 100, 1000 and -5, 0, +5), decimals without a leading zero, exponent notation and an invalid value. The examples use eql, so they catch a change between Integer and Float. Without the fix, 3 of its 6 examples fail. With the fix, the full suite passes (184 examples, 0 failures).

Fixes #273

The buckets parameter turned any value which did not look like
"digits.digits" into an integer with String#to_i. ".005, .01, .025"
became [0, 0, 0], so every observation fell into the +Inf bucket, and
"1e-3" became 1. A value which is not a number became 0 instead of
being refused.

Parse a bucket with Integer or Float, and raise ConfigError for a
value which is not a number.

Signed-off-by: Raphael Fakhri <153192858+RaphaelFakhri@users.noreply.github.com>

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix. .005 and 1e-3 now parse as 0.005 and 0.001, and a value that is not a number, such as abc, raises a ConfigError.

One problem remains. This change also changes the le label of some integer buckets that work on master. Please see the inline comments.

Comment thread lib/fluent/plugin/prometheus.rb Outdated
Comment on lines +659 to +660
# keep an integer as is, so that its `le` label stays "1", not "1.0"
e.match?(/\A\d+\z/) ? Integer(e, 10) : Float(e)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST: This changes the le label of integer buckets that work on master.

The old regexp did not escape ., so it also matched integers of three or more digits, and they became floats. Signed integers did not match it, so they stayed integers.

buckets le on master le with this PR
10, 100, 1000 "10", "100.0", "1000.0" "10", "100", "1000"
-5, 0, +5 "-5", "0", "5" "-5.0", "0", "5.0"

Prometheus 3 normalizes le on ingestion, so the old and new labels become the same there. But Prometheus 2 keeps the labels of the text format as is, so it gets new series. Then queries, recording rules and alerts that use le="100.0" stop matching.

Please keep the master values and labels in the table above, and keep the fixes for .005, 1e-3 and abc. One way is to parse each value with Float(e), and use the old result when it is equal to that value. Keep the old regexp as is, because escaping its . changes the labels again. I checked this suggestion with the existing specs, the cases in this PR and the cases above:

Suggested change
# keep an integer as is, so that its `le` label stays "1", not "1.0"
e.match?(/\A\d+\z/) ? Integer(e, 10) : Float(e)
# keep the old result if it is right, so that existing `le` labels such as "100.0" stay the same
value = Float(e)
legacy = e[/\A\d+.\d+\Z/] ? e.to_f : e.to_i
legacy == value ? legacy : value

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, applied in be8b6c5. 10, 100, 1000 and -5, 0, +5 keep the le labels from master, and .005, 1e-3 and abc keep the fix.

Comment on lines +26 to +28
it 'keeps integer and decimal buckets' do
expect(buckets_of('0.1, 1, 5, 10')).to eq([0.1, 1, 5, 10])
end

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHOULD: eq cannot catch the le change in lib/fluent/plugin/prometheus.rb, because [1.0] == [1] is true in Ruby. Please use eql and add the cases from that comment. With this PR as is, the two new examples fail:

Suggested change
it 'keeps integer and decimal buckets' do
expect(buckets_of('0.1, 1, 5, 10')).to eq([0.1, 1, 5, 10])
end
it 'keeps integer and decimal buckets' do
expect(buckets_of('0.1, 1, 5, 10')).to eql([0.1, 1, 5, 10])
end
it 'keeps buckets of three or more digits as floats, as before' do
expect(buckets_of('10, 100, 1000')).to eql([10, 100.0, 1000.0])
end
it 'keeps signed integer buckets as integers' do
expect(buckets_of('-5, 0, +5')).to eql([-5, 0, 5])
end

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in be8b6c5. The examples use eql, and the two new ones fail on the previous commit and pass now. bundle exec rspec passes (184 examples).

The previous change parsed every all-digit bucket as an Integer. On
master, buckets of three or more digits such as 100 matched the
unescaped "digits.digits" regexp and became floats, and signed integers
such as -5 stayed integers. Their le labels changed from "100.0" to
"100" and from "-5" to "-5.0", which creates new series on Prometheus 2.

Keep the old result when it equals the parsed Float value, and use eql
in the specs so that a change between 1 and 1.0 fails them.

Signed-off-by: Raphael Fakhri <153192858+RaphaelFakhri@users.noreply.github.com>

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update. I checked be8b6c5. 10, 100, 1000 and -5, 0, +5 keep the bucket values and le labels from master. .005 and 1e-3 still parse correctly, and abc still raises a ConfigError. The specs now use eql, so they catch a change between Integer and Float. I also updated the PR description to match be8b6c5. LGTM.

@Watson1978
Watson1978 merged commit 379987c into fluent:master Nov 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Histogram buckets such as .005 or 1e-3 are parsed as 0 or 1

2 participants