Skip to content

Send Content-Encoding from the WEBrick handlers - #278

Open
lbarrosby wants to merge 2 commits into
fluent:masterfrom
lbarrosby:fix-webrick-content-encoding
Open

lbarrosby wants to merge 2 commits into
fluent:masterfrom
lbarrosby:fix-webrick-content-encoding

Conversation

@lbarrosby

Copy link
Copy Markdown
Contributor

Follow-up to #275, from the review there.

When the plugin serves with WEBrick (an IPv6 bind, or <ssl> with extra_conf), the mount_proc handlers copied only Content-Type from the response tuple. With content_encoding gzip:

  • /metrics sent a gzip body with no Content-Encoding header.
  • /aggregated_metrics returned 500, because in this mode do_request is the plain Net::HTTP one, and Net::HTTP only decodes the worker body when the header is there. The aggregator got gzip bytes.

Both handlers now copy every header from the tuple, so they match the http_server helper path.

Test

New specs under when extra_conf is used request /metrics and /aggregated_metrics over WEBrick with content_encoding gzip. They check for a 200, for Content-Encoding: gzip, and that the body gunzips to the metrics. On master they fail with content-encoding nil on /metrics and a 500 on /aggregated_metrics.

$ bundle exec rake
186 examples, 0 failures

End to end with workers 2, bind ::1, content_encoding gzip:

master:  /aggregated_metrics 500, /metrics 200 with no Content-Encoding
branch:  /aggregated_metrics 200 enc=gzip, /metrics 200 enc=gzip

The WEBrick handlers copied only Content-Type from the response, so
with content_encoding gzip /metrics sent a gzip body without the
Content-Encoding header. Net::HTTP then did not decode the worker
responses and /aggregated_metrics returned 500.

Copy all the response headers in both handlers.

Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>
@Watson1978 Watson1978 added the bug label Nov 16, 2026

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up to #275.

I checked this on the PR head and on the merge result with the current master, which includes #275.

  • With <ssl> extra_conf and content_encoding gzip, master serves /metrics without Content-Encoding and /aggregated_metrics returns 500. With this PR both return 200 with Content-Encoding: gzip. This also holds with workers 2, and the aggregated output has the metrics of both workers.
  • The new specs fail on master and pass on this branch.

One change is needed for the default content_encoding identity. See the inline comment.

I could not test the IPv6 bind path because IPv6 is disabled on my machine. It uses the same handlers as the extra_conf path, though.

Comment thread lib/fluent/plugin/in_prometheus.rb Outdated
status, header, body = all_metrics
res.status = status
res['Content-Type'] = header['Content-Type']
header.each { |name, value| res[name] = value }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST: This also copies Content-Encoding: identity when content_encoding is identity, which is the default. On master, the WEBrick path sends no Content-Encoding for these responses.

This changes a setup that works on master. nginx does not compress an upstream response that already has a Content-Encoding header. I put nginx with gzip on; gzip_proxied any; gzip_types *; in front of <ssl> extra_conf with the default settings. On master, nginx compressed /metrics and /aggregated_metrics from 3060 to 538 bytes. With this PR, both stay at 3060 bytes and carry Content-Encoding: identity.

The http_server helper path already sends identity, but changing that is out of scope here. Please keep these points in both WEBrick handlers (this line and line 206):

  • With identity, send no Content-Encoding header, as on master.
  • With gzip, send Content-Encoding: gzip, as this PR does now.
  • On the 500 error response, send no Content-Encoding header, as on master.

Note that res['Content-Encoding'] = nil does not remove the header in WEBrick. It sends an empty Content-Encoding: header.

Please also add a spec for the identity case. It should check that res['content-encoding'] is nil on both /metrics and /aggregated_metrics.

On master the WEBrick handlers sent no Content-Encoding header. Keep
that for the default content_encoding identity, so that a proxy in
front can still compress the response, and send it only for gzip.

Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>
@lbarrosby

Copy link
Copy Markdown
Contributor Author

Thanks. Pushed 6ea07c4. Both WEBrick handlers now go through copy_webrick_headers, which skips Content-Encoding when its value is identity:

  • identity: no Content-Encoding header, as on master.
  • gzip: Content-Encoding: gzip, as before.
  • 500 response: its header has only Content-Type, so no Content-Encoding, as on master.

The header is skipped, not set to nil, so WEBrick does not send an empty Content-Encoding:. The http_server helper path is unchanged.

New spec with content_encoding identity uses the same <ssl> extra_conf setup and checks that res['content-encoding'] is nil on /metrics and /aggregated_metrics. It passes on master and fails on 9eef04a. bundle exec rake: 188 examples, 0 failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants