You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(webhooks): Return an error code when a webhook test can't connect - #9019
I have added information to docs/ if required so people know about the feature.
I have filled in the "Changes" section below.
I have filled in the "How did you test this code" section below.
Changes
In this PR, we surface non-HTTP errors during webhook testing as POSIX error names (e.g. ECONNREFUSED, EAI_NONAME), or OpenSSL certificate verification error names (e.g. X509_V_ERR_CERT_HAS_EXPIRED):
The code is derived only from the exception type, errno and OpenSSL verify code, and does not touch sensitive data. The codes do reveal coarse reachability, e.g. refused vs timed out. Response timing already gives that away, and SSRF validation still rejects internal addresses before any request is made.
How did you test this code?
Unit tests
Ran this branch locally (API + dashboard) with SSRF_ALLOWED_HOSTS=localhost and a private-CA HTTPS server on localhost:7000. curl --cacert to the endpoint succeeds, and the dashboard's "Test your webhook" shows the code above.
Ran get_request_error_code inside the released flagsmith/flagsmith:2.292.0 image (Python 3.13, requests 2.33.1, urllib3 2.8.0) against real failures.
Since #7550, a webhook test that fails before getting an HTTP response
returns the same message whatever went wrong, and the exception isn't
logged. Self-hosted users can't tell a TLS trust problem from DNS, a
proxy, a firewall or a timeout.
Add a `code` to that 400 response: the POSIX error name
(`ECONNREFUSED`, `ETIMEDOUT`, `ECONNRESET`, ...) or, for certificate
failures, the Node.js-style OpenSSL verification code
(`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `CERT_HAS_EXPIRED`, ...). The code
is derived from the exception type and errno only, so no hostnames,
addresses or ports reach the client.
No actionable comments were generated in the recent review.
Walkthrough
The change adds a helper that maps request exceptions to error codes. The webhook request error response includes the derived code. The serializer and OpenAPI schema document optional status and code fields. Unit tests cover error-code mappings and webhook responses.
Webhook test failures now report an error code, and the certificate and proxy-timeout cases raised earlier are handled and covered by unit tests. No actionable merge risk remains.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 6280127a-ce24-4081-a03d-29c2a86532f8
Commits
Reviewing files that changed from the base of the PR and between f835498 and 1417a19.
Files selected for processing: 6
api/core/network.py
api/tests/unit/core/test_network.py
api/tests/unit/webhooks/test_unit_webhooks.py
api/webhooks/serializers.py
api/webhooks/views.py
openapi.yaml
Files with no reviewable changes: 1
api/tests/unit/webhooks/test_unit_webhooks.py
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.
✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.93%. Comparing base (7ec6a0e) to head (1417a19). ⚠️ Report is 4 commits behind head on main.
❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
khvn26
changed the title
feat(webhooks): return an error code when a webhook test can't connect
feat(webhooks): Return an error code when a webhook test can't connect
Nov 19, 2026
The reason will be displayed to describe this comment to others. Learn more.
Why "as Node.js reports them"? I thought we'd updated the webhook testing process to be run through the python backend to make it as close as possible to the real thing.
The reason will be displayed to describe this comment to others. Learn more.
It's not about verification process, it's about presentation. Basically, the agent had "POSIX/Node.js-style errors" in the brief because they are the most universally comprehensive to my taste. Happy to remove any Node.js references if you'd like.
The reason will be displayed to describe this comment to others. Learn more.
1 issue found across 6 files
Confidence score: 5/5
In api/webhooks/serializers.py, the code help text says “POSIX error name,” but the values also include Node.js-style names, which could mislead API users. Update the help text to describe the values accurately.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="api/webhooks/serializers.py">
<violation number="1" location="api/webhooks/serializers.py:40">
P3: The `code` help_text says "POSIX error name", but the values come from `core/network.py` and include Node.js-style names such as `ENOTFOUND`, `ESOCKETTIMEDOUT`, and `ERR_TLS_CERT_ALTNAME_INVALID`. It can also be `UNKNOWN`. Update the help_text to say POSIX or Node.js-style, and list the fallback.</violation>
</file>
Reply with feedback, questions, or to request a fix.
The reason will be displayed to describe this comment to others. Learn more.
P3: The code help_text says "POSIX error name", but the values come from core/network.py and include Node.js-style names such as ENOTFOUND, ESOCKETTIMEDOUT, and ERR_TLS_CERT_ALTNAME_INVALID. It can also be UNKNOWN. Update the help_text to say POSIX or Node.js-style, and list the fallback.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At api/webhooks/serializers.py, line 40:
<comment>The `code` help_text says "POSIX error name", but the values come from `core/network.py` and include Node.js-style names such as `ENOTFOUND`, `ESOCKETTIMEDOUT`, and `ERR_TLS_CERT_ALTNAME_INVALID`. It can also be `UNKNOWN`. Update the help_text to say POSIX or Node.js-style, and list the fallback.</comment>
<file context>
@@ -29,5 +29,15 @@ class TestWebhookSuccessResponseSerializer(serializers.Serializer[None]):
+ code = serializers.CharField(
+ required=False,
+ help_text=(
+ "A POSIX error name, or a TLS error code, when "
+ "the webhook could not be reached."
+ ),
</file context>
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
Review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 63730013-119c-461c-a801-31c8bcbd2130
Commits
Reviewing files that changed from the base of the PR and between fe9b4cd and f835498.
Files selected for processing: 6
api/core/network.py
api/tests/unit/core/test_network.py
api/tests/unit/webhooks/test_unit_webhooks.py
api/webhooks/serializers.py
api/webhooks/views.py
openapi.yaml
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Minor · Functional Correctness · Quick win
Account for OpenSSL 3 verification codes.
On OpenSSL 3.0, verification code 24 means NO_ISSUER_PUBLIC_KEY; INVALID_CA is 79. This entry reports the wrong cause for code 24, while an invalid CA returns the generic EPROTO. Account for the runtime version and add tests for these mappings. Keep the existing expiry and issuer-trust mappings. (raw.githubusercontent.com)
The reason will be displayed to describe this comment to others. Learn more.
Minor · Functional Correctness · Quick win
Recognise timeouts inside proxy exception chains.
If an HTTPS proxy connection times out, urllib3 can wrap ConnectTimeoutError in ProxyError. Requests then raises requests.exceptions.ProxyError, so these top-level checks do not match. The chain classifier has no nested timeout check and returns UNKNOWN when the socket timeout has no errno. The dependency sources establish this wrapping path. (raw.githubusercontent.com)
Classify nested timeout types and add a proxy-timeout test. Exclude NewConnectionError from a ConnectTimeoutError check so refused connections still return ECONNREFUSED. (raw.githubusercontent.com)
- Name certificate failures with OpenSSL's X509_V_ERR_* constants,
numbered as in OpenSSL 3+, where 24 is NO_ISSUER_PUBLIC_KEY and
INVALID_CA is 79.
- Name DNS failures with their getaddrinfo EAI_* code, instead of
reporting every resolver error as ENOTFOUND.
- Report read timeouts, and timeouts connecting to a proxy, as
ETIMEDOUT.
- Omit `code` when the cause isn't recognised, rather than returning
UNKNOWN.
- Build real urllib3 objects in tests, removing the type ignores.
The reason will be displayed to describe this comment to others. Learn more.
2 issues found across 6 files (changes from recent commits).
Confidence score: 4/5
In api/core/network.py, unmapped certificate verification codes are reported as EPROTO, which misleads users about the trust failure. Return None for unmapped codes so they’re omitted.
In api/core/network.py, verify code 24 is mislabeled on Python builds linked to OpenSSL 1.1.1, obscuring invalid-CA failures. Select the name based on ssl.OPENSSL_VERSION_INFO.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="api/core/network.py">
<violation number="1" location="api/core/network.py:37">
P2: This labels `verify_code == 24` incorrectly on Python builds linked to OpenSSL 1.1.1, where code 24 is `X509_V_ERR_INVALID_CA`. Select the name based on `ssl.OPENSSL_VERSION_INFO` so invalid-CA failures are not misreported.</violation>
<violation number="2" location="api/core/network.py:70">
P2: An unmapped certificate verification code is reported as `EPROTO`, which tells the user this is a protocol error rather than a trust failure. Return `None` for unmapped verify codes so the code is omitted, as the commit message describes.</violation>
</file>
Reply with feedback, questions, or to request a fix.
The reason will be displayed to describe this comment to others. Learn more.
P2: An unmapped certificate verification code is reported as EPROTO, which tells the user this is a protocol error rather than a trust failure. Return None for unmapped verify codes so the code is omitted, as the commit message describes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At api/core/network.py, line 70:
<comment>An unmapped certificate verification code is reported as `EPROTO`, which tells the user this is a protocol error rather than a trust failure. Return `None` for unmapped verify codes so the code is omitted, as the commit message describes.</comment>
<file context>
@@ -10,63 +10,74 @@
# SSL errors subclass OSError, with an OpenSSL-specific errno.
if isinstance(exc, ssl.SSLCertVerificationError):
- return _CERT_VERIFY_ERROR_CODES.get(exc.verify_code, "EPROTO")
+ return _CERT_VERIFY_ERROR_NAMES.get(exc.verify_code, "EPROTO")
if isinstance(exc, ssl.SSLError):
return "EPROTO"
</file context>
The reason will be displayed to describe this comment to others. Learn more.
P2: This labels verify_code == 24 incorrectly on Python builds linked to OpenSSL 1.1.1, where code 24 is X509_V_ERR_INVALID_CA. Select the name based on ssl.OPENSSL_VERSION_INFO so invalid-CA failures are not misreported.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At api/core/network.py, line 37:
<comment>This labels `verify_code == 24` incorrectly on Python builds linked to OpenSSL 1.1.1, where code 24 is `X509_V_ERR_INVALID_CA`. Select the name based on `ssl.OPENSSL_VERSION_INFO` so invalid-CA failures are not misreported.</comment>
<file context>
@@ -10,63 +10,74 @@
+ 21: "X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE",
+ 22: "X509_V_ERR_CERT_CHAIN_TOO_LONG",
+ 23: "X509_V_ERR_CERT_REVOKED",
+ 24: "X509_V_ERR_NO_ISSUER_PUBLIC_KEY",
+ 25: "X509_V_ERR_PATH_LENGTH_EXCEEDED",
+ 26: "X509_V_ERR_INVALID_PURPOSE",
</file context>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apiIssue related to the REST APIfeatureNew feature or request
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for submitting a PR! Please check the boxes below:
docs/if required so people know about the feature.Changes
In this PR, we surface non-HTTP errors during webhook testing as POSIX error names (e.g.
ECONNREFUSED,EAI_NONAME), or OpenSSL certificate verification error names (e.g.X509_V_ERR_CERT_HAS_EXPIRED):The code is derived only from the exception type, errno and OpenSSL verify code, and does not touch sensitive data. The codes do reveal coarse reachability, e.g. refused vs timed out. Response timing already gives that away, and SSRF validation still rejects internal addresses before any request is made.
How did you test this code?
SSRF_ALLOWED_HOSTS=localhostand a private-CA HTTPS server onlocalhost:7000.curl --cacertto the endpoint succeeds, and the dashboard's "Test your webhook" shows the code above.get_request_error_codeinside the releasedflagsmith/flagsmith:2.292.0image (Python 3.13, requests 2.33.1, urllib3 2.8.0) against real failures.