Skip to content

security: stop logging raw request body in PINECONE_DEBUG_CURL (CodeQL #83) - #687

Open
jhamon wants to merge 1 commit into
mainfrom
security/pin-27-clear-text-logging-body
Open

jhamon wants to merge 1 commit into
mainfrom
security/pin-27-clear-text-logging-body

Conversation

@jhamon

@jhamon jhamon commented Jul 11, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes CodeQL code-scanning alert #83 py/clear-text-logging-sensitive-data (error/high) at pinecone/_internal/http_client.py _log_curl.

The PINECONE_DEBUG_CURL debug logger previously emitted the raw request body as -d '<body>'. CodeQL flags the body.decode(...) expression as clear-text logging of sensitive data.

Change

Stop logging the request body verbatim. Pinecone request bodies carry user vector and metadata payloads that may contain sensitive data (e.g. PII embedded in metadata), so even under the explicit PINECONE_DEBUG_CURL opt-in the body is now replaced with:

--data-binary @body.json  # <N> byte body omitted

This records the payload size and shows where to supply the body manually, while removing the clear-text path to the logger entirely. Sensitive headers (Api-Key, Authorization, Proxy-Authorization) remain redacted as before.

This removes the finding at the source rather than dismissing it — no code-scanning security_events scope needed.

Testing

Updated the two tests that asserted raw body content; added assertions that the body contents are absent and the byte-count placeholder is present.

uv run pytest tests/unit/test_debug_logging.py tests/unit/_internal/test_http_client_curl_debug.py tests/unit/test_http_client.py -k "curl or log_curl or redact or body"
28 passed, 69 deselected

Resolves PIN-27 (child of PIN-16).

🤖 Generated with Claude Code


Note

Low Risk
Debug-only, opt-in logging path with no change to request behavior; reduces exposure of user data in logs.

Overview
Stops logging raw HTTP request bodies when PINECONE_DEBUG_CURL is enabled, addressing CodeQL clear-text logging of sensitive data in _log_curl.

Instead of emitting -d '<decoded body>', the curl-equivalent debug line now uses --data-binary @body.json # <N> byte body omitted, so payload size is visible without writing vectors/metadata (possible PII) to logs. Sensitive header redaction is unchanged.

Unit tests were updated to assert bodies are absent from logs and the byte-count placeholder is present, including for HTTPClient.post integration.

Reviewed by Cursor Bugbot for commit c93062e. Bugbot is set up for automated code reviews on this repo. Configure here.

…G_CURL

Fixes CodeQL py/clear-text-logging-sensitive-data alert #83 at
pinecone/_internal/http_client.py _log_curl.

The debug curl logger previously emitted the raw request body via
`-d '<body>'`. Pinecone request bodies carry user vector and metadata
payloads that may contain sensitive data (e.g. PII in metadata), so the
body is no longer logged verbatim even under the PINECONE_DEBUG_CURL
opt-in. It is replaced with a `--data-binary @body.json  # N byte body
omitted` placeholder that records the size and shows where to supply the
payload manually. Sensitive headers remain redacted as before.

This removes the CodeQL finding at the source (no clear-text path to the
logger) rather than dismissing it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jhamon added a commit that referenced this pull request Oct 23, 2026
…ts docstrings (#790)

* gnhf 1: Corrected the #687 Field-equality sentence in both locations and completed a verified accuracy and style pass over pinecone/utils/filter_builder.py (Field/Condition/FilterBuilder).

* gnhf 2: Completed a verified accuracy and style pass over every Documents/AsyncDocuments method docstring (upsert, batch_upsert, search, fetch, delete, update, list), cutting stale limit values, the hedged 'observed (API 2026-07)' claim, and the server-explained replica caveat, while keeping sync and async byte-identical in Args/Returns/Raises.

* gnhf 3: Completed a verified accuracy and style pass over the score_by query classes (TextQuery, QueryStringQuery, DenseVectorQuery, SparseVectorQuery, EmbedQuery, SparseEmbedQuery) and their module docstring, removing the hedged 'observed (API 2026-07)' claim and every em dash.

* gnhf 4: Completed a verified accuracy and style pass over the documents models (Document, DocumentRecord, UpdateDocumentRecord, the six *DocumentsRequest bodies, and the response envelopes), cutting hard-coded limit values and every em dash.

* gnhf 5: Completed a verified accuracy and style pass over every public SchemaBuilder docstring (module, class, add_* methods, add_custom_field, build), cutting hard-coded limits, a backend-internal metric claim, the language-code dump, and every em dash in public prose, while keeping the PR #787 'structuring aid, server validates' framing.

* gnhf 6: Completed a verified accuracy and style pass over pinecone/models/indexes/schema.py (IndexSchema and every field/config class), cutting hard-coded limits, a trivia sentence about error wording, an overstated client-rejection claim, the 'validated' SchemaBuilder framing, and all 18 em dashes.

* gnhf 7: Added the first documents how-to page (docs/how-to/documents/schema-and-upsert.md: choosing fields, creating the index, upsert, batch_upsert, async visibility, async client) and linked it from a new 'Documents' toctree block in docs/index.rst, with every snippet run against a respx-mocked client.

* gnhf 8: Added the second documents how-to page, docs/how-to/documents/search.md, which covers choosing a score_by clause, the rules for combining clauses, include_fields and how results come back, and filters. It is linked from the existing Documents toctree in docs/index.rst, and every snippet ran against a respx-mocked client.

* gnhf 9: Added the third documents how-to page, docs/how-to/documents/manage.md, which covers fetch by ID and by paginated filter, partial update by ID or filter, delete, and listing IDs, and linked it from the Documents toctree. Every snippet ran against a respx-mocked client.

* gnhf 10: Reframed docs/how-to/integrated-records.md as the page for indexes created with create_for_model. It now opens by sending new users to the documents how-to pages (schema-and-upsert, search), and its em dashes are gone. With this change every part of the area's stop condition is met, and all verification passes.

* no-mistakes(document): Register new documents how-to pages in published URLs

* no-mistakes(document): Register documents search how-to in published URLs

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant