Python: preserve sensitive configuration key precision - #22396
Draft
yoff wants to merge 2 commits into
Draft
Conversation
Reproduce the Airflow-shaped loss of section/key identity through layered configuration getters. The concrete non-sensitive key is documented as SPURIOUS while concrete sensitive and dynamic keys remain positive controls. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The sensitive-name heuristic marks calls such as _get_config_value_from_secret_backend as secret sources. Interprocedural summaries then route those sources through every layered configuration getter result without retaining the guard correlation for each concrete section/key pair. This reproduced Airflow clear-text logging false positives for core.EXECUTOR and core.DAGS_FOLDER. Recognize fully resolved configuration lookups whose only value selectors are unique literal section and key arguments, and expose them as sanitizers to the clear-text logging, clear-text storage, and weak sensitive-data hashing analyses. The implementation checks every resolved target and preserves sensitive callee names and configuration-specific secret indicators. This belongs in SensitiveDataSources rather than core summary routing: the summarized path is topologically valid, while the missing fact is semantic precision for a name-based sensitive-source heuristic. Core dataflow therefore remains conservative for all other analyses. Dynamic or ambiguous selectors, sensitive concrete names, direct secret-named getters, and calls with additional value arguments remain flowing. Tests cover PASSWORD, FERNET_KEY, PASSWORD_FILE, dynamic keys, a sensitive fallback, and the Airflow-shaped safe routes. The refinement intentionally relies on conventional section/key parameter names and does not attempt arbitrary application registry reasoning. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Experimental draft stacked on #21925. This must not merge independently of its parent.
The base was revalidated immediately before creation at
github/codeql:yoff/python-shared-cfg-dataflow-flip=1a8e317b4a328bea1059453a2ab3ba6eada09e3f.Causal evidence
This is not an Airflow model correction. Python's generic
SensitiveFunctionCallheuristic marks_get_config_value_from_secret_backend(...)as asecretsource because the resolved function name matches the sensitive-name regular expression. Global summaries then preserve the topological return path through Airflow's_get_env_var_option/_get_secret_option,Configuration.get,getlist, andget_mandatory_list_value, but they do not retain the control correlation between(section, key) in self.sensitive_config_valuesand each caller's concrete arguments.Consequently,
core.EXECUTORandcore.DAGS_FOLDERcan inherit the secret source even though those pairs are not members of Airflow's runtime sensitive-key registry and cannot take the secret-backend branches. A secret-backend getter being reachable for some registered keys does not imply that every concrete key handled by the shared getter is sensitive.The first commit reproduces this with an Airflow-shaped local flow and a passing
SPURIOUSexpectation. The second commit removes that annotation after the precision fix.Scope
The change deliberately leaves core dataflow and summary routing unchanged: those paths are topologically valid and useful to other analyses. Instead,
SensitiveDataSourcesidentifies a known non-sensitive configuration lookup only when:sectionandkeyparameters;The clear-text logging, clear-text storage, and weak sensitive-data hashing customizations expose those nodes through their existing sanitizer extension points. There are no Airflow names or key values in the implementation.
Exact Airflow evidence
On a locally extracted CodeQL database for
apache/airflow@a9da0f7fb48dc7526b2745be3e8fe64e1c775da2:core.EXECUTORfamily atexecutor_loader.py:225/234is removed;core.DAGS_FOLDERfamily atutils/file.py:68/96is removed;(section, key)pairs; andThe DCA report's
cli_parser.py:64row is the motivatingcore.EXECUTORroute. The local extraction did not reproduce that exact sink location, so its evidence remains the DCA comparison plus the independently reproduced shared configuration route; the exact local database did reproduce and remove the related executor and file-path families above.Conservative controls
Tests retain flow for:
PASSWORD,FERNET_KEY, andPASSWORD_FILEselectors;Ambiguous or unresolved calls also remain conservative because every resolved target must satisfy the refinement.
Limitations
This is still a name-based heuristic refinement. It intentionally recognizes only configuration APIs using conventional
section/keyformal names and does not attempt arbitrary application registry reasoning. Calls with additional value inputs are excluded because the sanitizer blocks the complete lookup result, not one incoming edge. The broad Airflow result delta is why this remains an experimental draft despite the zero-overlap registry audit.Validation
python/ql/test/library-tests/dataflow/sensitive-datapython/ql/test/query-tests/Security/CWE-312-CleartextLoggingpython/ql/test/query-tests/Security/CWE-312-CleartextStoragepython/ql/test/query-tests/Security/CWE-312-CleartextStorage-py3python/ql/test/query-tests/Security/CWE-327-WeakSensitiveDataHashing