Skip to content

feat(agent): CSPM AI agent - ReAct loop, 6 tools, CLI, MCP server, security hardening - #397

Open
TFT444 wants to merge 19 commits into
devfrom
feat/355-cspm-ai-agent
Open

TFT444 wants to merge 19 commits into
devfrom
feat/355-cspm-ai-agent

Conversation

@TFT444

@TFT444 TFT444 commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds POST /api/agent/query - a natural-language CSPM Q&A endpoint powered by a ReAct loop over 6 tool functions (get_findings, get_compliance_posture, get_attack_paths, explain_finding, suggest_remediation, scan_subscription)
  • Adds openshield ask CLI command for terminal-based querying
  • Adds an MCP server exposing all 6 tools to Claude Desktop and GitHub Copilot
  • Hardens security: fail-closed tenant guard (empty tenant -> 403), subscription allowlist check on agent route, viewer-role exemption for Q&A POST, rollback safety on all DB tool functions, encoding fixes in tests

Changes by area

api/agent/

  • agent.py - ReAct loop with guardrails (max iterations, prompt injection filter, confidence scoring)
  • tools.py - 6 CSPM tool functions with tenant isolation; added db_conn.rollback() in all except blocks

api/routes/agent.py

  • POST /api/agent/query with rate limiting (20 req/min), JWT auth, fail-closed tenant guard, subscription allowlist check

api/app.py

  • _READ_ACCESSIBLE_POSTS exempts /api/agent/query from the write-role gate so viewer tokens can use Q&A

cli/ - openshield ask command with --subscription-id, --framework, --tenant-id flags

mcp_server.py - MCP server wiring all 6 tools for Claude Desktop / Copilot integration

tests/

  • Full coverage: test_agent_tools, test_agent_loop, test_agent_route, test_cli_ask, test_mcp_server (38 tests)
  • conftest: synthetic tenant injection hook so shared-secret-mode tests pass the tenant guard

Test plan

  • python -m pytest tests/test_agent_tools.py tests/test_agent_loop.py tests/test_agent_route.py tests/test_cli_ask.py tests/test_mcp_server.py -q - 38 passed
  • Tenant with no claim returns 403
  • Viewer role can POST to /api/agent/query (read-only Q&A)
  • Subscription not in allowlist returns 403

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…ests

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…equest body

The route was reading g.tenant_id and g.subscription_id which are never set
by verify_jwt. Fixed to read tenant_id from g.user['tenant'] (matches the
principal dict set by auth.py) and subscription_id from the validated
request body, consistent with how scans.py handles it.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…nt isolation test

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…opilot

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…ty, and role access

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
pip/aiohttp 3.14.5 UnknownUnknown
pip/certifi 2026.10.23 🟢 6.5
Details
CheckScoreReason
Code-Review🟢 6Found 4/6 approved changesets -- score normalized to 6
Security-Policy🟢 10security policy file detected
Binary-Artifacts🟢 10no binaries found in the repo
Maintained🟢 88 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 8
Token-Permissions🟢 10GitHub workflow tokens follow principle of least privilege
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Pinned-Dependencies🟢 5dependency not pinned by hash detected -- score normalized to 5
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 9license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Packaging🟢 10packaging workflow detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
pip/multidict 6.9.1 🟢 6.6
Details
CheckScoreReason
Code-Review⚠️ 0Found 0/22 approved changesets -- score normalized to 0
Maintained🟢 1030 commit(s) and 26 issue activity found in the last 90 days -- score normalized to 10
Binary-Artifacts🟢 10no binaries found in the repo
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 10license file detected
Fuzzing🟢 10project is fuzzed
Security-Policy🟢 10security policy file detected
Packaging🟢 10packaging workflow detected
SAST🟢 10SAST tool is run on all commits
Branch-Protection🟢 3branch protection is not maximal on development and all release branches
Signed-Releases🟢 85 out of the last 5 releases have a total of 5 signed artifacts.
pip/werkzeug 3.1.9 UnknownUnknown

Scanned Files

  • requirements.txt

…ial scan false positive

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…quirements-dev

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
@TFT444 TFT444 self-assigned this Oct 19, 2026
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…enerate locks, reformat code

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…P auth

Critical fixes:
- Sanitize LLM answer through clean_text before returning to API callers
- Validate parsed answer is a string before using it
- Gate scan_subscription to operator/admin roles; viewer bypass via agent route was a privilege escalation
- Add subscription allowlist check to MCP call_tool handler (was missing, HTTP route had it)
- Replace per-call psycopg2.connect in MCP with SimpleConnectionPool to avoid exhausting DB connections

Warning fixes:
- Cap tool result serialization at 4 KB to prevent unbounded prompt growth
- Deduplicate repeated tool calls within one agent loop via seen-set
- Return {"error": "db_error"} from get_findings/explain_finding on DB failure instead of empty list/None so LLM can report the real state

Info fixes:
- Add rationale to nosec B608 comment in tools.py
- Add mcp/__init__.py so local mcp/ package correctly shadows installed SDK for tests
- Expand test_mcp_server.py: subscription allowlist unit tests and call_tool coverage for tenant enforcement and blocked scan

Signed-off-by: Tanvir Farhad <tanvirfarhad007@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…rivy

- Remove unused import psycopg2 from mcp/server.py (ruff F401)
- Remove unused import os from tests/test_mcp_server.py (ruff F401)
- Extract _mcp_mocks variable to shorten two with-patch lines (ruff E501)
- Bump certifi 2026.6.17 -> 2026.10.23 (PYSEC-2026-4320, GHSA-6q48-6c7w-9gw2)
- Bump multidict 6.8.0 -> 6.9.1 (PYSEC-2026-4244)

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
from tests.conftest import _TEST_JWT_SECRET triggered a second module load,
producing a new random secret separate from the one the fixture injected into
app.config. Read the secret directly from app.config to guarantee the same
value is used for both signing and verification.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Use parenthesized context managers instead of backslash continuation,
matching the project's ruff format style.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>

@parthrohit22 parthrohit22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 641b928. The overall shape is good: tenant and subscription come from the request rather than the model, scanning needs an operator role, and there’s a cap on iterations. But the tools can’t work against our database yet.

  1. The queries target tables and columns that don’t exist. dev has no finding_lifecycles, finding_fingerprints, scan_rule_outcomes or compliance_scores table, and no tenant_id column anywhere. #326 · feat(automation): finding…’s schema doesn’t match either. It has fl.id, ff.id and resource_id_normalized, and no fl.finding_id, fl.outcome_id, fl.tenant_id, ff.fingerprint_id, ff.resource_id, or sro.outcome_id/severity/description/remediation.

Every tool falls into its except and returns db_error or an empty result. The agent then tells the user there are no findings, a false “all clear” from a security tool.
The tests use fake connections, so they don’t catch it. Please add a Postgres-backed test.

  1. scan_subscription would fail and skips scan admission. scans has no tenant_id or created_at, requires started_at, and uses lowercase 'pending'. Please go through the existing scan-creation path so the idempotency and admission checks apply.

  2. The MCP server bypasses API auth and roles. It connects straight to the database with one tenant from an environment variable. Anyone with the config can queue scans and read findings, and the subscription allowlist only covers scans. Please route it through the API with a token, or state this limitation clearly and remove the tenant-isolation claim from the README.

  3. The local mcp/ package shadows the mcp SDK. The sys.path/sys.modules workaround is fragile. Please rename it, e.g. openshield_mcp/.

  4. Malformed model output can cause a 500:

A bare JSON string makes "answer" in parsed a substring match, and then parsed["answer"] raises.
tuple(sorted(tool_args.items())) raises on list, dict or non-dict arguments.
The unparsed and no-key return paths skip clean_text.
confidence isn’t validated against HIGH/LOW.

  1. Smaller items:

framework is accepted but never used.
--tenant-id is in the description but doesn’t exist.
openshield ask isn’t installed as a command. Please add it as a subcommand in OWASP/openshield#394’s openshield/cli.py.
The conftest change injects a tenant into the shared app fixture for the whole suite. Please limit it to the agent tests.
Drop the requirements* edits (OWASP/openshield#401 covers them) and fix Ruff lint, which is still red.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants