Skip to content

Use _key instead of key in Kubernetes baseline lookups - #4432

Open
pyth0n1c wants to merge 1 commit into
developfrom
fix-k8s-baseline-lookup-key
Open

pyth0n1c wants to merge 1 commit into
developfrom
fix-k8s-baseline-lookup-key

Conversation

@pyth0n1c

Copy link
Copy Markdown
Collaborator

Details

Four Kubernetes baselines build an identity string in a field named key and write it with outputlookup. The KV Store lookup definitions in lookups/kvstore/k8s_*.yml declare _key, so key was stored as an ordinary column and Splunk generated a random _key per record.

This PR writes _key in the baselines and matches on _key in the detections that read the lookups.

Baselines

  • baseline_of_kubernetes_container_network_io
  • baseline_of_kubernetes_container_network_io_ratio
  • baseline_of_kubernetes_process_resource
  • baseline_of_kubernetes_process_resource_ratio

Detections (| lookup <name> key -> | lookup <name> _key)

  • kubernetes_anomalous_inbound_outbound_network_io
  • kubernetes_anomalous_inbound_to_outbound_network_io_ratio
  • kubernetes_process_with_anomalous_resource_utilisation
  • kubernetes_process_with_resource_ratio_anomalies

version is bumped and modification_date set to 2026-11-17 on all eight files. The two join type=left key network detections build their own key and do not use these lookups, so they are unchanged.

Not yet verified

  • The searches have not been run on a Splunk instance.
  • | lookup <name> _key only matches if _key is in the generated lookup's fields_list. The lookups/kvstore/k8s_*.yml files already list _key, but the generated transforms.conf should be checked in the built app.
  • The baselines still use outputlookup without append=true, so each run rebuilds the collection. This fixes the schema and key values but does not add upsert behavior.

Checklist

  • Validate name matches <platform>_<mitre att&ck technique>_<short description> nomenclature
  • CI/CD jobs passed ✔️
  • Validated SPL logic.
  • Validated tags, description, and how to implement.
  • Verified references match analytic.
  • Confirm updates to lookups are handled properly.

🤖 Generated with Claude Code

The four Kubernetes baselines built an identity string in a field named
`key` and wrote it with outputlookup. The matching KV Store lookup
definitions declare `_key`, so the identity was stored as an ordinary
column and Splunk generated a random `_key` for each record.

Write `_key` in the baselines and match on `_key` in the detections that
read the lookups. Bump version and modification_date on all eight files.

Co-Authored-By: Claude <noreply@anthropic.com>
@pyth0n1c

Copy link
Copy Markdown
Collaborator Author

Separate issue found while making this change (not fixed here, for discussion)

In baselines/baseline_of_kubernetes_process_resource_ratio.yml the mem:disk ratio divides memory by itself:

eval mem:disk = 'process.memory.utilization'/'process.memory.utilization'

The matching detection, kubernetes_process_with_resource_ratio_anomalies, computes it as memory over disk:

eval mem:disk = 'process.memory.utilization'/'process.disk.operations'

Effect: avg_mem:disk in k8s_process_resource_ratio_baseline is always 1 and stdev_mem:disk is 0 (or null). The detection compares a real memory/disk ratio against that baseline, so the mem:disk check can only fire on a deviation from 1, and it is not a meaningful anomaly signal.

Likely fix is changing the baseline's denominator to 'process.disk.operations'. I left it out of this PR because it changes what the baseline computes, and the baseline would need to be rebuilt before the detection compares against correct values.

Questions:

  • Is the self-division intentional? I couldn't find a reason for it. It looks like a typo from the change in c1b0b64.
  • Should it be fixed in this PR or a follow-up?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants