Kubernetes Unusual Decision by User Agent

 1[metadata]
 2creation_date = "2025/06/18"
 3integration = ["kubernetes"]
 4maturity = "production"
 5updated_date = "2025/07/21"
 6
 7[rule]
 8author = ["Elastic"]
 9description = """
10This rule detects unusual request responses in Kubernetes audit logs through the use of the
11"new_terms" rule type. In production environments, default API requests are typically made by
12system components or trusted users, who are expected to have a consistent user agent and
13allowed response annotations. By monitoring for anomalies in the username and response
14annotations, this rule helps identify potential unauthorized access or misconfigurations
15in the Kubernetes environment.
16"""
17index = ["logs-kubernetes.audit_logs-*"]
18language = "kuery"
19license = "Elastic License v2"
20name = "Kubernetes Unusual Decision by User Agent"
21note = """## Triage and analysis
22
23> **Disclaimer**:
24> This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.
25
26### Investigating Kubernetes Unusual Decision by User Agent
27
28Kubernetes orchestrates containerized applications, relying on API requests for operations. Typically, these requests originate from system components or trusted users with consistent user agents. Adversaries might exploit this by using atypical user agents to mask unauthorized access or misconfigurations. The detection rule identifies anomalies in user agents and response annotations, signaling potential threats in the Kubernetes environment.
29
30### Possible investigation steps
31
32- Review the Kubernetes audit logs for entries where the user_agent.original field is present to identify any unusual or unexpected user agents.
33- Cross-reference the identified user agents with known system components and trusted users to determine if the user agent is legitimate or potentially malicious.
34- Examine the kubernetes.audit.stage field for "ResponseComplete" entries to understand the context and outcome of the requests associated with the unusual user agent.
35- Investigate the source IP addresses and associated usernames in the audit logs to identify any patterns or anomalies that could indicate unauthorized access.
36- Check for any recent changes or deployments in the Kubernetes environment that might explain the presence of unusual user agents or unexpected behavior.
37- Assess the risk and impact of the detected anomaly by considering the sensitivity of the accessed resources and the permissions associated with the user account involved.
38
39### False positive analysis
40
41- System components or trusted users with legitimate but infrequent user agents may trigger the rule. To manage this, identify these user agents and add them to an exception list to prevent unnecessary alerts.
42- Automated scripts or tools used for maintenance or monitoring might use unique user agents. Regularly review these tools and update the exception list to include their user agents if they are verified as non-threatening.
43- New deployments or updates to Kubernetes components can introduce new user agents temporarily. Monitor these changes and adjust the rule exceptions accordingly to accommodate expected behavior during these periods.
44- Third-party integrations or plugins may use distinct user agents. Validate these integrations and, if deemed safe, include their user agents in the exception list to reduce false positives.
45
46### Response and remediation
47
48- Immediately isolate the affected Kubernetes node or cluster to prevent further unauthorized access or potential lateral movement by the adversary.
49- Review and terminate any suspicious or unauthorized sessions identified in the audit logs to cut off any active malicious activity.
50- Revoke and rotate credentials associated with the compromised user agent to prevent further unauthorized access using the same credentials.
51- Conduct a thorough review of the affected system's configurations and permissions to identify and rectify any misconfigurations or overly permissive access controls.
52- Implement additional monitoring and logging for the affected systems to detect any further anomalies or unauthorized activities promptly.
53- Escalate the incident to the security operations team for a comprehensive investigation and to determine if any data exfiltration or further compromise has occurred.
54- Update and enhance detection rules and alerts to better identify similar anomalies in user agents and response annotations in the future, ensuring quicker response times.
55"""
56risk_score = 21
57rule_id = "8a1db198-da6f-4500-b985-7fe2457300af"
58severity = "low"
59tags = [
60    "Domain: Kubernetes",
61    "Domain: Container",
62    "Use Case: Threat Detection",
63    "Data Source: Kubernetes",
64    "Tactic: Execution",
65    "Resources: Investigation Guide",
66]
67timestamp_override = "event.ingested"
68type = "new_terms"
69query = '''
70host.os.type:"linux" and event.dataset:"kubernetes.audit_logs" and kubernetes.audit.stage:"ResponseComplete" and user_agent.original:*
71'''
72
73[[rule.threat]]
74framework = "MITRE ATT&CK"
75
76[rule.threat.tactic]
77id = "TA0002"
78name = "Execution"
79reference = "https://attack.mitre.org/tactics/TA0002/"
80
81[rule.new_terms]
82field = "new_terms_fields"
83value = ["kubernetes.audit.annotations.authorization_k8s_io/decision", "kubernetes.audit.user.username", "user_agent.original"]
84
85[[rule.new_terms.history_window_start]]
86field = "history_window_start"
87value = "now-10d"