AWS Bedrock Detected Multiple Validation Exception Errors by a Single User
Identifies multiple validation exeception errors within AWS Bedrock. Validation errors occur when you run the InvokeModel or InvokeModelWithResponseStream APIs on a foundation model that uses an incorrect inference parameter or corresponding value. These errors also occur when you use an inference parameter for one model with a model that doesn't have the same API parameter. This could indicate attempts to bypass limitations of other approved models, or to force an impact on the environment by incurring exhorbitant costs.
Elastic rule (View on GitHub)
1[metadata]
2creation_date = "2024/09/11"
3integration = ["aws_bedrock"]
4maturity = "production"
5updated_date = "2025/01/17"
6min_stack_comments = "ES|QL rule type is still in technical preview as of 8.13, however this rule was tested successfully; integration in tech preview"
7min_stack_version = "8.13.0"
8
9[rule]
10author = ["Elastic"]
11description = """
12Identifies multiple validation exeception errors within AWS Bedrock. Validation errors occur when you run the InvokeModel or
13InvokeModelWithResponseStream APIs on a foundation model that uses an incorrect inference parameter or corresponding value.
14These errors also occur when you use an inference parameter for one model with a model that doesn't have the same API parameter.
15This could indicate attempts to bypass limitations of other approved models, or to force an impact on the environment by incurring
16exhorbitant costs.
17"""
18false_positives = ["Legitimate misunderstanding by users on accessing the bedrock models."]
19from = "now-60m"
20interval = "10m"
21language = "esql"
22license = "Elastic License v2"
23name = "AWS Bedrock Detected Multiple Validation Exception Errors by a Single User"
24references = [
25 "https://atlas.mitre.org/techniques/AML.T0015",
26 "https://atlas.mitre.org/techniques/AML.T0034",
27 "https://atlas.mitre.org/techniques/AML.T0046",
28 "https://www.elastic.co/security-labs/elastic-advances-llm-security"
29]
30risk_score = 73
31rule_id = "725a048a-88c5-4fc7-8677-a44fc0031822"
32note = """## Triage and analysis
33
34### Investigating AWS Bedrock Detected Multiple Validation Exception Errors by a Single User
35
36Amazon Bedrock is AWS’s managed service that enables developers to build and scale generative AI applications using large foundation models (FMs) from top providers.
37
38Bedrock offers a variety of pretrained models from Amazon (such as the Titan series), as well as models from providers like Anthropic, Meta, Cohere, and AI21 Labs.
39
40#### Possible investigation steps
41
42- Identify the user account that caused validation errors in accessing the Amazon Bedrock models.
43- Investigate other alerts associated with the user account during the past 48 hours.
44- Consider the time of day. If the user is a human (not a program or script), did the activity take place during a normal time of day?
45- Examine the account's attempts to access Amazon Bedrock models in the last 24 hours.
46- If you suspect the account has been compromised, scope potentially compromised assets by tracking Amazon Bedrock model access, prompts generated, and responses to the prompts by the account in the last 24 hours.
47
48### False positive analysis
49
50- Verify the user account that that caused validation errors is a legitimate misunderstanding by users on accessing the bedrock models.
51
52### Response and remediation
53
54- Initiate the incident response process based on the outcome of the triage.
55- Disable or limit the account during the investigation and response.
56- Identify the possible impact of the incident and prioritize accordingly; the following actions can help you gain context:
57 - Identify the account role in the cloud environment.
58 - Identify if the attacker is moving laterally and compromising other Amazon Bedrock Services.
59 - Identify any regulatory or legal ramifications related to this activity.
60 - Identify if any implication to resource billing.
61- Review the permissions assigned to the implicated user group or role behind these requests to ensure they are authorized and expected to access bedrock and ensure that the least privilege principle is being followed.
62- Determine the initial vector abused by the attacker and take action to prevent reinfection via the same vector.
63- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).
64"""
65setup = """## Setup
66
67This rule requires that AWS Bedrock Integration be configured. For more information, see the AWS Bedrock integration documentation:
68
69https://www.elastic.co/docs/current/integrations/aws_bedrock
70"""
71severity = "high"
72tags = [
73 "Domain: LLM",
74 "Data Source: AWS",
75 "Data Source: AWS Bedrock",
76 "Data Source: AWS S3",
77 "Use Case: Policy Violation",
78 "Mitre Atlas: T0015",
79 "Mitre Atlas: T0034",
80 "Mitre Atlas: T0046",
81 "Resources: Investigation Guide",
82]
83timestamp_override = "event.ingested"
84type = "esql"
85
86query = '''
87from logs-aws_bedrock.invocation-*
88// truncate the timestamp to a 1-minute window
89| eval target_time_window = DATE_TRUNC(1 minutes, @timestamp)
90| where gen_ai.response.error_code == "ValidationException"
91| keep user.id, gen_ai.request.model.id, cloud.account.id, gen_ai.response.error_code, target_time_window
92// count the number of users causing validation errors within a 1 minute window
93| stats total_denials = count(*) by target_time_window, user.id, cloud.account.id
94| where total_denials > 3
95'''
96
97[rule.investigation_fields]
98field_names = [
99 "target_time_window",
100 "user.id",
101 "cloud.account.id",
102 "total_denials"
103
104]
Triage and analysis
Investigating AWS Bedrock Detected Multiple Validation Exception Errors by a Single User
Amazon Bedrock is AWS’s managed service that enables developers to build and scale generative AI applications using large foundation models (FMs) from top providers.
Bedrock offers a variety of pretrained models from Amazon (such as the Titan series), as well as models from providers like Anthropic, Meta, Cohere, and AI21 Labs.
Possible investigation steps
- Identify the user account that caused validation errors in accessing the Amazon Bedrock models.
- Investigate other alerts associated with the user account during the past 48 hours.
- Consider the time of day. If the user is a human (not a program or script), did the activity take place during a normal time of day?
- Examine the account's attempts to access Amazon Bedrock models in the last 24 hours.
- If you suspect the account has been compromised, scope potentially compromised assets by tracking Amazon Bedrock model access, prompts generated, and responses to the prompts by the account in the last 24 hours.
False positive analysis
- Verify the user account that that caused validation errors is a legitimate misunderstanding by users on accessing the bedrock models.
Response and remediation
- Initiate the incident response process based on the outcome of the triage.
- Disable or limit the account during the investigation and response.
- Identify the possible impact of the incident and prioritize accordingly; the following actions can help you gain context:
- Identify the account role in the cloud environment.
- Identify if the attacker is moving laterally and compromising other Amazon Bedrock Services.
- Identify any regulatory or legal ramifications related to this activity.
- Identify if any implication to resource billing.
- Review the permissions assigned to the implicated user group or role behind these requests to ensure they are authorized and expected to access bedrock and ensure that the least privilege principle is being followed.
- Determine the initial vector abused by the attacker and take action to prevent reinfection via the same vector.
- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).
References
Related rules
- AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User
- AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request
- AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session
- AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session
- Unusual High Confidence Content Filter Blocks Detected