Azure OpenAI Insecure Output Handling

 1[metadata]
 2creation_date = "2025/02/25"
 3maturity = "production"
 4updated_date = "2025/04/07"
 5
 6[rule]
 7author = ["Elastic"]
 8description = """
 9Detects when Azure OpenAI requests result in zero response length, potentially indicating issues in output handling that
10might lead to security exploits such as data leaks or code execution. This can occur in cases where the API fails to
11handle outputs correctly under certain input conditions.
12"""
13false_positives = ["Queries that are designed to expect empty responses or benign system errors"]
14from = "now-60m"
15interval = "10m"
16language = "esql"
17license = "Elastic License v2"
18name = "Azure OpenAI Insecure Output Handling"
19note = """## Triage and analysis
20
21> **Disclaimer**:
22> This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.
23
24### Investigating Azure OpenAI Insecure Output Handling
25
26Azure OpenAI integrates AI capabilities into applications, enabling natural language processing tasks. However, improper output handling can lead to vulnerabilities, such as data leaks or unauthorized code execution. Adversaries might exploit these by crafting inputs that cause the API to mishandle responses. The detection rule identifies anomalies by flagging instances where API responses are unexpectedly empty, suggesting potential misuse or misconfiguration, especially when such events occur frequently.
27
28### Possible investigation steps
29
30- Review the logs for the specific Azure resource name flagged in the alert to understand the context and frequency of zero-length responses.
31- Examine the request lengths associated with the zero-length responses to identify any patterns or anomalies in the input data that might be causing the issue.
32- Check the cloud account ID associated with the alert to determine if there are any known issues or recent changes in configuration that could affect output handling.
33- Investigate the operation name "ChatCompletions_Create" to ensure that the API is being used as intended and that there are no unauthorized or unexpected uses.
34- Assess the overall environment for any recent updates or changes in the Azure OpenAI configuration that might have impacted output handling.
35
36### False positive analysis
37
38- Frequent legitimate requests with zero response length can occur during testing or development phases. To manage this, exclude known test environments or accounts from the detection rule by adding exceptions for specific cloud.account.id or azure.resource.name values.
39- Some applications may intentionally send requests that do not require a response, resulting in zero response length. Identify these applications and adjust the rule to exclude their specific azure.resource.name.
40- Network issues or temporary service disruptions can lead to zero-length responses. Monitor for patterns of such occurrences and consider excluding specific time frames or network segments if they are known to cause false positives.
41- Automated scripts or bots that interact with the API might generate zero-length responses as part of their normal operation. Identify these scripts and exclude their associated identifiers from the rule to prevent false alerts.
42
43### Response and remediation
44
45- Immediately isolate the affected Azure OpenAI resource to prevent further exploitation. This can be done by temporarily disabling the API or restricting access to it.
46- Review and validate the input handling mechanisms of the affected API to ensure they are robust against malformed or malicious inputs that could lead to insecure output handling.
47- Conduct a thorough audit of recent API requests and responses to identify any unauthorized access or data leaks. Pay special attention to requests with zero response length.
48- Implement additional logging and monitoring for the affected API to capture detailed information about requests and responses, which can help in identifying patterns or repeated attempts of exploitation.
49- Notify the security team and relevant stakeholders about the incident, providing them with detailed findings and any potential impact on data security.
50- If unauthorized access or data leakage is confirmed, follow the organization's incident response plan to notify affected parties and comply with any regulatory requirements.
51- Enhance detection capabilities by integrating anomaly detection tools that can identify unusual patterns in API usage, such as frequent zero-length responses, to prevent similar threats in the future.
52"""
53references = ["https://genai.owasp.org/llmrisk/llm02-insecure-output-handling"]
54risk_score = 21
55rule_id = "fb16f9ef-cb03-4234-adc2-44641f3b71ee"
56setup = """## Setup
57
58For more information on streaming events, see the Azure OpenAI documentation:
59
60https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/stream-monitoring-data-event-hubs
61"""
62severity = "low"
63tags = [
64    "Domain: LLM",
65    "Data Source: Azure OpenAI",
66    "Data Source: Azure Event Hubs",
67    "Use Case: Insecure Output Handling",
68    "Resources: Investigation Guide",
69]
70timestamp_override = "event.ingested"
71type = "esql"
72
73query = '''
74from logs-azure_openai.logs-*
75| where azure.open_ai.properties.response_length == 0 and azure.open_ai.result_signature == "200" and azure.open_ai.operation_name == "ChatCompletions_Create"
76| keep azure.open_ai.properties.request_length, azure.open_ai.result_signature, cloud.account.id, azure.resource.name
77| stats count = count() by azure.resource.name
78| where count >= 10
79| sort count desc
80'''