GCP Vertex AI Jailbreak Prompt Heuristics
Detects Vertex AI GenerateContent prompts that match common jailbreak / instruction-override heuristics (for example ignore previous instructions, DAN or developer mode, unrestricted-AI language, or explicit safety-filter bypass). Typical prompt_response_logs exports do not populate HARM_CATEGORY_JAILBREAK (unlike hate/harassment/dangerous ratings); Model Armor findings remain the preferred first-party jailbreak signal when that API is enabled.
Elastic rule (View on GitHub)
1[metadata]
2creation_date = "2026/10/02"
3integration = ["gcp_vertexai"]
4maturity = "production"
5min_stack_comments = "gcp_vertexai prompt_response_logs requires integration 1.4.0+ (Kibana ^9.2.0)."
6min_stack_version = "9.4.0"
7updated_date = "2026/11/06"
8
9[rule]
10author = ["Elastic"]
11description = """
12Detects Vertex AI GenerateContent prompts that match common jailbreak / instruction-override heuristics (for example
13ignore previous instructions, DAN or developer mode, unrestricted-AI language, or explicit safety-filter bypass).
14Typical prompt_response_logs exports do not populate HARM_CATEGORY_JAILBREAK (unlike hate/harassment/dangerous
15ratings); Model Armor findings remain the preferred first-party jailbreak signal when that API is enabled.
16"""
17false_positives = [
18 """
19 Approved red-team or evaluation prompts that deliberately include jailbreak strings. Exclude known test principals
20 or lower severity for those models.
21 """,
22]
23from = "now-60m"
24interval = "10m"
25language = "esql"
26license = "Elastic License v2"
27name = "GCP Vertex AI Jailbreak Prompt Heuristics"
28note = """## Triage and analysis
29
30### Investigating GCP Vertex AI Jailbreak Prompt Heuristics
31
32The user prompt matched common jailbreak / instruction-override language (for example "ignore
33previous instructions", DAN/"Do Anything Now", or "bypass the safety filter"). This is a content
34heuristic on prompt text — not a Model Armor finding or a Gemini `HARM_CATEGORY_JAILBREAK`
35safety rating (that category is often absent from prompt_response_logs). Read the response to
36see whether the model refused or complied.
37
38#### Possible investigation steps
39
40- Read `full_request.contents.parts.text` and `full_response.candidates.content.parts.text`. Confirm
41 intent is instruction override or filter bypass versus an innocent mention of the word
42 "jailbreak".
43- Note `finish_reason`: `STOP` with a helpful reply after a jailbreak attempt is higher priority
44 than a clear refusal; `SAFETY` means the provider blocked generation.
45- Correlate with `logs-gcp_vertexai.auditlogs-*` GenerateContent in the same window for `source.ip`
46 and `client.user.email`.
47- Check for related alerts: repeated refusals, elevated safety ratings, credentials in prompt, or
48 BLOCK_NONE from the same caller/model.
49
50### False positive analysis
51
52- Approved red-team, evaluation, or Model Armor test suites that deliberately include jailbreak
53 strings. Exclude known test principals or models.
54- Documentation that mentions "developer mode" or "without restrictions" in a non-adversarial
55 context. Read the full prompt before escalating.
56
57### Response and remediation
58
59- If not an approved test: investigate the calling principal/IP, restrict or rotate credentials, and
60 add application-side input filtering.
61- Prefer enabling Model Armor / AI Protection for a first-party jailbreak or prompt-injection signal
62 when available.
63"""
64references = [
65 "https://www.elastic.co/docs/reference/integrations/gcp_vertexai",
66 "https://cloud.google.com/blog/products/identity-security/introducing-ai-protection-security-for-the-ai-era",
67]
68risk_score = 47
69rule_id = "42cc9ad9-4ad1-4702-8abc-7d55096f6ee0"
70setup = """## Setup
71
72Requires GCP Vertex AI `prompt_response_logs`. For a higher-fidelity jailbreak signal, enable Model Armor / AI Protection
73and ingest those findings separately when available.
74"""
75severity = "medium"
76tags = [
77 "Domain: GenAI",
78 "Domain: Cloud",
79 "Data Source: GCP Vertex AI",
80 "Data Source: GCP",
81 "Data Source: Google Cloud Platform",
82 "Platform: GCP",
83 "Service: GCP Vertex AI",
84 "Use Case: Threat Detection",
85 "Tactic: Defense Evasion",
86 "Threat: Unauthorized AI Usage",
87 "Mitre Atlas: AML.T0051",
88 "Mitre Atlas: AML.T0054",
89 "Resources: Investigation Guide",
90 "Rule Type: ES|QL",
91]
92timestamp_override = "event.ingested"
93type = "esql"
94
95query = '''
96from logs-gcp_vertexai.prompt_response_logs-* metadata _id, _version, _index
97| where data_stream.dataset == "gcp_vertexai.prompt_response_logs"
98| eval request_text = to_lower(coalesce(mv_concat(gcp.vertexai.prompt_response_logs.full_request.contents.parts.text, " "), ""))
99| where
100 request_text rlike ".*ignore (all |your )?(previous|prior) instructions.*" or
101 request_text rlike ".*disregard (your |all |the )?(system|developer|prior).*" or
102 request_text rlike ".*(dan|stan|aim|dude) mode.*" or
103 request_text rlike ".*do anything now.*" or
104 request_text like "*bypass*safety*" or
105 request_text like "*bypass*content*filter*" or
106 request_text like "*bypass*content*polic*" or
107 request_text like "*no safety filter*" or
108 request_text rlike ".*without (any |your |content |safety )?restrictions?.*" or
109 request_text like "*no content restriction*" or
110 request_text like "*unrestricted ai*" or
111 request_text like "*jailbreak*" or
112 request_text like "*forget everything you were told*" or
113 request_text like "*you are dan*" or
114 request_text like "*developer mode*" or
115 request_text like "*override your safety*" or
116 request_text like "*hidden system prompt*" or
117 request_text like "*ignore the content policy*" or
118 request_text like "*ignore all previous*" or
119 request_text like "*ignore prior instructions*" or
120 request_text like "*ignore your previous*"
121| keep
122 _id,
123 _version,
124 _index,
125 @timestamp,
126 cloud.project.id,
127 gcp.vertexai.prompt_response_logs.model,
128 gcp.vertexai.prompt_response_logs.api_method,
129 gcp.vertexai.prompt_response_logs.full_request.contents.parts.text,
130 gcp.vertexai.prompt_response_logs.full_response.candidates.content.parts.text,
131 gcp.vertexai.prompt_response_logs.full_response.candidates.finish_reason
132'''
133
134
135[[rule.threat]]
136framework = "MITRE ATT&CK"
137[[rule.threat.technique]]
138id = "T1562"
139name = "Impair Defenses"
140reference = "https://attack.mitre.org/techniques/T1562/"
141
142
143[rule.threat.tactic]
144id = "TA0005"
145name = "Defense Evasion"
146reference = "https://attack.mitre.org/tactics/TA0005/"
147[[rule.threat_mappings]]
148framework = "MITRE ATLAS"
149version = "2026.08"
150[[rule.threat_mappings.threat]]
151framework = "MITRE ATLAS"
152[[rule.threat_mappings.threat.technique]]
153id = "AML.T0051"
154name = "LLM Prompt Injection"
155reference = "https://atlas.mitre.org/techniques/AML.T0051/"
156
157
158[rule.threat_mappings.threat.tactic]
159id = "AML.TA0005"
160name = "Execution"
161reference = "https://atlas.mitre.org/tactics/AML.TA0005/"
162[[rule.threat_mappings.threat]]
163framework = "MITRE ATLAS"
164[[rule.threat_mappings.threat.technique]]
165id = "AML.T0054"
166name = "LLM Jailbreak"
167reference = "https://atlas.mitre.org/techniques/AML.T0054/"
168
169
170[rule.threat_mappings.threat.tactic]
171id = "AML.TA0007"
172name = "Defense Evasion"
173reference = "https://atlas.mitre.org/tactics/AML.TA0007/"
174
175[rule.investigation_fields]
176field_names = [
177 "cloud.project.id",
178 "gcp.vertexai.prompt_response_logs.model",
179 "gcp.vertexai.prompt_response_logs.api_method",
180 "gcp.vertexai.prompt_response_logs.full_request.contents.parts.text",
181 "gcp.vertexai.prompt_response_logs.full_response.candidates.content.parts.text",
182 "gcp.vertexai.prompt_response_logs.full_response.candidates.finish_reason",
183]
Triage and analysis
Investigating GCP Vertex AI Jailbreak Prompt Heuristics
The user prompt matched common jailbreak / instruction-override language (for example "ignore
previous instructions", DAN/"Do Anything Now", or "bypass the safety filter"). This is a content
heuristic on prompt text — not a Model Armor finding or a Gemini HARM_CATEGORY_JAILBREAK
safety rating (that category is often absent from prompt_response_logs). Read the response to
see whether the model refused or complied.
Possible investigation steps
- Read
full_request.contents.parts.textandfull_response.candidates.content.parts.text. Confirm intent is instruction override or filter bypass versus an innocent mention of the word "jailbreak". - Note
finish_reason:STOPwith a helpful reply after a jailbreak attempt is higher priority than a clear refusal;SAFETYmeans the provider blocked generation. - Correlate with
logs-gcp_vertexai.auditlogs-*GenerateContent in the same window forsource.ipandclient.user.email. - Check for related alerts: repeated refusals, elevated safety ratings, credentials in prompt, or BLOCK_NONE from the same caller/model.
False positive analysis
- Approved red-team, evaluation, or Model Armor test suites that deliberately include jailbreak strings. Exclude known test principals or models.
- Documentation that mentions "developer mode" or "without restrictions" in a non-adversarial context. Read the full prompt before escalating.
Response and remediation
- If not an approved test: investigate the calling principal/IP, restrict or rotate credentials, and add application-side input filtering.
- Prefer enabling Model Armor / AI Protection for a first-party jailbreak or prompt-injection signal when available.
References
Related rules
- GCP Vertex AI Elevated Safety Rating on Response
- GCP Vertex AI Repeated Assistant Refusals
- GCP Vertex AI Response Blocked by Safety Filters
- GCP Vertex AI Safety Filters Set to BLOCK_NONE
- GCP Vertex AI Publisher Model Config Modified