AI Security: Jobs
The AI Security module includes 32 jobs that process events and detect threats in the OWASP Top 10 for LLM Applications categories.
The jobs run on a schedule. When threat indicators are detected, the relevant jobs create incidents in Incident Manager.detected.
Intended audience: information security analysts and SOC engineers
Jobs List
| Category | Job | Description |
|---|---|---|
| LLM01 | Prompt injection | Creates an incident from search results without using an external AI model (Incident example) |
| LLM01 | Prompt injection AI check | Further analyzes candidates using an external AI model and saves the analysis in the incident card |
| LLM03 | Unauthorized modification of training data | Detects training dataset files written, modified, or deleted by users who are not on the authorized users list |
| LLM03 | Unusual access to training data | Detects training dataset files read by users who are not on the authorized users list |
| LLM04 | Request context length limit exceeded | Detects requests whose input context exceeds the configured threshold (Incident example) |
| LLM04 | Generation time limit exceeded | Detects requests whose processing time exceeds the configured threshold, which may indicate a resource exhaustion attack |
| LLM04 | User token limit exceeded | Compares a user's token usage over 15 minutes with the average for the preceding 48 hours and triggers when usage exceeds the average by the configured factor |
| LLM04 | Total token limit exceeded | Monitors total token usage across all requests over 15 minutes against the configured threshold |
| LLM04 | Request rate threshold exceeded | Counts the number of requests per user per minute and compares it with the configured threshold |
| LLM05 | Unusual external agent connection | Detects outbound agent connections to destinations that are not on the allowed destinations list |
| LLM05/08 | Excessive permissions granted to agents | Detects dangerous combinations in permission configurations, including automatic approval without a denylist, overly broad command prefixes, and network access with automatic approval enabled |
| LLM05/08 | Deviation from the permission profile | Compares the current agent permission configuration with the agent_permission_baseline lookup, including approval mode, network access, sandbox mode, policy, and allowlists |
| LLM05/08 | Agent privilege escalation attempt | Detects failed agent actions and access denial messages such as permission denied, forbidden, and sandbox (Incident example) |
| LLM06 | Sensitive information disclosure | Analyzes model requests and responses for secrets, personal data, confidential business information, and web attack indicators by using a signature library (Incident example) |
| LLM06 | System prompt disclosure by the model | Detects characteristic markers of internal instructions and system prompts in model responses |
| LLM06 | Sensitive data leakage to external services | Detects secrets, personal data, and confidential information sent in requests to external providers |
| LLM07 | Dangerous command in a model response | Analyzes model responses for destructive, exfiltration, and privilege escalation commands (Incident example) |
| LLM07 | Dangerous process on an AI host | Detects processes launched with destructive, exfiltration, or malicious command lines |
| LLM07 | Dangerous action initiated by an agent | Detects a dangerous command when its parent process is an AI agent runtime |
| LLM07 | Correlation: prompt injection to action | Correlates a dangerous system action with an earlier prompt injection on the same host |
| LLM07 | Correlation: dangerous model response to action | Correlates a dangerous system action with an earlier model response containing a dangerous command on the same host |
| LLM10 | Successful model repository access by an unusual user | Detects access to model files by users who are not on the authorized users list |
| LLM10 | Multiple accesses to model files | Counts accesses to model weight files by user and host over a specified interval and triggers when the threshold is exceeded |
| LLM10 | Model artifact exfiltration by a process | Detects data transfer commands such as scp, rsync, rclone, and curl when combined with access to model weight files |
| LLM10 | Outbound transfer of model weights to an external network | Correlates large outbound connections with access to model files on the same host |
| LLM10 | API exfiltration | Compares a user's number of LLM API requests over 15 minutes with a 48-hour moving average; exceeding the average by the configured factor may indicate a model extraction attempt |
| Internal | Request preparation | Prepares events and text for semantic checks |
| Internal | Static check | Finds prompt injection signatures in requests and dangerous commands in model responses |
| Internal | Semantic check | Compares requests with reference examples of prompt injections, restriction bypasses, data disclosure, and dangerous actions |
| Internal | Merge results | Combines static and semantic matches for the same event |
| Internal | Populate the agent_permission_baseline lookup | Calculates current permission values for each agent daily and stores a baseline permission profile for the deviation detection rule |
Important
Thresholds configured in the jobs can be adjusted for the target deployment. Semantic check thresholds are configured separately for each reference example.