Configuring Data Collection

Table of Contents

• Overview
  • Data Collection Configuration Structure
• Configuration Procedure
  • Step 1. Configure Smart Monitor Data Collector
    • 1.1. Deploy the Pipeline Configurations
    • 1.2. Configure Keystore Keys
    • 1.3. Add the Pipelines to pipelines.yml
    • 1.4. Replace Configuration Placeholders
    • 1.5. Restart Logstash
  • Step 2. Configure Smart Beat Manager
    • 2.1. Deploy the Applications
    • 2.2. Configure Application Parameters
    • 2.3. Add Groups to serverclasses.yml
    • 2.4. Restart Smart Beat Manager
• Configuration-to-Index Mapping
• End-to-End Verification

Overview

The AI Security module provides separate data collection configurations for each source: local AI agent events and Claude Code and Codex permission configuration snapshots.

Data Collection Configuration Structure

ai_security/
├── sm_data_collector_pipelines/           # Input, filter, and output
│   ├── claude_code.conf                   # Claude Code sessions and Claude Desktop logs -> gen_ai_events
│   ├── codex.conf                         # Codex sessions -> gen_ai_events, gen_ai_cost
│   ├── permissions.conf                   # Agent permission snapshots -> gen_ai_permissions
├── opensearch/                            # gen_ai_* index templates
├── sample/                                # Input and normalized data samples
└── SmartBeatManager/
    └── apps/
        ├── filebeat_claude_code           # Filebeat: reads native Claude Code sessions and Claude Desktop logs
        ├── filebeat_codex                 # Filebeat: reads native Codex sessions
        ├── scriptbeat_claude_permissions  # Scriptbeat: captures ~/.claude/settings.json
        ├── scriptbeat_codex_permissions   # Scriptbeat: captures ~/.codex/config.toml

Configuration Procedure

Step 1. Configure Smart Monitor Data Collector

note

Smart Monitor Data Collector is based on Logstash. The name Logstash is also used throughout this document.

1.1. Deploy the Pipeline Configurations

Copy the contents of logstash_pipelines/ to /app/logstash/config/conf.d.

1.2. Configure Keystore Keys

The logstash.keystore file must contain the ES_PWD key, which stores the password of the logstash user in Smart Monitor:

sudo -u logstash /app/logstash/bin/logstash-keystore add ES_PWD

1.3. Add the Pipelines to pipelines.yml

Add the following configuration to /app/logstash/config/pipelines.yml:

### AI Security

- pipeline.id: claude_code
  path.config: "/app/logstash/config/conf.d/claude_code.conf"
  pipeline.workers: 1

- pipeline.id: codex
  path.config: "/app/logstash/config/conf.d/codex.conf"
  pipeline.workers: 1

- pipeline.id: permissions
  path.config: "/app/logstash/config/conf.d/permissions.conf"

1.4. Replace Configuration Placeholders

In the deployed *.conf files, replace the following placeholders:

• HOST:PORT in the output.opensearch block with the Smart Monitor address and port
• PORT in the input.beats block with the port used to receive data from Smart Beat

1.5. Restart Logstash

systemctl restart logstash

Step 2. Configure Smart Beat Manager

2.1. Deploy the Applications

Copy the directories from SmartBeatManager/apps/ to /app/smartBeatManager/apps, and copy the Beat binary distributions to /app/smartBeatManager/binaries.

2.2. Configure Application Parameters

In every Beat configuration, replace HOST:PORT with the Logstash address and port.

For local AI agents (filebeat_claude_code, filebeat_codex, scriptbeat_claude_permissions, and scriptbeat_codex_permissions), replace <HOME> with the user's home directory for the relevant operating system:

Operating System	Path
macOS	/Users/<user>
Linux	/home/<user>
Windows	C:\Users\<user>

2.3. Add Groups to serverclasses.yml

Add application groups for AI agent hosts and GPU hosts to /app/smartBeatManager/etc/serverclasses.yml.

2.4. Restart Smart Beat Manager

systemctl restart smartBeatManager

Configuration-to-Index Mapping

Configuration File	Target Index
claude_code.conf	gen_ai_events
codex.conf	gen_ai_events
permissions.conf	gen_ai_permissions

End-to-End Verification

After deploying all components:

1. Verify that new documents appear in gen_ai_permissions* in Smart Monitor.
2. Verify that events from the local Claude Code and Codex AI agents appear in gen_ai_events*.