Installing SAF Beat Manager

Please note!

Starting from version 4.0.0, the interaction scheme between SAF Beat and SAF Beat Manager has changed, but SAF Beat Manager maintains backward compatibility with legacy agents.

Starting from version 4.2.0, the interaction API has been updated without backward compatibility. The SAF version must be at least 4.2.0.

Starting from version 4.3.0, the clients.yml snapshot file is deprecated. Data storage has been migrated to an embedded database. During the first launch of SAF Beat Manager, data will be migrated from the clients.yml file, after which the file will be renamed to clients_old.yml.

Please note!

SAF Beat can work with any version of ElasticBeat, but if you plan to send these examples to OpenSearch, then the ElasticBeat version should be 7.12.1 (or 7.10.2 is a more stable version). If you plan to use Logstash, then the ElasticBeat version does not matter. Check on the official website.

Symbols

• ${SBM_HOME} is the installation path of SAF Beat Manager, usually it is /app/SAFBeatManager/

Installing SAF Beat Manager

Preparing to install SAF Beat Manager

SAF Beat Manager is available as part of the SAF installation archive in the directory ${SAF_INSTALLER}/utils/SAFBeatManager.

Please note!

It is recommended to install under the root user. For a user with limited rights, you can make a preset.

SAF Beat Manager Service Installation

SAF Beat Manager supports the following service management commands:

• install – installs the service
• install -d <string> | --directory <string> - installs the service in a specified directory. Default: SAF Beat Manager executable's directory
• install -u <string> | --user <string> - installs the service under specified user. Default: SAFBeatManager
• install -g <string> | --group <string> - installs the service under specified group. Default: SAFBeatManager
• install --ignore-selinux - installs the service without setting executable file permissions (if SELinux is present)
• install --ignore-systemd - installs the service without creating SAFBeatManager.service systemd file
• remove – removes the service (in older versions: delete)
• version – displays SAF Beat Manager version
• help – shows help for any command

Please note!

For the install command:

• If there was no user or group before, they will be created automatically. Including the default
• If you specify an existing user and a group, but the user is not a member of this group, an error will occur and the installation will stop.

The installer is only compatible with the systemd service initialization and management subsystem.

The command to install the service in systemd may look like this:

mkdir -p /app/SAFBeatManager/
cp ${SAF_INSTALLER}/utils/SAFBeatManager/SAFBeatManager-* /app/SAFBeatManager/SAFBeatManager
cd /app/SAFBeatManager/
./SAFBeatManager install

When using a non-standard directory (e.g., $DIR), the installation command becomes:

cd /app/SAFBeatManager/
./SAFBeatManager install --directory $DIR

When using a non-standard user (e.g., $USER), the installation command becomes:

cd /app/SAFBeatManager
./SAFBeatManager install --user $USER

If SELinux is used, the rights to the executable file are set automatically when installing the service.

If you don't have enough rights to change the SELinux settings, you can set the --ignore-selinux flag when installing the service and run the command yourself:

sudo chcon -Rv -u system_u -r object_r -t bin_t /app/SAFBeatManager/SAFBeatManager-*

Please note!

The version of SAF Beat Manager may differ, and the file may be named differently. Also, pay attention to the owner of the group after changes in the working directory, since the service runs under the specified user during installation.

If you lack sufficient permissions to configure the systemd service, you can use the --ignore-systemd flag during service installation. After installation, manually create the service file:

sudo touch /etc/systemd/system/SAFBeatManager.service

Modify the service configuration according to your requirements and save it to the created file:

[Unit]
Description=SAFBeatManager

[Service]
User=<SAFBeatManager_USER>
Group=<SAFBeatManager_GROUP>
WorkingDirectory=<SAFBeatManager_HOME>
ExecStart=<FULL_PATH_TO_EXECUTABLE_FILE_SAFBeatManager>
Restart=always

[Install]
WantedBy=multi-user.target

After creating the service, reload systemd configuration and enable autostart on server reboot:

systemctl daemon-reload
systemctl enable SAFBeatManager

Upon completing the installation, all necessary directories and SAF Beat configuration files will be created, allowing service configuration before its first start.

Remaining directories and files will be created during the service's first launch.

Example of starting a systemd-installed SAF Beat Manager service:

systemctl start SAFBeatManager.service

To verify the service status, execute:

systemctl status SAFBeatManager.service

Setting Up SAF Beat Manager

SAF Beat Manager Configuration File

Please note!

In the version of SAF Beat Manager 4.2.0, the name of the configuration file SAFBeatManager.yaml was changed to config.yaml. Both files are supported, but config.yaml will be considered a priority.

In SAF Beat Manager version 4.3.0:

• The updateSnapshot parameter is deprecated and will be ignored during operation

An example of the standard configuration of the configuration file config.yaml:

authorization:
  opensearch:
    ca_cert: ca-cert.pem
    host: https://127.0.0.1:9200
basic:
  appPath: ./apps
  binariesPath: ./binaries
  appsUnpackPath: ./config/apps
  binariesUnpackPath: ./bin
  portForSAFBeat: '7767'
  portForVisual: '7769'
  bindAddress: 127.0.0.1
  enableSnapshot: false
  enableDeletion: false
  timeoutDeletion: 24h
  disableGroupsCache: false
  badClientConnHours: 1h
logs:
  logPath: ./logs
  logMaxAge: 7d
  logRotate: 24h
ssl:
  enabled: true
  smartBeatTLSLevel: 0
  certPath: cert.pem
  keyPath: key.pem

The parameters are described below:

Parameter	Description	Default Value
authorization.opensearch.ca_cert	CA certificate in X509 format, used to connect to OpenSearch.	ca-cert.pem
authorization.opensearch.host	The address of one of the nodes of the OpenSearch server.	https://127.0.0.1:9200
basic.appPath	Directory for searching for application configurations (Elastic Beats configurations).	./apps
basic.binariesPath	Elastic Beats archive search directory.	./binaries
basic.appsUnpackPath	Directory for unpacking application archives on SAF Beat agents.	./config/apps
basic.binariesUnpackPath	Directory for unpacking Elastic Beats archives on SAF Beat agents.	./bin
basic.portForSAFBeat	RestAPI port for connecting SAF Beat agents.	7767
basic.portForVisual	RestAPI port for connecting SAF Web.	7769
basic.bindAddress	At which address to accept connections.	127.0.0.1
basic.enableSnapshot	Enabling snapshots to save a list of connected agents. During operation, the load on the hard drive increases.	false
basic.enableDeletion	Whether to delete old agent entries in the snapshot.	false
basic.timeoutDeletion	The lifetime of the agent record in the snapshot.	24h
basic.disableGroupsCache	Disabling the cache of application groups.	false
basic.badClientConnHours	The time after which it is assumed that there is no connection with the client.	1h
basic.disableClientBatch	Disabling batch data insertion by clients.	false
basic.clientBatchConfig.maxClients	The number of clients after which the recording takes place.	50
basic.clientBatchConfig.timeLimit	The time limit after which clients are recorded.	10s
logs.logPath	Logs directory.	./logs
logs.logMaxAge	Log storage time.	7d
logs.logRotate	The frequency of log rotation.	24h
ssl.enabled	Enables SSL.	true
ssl.SAFBeatTLSLevel	Certificate verification level: 0 – client certificate is not requested and is not required, 1 – agent certificate will be requested, but it is not required and will not be verified, 2 – agent certificate is required, but any valid agent certificate is acceptable. It will not be verified by the CA certificate, 3 – the agent certificate will not be requested, but if it is present, it will be verified by the CA certificate, 4 – an agent certificate is required, which is verified for compliance with the certificate.	0
ssl.certPath	Certificate file in x509 SAF Beat Manager format.	cert.pem
ssl.keyPath	Private key file in x509 SAF Beat Manager format.	key.pem

note

The ssl.SAFBeatTLSLevel parameter supports the following values:

• 0 – the client's certificate is not requested and is not required
• 1 – an agent certificate will be requested, but it is not required and will not be verified
• 2 – an agent certificate is required, but any valid agent certificate is acceptable. It will not be verified by the CA certificate
• 3 – the agent's certificate will not be requested, but if it is present, it will be verified against the CA certificate
• 4- an agent certificate is required, which is verified for compliance with the certificate

Install your SAF Beat Manager Certificates

Please note!

At the first launch, SAF Beat Manager generates its self-signed certificate and private key.

If you need to use ready-made certificates, then copy the CA certificate, the certificate and the SAF Beat Manager private key to the directory ${SBM_HOME}/etc/ssl/. You also need to configure the following settings:

• authorization.opensearch.ca_cert - path to the CA certificate or certificate chain in x509 format
• authorization.opensearch.host - the address of the OpenSearch server should start with https
• ssl.enabled - must be set to true
• ssl.smartBeatTLSLevel - select the required certificate verification mode
• ssl.CertPath - path to the SAF Beat Manager certificate file in x509 format
• ssl.keyPath - path to the SAF Beat Manager private key file in x509 format

It is recommended to generate certificates with the following parameters:

• keyUsage = critical, digitalSignature, keyEncipherment
• extendedKeyUsage = serverAuth, clientAuth
• Subject Alternative Name - specify all possible IP and domain names of the server where the agent is installed

Please note!

SAF Beat Manager must have access to the specified files and directories

Configuring log collection configurations for SAF Beat agents

To send data collection configuration files to agents, you need to configure the file $SBM_HOME/etc/serverclasses.yml, which can contain several groups of settings for agents. The available group parameters are listed below:

Parameter	Description
name	The name of the group. Required parameter.
filters	A list of filters for connected agents, the values of the fields ip, DNS name, hostname and guid of the agent can be used. You can use masks in the list. A logical OR is used between the conditions, i.e. it is sufficient to meet any of the listed conditions. Required parameter.
apps	List of configuration names (directory names) of ElasticBeats. Required parameter.
binaries	List of ElasticBeats archives that are required to perform the configurations listed in apps. You cannot specify multiple versions of the same ElasticBeat in the same group. Required parameter.
systems	A list of filters by the server platform on which the SAF Beat agent is running. The logical OR is used in the list, i.e. the agent falls into at least one of the filters. It is used as an additional filter with the filters field with the logical operation And.
excludes	List of agent filters to exclude from the group by ip, DNS name, hostname, guid. You can use masks in the list. A logical OR is used between the conditions, i.e. it is sufficient to meet any of the listed conditions. It is used as an additional filter with the filters field with the logical operation AND-NOT.

Please note!

In version 4.3.0 of SAF Beat Manager, for applications created through the SAF Beat management interface, a file with information about the application properties, including the type of ElasticBeat agent, will be stored.

For older applications created before version 4.3.0 that do not have a file with information about the application properties, backward compatibility has been preserved, that is, the name template will match <elasticbeat_name>_<apps_name>, where <elasticbeat_name> is the name of ElasticBeat in lowercase, <apps_name> is the name of the application.

Please note!

The DNS name is taken from the SAF Beat agent, for correct determination there must be a PTR record of the reverse DNS zone on the DNS server.

The following characters can be used as a mask in filters:

• * - asterisk sign, replaces any character with a length of 0 (missing character) to infinity
• ? - replaces any 1 character

Example of setting up server classes.yml:

groups:
  - name: linux_test
    filters:
      - 192.168.1.?/24
      - server-test-linux.localhost
    apps:
      - filebeat_nginx
      - auditbeat_info_system
      - execbeat_script_python
    binaries:
      - filebeat-7.10.1-linux-x86_64.tar.gz
      - auditbeat-8.9.0-linux-x86_64.tar.gz
      - execbeat-3.3.0-linux-x86_64.tar.gz
  - name: windows_test
    filters:
      - 10.10.*.*
      - 78d17a171-a3c4-4e27-9c20-bc7005db068e
      - *our-site.org*
    apps:
      - filebeat_logs_system
      - execbeat_xspider
    binaries:
      - filebeat-8.9.2-windows-x86_64.zip
      - execbeat-3.3.0-windows-x86_64.zip

After configuring the file $SBM_HOME/etc/serverclasses.yml, you need to download the binaries archives from the official website (there should be archives with a pre-compiled ElasticBeats executable file) of oss versions and place them in the directory $SBM_HOME/binaries/.

Please note!

It is recommended to use ElasticBeats versions of the same version that has Logstash installed. SAF version 4.2.0 uses Logstash version 8.9.0, so ElasticBeats is better to use versions 8.9.x.

Then you need to create the applications themselves (directories) as $SBM_HOME/etc/serverclasses.yml was specified in the $SBM_HOME/apps/ directory. Inside each application directory, all files and subdirectories will be sent to the agent. The name of the ElasticBeats configuration file must match the name used for the ElasticBeats application.

In the example below, there are applications execbeat_xspider, filebeat_nginx, filebeat_logs_system and others.

The execbeat_xspider application uses ElasticBeat - ExecBeat, the application name begins with execbeat\_, and the configuration file execbeat.ymlis located inside the directory. In this application, in addition to the configuration file forExecBeat, there are additional files, they will also be copied to SAF Beat agents.

The auditbeat_info_system application uses AuditBeat, so the name starts with auditbeat_, and the configuration file is called auditbeat.yml.

The contents of the ElasticBeats configuration files are filled in according to the official documentation of the corresponding ElasticBeats.

.
├── execbeat_xspider
│   ├── execbeat.yml
│   ├── xsipder_servers.ps1
│   ├── xsipder_skud.ps1
│   └── xsipder_switches.ps1
├── filebeat_nginx
│   └── filebeat.yml
├── filebeat_logs_system
│   └── filebeat.yml
├── auditbeat_info_system
│   └── auditbeat.yml
├── execbeat_script_python
│   └── execbeat.yml
└── metricbeat_logstash
    ├── metricbeat.yml
    └── modules.d
        ├── http.yml
        └── logstash-xpack.yml

The following is an example of the FileBeat configuration in the file bat.yml configuration file:

filebeat.inputs:
  - type: filestream
    id: logstash-logs
    enabled: true
    paths:
      - /app/logs/logstash/*.log

output.logstash:
  loadbalance: false
  hosts: ['server-logstash-01.com:5051', 'server-logstash-02.com:5051']
  ssl.certificate: '/app/smartBeat/cert/cert.pem'
  ssl.key: '/app/smartBeat/cert/key.pem'
  ssl.certificate_authorities: '/app/smartBeat/cert/ca-cert.pem'

The configuration above collects logs from the directory /app/logs/logstash/ with the file extension .log. The user running SAF Beat must have access to the files in this directory.

The collected data from the example above is sent to several Logstash servers via an encrypted SSL channel, the Logstash server is randomly selected and if the data is sent unsuccessfully, an attempt will be made to send it to another server.

To apply configurations, you can restart the systemd service or use the SAF Beat Manager API:

curl -k https://<SBM_HOST>:7769/reload -u sbm

Setting up apps.yml

Please note!

The apps.yml file is outdated and will be deleted in future versions.

The configuration file $SBM_HOME/etc/apps.yml is needed to configure the unpacking path on the configuration files agents. This file has a list of parameters:

• appname is the name of the configuration file
• path – the path of unpacking the file

Example of setting up apps.yml:

apps:
  - appname: scripts_exchange
    path: ./bin/scriptbeat-1.0.0-windows-x86_64/scripts
  - appname: scripts_activedirectory
    path: ./bin/scriptbeat-1.0.0-windows-x86_64/scripts

Setting up a connection to SA Web

To configure the management of SAF Beat Manager and SAF Beat using SA Web, you need to configure the parameters in the configuration file `$OSD_HOME/config/opensearch_dashboards.yml'.

Open the file and go to the SAF Beat Manager section and enter the correct settings. The configuration file already contains presets for these parameters. An example setup looks like this, where:

Parameter	Description
smart_monitor.safBeatManager.uri	Connection string to the SAF Beat Manager API.
smart_monitor.safBeatManager.uri.login	Username. Authentication verification on OpenSearch servers will be used.
smart_monitor.safBeatManager.password	User's password. It is recommended to use Keystore storage instead of an open password.
smart_monitor.safBeatManager.ssl.key	Path to the OpenSearch Dashboards private key in x509 format.
smart_monitor.safBeatManager.ssl.certificate	The path to the OpenSearch Dashboards certificate in x509 format.

An example of the settings is given below:

smart_monitor.safBeatManager.uri: 'https://127.0.0.1:7769'
smart_monitor.safBeatManager.login: safBeatManager
smart_monitor.safBeatManager.ssl.key: /app/opensearch-dashboards/config/node-key.pem
smart_monitor.safBeatManager.ssl.certificate: /app/opensearch-dashboards/config/node-cert.pem

Please note!

SAF already has a pre-installed user safBeatManager to work through the web interface with SAF Beat Manager. In the example above, the parameter saf.smart beats management.password is not specified, it is specified in the Keystore file. In order for authentication to work, you need to create a user in SAF in the menu item Main Menu - System Settings - Security Settings - Internal users.

Then add the password to the Keystore on the SA Web server. From the console, run:

sudo -u opensearch $OSD_HOME/bin/opensearch-dashboards-keystore add smart_monitor.safBeatManager.password

In SAF, the Keystore has a saved password from the user safBeatManager, so you can omit the command above if you plan to use the default values.

Save the changes and restart the OpenSearch-Dashboards service:

sudo systemctl restart opensearch-dashboards.service