Installing SAF Beat Manager

Please note!

Starting from version 4.0.0, the interaction scheme between SAF Beat and SAF Beat Manager has changed, but SAF Beat Manager remains generally compatible with older agents.

Since version 4.2.0, the interaction API has been updated without maintaining backward compatibility, the SAF version must be at least 4.2.0.

SAF Beat can work with any version of ElasticBeat, but if you plan to send these examples to OpenSearch, then the ElasticBeat version should be 7.12.1 (or 7.10.2 is a more stable version). If you plan to use Logstash, then the ElasticBeat version does not matter. Check on the official website.

Symbols

${SBM_HOME} is the installation path of SAF Beat Manager, usually it is /app/SAFBeatManager/

Installing SAF Beat Manager

Preparing to install SAF Beat Manager

SAF Beat Manager is available as part of the SAF installation archive in the directory ${SM_INSTALLER}/utils/SAFBeatManager. The installation must be performed under the root user. Create an empty directory and place the executable file SAFBeatManager-<version>.elf in this directory. For example, to the directory/app/SAFBeatManager. Then create a user under which the service will work, for example, sbm:

mkdir -p /app/SAFBeatManager
cd /app/SAFBeatManager
cp ${SM_INSTALLER}/utils/SAFBeatManager/SAFBeatManager-* /app/SAFBeatManager/
chmod +x ./SAFBeatManager-*
useradd --no-create-home --system --user-group sbm

Please note!

In some cases, the user's group of the same name may not be created automatically. Make sure that the user belongs to the group of the same name (in the example above - sbm), or create a group manually and add the user to it:

groupadd sbm
usermod -aG sbm sbm

Installing the SAF Beat Manager service

SAF Beat Manager supports the following commands to work with the service:

install – installs the service
install -u string | --user <string> - installs the service under a specific user
remove – deletes the service (in older versions, the delete command)
version - shows the version of SAF Beat Manager
help – help about any command

Please note!

If you specify the install command and the user in the system is not created in advance, for example, sbm, the installation will fail with an error. The installer is compatible only with the systemd service initialization and management subsystem.

The command to install the service in systemd may look like this:

cd /app/SAFBeatManager/
./SAFBeatManager install
chown -Rf sbm:sbm /app/SAFBeatManager/

If you plan to use a non-standard user, for example $USER, then the installation command will look like this:

cd /app/SAFBeatManager
./SAFBeatManager install --user $USER
chown -Rf $USER:$USER /app/SAFBeatManager/

If SELinux is used, configure the rights to the executable file:

sudo chcon -Rv -u system_u -r object_r -t bin_t /app/SAFBeatManager/SAFBeatManager-*

Please note!

The version of Smart Boot Manager may differ, so the file may be named differently. Also, pay attention to the owner of the group after changes in the working directory, since the service runs under the specified user during installation.

After the Smart Boot Manager service has been successfully installed, you can start the service. An example of launching the Smart Boot Manager service:

systemctl start saf-beat-manager.service

In order to find out if the service is working, you need to run the command:

systemctl status saf-beat-manager.service

Setting Up SAF Beat Manager

SAF Beat Manager will create all the necessary directories and files for correct operation at the first launch. If any file or directory is missing, it will be created the next time you run it.

SAF Beat Manager Configuration File

Please note!

In the version of SAF Beat Manager 4.2.0, the name of the configuration file SAFBeatManager.yaml was changed to config.yaml. Both files are supported, but config.yaml will be considered a priority.

An example of the standard configuration of the configuration file config.yaml:

authorization:
  opensearch:
    ca_cert: ca-cert.pem
    host: https://127.0.0.1:9200
basic:
  appPath: ./apps
  binariesPath: ./binaries
  appsUnpackPath: ./config/apps
  binariesUnpackPath: ./bin
  portForSmartBeat: '7767'
  portForVisual: '7769'
  bindAddress: 127.0.0.1
  enableSnapshot: false
  updateSnapshot: 10m
  enableDeletion: false
  timeoutDeletion: 24h
  disableGroupsCache: false
logs:
  logPath: ./logs
  logMaxAge: 7d
  logRotate: 24h
ssl:
  enabled: true
  smartBeatTLSLevel: 0
  certPath: cert.pem
  keyPath: key.pem

The parameters are described below:

Parameter	Description	Default Value
`authorization.opensearch.ca_cert`	CA certificate in X509 format, used to connect to OpenSearch.	`ca-cert.pem`
`authorization.opensearch.host`	The address of one of the nodes of the OpenSearch server.	`https://127.0.0.1:9200`
`basic.appPath`	Directory for searching for application configurations (Elastic Beats configurations).	`./apps`
`basic.binariesPath`	Elastic Beats archive search directory.	`./binaries`
`basic.appsUnpackPath`	Directory for unpacking application archives on SAF Beat agents.	`./config/apps`
`basic.binariesUnpackPath`	Directory for unpacking Elastic Beats archives on SAF Beat agents.	`./bin`
`basic.portForSmartBeat`	RestAPI port for connecting SAF Beat agents.	`7767`
`basic.portForVisual`	RestAPI port for connecting SAF Web.	`7769`
`basic.bindAddress`	At which address to accept connections.	`127.0.0.1`
`basic.enableSnapshot`	Enabling snapshots to save a list of connected agents. During operation, the load on the hard drive increases.	`false`
`basic.updateSnapshot`	The refresh rate of the snapshot.	`10m`
`basic.enableDeletion`	Whether to delete old agent entries in the snapshot.	`false`
`basic.timeoutDeletion`	The lifetime of the agent record in the snapshot.	`24h`
`basic.disableGroupsCache`	Disabling the cache of application groups.	`false`
`logs.logPath`	Logs directory.	`./logs`
`logs.logMaxAge`	Log storage time.	`7d`
`logs.logRotate`	The frequency of log rotation.	`24h`
`ssl.enabled`	Enables SSL.	`true`
`ssl.smartBeatTLSLevel`	Certificate verification level: 0 – client certificate is not requested and is not required, 1 – agent certificate will be requested, but it is not required and will not be verified, 2 – agent certificate is required, but any valid agent certificate is acceptable. It will not be verified by the CA certificate, 3 – the agent certificate will not be requested, but if it is present, it will be verified by the CA certificate, 4 – an agent certificate is required, which is verified for compliance with the certificate.	`0`
`ssl.certPath`	Certificate file in x509 SAF Beat Manager format.	`cert.pem`
`ssl.keyPath`	Private key file in x509 SAF Beat Manager format.	`key.pem`

Install your SAF Beat Manager Certificates

Please note!

At the first launch, SAF Beat Manager generates its self-signed certificate and private key.

If you need to use ready-made certificates, then copy the CA certificate, the certificate and the SAF Beat Manager private key to the directory ${SBM_HOME}/etc/ssl/. You also need to configure the following settings:

authorization.opensearch.ca_cert - path to the CA certificate or certificate chain in x509 format
authorization.opensearch.host - the address of the OpenSearch server should start with https
ssl.enabled - must be set to true
ssl.smartBeatTLSLevel - select the required certificate verification mode
ssl.CertPath - path to the SAF Beat Manager certificate file in x509 format
ssl.keyPath - path to the SAF Beat Manager private key file in x509 format

It is recommended to generate certificates with the following parameters:

keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
Subject Alternative Name - specify all possible IP and domain names of the server where the agent is installed

Please note!

SAF Beat Manager must have access to the specified files and directories

Configuring log collection configurations for SAF Beat agents

To send data collection configuration files to agents, you need to configure the file $SBM_HOME/etc/serverclasses.yml, which can contain several groups of settings for agents. The available group parameters are listed below:

Parameter	Description
`name`	The name of the group. Required parameter.
`filters`	A list of filters for connected agents, the values of the fields `ip`, `DNS name`, `hostname` and `guid` of the agent can be used. You can use masks in the list. A logical `OR` is used between the conditions, i.e. it is sufficient to meet any of the listed conditions. Required parameter.
`apps`	List of configuration names (directory names) of ElasticBeats. The name template must match `<elasticbeat_name>_<apps_name>`, where `<elasticbeat_name>` is the name of ElasticBeat in lowercase, `<apps_name>` is the name of the application. Required parameter.
`binaries`	List of ElasticBeats archives that are required to perform the configurations listed in `apps'. You cannot specify multiple versions of the same ElasticBeat in the same group. Required parameter.
`systems`	A list of filters by the server platform on which the SAF Beat agent is running. The logical `OR` is used in the list, i.e. the agent falls into at least one of the filters. It is used as an additional filter with the `filters` field with the logical operation `And'.
`excludes`	List of agent filters to exclude from the group by `ip`, `DNS name`, `hostname`, `guid`. You can use masks in the list. A logical `OR` is used between the conditions, i.e. it is sufficient to meet any of the listed conditions. It is used as an additional filter with the `filters` field with the logical operation `AND-NOT`.

Please note!

The DNS name is taken from the SmartBeat agent, for correct determination there must be a PTR record of the reverse DNS zone on the DNS server.

The following characters can be used as a mask in filters:

* - asterisk sign, replaces any character with a length of 0 (missing character) to infinity
? - replaces any 1 character

Example of setting up server classes.yml:

groups:
  - name: linux_test
    filters:
      - 192.168.1.?/24
      - server-test-linux.localhost
    apps:
      - filebeat_nginx
      - auditbeat_info_system
      - execbeat_script_python
    binaries:
      - filebeat-7.10.1-linux-x86_64.tar.gz
      - auditbeat-8.9.0-linux-x86_64.tar.gz
      - execbeat-3.3.0-linux-x86_64.tar.gz
  - name: windows_test
    filters:
      - 10.10.*.*
      - 78d17a171-a3c4-4e27-9c20-bc7005db068e
      - *our-site.org*
    apps:
      - filebeat_logs_system
      - execbeat_xspider
    binaries:
      - filebeat-8.9.2-windows-x86_64.zip
      - execbeat-3.3.0-windows-x86_64.zip

After configuring the file $SBM_HOME/etc/serverclasses.yml, you need to download the binaries archives from the official website (there should be archives with a pre-compiled ElasticBeats executable file) of oss versions and place them in the directory $SBM_HOME/binaries/.

Please note!

It is recommended to use ElasticBeats versions of the same version that has Logstash installed. SAF version 4.2.0 uses Logstash version 8.9.0, so ElasticBeats is better to use versions 8.9.x.

Then you need to create the applications themselves (directories) as $SBM_HOME/etc/serverclasses.yml was specified in the $SBM_HOME/apps/ directory. Inside each application directory, all files and subdirectories will be sent to the agent. The name of the ElasticBeats configuration file must match the name used for the ElasticBeats application.

In the example below, there are applications execbeat_xspider, filebeat_nginx, filebeat_logs_system and others.

The execbeat_xspider application uses ElasticBeat - ExecBeat, the application name begins with execbeat\_, and the configuration file execbeat.ymlis located inside the directory. In this application, in addition to the configuration file forExecBeat, there are additional files, they will also be copied to SAF Beat agents.

The auditbeat_info_system application uses AuditBeat, so the name starts with auditbeat_, and the configuration file is called auditbeat.yml.

The contents of the ElasticBeats configuration files are filled in according to the official documentation of the corresponding ElasticBeats.

.
├── execbeat_xspider
│   ├── execbeat.yml
│   ├── xsipder_servers.ps1
│   ├── xsipder_skud.ps1
│   └── xsipder_switches.ps1
├── filebeat_nginx
│   └── filebeat.yml
├── filebeat_logs_system
│   └── filebeat.yml
├── auditbeat_info_system
│   └── auditbeat.yml
├── execbeat_script_python
│   └── execbeat.yml
└── metricbeat_logstash
    ├── metricbeat.yml
    └── modules.d
        ├── http.yml
        └── logstash-xpack.yml

The following is an example of the FileBeat configuration in the file bat.yml configuration file:

filebeat.inputs:
  - type: filestream
    id: logstash-logs
    enabled: true
    paths:
      - /app/logs/logstash/*.log

output.logstash:
  loadbalance: false
  hosts: ['server-logstash-01.com:5051', 'server-logstash-02.com:5051']
  ssl.certificate: '/app/smartBeat/cert/cert.pem'
  ssl.key: '/app/smartBeat/cert/key.pem'
  ssl.certificate_authorities: '/app/smartBeat/cert/ca-cert.pem'

The configuration above collects logs from the directory /app/logs/logstash/ with the file extension .log. The user running SmartBeat must have access to the files in this directory.

The collected data from the example above is sent to several Logstash servers via an encrypted SSL channel, the Logstash server is randomly selected and if the data is sent unsuccessfully, an attempt will be made to send it to another server.

To apply configurations, you can restart the systemd service or use the SAF Beat Manager API:

curl -k https://<SBM_HOST>:7769/reload -u sbm

Setting up `apps.yml`

Please note!

The apps.yml file is outdated and will be deleted in future versions.

The configuration file $SBM_HOME/etc/apps.yml is needed to configure the unpacking path on the configuration files agents. This file has a list of parameters:

appname is the name of the configuration file
path – the path of unpacking the file

Example of setting up apps.yml:

apps:
  - appname: scripts_exchange
    path: ./bin/scriptbeat-1.0.0-windows-x86_64/scripts
  - appname: scripts_activedirectory
    path: ./bin/scriptbeat-1.0.0-windows-x86_64/scripts

Setting up a connection to SAF Web

To configure the management of Smart Boot Manager and SAF Beat using SAF Web, you need to configure the parameters in the configuration file `$OSD_HOME/config/opensearch_dashboards.yml'.

Open the file and go to the Smart Boot Manager section and enter the correct settings. The configuration file already contains presets for these parameters. An example setup looks like this, where:

Parameter	Description
`smart_monitor.smartbeatsmanagement.uri`	Connection string to the Smart Boot Manager API.
`smart_monitor.smartbeatsmanagement.uri.login`	Username. Authentication verification on OpenSearch servers will be used.
`smart_monitor.smartbeatsmanagement.password`	User's password. It is recommended to use Keystore storage instead of an open password.
`smart_monitor.smartbeatsmanagement.ssl.key`	Path to the OpenSearch Dashboards private key in x509 format.
`smart_monitor.smartbeatsmanagement.ssl.certificate`	The path to the OpenSearch Dashboards certificate in x509 format.

An example of the settings is given below:

smart_monitor.smartbeatsmanagement.uri: 'https://127.0.0.1:7769'
smart_monitor.smartbeatsmanagement.login: sbm
smart_monitor.smartbeatsmanagement.ssl.key: /app/opensearch-dashboards/config/node-key.pem
smart_monitor.smartbeatsmanagement.ssl.certificate: /app/opensearch-dashboards/config/node-cert.pem

Please note!

SAF already has a pre-installed user sbm to work through the web interface with Smart Boot Manager. In the example above, the parameter smart_monitor.smart beats management.password is not specified, it is specified in the Keystore file. In order for authentication to work, you need to create a user in SAF in the menu item Navigation Menu - System Settings - Security Settings - Internal users.

Then add the password to the Keystore on the SAF Web server. From the console, run:

sudo -u opensearch $OSD_HOME/bin/opensearch-dashboards-keystore add smart_monitor.smartbeatsmanagement.password

In SAF, the Keystore has a saved password from the user sbm, so you can omit the command above if you plan to use the default values.

Save the changes and restart the OpenSearch-Dashboards service:

sudo systemctl restart opensearch-dashboards.service

Symbols​

Installing SAF Beat Manager​

Preparing to install SAF Beat Manager​

Installing the SAF Beat Manager service​

Setting Up SAF Beat Manager​

SAF Beat Manager Configuration File​

Install your SAF Beat Manager Certificates​

Configuring log collection configurations for SAF Beat agents​

Setting up apps.yml​

Setting up a connection to SAF Web​

Symbols

Installing SAF Beat Manager

Preparing to install SAF Beat Manager

Installing the SAF Beat Manager service

Setting Up SAF Beat Manager

SAF Beat Manager Configuration File

Install your SAF Beat Manager Certificates

Configuring log collection configurations for SAF Beat agents

Setting up `apps.yml`

Setting up a connection to SAF Web