SA Data Collector Upgrade
Pre-Upgrade Preparation
Legend:
SM_INSTALLER
- directory where Search Anywhere Framework installation package is extractedLS_HOME
- Logstash installation directory (typically/app/logstash
)LS_CONFIG
- Logstash configuration directory (typically/app/logstash/config
)LS_DATA
- Logstash data directory (typically/app/data/logstash
)
SA Data Collector
is based on Logstash
and may be referred to as such in this document.
If uninterrupted data collection is critical, migrate current pipeline configurations to another Logstash instance and redirect all data collection to it.
In high-load installations, SA Data Collector
might not stop via the update script. Manually stop it before upgrading:
sudo systemctl stop logstash
Create backups of these directories:
LS_CONFIG
LS_DATA
mkdir -p /app/backup/logstash/
cp -r ${LS_CONFIG} /app/backup/logstash/config_$(date +%Y%m%d)/
cp -r ${LS_DATA} /app/backup/logstash/data_$(date +%Y%m%d)/
Upgrade Process
The script requires these pre-installed packages:
curl
zip
unzip
The automated update script streamlines the upgrade process and is located at $SAF_INSTALLER/logstash/update.sh
. When executing the script, you may specify a configuration file: $SAF_INSTALLER/logstash/example_config_dashboards.yaml
. The YAML file format matches the installation configuration file.
The update script supports the following runtime parameters:
-c, --config <path_to_config_file_yaml>
- specify upgrade configuration file-h, --help
- display command help
During execution, the script will backup:
systemd
service fileslogstash.yml
- configuration directory
to the temporary staging directory:
$SAF_INSTALLER/logstash/staging/
The update script does not modify:
data
directorylogs
directory
The staging
directory contents will be cleared if the script is rerun.
To perform the upgrade, execute:
$SAF_INSTALLER/logstash/update.sh
The script automatically detects these key server paths:
Logstash Home Directory
- Logstash installation path (typically/app/logstash
)Logstash Conf Directory
- Configuration files location (typically/app/logstash/config/
)Logstash Data Directory
- Data storage location (typically/app/data/logstash
)Logstash Logs Directory
- Log files location (typically/app/logs/logstash
)
================================================================================
SEARCH ANYWHERE FRAMEWORK INSTALL SCRIPT - LOGSTASH
================================================================================
Current working directory: /app/distr/saf_5.1/logstash
Current name of install's archive: logstash-oss-8.13.4-linux-x86_64.tar.gz
Current version of Logstash: 8.13.4
================================================================================
-- STEP 1. INSTALLATION DIRECTORIES
logstash.service file found. Will get necessary paths from there
Final Logstash home directory: /app/logstash
Final Logstash conf directory: /app/logstash/config
Final Logstash data directory: /app/data/logstash
Final Logstash logs directory: /app/logs/logstash
Is this correct? [y/n]:
After entering the directory paths, confirm your entries by pressing y
, or manually re-enter them by pressing n
.
The next step allows disabling all operations requiring privilege escalation to root
during the update.
A complete update cannot be performed without root
privileges. You must:
- Execute some operations as the
root
user - Run the installer as the SA Data Collector runtime user
- Perform final operations again as
root
after script completion
The installer will display the required operation list.
Failure to execute these commands before updating may result in data loss.
To launch the installer as the logstash
user:
sudo -u logstash $SAF_INSTALLER/logstash/update.sh
The first prompt will ask whether to disable commands requiring root
privilege escalation.
Next, you'll need to specify:
- The user
- The group
under which SA Data Collector operates.
Finally, the system will display a checklist of operations that must be completed before executing the update script.
-- STEP 2. DISABLE OPERATIONS WHERE ROOT RIGHTS ARE REQUIRED
Want to install SEARCH ANYWHERE FRAMEWORK without root privileges? [y/N]: y
Final you set the non-root flag to: y
Linux username for SEARCH ANYWHERE FRAMEWORK [logstash]:
Linux groupname for SEARCH ANYWHERE FRAMEWORK [logstash]:
Final SEARCH ANYWHERE FRAMEWORK username: logstash
Final SEARCH ANYWHERE FRAMEWORK groupname: logstash
Is this correct? [Y/n]:
ATTENTION!
Before installing SEARCH ANYWHERE FRAMEWORK, you must run the following commands as a user with root privileges.
Change user, group and directories as needed
systemctl stop logstash
chown -R logstash:logstash /app/logstash /app/logstash/config /app/data/logstash /app/logs/logstash /app/distr/saf_5.1/logstash
!!! Are you sure you have done everything from the list above? [y/N]:
At the final step, you must confirm that all root
user commands were executed.
Enter y
to proceed - otherwise, the update will be aborted.
The system will then perform pre-upgrade preparations. Before applying updates:
- You'll receive a final confirmation prompt
- No system-impacting changes will occur before this confirmation
!!! AT THIS POINT WE START TO MAKE CHANGES IN OPERATING SYSTEM !!!
Do you want to continue? [y/N]:
If you selected the update option without commands requiring root
privileges, the system will display an information block listing all commands that must be executed as the root
user.
================================================================================
ATTENTION!
After installing SEARCH ANYWHERE FRAMEWORK, you must run the following commands as a user with root privileges.
Change user, group and directories as needed.
cp /app/distr/saf_5.1/logstash/staging/services/logstash.service /etc/systemd/system/
chown root:root /etc/systemd/system/logstash.service
chcon -Rv -u system_u -r object_r -t bin_t /app/logstash/bin
chcon -Rv -u system_u -r object_r -t bin_t /app/logstash/jdk/bin
chown -R logstash:logstash /app/logstash /app/logstash/config /app/data/logstash /app/logs/logstash
systemctl daemon-reload
systemctl enable logstash
systemctl start logstash
systemctl status logstash
================================================================================
-- SEARCH ANYWHERE FRAMEWORK SUCCESSFULLY UPDATED!
-- For further instructions, please refer to the documentation: https://docs.saf-systems.com/
================================================================================
And this too: Upon successful completion of the update script, the corresponding text SEARCH ANYWHERE FRAMEWORK SUCCESSFULLY UPDATED
will be displayed.
Before running Logstash, it's recommended to check all pipeline
configurations using the following command:
sudo -u logstash $LS_HOME/bin/logstash --config.test_and_exit -f <pipeline-config>