SA Engine RE Configuration

SA Engine RE Startup Parameters

SA Engine RE startup parameters are specified in the application.properties configuration file, located in the same directory as the SA Engine RE utility.

Table of startup parameters:

Parameter	Description	Default Value
server.port	The port on which SA Engine RE will accept incoming connections.	18080
spring.servlet.multipart.max-request-size	Maximum file upload size in bytes.	100000000
server.tomcat.max-http-form-post-size	Maximum size for multipart/form-data requests in bytes.	104857600
sme.FileRotatePeriod	Request data file rotation time in seconds.	10000
sme.baseData	Directory for storing request data files.	./data
sme.directoryPollerInterval	Interval for checking the request data file storage directory in seconds.	50
sme.scriptConfigPath	Path to the scripts.yaml configuration file.	./scripts.yaml
sme.LogStringSize	Log string size.	300
sme.RapidDelete	Whether to delete request data files.	True
sme.redis.url	Address for connecting to Redis.	"127.0.0.1"
sme.redis.port	Port for connecting to Redis.	6379
sme.redis.pass	Password for connecting to Redis.	""
logging.file.path	Path for storing logs.	logs
server.ssl.enabled	Whether to use SSL/TLS for incoming connections.	true
server.ssl.certificate	Path to the public key (certificate) of the current SA Engine RE server in PKCS8 format.	/app/opensearch/config/node-cert.pem
server.ssl.certificate-private-key	Path to the private key of the current SA Engine RE server in PKCS8 format.	/app/opensearch/config/node-key.pem
server.ssl.trust-certificate	Path to the Certificate Authority (CA) chain in PKCS8 format.	/app/opensearch/config/ca-cert.pem
server.ssl.client-auth	Certificate validation mode, can be need, none, want.	need

General SA Engine RE Connection Settings

Table describing the general settings for connecting to SA Engine RE:

Setting	Description	Default Value	Setting Type
sme.core.remote_script.user	Username used to make requests to SA Engine RE.	sme_re_user	Cluster Setting
sme.core.remote_script.password	Password of the user used to make requests to SA Engine RE.		Keystore Setting
sme.core.remote_script.enable_ssl	Flag indicating whether to use SSL for connecting to SA Engine RE from the script command in search and in ScriptAction in the task scheduler.	true	Cluster Setting

Information for Running Scripts in Search

Connecting to SA Engine RE from the script Command in Search

Table describing cluster settings for connecting to SA Engine RE from the script command:

Setting	Description	Default Value
sme.core.remote_script.base_path	Path to the folder where scripts will be uploaded.	"/app/opensearch/utils/scripts/"
sme.core.remote_script.port	Port on which SA Engine RE is running.	18080
sme.core.remote_script.url	URL where SA Engine RE is running.	"https://127.0.0.1"
sme.core.remote_script.base_interpreter_name	Default interpreter name to use the script command without the intr parameter.	"bash"
sme.core.remote_script.interpreters	List of available interpreters, as an array of strings. String format: "interpreter name"::"path to interpreter".	["python3::/app/opensearch/utils/python/bin/python3","bash::bash"]

Creating a Script

To use a script in search, you must place it in the folder specified in the sme.core.remote_script.base_path setting on all servers in the cluster, ensure the file is executable, and has opensearch:opensearch permissions.

To return data from the script for further processing in the search, output the result in JSON format.

Example for Python:

print(scriptResultJson)

If other commands are executed before the script command, their result is saved to a temporary file in the folder specified in sme.baseData. The name of this file will be passed to the script as a command-line argument.

Example for Python:

import sys

if __name__ == "__main__":
    # open the file with data from the request
    inputDataFile = open(sys.argv[1], "r")

    # read the file with data from the request
    inputData = inputDataFile.read()

    inputDataFile.close() # close the file after reading

Information for Running Scripts in the Task Scheduler

Connecting to SA Engine RE from ScriptAction in the Task Scheduler

Settings can be retrieved via the following path:

GET _core/settings/job-scheduler

Table describing the settings for connecting to SA Engine RE from ScriptAction:

Setting	Description	Default Value
sme-re_port	Port on which SA Engine RE is running.	18080
sme-re_host	Host on which SA Engine RE is running.	"localhost"

Creating a Script

You can run any script in the task scheduler.

If you need to pass sensitive data to the script, you can use the environment variables feature. This feature allows you to store sensitive data in the keystore so that SA Engine RE adds them to environment variables when the script is run.

Algorithm for configuring environment variables:

1. Store the necessary variables to be passed to the script in the keystore on all cluster servers, using the variable jobscheduler.script.pass.<variable_name>
2. Specify in the task scheduler settings which task can use which variables from the keystore. To do this, execute the following command:

PUT _core/settings/job-scheduler
{
  "script_environments": {
    "<task_id>": ["<variable_name>"]
  }
}

Now the scheduler will instruct SA Engine RE to run the script with the variables in the environment.

3. In the script, include code to retrieve the variable from the environment.

Example for Python:

import os

if __name__ == "__main__":
    # get the variable from the environment
    try:
        envParamData = os.getenv("<variable_name>")
    except Exception as e:
        envParamData = e