Skip to main content
Version: 6.2

Job Scheduler API

Job​

A job contains a list of active actions that process search results. The search runs according to the schedule specified when the job is created. The API lets you retrieve, create, update, and delete jobs.

Job Structure​

NameSyntaxDescription
name<string>Job name.
title<string>Job title.
description<string>Job description.
throttle<object>Job throttling (More details Throttle).
actions<array>List of active actions. Each active action is an object (More details Actions API).
global_params<object>Global job parameters in key-value format.
search<object>Search parameters that are executed when the job runs (More details Search).
run_with_roles<array>List of roles that can execute the job.
schedule<object>Job execution schedule (More details Schedule).
enabled<bool>Job status (enabled, disabled).
lock_duration_seconds<int>Time interval from the executed query to the next execution.
jitter<float>Random time offset.
_permissions<object>Job access permissions.

Throttle​

NameTypeDescription
enabled<bool>Throttling state: true means enabled, false means disabled. The default is false.
time<string>Throttling time.
fields<array>List of fields used for throttling. Field names are passed as strings.
Important

The throttle.enabled field is available starting with version 6.2.0.

Throttling

Throttling filters out similar search results based on specified fields for a selected period.

The Throttling in Job Scheduler page describes how throttling works. The Throttling API page describes operations for manual suppressions and exclusions.

Schedule​

NameSyntaxDescription
cron<object>Schedule task via Cron.
interval<object>Schedule task via interval values.
warning

The cron and interval parameters are mutually exclusive. If the cron field is specified, then the interval field must be absent (and vice versa).

Cron​

NameSyntaxDescription
expression<string>Cron expression.
timezone<string>Time zone.

Interval​

NameSyntaxDescription
start_time<string>Time from which the interval count starts.
period<int>Interval length.
unit<string>Unit of measurement. For example, MINUTES, HOURS, DAYS, etc.
NameSyntaxDescription
query<string>Query.
time_field<string>Name of the field containing the timestamp.
earliest<string>Start time of the search.
latest<string>End time of the search.

CRUD for Job​

Retrieve a List of All Jobs​

GET _core/job_scheduler/jobs/
info

Starting with version 6.1.0, the request for retrieving the list of all scheduler tasks returns statistics for the five most recent runs of each task.

The run statistics contain the following fields:

NameSyntaxDescription
message<string>A text message describing the task execution result.
execution_time<number>The task execution time in milliseconds.
status<string>The task execution status.
start_time<string>The timestamp when the task execution started.

Retrieve a Job by ID​

GET _core/job_scheduler/jobs/{job_id}
info

Starting with version 6.1.0, when requesting the list of scheduler tasks, the user receives statistics for the five most recent runs of each task.

The run statistics have the following structure:

NameSyntaxDescription
message<string>A text message describing the task execution result.
execution_time<number>The task execution time in seconds.
status<string>The task execution status.
start_time<string>The timestamp when the task execution started.

Add a New Job​

POST _core/job_scheduler/jobs/
{
  "title": "Testing Email c smtpConnection",
  "description": "test description",
  "throttle": {
    "time": "5m",
    "fields": []
  },
  "actions": [
    {
      "script": {
        "context": "/app/opensearch/utils/python/bin/python3 /app/opensearch/utils/scripts/file_write_script.py",
        "is_once": false,
        "local_params": {}
      }
    }
  ],
  "global_params": {
    "key": "value"
  },
  "search": {
    "sme": {
      "query": "source internal_audit-* qsize=5;",
      "time_field": "@timestamp",
      "earliest": "now-24h",
      "latest": "now"
    }
  },
  "run_with_roles": [],
  "name": "Testirovanie_Email_c_smtpConnection",
  "schedule": {
    "cron": {
      "expression": "* * * * *",
      "timezone": "Europe/Moscow"
    }
  },
  "enabled": false,
  "lock_duration_seconds": 30,
  "_permissions": {
    "owner": "admin",
    "read": {
      "roles": [],
      "users": []
    },
    "write": {
      "roles": [],
      "users": []
    }
  }
}

Updating the task​

PUT _core/job_scheduler/job/{job_id}
{
  "title": "Testing Email with smtpConnection",
  "throttle": {
    "time": "5m",
    "fields": []
  },
  "actions": [
    {
      "script": {
        "context": "/app/opensearch/utils/python/bin/python3 /app/opensearch/utils/scripts/file_write_script.py",
        "is_once": false,
        "local_params": {}
      }
    }
  ],
  "global_params": {},
  "search": {
    "sme": {
      "query": "source internal_audit-* qsize=5;",
      "time_field": "@timestamp",
      "earliest": "now-24h",
      "latest": "now"
    }
  },
  "run_with_roles": [],
  "name": "Testing_Email_with_smtpConnection",
  "schedule": {
    "cron": {
      "expression": "* * * * *",
      "timezone": "Asia/Dubai"
    }
  },
  "enabled": false,
  "lock_duration_seconds": 30,
  "_permissions": {
    "owner": "admin",
    "read": {
      "roles": [],
      "users": []
    },
    "write": {
      "roles": [],
      "users": []
    }
  }
}

Deleting a Job​

DELETE _core/job_scheduler/jobs/{job_id}

Writing to a Database​

To configure the Write to DB active action, first create JDBC users and connections.

JDBC User​

NameSyntaxDescription
name<string>Arbitrary name for identifying the JDBC user.
username<string>User login used for database authentication.
password_id<string>Keystore key identifier. The password is obtained using a key with the prefix jobscheduler.jdbc.password.

Example:

{
  "name": "name",
  "username": "username",
  "password_id": "passwordId"
}

Connection​

NameSyntaxDescription
name<string>Database name.
driver<object>Database driver (e.g., postgres, clickhouse).
host<string>Host address where the database is located (e.g., localhost or an IP address).
port<string>Port number used to connect to the database.
db<string>Name of the database to which the connection is made.

Example:

{
  "name": "myDatabase",
  "driver": "postgres",
  "host": "localhost",
  "port": "5432",
  "db": "myDbName"
}

CRUD for JDBC User​

Retrieve a List of All Users​

GET _core/job_scheduler/jdbc_user/

Retrieve a User by ID​

GET _core/job_scheduler/jdbc_user/{jdbc_user_id}

Add a New User​

POST _core/job_scheduler/jdbc_user/
{
  "name": "name",
  "username": "username",
  "password_id": "passwordId"
}

Update a User​

PUT _core/job_scheduler/jdbc_user/{jdbc_user_id}
{
  "name": "name",
  "username": "username",
  "password_id": "passwordId"
}
Attention!

The password_id parameter must contain not the password itself, but the key identifier in Keystore. The key must have the prefix jobscheduler.jdbc.password..

info

The name parameter is used to specify an arbitrary name for the JDBC user, while username specifies the login used to connect to the database.

Update a User​

PUT _core/job_scheduler/jdbc_user/{jdbc_user_id}
{
  "name": "name",
  "username": "username",
  "password_id": "passwordId"
}

Delete a User​

DELETE _core/job_scheduler/jdbc_user/{jdbc_user_id}

User Password Setup​

POST _core/keystore/jobscheduler.jdbc.password.{password_id}
{
  "value" : "password"
}

CRUD for Connection​

Retrieve a List of All Connections​

GET _core/job_scheduler/jdbc_connection/

Retrieve a Connection by ID​

GET _core/job_scheduler/jdbc_connection/{connection_id}

Add a New Connection​

POST _core/job_scheduler/jdbc_connection
{
  "name": "myDatabase",
  "driver": "postgres",
  "host": "localhost",
  "port": "5432",
  "db": "myDbName"
}
warning

The driver parameter, indicating which JDBC driver will be used, can accept the following values:

  • postgres
  • mssql
  • oracle
  • hive
  • sap
  • clickhouse
info

The name parameter defines an arbitrary name for identifying the database, while db specifies its exact system name for establishing a connection.

Update a Connection​

PUT _core/job_scheduler/jdbc_connection/{connection_id}
{
  "name": "myDatabase",
  "driver": "postgres",
  "host": "localhost",
  "port": "5432",
  "db": "myDbName"
}

Delete a Connection​

DELETE _core/job_scheduler/jdbc_connection/{connection_id}

Mail​

The Mail settings are necessary for creating the corresponding active action and for sending emails to a group of recipients. To do this, you must first create the necessary number of recipients and then create a group from them. Using the API, the user can configure recipients, recipient groups, and SMTP.

warning

If smtp_id is not specified in the Mail Action, the sender settings will be taken from the .sm_settings index.

warning

If the recipient_group_id parameter is not specified in the Mail Action, the recipient will be the user specified in the to parameter.

Recipient

NameSyntaxDescription
name<string>Recipient name.
description<string>Recipient description.
type<string>Address type.
address<string>Recipient's email address.

Example:

{
  "name": "name",
  "description": "description",
  "type": "EMAIL",
  "address": "test@test.ru"
}

Recipient Group​

NameSyntaxDescription
name<string>Recipient group name.
description<string>Recipient group description.
recipient_ids<array>IDs of recipients recorded in the recipient index.
recipient_emails<array>List of recipient email addresses not recorded in the recipient index.

Example:

{
  "name": "name",
  "description": "description",
  "recipient_ids": ["id1", "id2"],
  "recipient_emails": ["test@test.org", "test2@test.org"]
}

SMTP​

The SMTP structure is as follows:

NameSyntaxDescription
name<string>SMTP server name.
description<string>SMTP server description.
host<string>SMTP server host address.
port<string>Port on which the SMTP server operates (e.g., 12345).
user<string>Username for authentication on the SMTP server.
user_email<string>Email address associated with the SMTP server account.
password_id<string>Keystore key identifier. The password is obtained using a key with the prefix jobscheduler.email.password..
timeout<int>Timeout for the connection and each SMTP server response, in milliseconds. The default is 5000. A value of 0 disables the timeout. Negative values are rejected when the configuration is saved.
proto<string>Protocol used to connect to the SMTP server (e.g., ssl).
System SMTP Configuration Password

If the system SMTP configuration does not specify password_id, the password is loaded from Keystore using the jobscheduler.email.pass key. If password_id is specified, the jobscheduler.email.password.{password_id} key is used.

{
  "name": "name",
  "description": "description",
  "host": "host",
  "port": "12345",
  "user": "user",
  "user_email": "test@test.com",
  "password_id": "passwordId",
  "timeout": 5000,
  "proto": ""
}

CRUD for Recipient​

Retrieve a List of All Recipients​

GET _core/job_scheduler/recipients

Retrieve a Recipient by ID​

GET _core/job_scheduler/recipients/{recipient_id}

Add a New Recipient​

POST _core/job_scheduler/recipients
{
  "name": "name",
  "description": "description",
  "type": "EMAIL",
  "address": "test@test.org"
}

Update a Recipient​

PUT _core/job_scheduler/recipients/{recipient_id}
{
  "name": "name",
  "description": "description",
  "type": "EMAIL",
  "address": "test@test.org"
}

Delete a Recipient​

DELETE _core/job_scheduler/recipients/{recipient_id}

CRUD for Recipient Group​

Retrieve a List of All Recipient Groups​

GET _core/job_scheduler/recipient_groups

Retrieve a Recipient Group by ID​

GET _core/job_scheduler/recipient_groups/{recipient_group_id}

Add a New Recipient Group​

POST _core/job_scheduler/recipient_groups
{
  "name": "name",
  "description": "description",
  "recipient_ids": [
    "id1",
    "id2"
  ],
  "recipient_emails": [
    "test@test.org",
    "test2@test.org"
  ]
}

Update a Recipient Group​

PUT _core/job_scheduler/recipient_groups
{
  "name": "name",
  "description": "description",
  "recipient_ids": [
    "id1",
    "id2"
  ],
  "recipient_emails": [
    "test@test.org",
    "test2@test.org"
  ]
}

Delete a Recipient Group​

DELETE _core/job_scheduler/recipient_groups/{recipient_group_id}

CRUD for SMTP​

Retrieve a List of All SMTP​

GET _core/job_scheduler/smtp

Retrieve an SMTP by ID​

GET _core/job_scheduler/smtp/{smtp_id}

Add a New SMTP​

POST _core/job_scheduler/smtp
{
  "name": "name",
  "description": "description",
  "host": "host",
  "port": "12345",
  "user": "user",
  "user_email": "test@test.com",
  "password_id": "passwordId",
  "timeout": 5000,
  "proto": ""
}

On successful creation, the response contains the SMTP configuration identifier in id:

{
  "message": "Config with id 'CSksd1SABJ12O_DKcsu4BP' CREATED",
  "id": "CSksd1SABJ12O_DKcsu4BP"
}

Update an SMTP​

PUT _core/job_scheduler/smtp/{smtp_id}
{
  "name": "name",
  "description": "description",
  "host": "host",
  "port": "12345",
  "user": "user",
  "user_email": "test@test.com",
  "password_id": "passwordId",
  "timeout": 5000,
  "proto": ""
}

Delete an SMTP​

DELETE _core/job_scheduler/smtp/{smtp_id}

Test the Connection to the SMTP Server​

You can test a saved SMTP configuration or supply parameters in the request body without saving them. Testing a saved configuration requires read permission for it; testing parameters supplied in the request body requires permission to create SMTP configurations.

No email is sent during the test. The test connects to the SMTP server, performs AUTH authentication, checks the sender address with MAIL FROM, and resets the mail transaction with RSET.

Each network operation, whether connecting or waiting for a command response, is limited to 300,000 ms. There is no overall time limit for the test.

Test a Saved SMTP Configuration​

POST /_core/job_scheduler/smtp/test_smtp/{id}

id is the identifier of the saved SMTP configuration. The password is loaded from Keystore using its password_id.

Test an SMTP Configuration Using Request Body Parameters​

POST /_core/job_scheduler/smtp/test_smtp/
{
  "host": "127.0.0.1",
  "port": "25",
  "proto": "none",
  "user_email": "tester@example.org",
  "user": "tester",
  "password": "qwert12345",
  "timeout": 30000
}

Request parameters:

NameTypeRequiredDescription
host<string>YesSMTP server host
port<string>YesSMTP server port
proto<string>YesConnection security protocol: none, tls, or ssl
user_email<string>YesEmail address
user<string>NoUsername. user and password must be provided together
password<string>NoSMTP user password. user and password must be provided together
timeout<int>NoTimeout for the connection and each SMTP server response, in milliseconds
Test Timeout

If timeout is omitted, the default is 5000. Negative values are rejected. If the value is 0 or greater than 300000, each network operation is limited to 300,000 ms.

Unknown fields in the request body are ignored.

Response Format​

The response contains these fields:

NameTypeDescription
status<string>Test result: success or reason for failure
stage<string>Stage at which the test ended
message<string>Message describing the test result

Possible status values:

ValueDescription
successTest completed successfully with authentication
success_without_authTest completed successfully without authentication
invalid_configurationSMTP configuration parameters are invalid
connection_failedCould not connect to the SMTP server
secure_failedCould not establish a secure connection
authentication_not_supportedSMTP server does not support authentication
authentication_failedAuthentication failed
sender_rejectedSMTP server rejected the sender address
timeoutOperation timed out
protocol_errorSMTP protocol error
internal_errorInternal test error

Possible stage values:

ValueDescription
connectConnect to the SMTP server
secureEstablish a secure connection
greetingReceive the SMTP server greeting
authAuthenticate with the SMTP server
senderCheck the sender address using MAIL FROM

For invalid_configuration and internal_error, the stage field is absent.

The HTTP response code depends on the test result:

Codestatus value
200All values except invalid_configuration and internal_error
400invalid_configuration
500internal_error

Example of a successful response:

{
  "status": "success",
  "stage": "sender",
  "message": "SMTP configuration check completed successfully"
}

Example of an unsuccessful response with HTTP 200:

{
  "status": "connection_failed",
  "stage": "connect",
  "message": "Could not connect to the SMTP server"
}

Job Statistics​

Retrieving Scheduler Task Execution Statistics​

GET _core/job_scheduler/statistic/{job_id}
ParameterTypeDescription
job_idparamTask identifier.
job_typequeryTask type (META, LOCAL).
sizequeryNumber of records to return.
statusqueryStatus of the returned statistics entries (error, warning, normal).
earliestqueryStart of the time range.
latestqueryEnd of the time range.

Retrieving Statistics for a Specific Active Action​

GET _core/job_scheduler/statistic/{job_id}/{run_id}
ParameterTypeDescription
job_idparamJob identifier.
run_idparamRun identifier.
job_typequeryJob type (META, LOCAL).
actionqueryActive action type.
numberqueryPosition of the active action in the job's action list.