peval

Description

Performs various operations on the data. It is based on internal storage mechanisms.

danger

Using peval in a query is allowed if it is preceded only by commands that also work with internal storage mechanisms. These include source, inputlookup and search. This condition must also be met for all subqueries in the query.

Syntax

 | peval <field>=<expression>["," <field>=<expression>]...

Parameter	Syntax	Description
field	<string>	Name of the field with the result of the operation.
expression	<expression>	The operation (computation) or a set of operations to be performed to create the new field.

Supported expression Operations

Operation Type	Syntax	Description
Arithmetic Operations	(+|-|*|/)	Arithmetic operations are applied to numeric fields of numeric type.
Logical Operations	(==|!=|>|>=|<|<=) (AND|OR|NOT)	Logical operations include comparisons and logical operators that return boolean values.
Conditional Operations	cidrmatch coalesce like nullif case if validate	Conditional operations provide additional capabilities for data processing.
Mathematical Operations	abs ceil exact exp floor ln log pi pow round sigfig sqrt	Mathematical operations with numerical values for various computations.
Time Operations	now strftime strptime	Time operations related to processing, managing, or analyzing data within a time interval.
Data Type Identification Operations	isbool isint isnotnull isnull isnum isstr typeof	Operations where the system or program evaluates the data type of a variable, object, or expression.
Data Conversion Operations	printf tobool tonumber tostring	These are processes of converting data into different types.
Statistical Operations	max min	These are operations that are defined for an object or type of objects but are not executed for an instance of an object.
Text Operations	len lower ltrim replace rtrim substr trim upper	These are sets of actions performed to analyze, transform, format, store, or process textual data.
Multivalue Operations	mvappend mvcount mvdedup mvjoin mvrange mvsort mvzip split	Operations designed for working with multivalue fields or creating and processing such arrays.
Regular Expressions	regex	Regular expression search operations.
Painless Scripts	_script	Operations described in the Painless scripting language.

Data Types

Peval operates with the following basic data types and attempts to retain the original or assigned data type in the response:

• string
• numeric
• boolean
• null
• multivalue

info

Multivalues represent arrays of values, where each element is a basic data type.

OpenSearch Specifics

• Scripts in OpenSearch handle numeric field types or keyword types for text fields. For text fields, you need to append <field name>.keyword
• Regular expressions must be wrapped in the regex function to ensure proper OpenSearch script handling:
  • Syntax: regex("<regular expression>")
• Allows executing custom painless scripts during command execution using the _script function:
  • Syntax: _script("<script>", (<arg_name>, <arg_value>)...)

Query Examples

Example 1

Concatenating fields with a string separator.

Example №1

source nix_events-*
| search host="SIN-*"
| peval agent= agent + ":" + port
| search agent="*log*77"
| aggs values(per_number) by agent

In this example, the system filters events where host has the prefix SIN-*. A new field agent is then created by concatenating the string values of agent and port with a : separator. The query then selects events where the agent field contains the substring log and ends with 77. Finally, an aggregation is performed to sum the values of the per_number field for each unique agent.

Sample input data:

host	agent	port	per_number
SIN-WS-01	nginx-log	77	smith
SIN-WS-02	nginx-log	77	dawson
DXB-DB-02	syslog	53363	smith
SIN-DB-03	syslog	57177	wright
SIN-WS-03	nginx-error	77	wright

The query execution result may be the following table:

agent	values (per_number)
nginx-log:77	smith dawson
syslog:57177	wright

---

Example 2

Calculating the difference between numeric fields.

Example №2

... | peval temp_diff=ups_adv_battery_temperature - env_temperature
| where temp_diff > 10

In this example, the command calculates the difference between the numeric fields ups_adv_battery_temperature and env_temperature, writing the result to temp_diff. The where filter retains events where the difference exceeds 10 degrees.

Sample input data:

ups_adv_battery_temperature	env_temperature
35	20
30	22
28	25

The query execution result may be the following table:

ups_adv_battery_temperature	env_temperature	ups_adv_battery_temperature
35	20	15
30	22	8

---

Example 3

Classifying based on the value of a numeric field using Painless scripts.

Example №3

... | peval res=_script("if (doc['total_price'].value < 1000 ) { return 'low'; } else if (doc['total_price'].value < 2000) { return 'medium'; } else { return 'high'; }", (total_price, total_price))

In this example, the command classifies orders by the value of the numeric field total_price via doc['total_price'].value as low (less than 1000), medium (from 1000 to 1999), or high (2000 and above) using a Painless script with the _script function.

The query execution result may be the following table:

user_id	items_quantity	operation_status	total_price	res
1	3	success	1815	medium
2	7	success	4073	high
3	10	success	5520	high
4	1	success	599	low

---

Example 4

Determining the status level based on the error count and threshold value for each document using the calcStatusByErrors script.

... | peval level=calcStatusByErrors(errorCount, 3)

The query execution result may be the following table:

errorCount	level	service
3	warning	auth
0	ok	cache
10	critical	search
5	critical	billing

---

Example 5

Creating an array of statuses using multivalue operations:

Example №4

| peval status_list=mvappend(case(http.response.status_code == 200, "OK", http.response.status_code == 404, "Not Found", "Error"), tostring(http.response.status_code))

In this example, the system processes numeric HTTP statuses http.response.status_code into an array status_list containing a textual description and the original code as a string.

The query execution result may be the following table:

source.address	destination.address	http.request.method	http.request.status	status_list
172.24.76.116	192.168.12.17	GET	200	OK 200
172.23.10.193	192.168.12.98	GET	404	Not Found 404
172.27.99.179	192.168.12.1	GET	403	Error 403

---

Example 6

Extracting a text value from the host.name field and converting its first part to lowercase.

Example №5

... | peval hn=lower(mvindex(split(host.name, "."), 0))

In this example, the split function divides the string host.name into a multivalue field hn using the . delimiter, and the mvindex function extracts the element at index 0 from this field. The lower function then converts this element to lowercase.

The query execution result may be the following table:

user	action	host	hn	hn (without function mvindex)
smith	added-user-account	WS-01.work.local	ws-01	ws-01 local work
holmes	changed-password	WS-02.work.local	ws-02	ws-01 local work
wright	removed-member-from-group	WS-03.work.local	ws-03	ws-01 local work

---

Example 7

Creating a field to classify events based on HTTP response status:

Example №6

source web_indexes
| peval name=if(isnull(user.name), "None", user.name)

In this example, the if function checks the condition isnull (user.name), returning true if the field contains an empty value. If the condition is true, the string None is written to the name field; if false, the original value from user.name is written.

The query execution result may be the following table:

source	destination	path	status	user	user (before the expression if)
172.26.0.240	192.168.12.100	/veil.jpg	200	Smith	Smith
172.26.0.241	192.168.12.101	/number/seashore	404	None	null
172.26.0.242	192.168.12.102	/advice	403	None	null
172.26.0.243	192.168.12.103	/walk/stove	200	Holmes	Holmes