Skip to main content

Evaluating fields with multiple values

A multi-value field is a field that contains more than one value. For example, an email log contains multi-valued fields “To:”, “Cc:”, or an online store log contains a field with a list of products.

Commands for working with multi-value fields

mvappend​

Appending fields.

The search query will create a new field event_info, which will collect information about the values ​​from other fields. The number of parameters supplied to the mvappend function may have a different number other than zero.

source wineventlog-*
| table user.name, source.ip, event.action
| eval event_info = mvappend(user.name, source.ip, event.action)

mvcount​

Counting the number of values in multi-valued fields.

Multi-valued fields can be counted. There is a command called mvcount for this. The search query will create a list of events in the event.action field, grouping them by the source.ip field, and then the mvcount command is executed, which will count the number of values in the event.action field.

source wineventlog-*
| stats values(event.action) as event.action by source.ip
| eval source_number=mvcount(event.action)

mvfilter​

Filtering multi-valued fields.

The request will create new fields logged_in and logged using the mvfilter and match functions, which will help select the desired values in the event.action field according to patterns. Various matching functions are used within the mvfilter command (e.g. match, cidrmatch, in, etc.).

source wineventlog-*
| stats values(event.action) as event.action by source.ip
| eval source_number = mvcount(event.action)
| eval logged_in = mvfilter(match(event.action, "logged-in"))
| eval logged = mvfilter(match(event.action, "logged-(.*)"))

mvjoin​

Takes as input the name of a multi-valued field, the value of the concatenator and returns a string with all the values through the concatenator.

The search query searches the index for the number of events (store orders) among the first 100,000 values, grouping orders by user ID (user_id), then concatenates the values of the product field (the list of products in the order) using the concatenator ,. When the table is displayed, the displayed values will be sorted. The product column has the values sorted. The result column has the actual order of the values in the product field. The actual order of values is important when working with value indexes. Therefore, it is recommended to use the mvsort command to sort field values.

source food_orders qsize=100000
| stats count, values(items) as product by user_id
| eval result=mvjoin(product, ", ")
| table user_id, product, result

mvsort​

Takes the name of a multivalued field as input and performs sorting within it.

The search query searches the index for the number of events (orders) among the first 100,000 values, grouping orders by user ID (user_id), then sorts (mvsort) the values inside the product field (list of products in the order), then performs concatenation values of the product field (list of products in the order) using the concatenator ,.

source food_orders qsize=100000
| stats count, values(items) as product by user_id
| eval product=mvsort(product)
| eval result=mvjoin(product, ", ")
| table user_id, product, result

mvfind​

Takes a value as input and returns the index of the first match found. The index starts at 0.

The search query searches the index for the number of events (orders) among the first 100,000 values, grouping orders by user ID (user_id), then sorts (mvsort) the values inside the product field (list of products in the order), and returns the index the first match found for the pattern "C(.*)" in the product field. The numbering of indices of elements of a multivalued field starts from null.

source food_orders qsize=100000
| stats count, values(items) as product by user_id
| eval product=mvsort(product)
| eval result=mvfind(product, "C(.*)")
| table result, user_id, product

mvindex​

The mvindex function takes as input a parameter and the number of the desired index in a multi-valued field. Returns the value or null if not found.

The query searches the index for the number of events (orders) among the first 100,000 values, grouping orders by user ID (user_id), then sorts (mvsort) the values inside the product field (list of products in the order), and returns the index of the first found match according to the pattern "C(.*)" in the product field, and the result_index field is assigned the value of the 4th element of the product field.

source food_orders qsize=100000
| stats count, values(items) as product by user_id
| eval product=mvsort(product)
| eval result=mvfind(product, "C(.*)")
| eval result_index=mvindex(product, 4)
| table result, result_index, user_id, product

mvrange​

The mvrange function takes as input the start, end, and increment interval parameters (optional). Returns a multi-valued field with a list of numbers according to the specified parameters.

source food_orders qsize=100000
| eval range=mvrange(1,20,1)
| table range

mvzip​

The mvzip function takes multivalued fields as input and performs concatenation (comma separated) of the values of the first field with the second (according to the index order of the values of the first field). If one of the multi-valued fields has more values than the others, concatenation over those values will not be performed.

The search query searches the index for the number of events (store orders) among the first 100,000 values, grouping orders by user ID (user_id), then sorting (mvsort) the values inside the product field (list of products in the order), then a multi-valued field (range) is created with a range of values from 1 to 50, then a final list (rezult_zip) is formed from the values of the range and product fields, then “;” is added to the last value of the result_zip field (the result_append field), then, since the ranges of field values are different, the “extra” numbers are removed and we get the final list - the result_replace field.

source food_orders qsize=100000
| stats count, values(items) as product by user_id
| eval product=mvsort(product)
| eval range=mvrange(1,50,1)
| eval result_zip=mvzip(range,product)
| eval result_join=mvjoin(result_zip, "; ")
| eval result_append=mvappend(result_join + ";")
| eval result_replace = replace(result_append, "[0-9]+;", "")
| table user_id, result_zip, result_join, result_append, result_replace

split​

The split function takes a string and a separator as input. Returns a multi-valued field formatted with a delimiter.

The next query will perform the same operations as the previous one, but the result field result_replace will be split using the delimiter pattern ; and the values of the result_split field will be sorted.

source food_orders qsize=100000
| stats count, values(items) as product by user_id
| eval product=mvsort(product)
| eval range=mvrange(1,50,1)
| eval result_zip=mvzip(range,product)
| eval result_join=mvjoin(result_zip, "; ")
| eval result_append=mvappend(result_join + ";")
| eval result_replace=replace(result_append, "[0-9]+;", "")
| eval result_split=split(result_replace, "; ")
| eval result_split=mvsort(result_split)
| table user_id, result_zip, result_join, result_append, result_replace, result_split

mvdedup​

Deduplication of values in multi-valued fields.

The search query searches the index for the number of events (orders) among the first 100,000 values, grouping orders by user ID (user_id). Unlike the values command, which stores only unique values in a field, the list command produces a list of all values. Therefore, duplicates will appear in the product field. The mvdedup command will remove duplicate values in the product field, and assign the execution result to the result_dedup field.

source food_orders qsize=100000
| stats count, list(items) as product by user_id
| eval result_dedup=mvdedup(product)
| table user_id, product, result_dedup