dedup
Description
Keeps only unique records in the results based on specified fields.
Syntax
| dedup [<int>] <field-list> [sortby <sort-by-clause>]
Required Arguments
| Parameter | Syntax | Description |
|---|---|---|
<field-list> | <field> [, <field>] | List of fields by which deduplication should occur. |
Optional Arguments
| Parameter | Syntax | Default | Description |
|---|---|---|---|
maxnum | <int> | No limit | Maximum number of combinations for deduplication. |
[sortby <sort-by-clause>] | sortby <global-sort-options> (-|+)<sort-field> [(-|+)<sort-field> ...] | Specifying the sorting description. |
Sorting Options
| Parameter | Syntax | Default | Description |
|---|---|---|---|
<global-sort-options> | +|- | + | Sort + ascending, - descending. |
<sort-field> | <field> | auto(<field>) | str(<field>) | ip(<field>) | num(<field>) | Sorting type description. |
Sorting Types
| Parameter | Syntax | Description | Notes |
|---|---|---|---|
<field> | <field> | Name of the field for sorting. | |
<auto> | auto(<field>) | Automatically determines the sorting method. | |
ip | ip(<field>) | Sorts for IP addresses. | |
num | num(<field>) | Sorts the field as a number. | If the field does not contain a numeric value, the system will return an error. |
str | str(<field>) | Sorts the field as text. | If the field contains another data type (numbers, dates, boolean value), the field values will be converted to text. |
info
The nature of sorting depends on the data type (text—alphabetical order, numbers—ascending/descending, dates—earlier/later, etc.).
Query Examples
Example #1
source radius_logs
| dedup event
Example #2
source radius_logs
| dedup 3 event
Example #3
source radius_logs
| dedup event sortby -event +@timestamp
Example #4
source radius_logs
| dedup event sortby num(event)