Overview

This is the documentation of ECS version 1.7.0.

What is ECS?

The Elastic Common Schema (ECS) is an open source specification, developed with support from the Elastic user community. ECS defines a common set of fields to be used when storing event data in Elasticsearch, such as logs and metrics.

ECS specifies field names and Elasticsearch datatypes for each field, and provides descriptions and example usage. ECS also groups fields into ECS levels, which are used to signal how much a field is expected to be present. You can learn more about ECS levels in [ecs-guidelines]. Finally, ECS also provides a set of naming guidelines for adding custom fields.

The goal of ECS is to enable and encourage users of Elasticsearch to normalize their event data, so that they can better analyze, visualize, and correlate the data represented in their events. ECS has been scoped to accommodate a wide variety of events, spanning:

  • Event sources: whether the source of your event is an Elastic product, a third- party product, or a custom application built by your organization.

  • Ingestion architectures: whether the ingestion path for your events includes Beats processors, Logstash, Elasticsearch ingest node, all of the above, or none of the above.

  • Consumers: whether consumed by API, Kibana queries, dashboards, apps, or other means.

New to ECS?

If you’re new to ECS and looking for an introduction to its benefits and examples of the core concepts, [ecs-getting-started] is a great place to start.

My events don’t map with ECS

ECS is a permissive schema. If your events have additional data that cannot be mapped to ECS, you can simply add them to your events, using custom field names.

Maturity

ECS improvements are released following Semantic Versioning. Major ECS releases are planned to be aligned with major Elastic Stack releases.

Any feedback on the general structure, missing fields, or existing fields is appreciated. For contributions please read the Contribution Guidelines.

Using ECS

ECS fields follow a series of guidelines, to ensure a consistent and predictable feel, across various use cases.

If you’re new to ECS and would like an introduction on implementing and using the schema, check out the [ecs-getting-started] guide.

Whether you’re trying to recall a field name, implementing a solution that follows ECS, or proposing a change to the schema, the [ecs-guidelines] and [ecs-conventions] will help get you there.

If you’re wondering how to best capture event details that don’t map to existing ECS fields, head over to [ecs-custom-fields-in-ecs].

Mapping network events provides a detailed walk-through of how to best map and categorize an example network event to the schema.

Getting Started

ECS enables and encourages users to normalize event data in order to better analyze, visualize, and correlate their events. Collected events can be normalized at ingest time, consistently searched across indices, and visualized predictably.

Note that when adopting an Elastic solution, such as Observability or Security, all events will map to ECS out of the box. Elastic provides an extensive set of integrations to simplify ingesting your data sources.

If you rely on custom data pipelines and/or building content around specific needs, ECS can still help to alleviate the challenges of searching, analyzing, and visualizing across your data. Let’s see how using a common schema can simplify the search experience, and then take a look at how an event’s contents can be mapped to ECS field sets.

With ECS defining a normalized schema across all of your data sources, querying against those sources is simplified. Consider searching for a particular source IP address prior to adopting ECS. All the various data sources and their field mappings would need to be considered in your query:

src:10.42.42.42 OR client_ip:10.42.42.42 OR apache.access.remote_ip:10.42.42.42 OR
context.user.ip:10.42.42.42 OR src_ip:10.42.42.42

With all sources mapped to ECS, the query becomes much simpler:

source.ip:10.42.42.42

Not only does this simplify writing queries, but saved queries shared with other users become much more obvious. To gain familiarity with ECS fields, you can also take a look at the [ecs-field-reference] section.

Unified Visualizations

With normalized data from different data sources, building insightful visualizations across sources is simple. From a single, centralized dashboard, events from web servers, IDS/IPS devices, and firewalls can be aggregated and visualized, and enhanced with drill-downs, and pivoting for delving into deeper investigations. Centralized monitoring of diverse data sources is straightforward with normalized ECS data.

Simplify visualization using ECS

Translating Data Sources

To align events to ECS, some sort of parsing will usually be necessary to transform the contents of the original event into the relevant ECS fields. Depending on how you’ve designed your Elastic Stack data ingestion pipelines, the amount of work to parse your events will vary.

For example, an Apache web server log event:

10.42.42.42 - - [15/Jul/2020:20:48:32 +0000] "GET /content HTTP/1.1" 200 2571 "-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/83.0.4103.106 Safari/537.36"

In order to map this event to ECS, the contents of the event is associated with the appropriate ECS fields.

Field Name Value

@timestamp

2020-07-15T20:20:48.000Z

event.original

10.42.42.42 - - [15/Jul/2020:20:48:32 +0000] "GET /content HTTP/1.1" 200 2571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36

http.request.method

GET

http.response.body.bytes

2571

http.response.status_code

200

http.version

1.1

message

GET /content HTTP/1.1" 200 2571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36

source.address

10.42.42.42

source.ip

10.42.42.42

url.original

/content

user_agent.original

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36

Beyond extracting values that are present in the original event, we also populate other fields to provide additional context about the event itself.

  • ecs.version: States which version of ECS the ingest pipeline was developed against.

  • event.dataset and event.module: Answers "where is this event from" and are expected to have a hardcoded value per pipeline, per source.

  • event.kind, event.category, event.type, and event.outcome: The [ecs-category-field-values-reference] should also be hardcoded using knowledge of each type of event the source emits. The contents of these fields are limited to the specifically allowed values detailed in the ECS documentation.

Field Name Value

ecs.version

1.5.0

event.module

apache

event.dataset

apache.access

event.kind

event

event.category

[ "network", "web" ]

event.type

[ "access" ]

event.outcome

success

Lastly, existing field values can be interpreted or enriched using a processor, with the results populating additional fields in the final event.

  • The user_agent processor extracts details from the original user agent string, user_agent.original.

  • IP fields like source.ip can provide enrichment using the geoip processor to add information about the location and autonomous system number (ASN) associated with an IP address.

  • The registered domain processor reads a field containing a hostname and writes the registered domain to another field

  • each event with metadata from the machine’s hosting provider (cloud) and/or from the host machine (host).

Here are some examples of additional fields processed by metadata or parser processors.

Field Name Value Processor

host.architecture

x86_64

add_host_metadata

host.hostname

mbp.example.com

add_host_metadata

host.ip

[ "192.168.1.100" ]

add_host_metadata

host.os.family

darwin

add_host_metadata

host.os.kernel

19.4.0

add_host_metadata

host.os.name

Mac OS X

add_host_metadata

host.os.version

10.15.4

add_host_metadata

user_agent.name

Chrome

user_agent

user_agent.os.full

Mac OS X 10.15.4

user_agent

user_agent.os.name

Mac OS X

user_agent

user_agent.os.version

10.15.4

user_agent

user_agent.version

83.0.4103.106

user_agent

Field Mapping Reference Guides

We’ve covered at a high level how to map your events to ECS. Now if you’d like your events to render well in the Elastic solutions, check out the reference guides below to learn more about each:

Guidelines and Best Practices

The ECS schema serves best when you follow schema guidelines and best practices.

ECS Field Levels

ECS defines "Core" and "Extended" fields.

  • Core fields. Fields that are most common across all use cases are defined as core fields.

    These generalized fields are used by analysis content (searches, visualizations, dashboards, alerts, machine learning jobs, reports) across use cases. Analysis content designed to operate on these fields should work properly on data from any relevant source.

    Focus on populating these fields first.

  • Extended fields. Any field that is not a core field is defined as an extended field. Extended fields may apply to more narrow use cases, or may be more open to interpretation depending on the use case. Extended fields are more likely to change over time.

Each ECS field in a table is identified as core or extended.

General guidelines

  • The document MUST have the @timestamp field.

  • defined for an ECS field.

  • Use the ecs.version field to define which version of ECS is used.

  • Map as many fields as possible to ECS.

Guidelines for field names
  • Field names must be lower case

  • Combine words using underscore

  • No special characters except underscore

  • Use present tense unless field describes historical information.

  • Use singular and plural names properly to reflect the field content.

    • For example, use requests_per_sec rather than request_per_sec.

  • Use prefixes for all fields, except for the base fields.

    • For example, all host fields are prefixed with host.. Such a grouping is called a field set.

  • Nest fields inside a field set with dots

    • The document structure should be nested JSON objects. If you use Beats or Logstash, the nesting of JSON objects is done for you automatically. If you’re ingesting to Elasticsearch using the API, your fields must be nested objects, not strings containing dots.

    • See [dot-notation] for more details.

  • General to specific. Organise the nesting of field sets from general to specific, to allow grouping fields into objects with a prefix like host.*.

  • Avoid repetition or stuttering of words

    • If part of the field name is already in the name of the field set, avoid repeating it. Example: host.host_ip should be host.ip.

    • Exceptions can be made, when changing the name of the field would break a strong convention in the community. Example: host.hostname is an exception to this rule.

  • Avoid abbreviations when possible

    • Exceptions can be made, when the name used for the concept is too strongly in favor of the abbreviation. Example: ip fields, or field sets such as os, geo.

Conventions

The implementation of ECS follows a few conventions. Understanding them will help you understand ECS better.

Datatype for integers

Unless otherwise noted, the datatype used for integer fields should be long.

IDs and most codes are keywords, not integers

Despite the fact that IDs and codes (such as error codes) are often integers, this is not always the case. Since we want to make it possible to map as many systems and data sources to ECS as possible, we default to using the keyword type for IDs and codes.

Some specific kinds of codes are always integers, like HTTP status codes. If those have a specific corresponding specific field (as HTTP status does), its type can safely be an integer type. But generic fields like error.code cannot have this guarantee, and are therefore keyword.

Text indexing and multi-fields

Elasticsearch can index text using datatypes:

  • text Text indexing allows for full text search, or searching arbitrary words that are part of the field.

  • keyword Keyword indexing offers faster exact match filtering, prefix search (like autocomplete),

Default Elasticsearch convention for indexing text fields

Unless your index mapping or index template specifies otherwise (as the ECS index template does), Elasticsearch indexes a text field as text at the canonical field name, and indexes a second time as keyword, nested in a multi-field.

Default Elasticsearch convention:

  • Canonical field: myfield is text

  • Multi-field: myfield.keyword is keyword

ECS convention for indexing text fields

ECS flips the above convention around.

For monitoring use cases, keyword indexing is needed almost exclusively, with full text search needed on very few fields. Moreover, indexing for full text search on lots of fields, where it’s not expected to be used is wasteful of resources.

Given these two premises, ECS defaults all text indexing to keyword datatype (with very few exceptions). Any use case that requires full text search indexing on additional fields for full text search. Doing so does not conflict with ECS, as the canonical field name will remain keyword indexed.

So the ECS multi-field convention for text is:

  • Canonical field: myfield is keyword

  • Multi-field: myfield.text is text

Exceptions

The only exceptions to this convention are fields message and error.message, which are indexed for full text search only, with no multi-field. These two fields don’t follow the new convention because they are deemed too big of a breaking change with these two widely used fields in Beats.

Any future field that will be indexed for full text search in ECS will however follow the multi-field convention where text indexing is nested in the multi-field.

Custom Fields

ECS defines fields, their datatypes and their usage, and classifies them in "core" and "extended" levels.

However, ECS does not define anything about custom fields. By definition, they are additional fields, exactly as the user or the integration defines them, independently of ECS.

Users and integrations are welcome to capture additional information in their events, as custom fields. This flexibility is by design, and ensures that no one is ever blocked by something not being supported by ECS yet.

ECS is under active development, however. Adding custom fields carries a small risk of conflicting with a future ECS field. There are ways of modeling custom fields that will lead to lower chances of conflict with future versions of ECS. This section outlines a few of these strategies.

Modeling to Reduce Chances of Conflict

The labels Field

Any time a data source has a few extra fields that can be modeled with the keyword data type, the simplest way to capture them is with the ECS field labels.

Example:

{ "labels": { "foo_id": "beef42", "env": "production" },
  "message": "...",
  "event": { ... }
}

If labels doesn’t work for your use case, here’s a few more tips to avoid conflicts.

Proper Names

ECS tries to model information by using the name of concepts, and avoids proper names such as tool names, project names or company names. By extension, nesting custom fields under a proper name is a relatively safe approach to adding custom fields. This is the approach taken by Filebeats modules, for example.

As an example, an HTTP log from HAProxy will contain typical HTTP information, as well as proxy details and statistics. The standard HTTP information can be captured in the ECS field sets http and url, and the extra details in a custom haproxy section:

{ "http": { "request": { "method": "get", ... },
            "response": { "status_code": 200, ... } },
  "url": { "original": "/favicon.ico", ... },
  "haproxy": { "frontend_name": "myfrontend", "backend_name": "mybackend_prod",
               "backend_queue": 0, ... }
}
Capitalization

ECS strives for a consistent feel by using nesting to group related concepts, and underscores to join words. Following these guidelines for custom fields ensures you preserve the same consistent experience throughout your schemas.

Note however, that breaking away from these guidelines for your custom fields can be used to your advantage. It can be a good way to differentiate between ECS fields and custom fields. Since ECS doesn’t use capitalization for field names, this approach virtually guarantees that custom fields will not conflict with future ECS fields.

Common proxy concepts could modelled via a capitalized, but generic concept name.

HAProxy example:

{ "http": { "request": { "method": "get", ... } },
  "url": { "original": "/favicon.ico", ... },
  "Proxy": { "FrontendName": "myfrontend", "BackendName": "mybackend_prod" },
  "event": { "module": "haproxy" }
}

NGINX example:

{ "http": { "request": { "method": "get", ... } },
  "url": { "original": "/favicon.ico", ... },
  "Proxy": { "FrontendName": "another_frontend", "BackendName": "another_backend_prod" },
  "event": { "module": "nginx" }
}

The above demonstrates that using a common concept name in custom fields can still be beneficial to correlate among multiple sources that populate them. Using capitalization ensures a future version of ECS that defines a proxy field set will not conflict.

Here’s a sample event, during a migration from the custom field, to using a new equivalent ECS field set:

{ "http": { "request": { "method": "get", ... } },
  "Proxy": { "FrontendName": "myfrontend", "BackendName": "mybackend_prod" },
  "proxy": { "frontend_name": "myfrontend", "backend_name": "mybackend_prod" }
}

The above will look strange during the migration. However the ability to start populating ECS fields while custom fields are still present in your events makes it possible to decouple the upgrade to a new version of ECS from the time you adjust your pipelines and analysis content.

Mapping Network Events

Network events capture the details of one device communicating with another. The initiator is referred to as the source, and the recipient as the destination. Depending on the data source, a network event can contain details of addresses, protocols, headers, and device roles.

This guide describes the different field sets available for network-related events in ECS and provides direction on the ECS best practices for mapping to them.

Source and destination baseline

When an event contains details about the sending and receiving hosts, the baseline for capturing these values will be the source and destination fields.

Some events may also indicate each host’s role in the exchange: client or server. When this information is available, the client and server fields should be used in addition to the source and destination fields. The fields and values mapped under source/destination should be copied under client/server.

Network event mapping example

Below is a DNS network event. The source device (192.168.86.222) makes a DNS query, acting as the client and the DNS server is the destination (192.168.86.1).

Note this event contains additional details that would populate additional fields (such as the [ecs-dns]) if this was a complete mapping example. These additional fields are omitted here to focus on the network details.

{
  "ts":1599775747.53056,
  "uid":"CYqFPH3nOAa0kPxA0d",
  "id.orig_h":"192.168.86.222",
  "id.orig_p":54162,
  "id.resp_h":"192.168.86.1",
  "id.resp_p":53,
  "proto":"udp",
  "trans_id":28899,
  "rtt":0.02272200584411621,
  "query":"example.com",
  "qclass":1,
  "qclass_name":"C_INTERNET",
  "qtype":1,
  "qtype_name":"A",
  "rcode":0,
  "rcode_name":"NOERROR",
  "AA":false,
  "TC":false,
  "RD":true,
  "RA":true,
  "Z":0,
  "answers":["93.184.216.34"],
  "TTLs":[21209.0],
  "rejected":false
}

Source and destination fields

First, the source.* and destination.* field sets are populated:

  "source": {
    "ip": "192.168.86.222",
    "port": 54162
  }
  "destination": {
    "ip": "192.168.86.1",
    "port": 53
  }

Client and server fields

Looking back at the original event, it shows the source device is the DNS client and the destination device is the DNS server. The values mapped under source and destination are copied and mapped under client and server, respectively:

  "client": {
    "ip": "192.168.86.222",
    "port": 64734
  }
  "server": {
    "ip": "192.168.86.1",
    "port": 53
  }

Mapping both pairs of field sets gives query visibility of the same network transaction in two ways.

  • source.ip:192.168.86.222 returns all events sourced from 192.168.86.222, regardless its role in a transaction

  • client.ip:192.168.86.222 returns all events with host 192.168.86.222 acting as a client

The same applies for the destination and server fields:

  • destination.ip:192.168.86.1 returns all events destined to 192.168.86.1

  • server.ip:192.168.86.1 returns all events with 192.168.86.1 acting as the server

It’s important to note that while the values for the source and destination fields may reverse between events in a single network transaction, the values for client and server typically will not. The following two tables demonstrate how two DNS transactions involving two clients and one server would map to source.ip/destination.ip vs. client.ip/server.ip:

Table 1. Source/Destination
source.ip destination.ip event

192.168.86.222

192.168.86.1

DNS query request 1

192.168.86.1

192.168.86.222

DNS answer response 1

192.168.86.42

192.168.86.1

DNS answer request 2

192.168.86.1

192.168.86.42

DNS answer request 2

Table 2. Client/Server
client.ip server.ip event

192.168.86.222

192.168.86.1

DNS query request 1

192.168.86.222

192.168.86.1

DNS answer response 1

192.168.86.42

192.168.86.1

DNS query request 2

192.168.86.42

192.168.86.1

DNS answer response 2

The related.ip field captures all the IPs present in the event in a single array:

  "related": {
    "ip": [
      "192.168.86.222",
      "192.168.86.1",
      "93.184.216.34"
    ]
  }

The related fields are meant to facilitate pivoting. Since these IP addresses can appear in many different fields (source.ip, destination.ip, client.ip, server.ip, etc.), you can search for the IP trivially no matter what field it appears using a single query, e.g. related.ip:192.168.86.222.

Network events are not only limited to using related.ip. If hostnames or other host identifiers were present in the event, related.hosts should be populated too.

Categorization using event fields

When considering the event categorization fields, the category and type fields are populated using their respective allowed values which best classify the source network event.

  "event": {
    "category": [
      "network"
    ],
    "type": [
      "connection",
      "protocol"
    ],
    "kind": "event"
  }

Most event.category/event.type ECS pairings are complete on their own. However, the pairing of event.category:network and event.type:protocol is an exception. When these two fields/value pairs both used to categorize an event, the network.protocol field should also be populated:

  "network": {
    "protocol": "dns",
    "type": "ipv4",
    "transport": "udp"
  }

Result

Putting everything together covered so far, we have a final ECS-mapped event:

{
  "event": {
    "category": [
      "network"
    ],
    "type": [
      "connection",
      "protocol"
    ],
    "kind": "event"
  },
  "network": {
    "protocol": "dns",
    "type": "ipv4",
    "transport": "udp"
  },
  "source": {
    "ip": "192.168.86.222",
    "port": 54162
  },
  "destination": {
    "ip": "192.168.86.1",
    "port": 53
  },
  "client": {
    "ip": "192.168.86.222",
    "port": 64734
  },
  "server": {
    "ip": "192.168.86.1",
    "port": 53
  },
  "related": {
    "ip": [
      "192.168.86.222",
      "192.168.86.1",
      "93.184.216.34"
    ]
  },
  "dns": { ... }, <= Again, not diving into the DNS fields here but included for completeness.
  "zeek": { "ts":1599775747.53056, ... } <= Original fields can optionally be kept around as custom fields.
}

ECS Field Reference

This is the documentation of ECS version 1.7.0-1.

ECS defines multiple groups of related fields. They are called "field sets". The Base field set is the only one whose fields are defined at the root of the event.

all fields are defined.

Field Sets

Field Set Description

Base

All fields defined directly at the root of the events.

Agent

Fields about the monitoring agent.

Autonomous System

Fields describing an Autonomous System (Internet routing prefix).

Client

Fields about the client side of a network connection, used with server.

Cloud

Fields about the cloud resource.

Code Signature

These fields contain information about binary code signatures.

Container

Fields describing the container that generated this event.

Destination

Fields about the destination side of a network connection, used with source.

DLL

These fields contain information about code libraries dynamically loaded into processes.

DNS

Fields describing DNS queries and answers.

ECS

Meta-information specific to ECS.

Error

Fields about errors of any kind.

Event

Fields breaking down the event details.

File

Fields describing files.

Geo

Fields describing a location.

Group

User’s group relevant to the event.

Hash

Hashes, usually file hashes.

Host

Fields describing the relevant computing instance.

HTTP

Fields describing an HTTP request.

Interface

Fields to describe observer interface information.

Log

Details about the event’s logging mechanism.

Network

Fields describing the communication path over which the event happened.

Observer

Fields describing an entity observing the event from outside the host.

Organization

Fields describing the organization or company the event is associated with.

Operating System

OS fields contain information about the operating system.

Package

These fields contain information about an installed software package.

PE Header

These fields contain Windows Portable Executable (PE) metadata.

Process

These fields contain information about a process.

Registry

Fields related to Windows Registry operations.

Related

Fields meant to facilitate pivoting around a piece of data.

Rule

Fields to capture details about rules used to generate alerts or other notable events.

Server

Fields about the server side of a network connection, used with client.

Service

Fields describing the service for or from which the data was collected.

Source

Fields about the source side of a network connection, used with destination.

Threat

Fields to classify events and alerts according to a threat taxonomy.

TLS

Fields describing a TLS connection.

Tracing

Fields related to distributed tracing.

URL

Fields that let you store URLs in various forms.

User

Fields to describe the user relevant to the event.

User agent

Fields to describe a browser user_agent string.

VLAN

Fields to describe observed VLAN information.

Vulnerability

Fields to describe the vulnerability relevant to an event.

x509 Certificate

These fields contain x509 certificate metadata.

Base Fields

The base field set contains all fields which are at the root of the events. These fields are common across all types of events.

Field Description Level

@timestamp

Date/time when the event originated.

This is the date/time extracted from the event, typically representing when the event was generated by the source.

If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline.

Required field for all events.

type: date

example: 2016-05-23T08:05:34.853Z

core

labels

Custom key/value pairs.

Can be used to add meta information to events. Should not contain nested objects. All values are stored as keyword.

Example: docker and k8s labels.

type: object

example: {"application": "foo-bar", "env": "production"}

core

message

For log events the message field contains the log message, optimized for viewing in a log viewer.

For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event.

If multiple messages exist, they can be combined into one message.

type: text

example: Hello World

core

tags

List of keywords used to tag each event.

type: keyword

Note: this field should contain an array of values.

example: ["production", "env2"]

core

Agent Fields

The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host.

Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken.

Field Description Level

agent.build.original

Extended build information for the agent.

This field is intended to contain any build information that a data source may provide, no specific formatting is required.

type: keyword

example: metricbeat version 7.6.0 (amd64), libbeat 7.6.0 [6a23e8f8f30f5001ba344e4e54d8d9cb82cb107c built 2020-02-05 23:10:10 +0000 UTC]

core

agent.ephemeral_id

Ephemeral identifier of this agent (if one exists).

This id normally changes across restarts, but agent.id does not.

type: keyword

example: 8a4f500f

extended

agent.id

Unique identifier of this agent (if one exists).

Example: For Beats this would be beat.id.

type: keyword

example: 8a4f500d

core

agent.name

Custom name of the agent.

This is a name that can be given to an agent. This can be helpful if for example two Filebeat instances are running on the same host but a human readable separation is needed on which Filebeat instance data is coming from.

If no name is given, the name is often left empty.

type: keyword

example: foo

core

agent.type

Type of the agent.

The agent type always stays the same and should be given by the agent used. In case of Filebeat the agent would always be Filebeat also if two Filebeat instances are run on the same machine.

type: keyword

example: filebeat

core

agent.version

Version of the agent.

type: keyword

example: 6.0.0-rc2

core

Autonomous System Fields

An autonomous system (AS) is a collection of connected Internet Protocol (IP) routing prefixes under the control of one or more network operators on behalf of a single administrative entity or domain that presents a common, clearly defined routing policy to the internet.

Field Description Level

as.number

Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.

type: long

example: 15169

extended

as.organization.name

Organization name.

type: keyword

Multi-fields:

* as.organization.name.text (type: text)

example: Google LLC

extended

The as fields are expected to be nested at: client.as, destination.as, server.as, source.as.

Note also that the as fields are not expected to be used directly at the root of the events.

Client Fields

A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records.

For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events.

Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately.

Field Description Level

client.address

Some event client addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the .address field.

Then it should be duplicated to .ip or .domain, depending on which one it is.

type: keyword

extended

client.bytes

Bytes sent from the client to the server.

type: long

example: 184

core

client.domain

Client domain.

type: keyword

core

client.ip

IP address of the client (IPv4 or IPv6).

type: ip

core

client.mac

MAC address of the client.

type: keyword

core

client.nat.ip

Translated IP of source based NAT sessions (e.g. internal client to internet).

Typically connections traversing load balancers, firewalls, or routers.

type: ip

extended

client.nat.port

Translated port of source based NAT sessions (e.g. internal client to internet).

Typically connections traversing load balancers, firewalls, or routers.

type: long

extended

client.packets

Packets sent from the client to the server.

type: long

example: 12

core

client.port

Port of the client.

type: long

core

client.registered_domain

The highest registered client domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: example.com

extended

client.subdomain

The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: east

extended

client.top_level_domain

The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: co.uk

extended

Nested fields Description

client.as.*

Fields describing an Autonomous System (Internet routing prefix).

client.geo.*

Fields describing a location.

client.user.*

Fields to describe the user relevant to the event.

Cloud Fields

Fields related to the cloud or infrastructure the events are coming from.

Field Description Level

cloud.account.id

The cloud account or organization id used to identify different entities in a multi-tenant environment.

Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.

type: keyword

example: 666777888999

extended

cloud.account.name

The cloud account name or alias used to identify different entities in a multi-tenant environment.

Examples: AWS account name, Google Cloud ORG display name.

type: keyword

example: elastic-dev

extended

cloud.availability_zone

Availability zone in which this host is running.

type: keyword

example: us-east-1c

extended

cloud.instance.id

Instance ID of the host machine.

type: keyword

example: i-1234567890abcdef0

extended

cloud.instance.name

Instance name of the host machine.

type: keyword

extended

cloud.machine.type

Machine type of the host machine.

type: keyword

example: t2.medium

extended

cloud.project.id

The cloud project identifier.

Examples: Google Cloud Project id, Azure Project id.

type: keyword

example: my-project

extended

cloud.project.name

The cloud project name.

Examples: Google Cloud Project name, Azure Project name.

type: keyword

example: my project

extended

cloud.provider

Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.

type: keyword

example: aws

extended

cloud.region

Region in which this host is running.

type: keyword

example: us-east-1

extended

Code Signature Fields

These fields contain information about binary code signatures.

Field Description Level

code_signature.exists

Boolean to capture if a signature is present.

type: boolean

example: true

core

code_signature.status

Additional information about the certificate status.

This is useful for logging cryptographic errors with the certificate validity or trust status. Leave unpopulated if the validity or trust of the certificate was unchecked.

type: keyword

example: ERROR_UNTRUSTED_ROOT

extended

code_signature.subject_name

Subject name of the code signer

type: keyword

example: Microsoft Corporation

core

code_signature.trusted

Stores the trust status of the certificate chain.

Validating the trust of the certificate chain may be complicated, and this field should only be populated by tools that actively check the status.

type: boolean

example: true

extended

code_signature.valid

Boolean to capture if the digital signature is verified against the binary content.

Leave unpopulated if a certificate was unchecked.

type: boolean

example: true

extended

The code_signature fields are expected to be nested at: dll.code_signature, file.code_signature, process.code_signature.

Note also that the code_signature fields are not expected to be used directly at the root of the events.

Container Fields

Container fields are used for meta information about the specific container that is the source of information.

These fields help correlate data based containers from any runtime.

Field Description Level

container.id

Unique container id.

type: keyword

core

container.image.name

Name of the image the container was built on.

type: keyword

extended

container.image.tag

Container image tags.

type: keyword

Note: this field should contain an array of values.

extended

container.labels

Image labels.

type: object

extended

container.name

Container name.

type: keyword

extended

container.runtime

Runtime managing this container.

type: keyword

example: docker

extended

Destination Fields

Destination fields capture details about the receiver of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction.

Destination fields are usually populated in conjunction with source fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated.

Field Description Level

destination.address

Some event destination addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the .address field.

Then it should be duplicated to .ip or .domain, depending on which one it is.

type: keyword

extended

destination.bytes

Bytes sent from the destination to the source.

type: long

example: 184

core

destination.domain

Destination domain.

type: keyword

core

destination.ip

IP address of the destination (IPv4 or IPv6).

type: ip

core

destination.mac

MAC address of the destination.

type: keyword

core

destination.nat.ip

Translated ip of destination based NAT sessions (e.g. internet to private DMZ)

Typically used with load balancers, firewalls, or routers.

type: ip

extended

destination.nat.port

Port the source session is translated to by NAT Device.

Typically used with load balancers, firewalls, or routers.

type: long

extended

destination.packets

Packets sent from the destination to the source.

type: long

example: 12

core

destination.port

Port of the destination.

type: long

core

destination.registered_domain

The highest registered destination domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: example.com

extended

destination.subdomain

The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: east

extended

destination.top_level_domain

The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: co.uk

extended

Nested fields Description

destination.as.*

Fields describing an Autonomous System (Internet routing prefix).

destination.geo.*

Fields describing a location.

destination.user.*

Fields to describe the user relevant to the event.

DLL Fields

These fields contain information about code libraries dynamically loaded into processes.

Many operating systems refer to "shared code libraries" with different names, but this field set refers to all of the following:

  • Dynamic-link library (.dll) commonly used on Windows

  • Shared Object (.so) commonly used on Unix-like operating systems

  • Dynamic library (.dylib) commonly used on macOS

Field Description Level

dll.name

Name of the library.

This generally maps to the name of the file on disk.

type: keyword

example: kernel32.dll

core

dll.path

Full file path of the library.

type: keyword

example: C:\Windows\System32\kernel32.dll

extended

Nested fields Description

dll.code_signature.*

These fields contain information about binary code signatures.

dll.hash.*

Hashes, usually file hashes.

dll.pe.*

These fields contain Windows Portable Executable (PE) metadata.

DNS Fields

Fields describing DNS queries and answers.

DNS events should either represent a single DNS query prior to getting answers (dns.type:query) or they should represent a full exchange and contain the query details as well as all of the answers that were provided for this query (dns.type:answer).

Field Description Level

dns.answers

An array containing an object for each answer section returned by the server.

The main keys that should be present in these objects are defined by ECS. Records that have more information may contain more keys than what ECS defines.

Not all DNS data sources give all details about DNS answers. At minimum, answer objects must contain the data key. If more information is available, map as much of it to ECS as possible, and add any additional fields to the answer objects as custom fields.

type: object

Note: this field should contain an array of values.

extended

dns.answers.class

The class of DNS data contained in this resource record.

type: keyword

example: IN

extended

dns.answers.data

The data describing the resource.

The meaning of this data depends on the type and class of the resource record.

type: keyword

example: 10.10.10.10

extended

dns.answers.name

The domain name to which this resource record pertains.

If a chain of CNAME is being resolved, each answer’s name should be the one that corresponds with the answer’s data. It should not simply be the original question.name repeated.

type: keyword

example: www.example.com

extended

dns.answers.ttl

The time interval in seconds that this resource record may be cached before it should be discarded. Zero values mean that the data should not be cached.

type: long

example: 180

extended

dns.answers.type

The type of data contained in this resource record.

type: keyword

example: CNAME

extended

dns.header_flags

Array of 2 letter DNS header flags.

Expected values are: AA, TC, RD, RA, AD, CD, DO.

type: keyword

Note: this field should contain an array of values.

example: ["RD", "RA"]

extended

dns.id

The DNS packet identifier assigned by the program that generated the query. The identifier is copied to the response.

type: keyword

example: 62111

extended

dns.op_code

The DNS operation code that specifies the kind of query in the message. This value is set by the originator of a query and copied into the response.

type: keyword

example: QUERY

extended

dns.question.class

The class of records being queried.

type: keyword

example: IN

extended

dns.question.name

The name being queried.

If the name field contains non-printable characters (below 32 or above 126), those characters should be represented as escaped base 10 integers (\DDD). Back slashes and quotes should be escaped. Tabs, carriage returns, and line feeds should be converted to \t, \r, and \n respectively.

type: keyword

example: www.example.com

extended

dns.question.registered_domain

The highest registered domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: example.com

extended

dns.question.subdomain

The subdomain is all of the labels under the registered_domain.

If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: www

extended

dns.question.top_level_domain

The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: co.uk

extended

dns.question.type

The type of record being queried.

type: keyword

example: AAAA

extended

dns.resolved_ip

Array containing all IPs seen in answers.data.

The answers array can be difficult to use, because of the variety of data formats it can contain. Extracting all IP addresses seen in there to dns.resolved_ip makes it possible to index them as IP addresses, and makes them easier to visualize and query for.

type: ip

Note: this field should contain an array of values.

example: ["10.10.10.10", "10.10.10.11"]

extended

dns.response_code

The DNS response code.

type: keyword

example: NOERROR

extended

dns.type

The type of DNS event captured, query or answer.

If your source of DNS events only gives you DNS queries, you should only create dns events of type dns.type:query.

If your source of DNS events gives you answers as well, you should create one event per query (optionally as soon as the query is seen). And a second event containing all query details as well as an array of answers.

type: keyword

example: answer

extended

ECS Fields

Meta-information specific to ECS.

Field Description Level

ecs.version

ECS version this event conforms to. ecs.version is a required field and must exist in all events.

When querying across multiple indices — which may conform to slightly different ECS versions — this field lets integrations adjust to the schema version of the events.

type: keyword

example: 1.0.0

core

Error Fields

These fields can represent errors of any kind.

Use them for errors that happen while fetching events or in cases where the event itself contains an error.

Field Description Level

error.code

Error code describing the error.

type: keyword

core

error.id

Unique identifier for the error.

type: keyword

core

error.message

Error message.

type: text

core

error.stack_trace

The stack trace of this error in plain text.

type: keyword

Multi-fields:

* error.stack_trace.text (type: text)

extended

error.type

The type of the error, for example the class name of the exception.

type: keyword

example: java.lang.NullPointerException

extended

Event Fields

The event fields are used for context information about the log or metric event itself.

A log is defined as an event containing details of something that happened. Log events must include the time at which the thing happened. Examples of log events include a process starting on a host, a network packet being sent from a source to a destination, or a network connection between a client and a server being initiated or closed. A metric is defined as an event containing one or more numerical measurements and the time at which the measurement was taken. Examples of metric events include memory pressure measured on a host and device temperature. See the event.kind definition in this section for additional details about metric and state events.

Field Description Level

event.action

The action captured by the event.

This describes the information in the event. It is more specific than event.category. Examples are group-add, process-started, file-created. The value is normally defined by the implementer.

type: keyword

example: user-password-change

core

event.category

This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy.

event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory.

This field is an array. This will allow proper categorization of some events that fall in multiple categories.

type: keyword

Note: this field should contain an array of values.

Important: The field value must be one of the following:

authentication, configuration, database, driver, file, host, iam, intrusion_detection, malware, network, package, process, web

To learn more about when to use which value, visit the page allowed values for event.category

core

event.code

Identification code for this event, if one exists.

Some event sources use event codes to identify messages unambiguously, regardless of message language or wording adjustments over time. An example of this is the Windows Event ID.

type: keyword

example: 4648

extended

event.created

event.created contains the date/time when the event was first read by an agent, or by your pipeline.

This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event.

In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent’s or pipeline’s ability to keep up with your event source.

In case the two timestamps are identical, @timestamp should be used.

type: date

example: 2016-05-23T08:05:34.857Z

core

event.dataset

Name of the dataset.

If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from.

It’s recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name.

type: keyword

example: apache.access

core

event.duration

Duration of the event in nanoseconds.

If event.start and event.end are known this value should be the difference between the end and start time.

type: long

core

event.end

event.end contains the date when the event ended or when the activity was last observed.

type: date

extended

event.hash

Hash (perhaps logstash fingerprint) of raw field to be able to demonstrate log integrity.

type: keyword

example: 123456789012345678901234567890ABCD

extended

event.id

Unique ID to describe the event.

type: keyword

example: 8a4f500d

core

event.ingested

Timestamp when an event arrived in the central data store.

This is different from @timestamp, which is when the event originally occurred. It’s also different from event.created, which is meant to capture the first time an agent saw the event.

In normal conditions, assuming no tampering, the timestamps should chronologically look like this: @timestamp < event.created < event.ingested.

type: date

example: 2016-05-23T08:05:35.101Z

core

event.kind

This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy.

event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events.

The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not.

type: keyword

Important: The field value must be one of the following:

alert, event, metric, state, pipeline_error, signal

To learn more about when to use which value, visit the page allowed values for event.kind

core

event.module

Name of the module this data is coming from.

If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), event.module should contain the name of this module.

type: keyword

example: apache

core

event.original

Raw text message of entire event. Used to demonstrate log integrity.

This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from _source.

type: keyword

example: Sep 19 08:26:10 host CEF:0&#124;Security&#124; threatmanager&#124;1.0&#124;100&#124; worm successfully stopped&#124;10&#124;src=10.0.0.1 dst=2.1.2.2spt=1232

core

event.outcome

This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy.

event.outcome simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event.

Note that when a single transaction is described in multiple events, each event may populate different values of event.outcome, according to their perspective.

Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer.

Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with event.type:info, or any events for which an outcome does not make logical sense.

type: keyword

Important: The field value must be one of the following:

failure, success, unknown

To learn more about when to use which value, visit the page allowed values for event.outcome

core

event.provider

Source of the event.

Event transports such as Syslog or the Windows Event Log typically mention the source of an event. It can be the name of the software that generated the event (e.g. Sysmon, httpd), or of a subsystem of the operating system (kernel, Microsoft-Windows-Security-Auditing).

type: keyword

example: kernel

extended

event.reason

Reason why this event happened, according to the source.

This describes the why of a particular action or outcome captured in the event. Where event.action captures the action from the event, event.reason describes why that action was taken. For example, a web proxy with an event.action which denied the request may also populate event.reason with the reason why (e.g. blocked site).

type: keyword

example: Terminated an unexpected process

extended

event.reference

Reference URL linking to additional information about this event.

This URL links to a static definition of the this event. Alert events, indicated by event.kind:alert, are a common use case for this field.

type: keyword

example: https://system.example.com/event/#0001234

extended

event.risk_score

Risk score or priority of the event (e.g. security solutions). Use your system’s original value here.

type: float

core

event.risk_score_norm

Normalized risk score or priority of the event, on a scale of 0 to 100.

This is mainly useful if you use more than one system that assigns risk scores, and you want to see a normalized value across all systems.

type: float

extended

event.sequence

Sequence number of the event.

The sequence number is a value published by some event sources, to make the exact ordering of events unambiguous, regardless of the timestamp precision.

type: long

extended

event.severity

The numeric severity of the event according to your event source.

What the different severity values mean can be different between sources and use cases. It’s up to the implementer to make sure severities are consistent across events from the same source.

The Syslog severity belongs in log.syslog.severity.code. event.severity is meant to represent the severity according to the event source (e.g. firewall, IDS). If the event source does not publish its own severity, you may optionally copy the log.syslog.severity.code to event.severity.

type: long

example: 7

core

event.start

event.start contains the date when the event started or when the activity was first observed.

type: date

extended

event.timezone

This field should be populated when the event’s timestamp does not include timezone information already (e.g. default Syslog timestamps). It’s optional otherwise.

Acceptable timezone formats are: a canonical ID (e.g. "Europe/Amsterdam"), abbreviated (e.g. "EST") or an HH:mm differential (e.g. "-05:00").

type: keyword

extended

event.type

This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy.

event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization.

This field is an array. This will allow proper categorization of some events that fall in multiple event types.

type: keyword

Note: this field should contain an array of values.

Important: The field value must be one of the following:

access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, info, installation, protocol, start, user

To learn more about when to use which value, visit the page allowed values for event.type

core

event.url

URL linking to an external system to continue investigation of this event.

This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by event.kind:alert, are a common use case for this field.

type: keyword

example: https://mysystem.example.com/alert/5271dedb-f5b0-4218-87f0-4ac4870a38fe

extended

File Fields

A file is defined as a set of information that has been created on, or has existed on a filesystem.

File objects can be associated with host events, network events, and/or file events (e.g., those produced by File Integrity Monitoring [FIM] products or services). File fields provide details about the affected file associated with the event or metric.

Field Description Level

file.accessed

Last time the file was accessed.

Note that not all filesystems keep track of access time.

type: date

extended

file.attributes

Array of file attributes.

Attributes names will vary by platform. Here’s a non-exhaustive list of values that are expected in this field: archive, compressed, directory, encrypted, execute, hidden, read, readonly, system, write.

type: keyword

Note: this field should contain an array of values.

example: ["readonly", "system"]

extended

file.created

File creation time.

Note that not all filesystems store the creation time.

type: date

extended

file.ctime

Last time the file attributes or metadata changed.

Note that changes to the file content will update mtime. This implies ctime will be adjusted at the same time, since mtime is an attribute of the file.

type: date

extended

file.device

Device that is the source of the file.

type: keyword

example: sda

extended

file.directory

Directory where the file is located. It should include the drive letter, when appropriate.

type: keyword

example: /home/alice

extended

file.drive_letter

Drive letter where the file is located. This field is only relevant on Windows.

The value should be uppercase, and not include the colon.

type: keyword

example: C

extended

file.extension

File extension, excluding the leading dot.

Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz").

type: keyword

example: png

extended

file.gid

Primary group ID (GID) of the file.

type: keyword

example: 1001

extended

file.group

Primary group name of the file.

type: keyword

example: alice

extended

file.inode

Inode representing the file in the filesystem.

type: keyword

example: 256383

extended

file.mime_type

MIME type should identify the format of the file or stream of bytes using IANA official types, where possible. When more than one type is applicable, the most specific type should be used.

type: keyword

extended

file.mode

Mode of the file in octal representation.

type: keyword

example: 0640

extended

file.mtime

Last time the file content was modified.

type: date

extended

file.name

Name of the file including the extension, without the directory.

type: keyword

example: example.png

extended

file.owner

File owner’s username.

type: keyword

example: alice

extended

file.path

Full path to the file, including the file name. It should include the drive letter, when appropriate.

type: keyword

Multi-fields:

* file.path.text (type: text)

example: /home/alice/example.png

extended

file.size

File size in bytes.

Only relevant when file.type is "file".

type: long

example: 16384

extended

file.target_path

Target path for symlinks.

type: keyword

Multi-fields:

* file.target_path.text (type: text)

extended

file.type

File type (file, dir, or symlink).

type: keyword

example: file

extended

file.uid

The user ID (UID) or security identifier (SID) of the file owner.

type: keyword

example: 1001

extended

Nested fields Description

file.code_signature.*

These fields contain information about binary code signatures.

file.hash.*

Hashes, usually file hashes.

file.pe.*

These fields contain Windows Portable Executable (PE) metadata.

file.x509.*

These fields contain x509 certificate metadata.

Geo Fields

Geo fields can carry data about a specific location related to an event.

This geolocation information can be derived from techniques such as Geo IP, or be user-supplied.

Field Description Level

geo.city_name

City name.

type: keyword

example: Montreal

core

geo.continent_name

Name of the continent.

type: keyword

example: North America

core

geo.country_iso_code

Country ISO code.

type: keyword

example: CA

core

geo.country_name

Country name.

type: keyword

example: Canada

core

geo.location

Longitude and latitude.

type: geo_point

example: { "lon": -73.614830, "lat": 45.505918 }

core

geo.name

User-defined description of a location, at the level of granularity they care about.

Could be the name of their data centers, the floor number, if this describes a local physical entity, city names.

Not typically used in automated geolocation.

type: keyword

example: boston-dc

extended

geo.region_iso_code

Region ISO code.

type: keyword

example: CA-QC

core

geo.region_name

Region name.

type: keyword

example: Quebec

core

The geo fields are expected to be nested at: client.geo, destination.geo, host.geo, observer.geo, server.geo, source.geo.

Note also that the geo fields are not expected to be used directly at the root of the events.

Group Fields

The group fields are meant to represent groups that are relevant to the event.

Field Description Level

group.domain

Name of the directory the group is a member of.

For example, an LDAP or Active Directory domain name.

type: keyword

extended

group.id

Unique identifier for the group on the system/platform.

type: keyword

extended

group.name

Name of the group.

type: keyword

extended

The group fields are expected to be nested at: user.group.

Note also that the group fields may be used directly at the root of the events.

Hash Fields

The hash fields represent different hash algorithms and their values.

Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for other hashes by lowercasing the hash algorithm name and using underscore separators as appropriate (snake case, e.g. sha3_512).

Field Description Level

hash.md5

MD5 hash.

type: keyword

extended

hash.sha1

SHA1 hash.

type: keyword

extended

hash.sha256

SHA256 hash.

type: keyword

extended

hash.sha512

SHA512 hash.

type: keyword

extended

The hash fields are expected to be nested at: dll.hash, file.hash, process.hash.

Note also that the hash fields are not expected to be used directly at the root of the events.

Host Fields

A host is defined as a general computing instance.

ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.

Field Description Level

host.architecture

Operating system architecture.

type: keyword

example: x86_64

core

host.domain

Name of the domain of which the host is a member.

For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.

type: keyword

example: CONTOSO

extended

host.hostname

Hostname of the host.

It normally contains what the hostname command returns on the host machine.

type: keyword

core

host.id

Unique host id.

As hostname is not always unique, use values that are meaningful in your environment.

Example: The current usage of beat.name.

type: keyword

core

host.ip

Host ip addresses.

type: ip

Note: this field should contain an array of values.

core

host.mac

Host mac addresses.

type: keyword

Note: this field should contain an array of values.

core

host.name

Name of the host.

It can contain what hostname returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.

type: keyword

core

host.type

Type of host.

For Cloud providers this can be the machine type like t2.medium. If vm, this could be the container, for example, or other information meaningful in your environment.

type: keyword

core

host.uptime

Seconds the host has been up.

type: long

example: 1325

extended

Nested fields Description

host.geo.*

Fields describing a location.

host.os.*

OS fields contain information about the operating system.

host.user.*

Fields to describe the user relevant to the event.

HTTP Fields

Fields related to HTTP activity. Use the url field set to store the url of the request.

Field Description Level

http.request.body.bytes

Size in bytes of the request body.

type: long

example: 887

extended

http.request.body.content

The full HTTP request body.

type: keyword

Multi-fields:

* http.request.body.content.text (type: text)

example: Hello world

extended

http.request.bytes

Total size in bytes of the request (body and headers).

type: long

example: 1437

extended

http.request.method

HTTP request method.

Prior to ECS 1.6.0 the following guidance was provided:

"The field value must be normalized to lowercase for querying."

As of ECS 1.6.0, the guidance is deprecated because the original case of the method may be useful in anomaly detection. Original case will be mandated in ECS 2.0.0

type: keyword

example: GET, POST, PUT, PoST

extended

http.request.mime_type

Mime type of the body of the request.

This value must only be populated based on the content of the request body, not on the Content-Type header. Comparing the mime type of a request with the request’s Content-Type header can be helpful in detecting threats or misconfigured clients.

type: keyword

example: image/gif

extended

http.request.referrer

Referrer for this HTTP request.

type: keyword

example: https://blog.example.com/

extended

http.response.body.bytes

Size in bytes of the response body.

type: long

example: 887

extended

http.response.body.content

The full HTTP response body.

type: keyword

Multi-fields:

* http.response.body.content.text (type: text)

example: Hello world

extended

http.response.bytes

Total size in bytes of the response (body and headers).

type: long

example: 1437

extended

http.response.mime_type

Mime type of the body of the response.

This value must only be populated based on the content of the response body, not on the Content-Type header. Comparing the mime type of a response with the response’s Content-Type header can be helpful in detecting misconfigured servers.

type: keyword

example: image/gif

extended

http.response.status_code

HTTP response status code.

type: long

example: 404

extended

http.version

HTTP version.

type: keyword

example: 1.1

extended

Interface Fields

The interface fields are used to record ingress and egress interface information when reported by an observer (e.g. firewall, router, load balancer) in the context of the observer handling a network connection. In the case of a single observer interface (e.g. network sensor on a span port) only the observer.ingress information should be populated.

Field Description Level

interface.alias

Interface alias as reported by the system, typically used in firewall implementations for e.g. inside, outside, or dmz logical interface naming.

type: keyword

example: outside

extended

interface.id

Interface ID as reported by an observer (typically SNMP interface ID).

type: keyword

example: 10

extended

interface.name

Interface name as reported by the system.

type: keyword

example: eth0

extended

The interface fields are expected to be nested at: observer.egress.interface, observer.ingress.interface.

Note also that the interface fields are not expected to be used directly at the root of the events.

Log Fields

Details about the event’s logging mechanism or logging transport.

The log.* fields are typically populated with details about the logging mechanism used to create and/or transport the event. For example, syslog details belong under log.syslog.*.

The details specific to your event source are typically not logged under log.*, but rather in event.* or in other ECS fields.

Field Description Level

log.file.path

Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate.

If the event wasn’t read from a log file, do not populate this field.

type: keyword

example: /var/log/fun-times.log

extended

log.level

Original log level of the log event.

If the source of the event provides a log level or textual severity, this is the one that goes in log.level. If your source doesn’t specify one, you may put your event transport’s severity here (e.g. Syslog severity).

Some examples are warn, err, i, informational.

type: keyword

example: error

core

log.logger

The name of the logger inside an application. This is usually the name of the class which initialized the logger, or can be a custom name.

type: keyword

example: org.elasticsearch.bootstrap.Bootstrap

core

log.origin.file.line

The line number of the file containing the source code which originated the log event.

type: integer

example: 42

extended

log.origin.file.name

The name of the file containing the source code which originated the log event.

Note that this field is not meant to capture the log file. The correct field to capture the log file is log.file.path.

type: keyword

example: Bootstrap.java

extended

log.origin.function

The name of the function or method which originated the log event.

type: keyword

example: init

extended

log.original

This is the original log message and contains the full log message before splitting it up in multiple parts.

In contrast to the message field which can contain an extracted part of the log message, this field contains the original, full log message. It can have already some modifications applied like encoding or new lines removed to clean up the log message.

This field is not indexed and doc_values are disabled so it can’t be queried but the value can be retrieved from _source.

type: keyword

example: Sep 19 08:26:10 localhost My log

core

log.syslog

The Syslog metadata of the event, if the event was transmitted via Syslog. Please see RFCs 5424 or 3164.

type: object

extended

log.syslog.facility.code

The Syslog numeric facility of the log event, if available.

According to RFCs 5424 and 3164, this value should be an integer between 0 and 23.

type: long

example: 23

extended

log.syslog.facility.name

The Syslog text-based facility of the log event, if available.

type: keyword

example: local7

extended

log.syslog.priority

Syslog numeric priority of the event, if available.

According to RFCs 5424 and 3164, the priority is 8 * facility + severity. This number is therefore expected to contain a value between 0 and 191.

type: long

example: 135

extended

log.syslog.severity.code

The Syslog numeric severity of the log event, if available.

If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source’s numeric severity should go to event.severity. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to event.severity.

type: long

example: 3

extended

log.syslog.severity.name

The Syslog numeric severity of the log event, if available.

If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source’s text severity should go to log.level. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to log.level.

type: keyword

example: Error

extended

Network Fields

The network is defined as the communication path over which a host or network event happens.

The network.* fields should be populated with details about the network activity associated with an event.

Field Description Level

network.application

A name given to an application level protocol. This can be arbitrarily assigned for things like microservices, but also apply to things like skype, icq, facebook, twitter. This would be used in situations where the vendor or service can be decoded such as from the source/dest IP owners, ports, or wire format.

The field value must be normalized to lowercase for querying. See the documentation section "Implementing ECS".

type: keyword

example: aim

extended

network.bytes

Total bytes transferred in both directions.

If source.bytes and destination.bytes are known, network.bytes is their sum.

type: long

example: 368

core

network.community_id

A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows.

Learn more at https://github.com/corelight/community-id-spec.

type: keyword

example: 1:hO+sN4H+MG5MY/8hIrXPqc4ZQz0=

extended

network.direction

Direction of the network traffic.

Recommended values are:

* ingress

* egress

* inbound

* outbound

* internal

* external

* unknown

When mapping events from a host-based monitoring context, populate this field from the host’s point of view, using the values "ingress" or "egress".

When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external".

Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers.

type: keyword

example: inbound

core

network.forwarded_ip

Host IP address when the source IP address is the proxy.

type: ip

example: 192.1.1.2

core

network.iana_number

IANA Protocol Number (https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml). Standardized list of protocols. This aligns well with NetFlow and sFlow related logs which use the IANA Protocol Number.

type: keyword

example: 6

extended

network.inner

Network.inner fields are added in addition to network.vlan fields to describe the innermost VLAN when q-in-q VLAN tagging is present. Allowed fields include vlan.id and vlan.name. Inner vlan fields are typically used when sending traffic with multiple 802.1q encapsulations to a network sensor (e.g. Zeek, Wireshark.)

type: object

extended

network.name

Name given by operators to sections of their network.

type: keyword

example: Guest Wifi

extended

network.packets

Total packets transferred in both directions.

If source.packets and destination.packets are known, network.packets is their sum.

type: long

example: 24

core

network.protocol

L7 Network protocol name. ex. http, lumberjack, transport protocol.

The field value must be normalized to lowercase for querying. See the documentation section "Implementing ECS".

type: keyword

example: http

core

network.transport

Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.)

The field value must be normalized to lowercase for querying. See the documentation section "Implementing ECS".

type: keyword

example: tcp

core

network.type

In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc

The field value must be normalized to lowercase for querying. See the documentation section "Implementing ECS".

type: keyword

example: ipv4

core

Nested fields Description

network.inner.vlan.*

Fields to describe observed VLAN information.

network.vlan.*

Fields to describe observed VLAN information.

Observer Fields

An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics.

This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, web proxies, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS.

Field Description Level

observer.egress

Observer.egress holds information like interface number and name, vlan, and zone information to classify egress traffic. Single armed monitoring such as a network sensor on a span port should only use observer.ingress to categorize traffic.

type: object

extended

observer.egress.zone

Network zone of outbound traffic as reported by the observer to categorize the destination area of egress traffic, e.g. Internal, External, DMZ, HR, Legal, etc.

type: keyword

example: Public_Internet

extended

observer.hostname

Hostname of the observer.

type: keyword

core

observer.ingress

Observer.ingress holds information like interface number and name, vlan, and zone information to classify ingress traffic. Single armed monitoring such as a network sensor on a span port should only use observer.ingress to categorize traffic.

type: object

extended

observer.ingress.zone

Network zone of incoming traffic as reported by the observer to categorize the source area of ingress traffic. e.g. internal, External, DMZ, HR, Legal, etc.

type: keyword

example: DMZ

extended

observer.ip

IP addresses of the observer.

type: ip

Note: this field should contain an array of values.

core

observer.mac

MAC addresses of the observer

type: keyword

Note: this field should contain an array of values.

core

observer.name

Custom name of the observer.

This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization.

If no custom name is needed, the field can be left empty.

type: keyword

example: 1_proxySG

extended

observer.product

The product name of the observer.

type: keyword

example: s200

extended

observer.serial_number

Observer serial number.

type: keyword

extended

observer.type

The type of the observer the data is coming from.

There is no predefined list of observer types. Some examples are forwarder, firewall, ids, ips, proxy, poller, sensor, APM server.

type: keyword

example: firewall

core

observer.vendor

Vendor name of the observer.

type: keyword

example: Symantec

core

observer.version

Observer version.

type: keyword

core

Nested fields Description

observer.egress.interface.*

Fields to describe observer interface information.

observer.egress.vlan.*

Fields to describe observed VLAN information.

observer.geo.*

Fields describing a location.

observer.ingress.interface.*

Fields to describe observer interface information.

observer.ingress.vlan.*

Fields to describe observed VLAN information.

observer.os.*

OS fields contain information about the operating system.

Organization Fields

The organization fields enrich data with information about the company or entity the data is associated with.

These fields help you arrange or filter data stored in an index by one or multiple organizations.

Field Description Level

organization.id

Unique identifier for the organization.

type: keyword

extended

organization.name

Organization name.

type: keyword

Multi-fields:

* organization.name.text (type: text)

extended

Operating System Fields

The OS fields contain information about the operating system.

Field Description Level

os.family

OS family (such as redhat, debian, freebsd, windows).

type: keyword

example: debian

extended

os.full

Operating system name, including the version or code name.

type: keyword

Multi-fields:

* os.full.text (type: text)

example: Mac OS Mojave

extended

os.kernel

Operating system kernel version as a raw string.

type: keyword

example: 4.4.0-112-generic

extended

os.name

Operating system name, without the version.

type: keyword

Multi-fields:

* os.name.text (type: text)

example: Mac OS X

extended

os.platform

Operating system platform (such centos, ubuntu, windows).

type: keyword

example: darwin

extended

os.version

Operating system version as a raw string.

type: keyword

example: 10.14.1

extended

The os fields are expected to be nested at: host.os, observer.os, user_agent.os.

Note also that the os fields are not expected to be used directly at the root of the events.

Package Fields

These fields contain information about an installed software package. It contains general information about a package, such as name, version or size. It also contains installation details, such as time or location.

Field Description Level

package.architecture

Package architecture.

type: keyword

example: x86_64

extended

package.build_version

Additional information about the build version of the installed package.

For example use the commit SHA of a non-released package.

type: keyword

example: 36f4f7e89dd61b0988b12ee000b98966867710cd

extended

package.checksum

Checksum of the installed package for verification.

type: keyword

example: 68b329da9893e34099c7d8ad5cb9c940

extended

package.description

Description of the package.

type: keyword

example: Open source programming language to build simple/reliable/efficient software.

extended

package.install_scope

Indicating how the package was installed, e.g. user-local, global.

type: keyword

example: global

extended

package.installed

Time when package was installed.

type: date

extended

package.license

License under which the package was released.

Use a short name, e.g. the license identifier from SPDX License List where possible (https://spdx.org/licenses/).

type: keyword

example: Apache License 2.0

extended

package.name

Package name

type: keyword

example: go

extended

package.path

Path where the package is installed.

type: keyword

example: /usr/local/Cellar/go/1.12.9/

extended

package.reference

Home page or reference URL of the software in this package, if available.

type: keyword

example: https://golang.org

extended

package.size

Package size in bytes.

type: long

example: 62231

extended

package.type

Type of package.

This should contain the package file type, rather than the package manager name. Examples: rpm, dpkg, brew, npm, gem, nupkg, jar.

type: keyword

example: rpm

extended

package.version

Package version

type: keyword

example: 1.12.9

extended

PE Header Fields

These fields contain Windows Portable Executable (PE) metadata.

Field Description Level

pe.architecture

CPU architecture target for the file.

type: keyword

example: x64

extended

pe.company

Internal company name of the file, provided at compile-time.

type: keyword

example: Microsoft Corporation

extended

pe.description

Internal description of the file, provided at compile-time.

type: keyword

example: Paint

extended

pe.file_version

Internal version of the file, provided at compile-time.

type: keyword

example: 6.3.9600.17415

extended

pe.imphash

A hash of the imports in a PE file. An imphash — or import hash — can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html.

type: keyword

example: 0c6803c4e922103c4dca5963aad36ddf

extended

pe.original_file_name

Internal name of the file, provided at compile-time.

type: keyword

example: MSPAINT.EXE

extended

pe.product

Internal product name of the file, provided at compile-time.

type: keyword

example: Microsoft® Windows® Operating System

extended

The pe fields are expected to be nested at: dll.pe, file.pe, process.pe.

Note also that the pe fields are not expected to be used directly at the root of the events.

Process Fields

These fields contain information about a process.

These fields can help you correlate metrics information with a process id/name from a log message. The process.pid often stays in the metric itself and is copied to the global field for correlation.

Field Description Level

process.args

Array of process arguments, starting with the absolute path to the executable.

May be filtered to protect sensitive information.

type: keyword

Note: this field should contain an array of values.

example: ["/usr/bin/ssh", "-l", "user", "10.0.0.16"]

extended

process.args_count

Length of the process.args array.

This field can be useful for querying or performing bucket analysis on how many arguments were provided to start a process. More arguments may be an indication of suspicious activity.

type: long

example: 4

extended

process.command_line

Full command line that started the process, including the absolute path to the executable, and all arguments.

Some arguments may be filtered to protect sensitive information.

type: keyword

Multi-fields:

* process.command_line.text (type: text)

example: /usr/bin/ssh -l user 10.0.0.16

extended

process.entity_id

Unique identifier for the process.

The implementation of this is specified by the data source, but some examples of what could be used here are a process-generated UUID, Sysmon Process GUIDs, or a hash of some uniquely identifying components of a process.

Constructing a globally unique identifier is a common practice to mitigate PID reuse as well as to identify a specific process over time, across multiple monitored hosts.

type: keyword

example: c2c455d9f99375d

extended

process.executable

Absolute path to the process executable.

type: keyword

Multi-fields:

* process.executable.text (type: text)

example: /usr/bin/ssh

extended

process.exit_code

The exit code of the process, if this is a termination event.

The field should be absent if there is no exit code for the event (e.g. process start).

type: long

example: 137

extended

process.name

Process name.

Sometimes called program name or similar.

type: keyword

Multi-fields:

* process.name.text (type: text)

example: ssh

extended

process.pgid

Identifier of the group of processes the process belongs to.

type: long

extended

process.pid

Process id.

type: long

example: 4242

core

process.ppid

Parent process' pid.

type: long

example: 4241

extended

process.start

The time the process started.

type: date

example: 2016-05-23T08:05:34.853Z

extended

process.thread.id

Thread ID.

type: long

example: 4242

extended

process.thread.name

Thread name.

type: keyword

example: thread-0

extended

process.title

Process title.

The proctitle, some times the same as process name. Can also be different: for example a browser setting its title to the web page currently opened.

type: keyword

Multi-fields:

* process.title.text (type: text)

extended

process.uptime

Seconds the process has been up.

type: long

example: 1325

extended

process.working_directory

The working directory of the process.

type: keyword

Multi-fields:

* process.working_directory.text (type: text)

example: /home/alice

extended

The process fields are expected to be nested at: process.parent.

Note also that the process fields may be used directly at the root of the events.

Nested fields Description

process.code_signature.*

These fields contain information about binary code signatures.

process.hash.*

Hashes, usually file hashes.

process.parent.*

These fields contain information about a process.

process.pe.*

These fields contain Windows Portable Executable (PE) metadata.

Registry Fields

Fields related to Windows Registry operations.

Field Description Level

registry.data.bytes

Original bytes written with base64 encoding.

For Windows registry operations, such as SetValueEx and RegQueryValueEx, this corresponds to the data pointed by lp_data. This is optional but provides better recoverability and should be populated for REG_BINARY encoded values.

type: keyword

example: ZQBuAC0AVQBTAAAAZQBuAAAAAAA=

extended

registry.data.strings

Content when writing string types.

Populated as an array when writing string data to the registry. For single string registry types (REG_SZ, REG_EXPAND_SZ), this should be an array with one string. For sequences of string with REG_MULTI_SZ, this array will be variable length. For numeric data, such as REG_DWORD and REG_QWORD, this should be populated with the decimal representation (e.g "1").

type: keyword

Note: this field should contain an array of values.

example: ["C:\rta\red_ttp\bin\myapp.exe"]

core

registry.data.type

Standard registry type for encoding contents

type: keyword

example: REG_SZ

core

registry.hive

Abbreviated name for the hive.

type: keyword

example: HKLM

core

registry.key

Hive-relative path of keys.

type: keyword

example: SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\winword.exe

core

registry.path

Full path, including hive, key and value

type: keyword

example: HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\winword.exe\Debugger

core

registry.value

Name of the value written.

type: keyword

example: Debugger

core

This field set is meant to facilitate pivoting around a piece of data.

Some pieces of information can be seen in many places in an ECS event. To facilitate searching for them, store an array of all seen values to their corresponding field in related..

A concrete example is IP addresses, which can be under host, observer, source, destination, client, server, and network.forwarded_ip. If you append all IPs to related.ip, you can then search for a given IP trivially, no matter where it appeared, by querying related.ip:192.0.2.15.

Field Description Level

related.hash

All the hashes seen on your event. Populating this field, then using it to search for hashes can help in situations where you’re unsure what the hash algorithm is (and therefore which key name to search).

type: keyword

Note: this field should contain an array of values.

extended

related.hosts

All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases.

type: keyword

Note: this field should contain an array of values.

extended

related.ip

All of the IPs seen on your event.

type: ip

Note: this field should contain an array of values.

extended

related.user

All the user names seen on your event.

type: keyword

Note: this field should contain an array of values.

extended

Rule Fields

Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other notable events.

Examples of data sources that would populate the rule fields include: network admission control platforms, network or host IDS/IPS, network firewalls, web application firewalls, url filters, endpoint detection and response (EDR) systems, etc.

Field Description Level

rule.author

Name, organization, or pseudonym of the author or authors who created the rule used to generate this event.

type: keyword

Note: this field should contain an array of values.

example: ["Star-Lord"]

extended

rule.category

A categorization value keyword used by the entity using the rule for detection of this event.

type: keyword

example: Attempted Information Leak

extended

rule.description

The description of the rule generating the event.

type: keyword

example: Block requests to public DNS over HTTPS / TLS protocols

extended

rule.id

A rule ID that is unique within the scope of an agent, observer, or other entity using the rule for detection of this event.

type: keyword

example: 101

extended

rule.license

Name of the license under which the rule used to generate this event is made available.

type: keyword

example: Apache 2.0

extended

rule.name

The name of the rule or signature generating the event.

type: keyword

example: BLOCK_DNS_over_TLS

extended

rule.reference

Reference URL to additional information about the rule used to generate this event.

The URL can point to the vendor’s documentation about the rule. If that’s not available, it can also be a link to a more general page describing this type of alert.

type: keyword

example: https://en.wikipedia.org/wiki/DNS_over_TLS

extended

rule.ruleset

Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member.

type: keyword

example: Standard_Protocol_Filters

extended

rule.uuid

A rule ID that is unique within the scope of a set or group of agents, observers, or other entities using the rule for detection of this event.

type: keyword

example: 1100110011

extended

rule.version

The version / revision of the rule being used for analysis.

type: keyword

example: 1.1

extended

Server Fields

A Server is defined as the responder in a network connection for events regarding sessions, connections, or bidirectional flow records.

For TCP events, the server is the receiver of the initial SYN packet(s) of the TCP connection. For other protocols, the server is generally the responder in the network transaction. Some systems actually use the term "responder" to refer the server in TCP connections. The server fields describe details about the system acting as the server in the network event. Server fields are usually populated in conjunction with client fields. Server fields are generally not populated for packet-level events.

Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately.

Field Description Level

server.address

Some event server addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the .address field.

Then it should be duplicated to .ip or .domain, depending on which one it is.

type: keyword

extended

server.bytes

Bytes sent from the server to the client.

type: long

example: 184

core

server.domain

Server domain.

type: keyword

core

server.ip

IP address of the server (IPv4 or IPv6).

type: ip

core

server.mac

MAC address of the server.

type: keyword

core

server.nat.ip

Translated ip of destination based NAT sessions (e.g. internet to private DMZ)

Typically used with load balancers, firewalls, or routers.

type: ip

extended

server.nat.port

Translated port of destination based NAT sessions (e.g. internet to private DMZ)

Typically used with load balancers, firewalls, or routers.

type: long

extended

server.packets

Packets sent from the server to the client.

type: long

example: 12

core

server.port

Port of the server.

type: long

core

server.registered_domain

The highest registered server domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: example.com

extended

server.subdomain

The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: east

extended

server.top_level_domain

The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: co.uk

extended

Nested fields Description

server.as.*

Fields describing an Autonomous System (Internet routing prefix).

server.geo.*

Fields describing a location.

server.user.*

Fields to describe the user relevant to the event.

Service Fields

The service fields describe the service for or from which the data was collected.

These fields help you find and correlate logs for a specific service and version.

Field Description Level

service.ephemeral_id

Ephemeral identifier of this service (if one exists).

This id normally changes across restarts, but service.id does not.

type: keyword

example: 8a4f500f

extended

service.id

Unique identifier of the running service. If the service is comprised of many nodes, the service.id should be the same for all nodes.

This id should uniquely identify the service. This makes it possible to correlate logs and metrics for one specific service, no matter which particular node emitted the event.

Note that if you need to see the events from one specific host of the service, you should filter on that host.name or host.id instead.

type: keyword

example: d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6

core

service.name

Name of the service data is collected from.

The name of the service is normally user given. This allows for distributed services that run on multiple hosts to correlate the related instances based on the name.

In the case of Elasticsearch the service.name could contain the cluster name. For Beats the service.name is by default a copy of the service.type field if no name is specified.

type: keyword

example: elasticsearch-metrics

core

service.node.name

Name of a service node.

This allows for two nodes of the same service running on the same host to be differentiated. Therefore, service.node.name should typically be unique across nodes of a given service.

In the case of Elasticsearch, the service.node.name could contain the unique node name within the Elasticsearch cluster. In cases where the service doesn’t have the concept of a node name, the host name or container name can be used to distinguish running instances that make up this service. If those do not provide uniqueness (e.g. multiple instances of the service running on the same host) - the node name can be manually set.

type: keyword

example: instance-0000000016

extended

service.state

Current state of the service.

type: keyword

core

service.type

The type of the service data is collected from.

The type can be used to group and correlate logs and metrics from one service type.

Example: If logs or metrics are collected from Elasticsearch, service.type would be elasticsearch.

type: keyword

example: elasticsearch

core

service.version

Version of the service the data was collected from.

This allows to look at a data set only for a specific version of a service.

type: keyword

example: 3.2.4

core

Source Fields

Source fields capture details about the sender of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction.

Source fields are usually populated in conjunction with destination fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated.

Field Description Level

source.address

Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the .address field.

Then it should be duplicated to .ip or .domain, depending on which one it is.

type: keyword

extended

source.bytes

Bytes sent from the source to the destination.

type: long

example: 184

core

source.domain

Source domain.

type: keyword

core

source.ip

IP address of the source (IPv4 or IPv6).

type: ip

core

source.mac

MAC address of the source.

type: keyword

core

source.nat.ip

Translated ip of source based NAT sessions (e.g. internal client to internet)

Typically connections traversing load balancers, firewalls, or routers.

type: ip

extended

source.nat.port

Translated port of source based NAT sessions. (e.g. internal client to internet)

Typically used with load balancers, firewalls, or routers.

type: long

extended

source.packets

Packets sent from the source to the destination.

type: long

example: 12

core

source.port

Port of the source.

type: long

core

source.registered_domain

The highest registered source domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: example.com

extended

source.subdomain

The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: east

extended

source.top_level_domain

The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: co.uk

extended

Nested fields Description

source.as.*

Fields describing an Autonomous System (Internet routing prefix).

source.geo.*

Fields describing a location.

source.user.*

Fields to describe the user relevant to the event.

Threat Fields

Fields to classify events and alerts according to a threat taxonomy such as the MITRE ATT&CK® framework.

These fields are for users to classify alerts from all of their sources (e.g. IDS, NGFW, etc.) within a common taxonomy. The threat.tactic.* are meant to capture the high level category of the threat (e.g. "impact"). The threat.technique.* fields are meant to capture which kind of approach is used by this detected threat, to accomplish the goal (e.g. "endpoint denial of service").

Field Description Level

threat.framework

Name of the threat framework used to further categorize and classify the tactic and technique of the reported threat. Framework classification can be provided by detecting systems, evaluated at ingest time, or retrospectively tagged to events.

type: keyword

example: MITRE ATT&CK

extended

threat.tactic.id

The id of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/ )

type: keyword

Note: this field should contain an array of values.

example: TA0002

extended

threat.tactic.name

Name of the type of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/)

type: keyword

Note: this field should contain an array of values.

example: Execution

extended

threat.tactic.reference

The reference url of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/ )

type: keyword

Note: this field should contain an array of values.

example: https://attack.mitre.org/tactics/TA0002/

extended

threat.technique.id

The id of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/)

type: keyword

Note: this field should contain an array of values.

example: T1059

extended

threat.technique.name

The name of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/)

type: keyword

Multi-fields:

* threat.technique.name.text (type: text)

Note: this field should contain an array of values.

example: Command and Scripting Interpreter

extended

threat.technique.reference

The reference url of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/)

type: keyword

Note: this field should contain an array of values.

example: https://attack.mitre.org/techniques/T1059/

extended

threat.technique.subtechnique.id

The full id of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. https://attack.mitre.org/techniques/T1059/001/)

type: keyword

Note: this field should contain an array of values.

example: T1059.001

extended

threat.technique.subtechnique.name

The name of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. https://attack.mitre.org/techniques/T1059/001/)

type: keyword

Multi-fields:

* threat.technique.subtechnique.name.text (type: text)

Note: this field should contain an array of values.

example: PowerShell

extended

threat.technique.subtechnique.reference

The reference url of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. https://attack.mitre.org/techniques/T1059/001/)

type: keyword

Note: this field should contain an array of values.

example: https://attack.mitre.org/techniques/T1059/001/

extended

TLS Fields

Fields related to a TLS connection. These fields focus on the TLS protocol itself and intentionally avoids in-depth analysis of the related x.509 certificate files.

Field Description Level

tls.cipher

String indicating the cipher used during the current connection.

type: keyword

example: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256

extended

tls.client.certificate

PEM-encoded stand-alone certificate offered by the client. This is usually mutually-exclusive of client.certificate_chain since this value also exists in that list.

type: keyword

example: MII...

extended

tls.client.certificate_chain

Array of PEM-encoded certificates that make up the certificate chain offered by the client. This is usually mutually-exclusive of client.certificate since that value should be the first certificate in the chain.

type: keyword

Note: this field should contain an array of values.

example: ["MII...", "MII..."]

extended

tls.client.hash.md5

Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: 0F76C7F2C55BFD7D8E8B8F4BFBF0C9EC

extended

tls.client.hash.sha1

Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: 9E393D93138888D288266C2D915214D1D1CCEB2A

extended

tls.client.hash.sha256

Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: 0687F666A054EF17A08E2F2162EAB4CBC0D265E1D7875BE74BF3C712CA92DAF0

extended

tls.client.issuer

Distinguished name of subject of the issuer of the x.509 certificate presented by the client.

type: keyword

example: CN=Example Root CA, OU=Infrastructure Team, DC=example, DC=com

extended

tls.client.ja3

A hash that identifies clients based on how they perform an SSL/TLS handshake.

type: keyword

example: d4e5b18d6b55c71272893221c96ba240

extended

tls.client.not_after

Date/Time indicating when client certificate is no longer considered valid.

type: date

example: 2021-01-01T00:00:00.000Z

extended

tls.client.not_before

Date/Time indicating when client certificate is first considered valid.

type: date

example: 1970-01-01T00:00:00.000Z

extended

tls.client.server_name

Also called an SNI, this tells the server which hostname to which the client is attempting to connect to. When this value is available, it should get copied to destination.domain.

type: keyword

example: www.elastic.co

extended

tls.client.subject

Distinguished name of subject of the x.509 certificate presented by the client.

type: keyword

example: CN=myclient, OU=Documentation Team, DC=example, DC=com

extended

tls.client.supported_ciphers

Array of ciphers offered by the client during the client hello.

type: keyword

Note: this field should contain an array of values.

example: ["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "..."]

extended

tls.curve

String indicating the curve used for the given cipher, when applicable.

type: keyword

example: secp256r1

extended

tls.established

Boolean flag indicating if the TLS negotiation was successful and transitioned to an encrypted tunnel.

type: boolean

extended

tls.next_protocol

String indicating the protocol being tunneled. Per the values in the IANA registry (https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids), this string should be lower case.

type: keyword

example: http/1.1

extended

tls.resumed

Boolean flag indicating if this TLS connection was resumed from an existing TLS negotiation.

type: boolean

extended

tls.server.certificate

PEM-encoded stand-alone certificate offered by the server. This is usually mutually-exclusive of server.certificate_chain since this value also exists in that list.

type: keyword

example: MII...

extended

tls.server.certificate_chain

Array of PEM-encoded certificates that make up the certificate chain offered by the server. This is usually mutually-exclusive of server.certificate since that value should be the first certificate in the chain.

type: keyword

Note: this field should contain an array of values.

example: ["MII...", "MII..."]

extended

tls.server.hash.md5

Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: 0F76C7F2C55BFD7D8E8B8F4BFBF0C9EC

extended

tls.server.hash.sha1

Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: 9E393D93138888D288266C2D915214D1D1CCEB2A

extended

tls.server.hash.sha256

Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: 0687F666A054EF17A08E2F2162EAB4CBC0D265E1D7875BE74BF3C712CA92DAF0

extended

tls.server.issuer

Subject of the issuer of the x.509 certificate presented by the server.

type: keyword

example: CN=Example Root CA, OU=Infrastructure Team, DC=example, DC=com

extended

tls.server.ja3s

A hash that identifies servers based on how they perform an SSL/TLS handshake.

type: keyword

example: 394441ab65754e2207b1e1b457b3641d

extended

tls.server.not_after

Timestamp indicating when server certificate is no longer considered valid.

type: date

example: 2021-01-01T00:00:00.000Z

extended

tls.server.not_before

Timestamp indicating when server certificate is first considered valid.

type: date

example: 1970-01-01T00:00:00.000Z

extended

tls.server.subject

Subject of the x.509 certificate presented by the server.

type: keyword

example: CN=www.example.com, OU=Infrastructure Team, DC=example, DC=com

extended

tls.version

Numeric part of the version parsed from the original string.

type: keyword

example: 1.2

extended

tls.version_protocol

Normalized lowercase protocol name parsed from original string.

type: keyword

example: tls

extended

Nested fields Description

tls.client.x509.*

These fields contain x509 certificate metadata.

tls.server.x509.*

These fields contain x509 certificate metadata.

Tracing Fields

Distributed tracing makes it possible to analyze performance throughout a microservice architecture all in one view. This is accomplished by tracing all of the requests - from the initial web request in the front-end service - to queries made through multiple back-end services.

Field Description Level

span.id

Unique identifier of the span within the scope of its trace.

A span represents an operation within a transaction, such as a request to another service, or a database query.

type: keyword

example: 3ff9a8981b7ccd5a

extended

trace.id

Unique identifier of the trace.

A trace groups multiple events like transactions that belong together. For example, a user request handled by multiple inter-connected services.

type: keyword

example: 4bf92f3577b34da6a3ce929d0e0e4736

extended

transaction.id

Unique identifier of the transaction within the scope of its trace.

A transaction is the highest level of work measured within a service, such as a request to a server.

type: keyword

example: 00f067aa0ba902b7

extended

URL Fields

URL fields provide support for complete or partial URLs, and supports the breaking down into scheme, domain, path, and so on.

Field Description Level

url.domain

Domain of the url, such as "www.elastic.co".

In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the domain field.

type: keyword

example: www.elastic.co

extended

url.extension

The field contains the file extension from the original request url.

The file extension is only set if it exists, as not every url has a file extension.

The leading period must not be included. For example, the value must be "png", not ".png".

type: keyword

example: png

extended

url.fragment

Portion of the url after the #, such as "top".

The # is not part of the fragment.

type: keyword

extended

url.full

If full URLs are important to your use case, they should be stored in url.full, whether this field is reconstructed or present in the event source.

type: keyword

Multi-fields:

* url.full.text (type: text)

example: https://www.elastic.co:443/search?q=elasticsearch#top

extended

url.original

Unmodified original url as seen in the event source.

Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path.

This field is meant to represent the URL as it was observed, complete or not.

type: keyword

Multi-fields:

* url.original.text (type: text)

example: https://www.elastic.co:443/search?q=elasticsearch#top or /search?q=elasticsearch

extended

url.password

Password of the request.

type: keyword

extended

url.path

Path of the request, such as "/search".

type: keyword

extended

url.port

Port of the request, such as 443.

type: long

example: 443

extended

url.query

The query field describes the query string of the request, such as "q=elasticsearch".

The ? is excluded from the query string. If a URL contains no ?, there is no query field. If there is a ? but no query, the query field exists with an empty string. The exists query can be used to differentiate between the two cases.

type: keyword

extended

url.registered_domain

The highest registered url domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: example.com

extended

url.scheme

Scheme of the request, such as "https".

Note: The : is not part of the scheme.

type: keyword

example: https

extended

url.subdomain

The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: east

extended

url.top_level_domain

The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: co.uk

extended

url.username

Username of the request.

type: keyword

extended

User Fields

The user fields describe information about the user that is relevant to the event.

Fields can have one entry or multiple entries. If a user has more than one id, provide an array that includes all of them.

Field Description Level

user.domain

Name of the directory the user is a member of.

For example, an LDAP or Active Directory domain name.

type: keyword

extended

user.email

User email address.

type: keyword

extended

user.full_name

User’s full name, if available.

type: keyword

Multi-fields:

* user.full_name.text (type: text)

example: Albert Einstein

extended

user.hash

Unique user hash to correlate information for a user in anonymized form.

Useful if user.id or user.name contain confidential information and cannot be used.

type: keyword

extended

user.id

Unique identifier of the user.

type: keyword

core

user.name

Short name or login of the user.

type: keyword

Multi-fields:

* user.name.text (type: text)

example: albert

core

user.roles

Array of user roles at the time of the event.

type: keyword

Note: this field should contain an array of values.

example: ["kibana_admin", "reporting_user"]

extended

The user fields are expected to be nested at: client.user, destination.user, host.user, server.user, source.user.

Note also that the user fields may be used directly at the root of the events.

Nested fields Description

user.group.*

User’s group relevant to the event.

User agent Fields

The user_agent fields normally come from a browser request.

They often show up in web service logs coming from the parsed user agent string.

Field Description Level

user_agent.device.name

Name of the device.

type: keyword

example: iPhone

extended

user_agent.name

Name of the user agent.

type: keyword

example: Safari

extended

user_agent.original

Unparsed user_agent string.

type: keyword

Multi-fields:

* user_agent.original.text (type: text)

example: Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1

extended

user_agent.version

Version of the user agent.

type: keyword

example: 12.0

extended

Nested fields Description

user_agent.os.*

OS fields contain information about the operating system.

VLAN Fields

The VLAN fields are used to identify 802.1q tag(s) of a packet, as well as ingress and egress VLAN associations of an observer in relation to a specific packet or connection.

Network.vlan fields are used to record a single VLAN tag, or the outer tag in the case of q-in-q encapsulations, for a packet or connection as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic.

Network.inner VLAN fields are used to report inner q-in-q 802.1q tags (multiple 802.1q encapsulations) as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic. Network.inner VLAN fields should only be used in addition to network.vlan fields to indicate q-in-q tagging.

Observer.ingress and observer.egress VLAN values are used to record observer specific information when observer events contain discrete ingress and egress VLAN information, typically provided by firewalls, routers, or load balancers.

Field Description Level

vlan.id

VLAN ID as reported by the observer.

type: keyword

example: 10

extended

vlan.name

Optional VLAN name as reported by the observer.

type: keyword

example: outside

extended

The vlan fields are expected to be nested at: network.inner.vlan, network.vlan, observer.egress.vlan, observer.ingress.vlan.

Note also that the vlan fields are not expected to be used directly at the root of the events.

Vulnerability Fields

The vulnerability fields describe information about a vulnerability that is relevant to an event.

Field Description Level

vulnerability.category

The type of system or architecture that the vulnerability affects. These may be platform-specific (for example, Debian or SUSE) or general (for example, Database or Firewall). For example (Qualys vulnerability categories)

This field must be an array.

type: keyword

Note: this field should contain an array of values.

example: ["Firewall"]

extended

vulnerability.classification

The classification of the vulnerability scoring system. For example (https://www.first.org/cvss/)

type: keyword

example: CVSS

extended

vulnerability.description

The description of the vulnerability that provides additional context of the vulnerability. For example (Common Vulnerabilities and Exposure CVE description)

type: keyword

Multi-fields:

* vulnerability.description.text (type: text)

example: In macOS before 2.12.6, there is a vulnerability in the RPC...

extended

vulnerability.enumeration

The type of identifier used for this vulnerability. For example (https://cve.mitre.org/about/)

type: keyword

example: CVE

extended

vulnerability.id

The identification (ID) is the number portion of a vulnerability entry. It includes a unique identification number for the vulnerability. For example (Common Vulnerabilities and Exposure CVE ID

type: keyword

example: CVE-2019-00001

extended

vulnerability.reference

A resource that provides additional information, context, and mitigations for the identified vulnerability.

type: keyword

example: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111

extended

vulnerability.report_id

The report or scan identification number.

type: keyword

example: 20191018.0001

extended

vulnerability.scanner.vendor

The name of the vulnerability scanner vendor.

type: keyword

example: Tenable

extended

vulnerability.score.base

Scores can range from 0.0 to 10.0, with 10.0 being the most severe.

Base scores cover an assessment for exploitability metrics (attack vector, complexity, privileges, and user interaction), impact metrics (confidentiality, integrity, and availability), and scope. For example (https://www.first.org/cvss/specification-document)

type: float

example: 5.5

extended

vulnerability.score.environmental

Scores can range from 0.0 to 10.0, with 10.0 being the most severe.

Environmental scores cover an assessment for any modified Base metrics, confidentiality, integrity, and availability requirements. For example (https://www.first.org/cvss/specification-document)

type: float

example: 5.5

extended

vulnerability.score.temporal

Scores can range from 0.0 to 10.0, with 10.0 being the most severe.

Temporal scores cover an assessment for code maturity, remediation level, and confidence. For example (https://www.first.org/cvss/specification-document)

type: float

extended

vulnerability.score.version

The National Vulnerability Database (NVD) provides qualitative severity rankings of "Low", "Medium", and "High" for CVSS v2.0 base score ranges in addition to the severity ratings for CVSS v3.0 as they are defined in the CVSS v3.0 specification.

CVSS is owned and managed by FIRST.Org, Inc. (FIRST), a US-based non-profit organization, whose mission is to help computer security incident response teams across the world. For example (https://nvd.nist.gov/vuln-metrics/cvss)

type: keyword

example: 2.0

extended

vulnerability.severity

The severity of the vulnerability can help with metrics and internal prioritization regarding remediation. For example (https://nvd.nist.gov/vuln-metrics/cvss)

type: keyword

example: Critical

extended

x509 Certificate Fields

This implements the common core fields for x509 certificates. This information is likely logged with TLS sessions, digital signatures found in executable binaries, S/MIME information in email bodies, or analysis of files on disk.

When the certificate relates to a file, use the fields at file.x509. When hashes of the DER-encoded certificate are available, the hash data set should be populated as well (e.g. file.hash.sha256).

Events that contain certificate information about network connections, should use the x509 fields under the relevant TLS fields: tls.server.x509 and/or tls.client.x509.

Field Description Level

x509.alternative_names

List of subject alternative names (SAN). Name types vary by certificate authority and certificate type but commonly contain IP addresses, DNS names (and wildcards), and email addresses.

type: keyword

Note: this field should contain an array of values.

example: *.elastic.co

extended

x509.issuer.common_name

List of common name (CN) of issuing certificate authority.

type: keyword

Note: this field should contain an array of values.

example: Example SHA2 High Assurance Server CA

extended

x509.issuer.country

List of country © codes

type: keyword

Note: this field should contain an array of values.

example: US

extended

x509.issuer.distinguished_name

Distinguished name (DN) of issuing certificate authority.

type: keyword

example: C=US, O=Example Inc, OU=www.example.com, CN=Example SHA2 High Assurance Server CA

extended

x509.issuer.locality

List of locality names (L)

type: keyword

Note: this field should contain an array of values.

example: Mountain View

extended

x509.issuer.organization

List of organizations (O) of issuing certificate authority.

type: keyword

Note: this field should contain an array of values.

example: Example Inc

extended

x509.issuer.organizational_unit

List of organizational units (OU) of issuing certificate authority.

type: keyword

Note: this field should contain an array of values.

example: www.example.com

extended

x509.issuer.state_or_province

List of state or province names (ST, S, or P)

type: keyword

Note: this field should contain an array of values.

example: California

extended

x509.not_after

Time at which the certificate is no longer considered valid.

type: date

example: 2020-07-16 03:15:39

extended

x509.not_before

Time at which the certificate is first considered valid.

type: date

example: 2019-08-16 01:40:25

extended

x509.public_key_algorithm

Algorithm used to generate the public key.

type: keyword

example: RSA

extended

x509.public_key_curve

The curve used by the elliptic curve public key algorithm. This is algorithm specific.

type: keyword

example: nistp521

extended

x509.public_key_exponent

Exponent used to derive the public key. This is algorithm specific.

type: long

example: 65537

extended

x509.public_key_size

The size of the public key space in bits.

type: long

example: 2048

extended

x509.serial_number

Unique serial number issued by the certificate authority. For consistency, if this value is alphanumeric, it should be formatted without colons and uppercase characters.

type: keyword

example: 55FBB9C7DEBF09809D12CCAA

extended

x509.signature_algorithm

Identifier for certificate signature algorithm. We recommend using names found in Go Lang Crypto library. See https://github.com/golang/go/blob/go1.14/src/crypto/x509/x509.go#L337-L353.

type: keyword

example: SHA256-RSA

extended

x509.subject.common_name

List of common names (CN) of subject.

type: keyword

Note: this field should contain an array of values.

example: shared.global.example.net

extended

x509.subject.country

List of country © code

type: keyword

Note: this field should contain an array of values.

example: US

extended

x509.subject.distinguished_name

Distinguished name (DN) of the certificate subject entity.

type: keyword

example: C=US, ST=California, L=San Francisco, O=Example, Inc., CN=shared.global.example.net

extended

x509.subject.locality

List of locality names (L)

type: keyword

Note: this field should contain an array of values.

example: San Francisco

extended

x509.subject.organization

List of organizations (O) of subject.

type: keyword

Note: this field should contain an array of values.

example: Example, Inc.

extended

x509.subject.organizational_unit

List of organizational units (OU) of subject.

type: keyword

Note: this field should contain an array of values.

extended

x509.subject.state_or_province

List of state or province names (ST, S, or P)

type: keyword

Note: this field should contain an array of values.

example: California

extended

x509.version_number

Version of x509 format.

type: keyword

example: 3

extended

The x509 fields are expected to be nested at: file.x509, tls.client.x509, tls.server.x509.

Note also that the x509 fields are not expected to be used directly at the root of the events.

ECS Categorization Fields

Warning
This section of ECS is in beta and is subject to change. These allowed values are still under active development. Additional values will be published gradually, and some of the values or relationships described here may change. Users who want to provide feedback, or who want to have a look at upcoming allowed values can visit this public feedback document https://ela.st/ecs-categories-draft.

At a high level, ECS provides fields to classify events in two different ways: "Where it’s from" (e.g., event.module, event.dataset, agent.type, observer.type, etc.), and "What it is." The categorization fields hold the "What it is" information, independent of the source of the events.

ECS defines four categorization fields for this purpose, each of which falls under the event.* field set.

Categorization Fields

Note
If your events don’t match any of these categorization values, you should leave the fields empty. This will ensure you can start populating the fields once the appropriate categorization values are published, in a later release.

ECS Categorization Field: event.kind

This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy.

event.kind gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events.

The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not.

Warning
After the beta period for categorization, only the allowed categorization values listed in the ECS repository and official ECS documentation should be considered official. Use of any other values may result in incompatible implementations that will require subsequent breaking changes.

Allowed Values

alert

This value indicates an event that describes an alert or notable event, triggered by a detection rule.

event.kind:alert is often populated for events coming from firewalls, intrusion detection systems, endpoint detection and response systems, and so on.

event

This value is the most general and most common value for this field. It is used to represent events that indicate that something happened.

metric

This value is used to indicate that this event describes a numeric measurement taken at given point in time.

Examples include CPU utilization, memory usage, or device temperature.

Metric events are often collected on a predictable frequency, such as once every few seconds, or once a minute, but can also be used to describe ad-hoc numeric metric queries.

state

The state value is similar to metric, indicating that this event describes a measurement taken at given point in time, except that the measurement does not result in a numeric value, but rather one of a fixed set of categorical values that represent conditions or states.

Examples include periodic events reporting Elasticsearch cluster state (green/yellow/red), the state of a TCP connection (open, closed, fin_wait, etc.), the state of a host with respect to a software vulnerability (vulnerable, not vulnerable), and the state of a system regarding compliance with a regulatory standard (compliant, not compliant).

Note that an event that describes a change of state would not use event.kind:state, but instead would use event.kind:event since a state change fits the more general event definition of something that happened.

State events are often collected on a predictable frequency, such as once every few seconds, once a minute, once an hour, or once a day, but can also be used to describe ad-hoc state queries.

pipeline_error

This value indicates that an error occurred during the ingestion of this event, and that event data may be missing, inconsistent, or incorrect. event.kind:pipeline_error is often associated with parsing errors.

signal

This value is used by the Elastic SIEM app to denote an Elasticsearch document that was created by a SIEM detection engine rule.

A signal will typically trigger a notification that something meaningful happened and should be investigated.

Usage of this value is reserved, and pipelines should not populate event.kind with the value "signal".

ECS Categorization Field: event.category

This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy.

event.category represents the "big buckets" of ECS categories. For example, filtering on event.category:process yields all events relating to process activity. This field is closely related to event.type, which is used as a subcategory.

This field is an array. This will allow proper categorization of some events that fall in multiple categories.

Warning
After the beta period for categorization, only the allowed categorization values listed in the ECS repository and official ECS documentation should be considered official. Use of any other values may result in incompatible implementations that will require subsequent breaking changes.

Allowed Values

authentication

Events in this category are related to the challenge and response process in which credentials are supplied and verified to allow the creation of a session. Common sources for these logs are Windows event logs and ssh logs. Visualize and analyze events in this category to look for failed logins, and other authentication-related activity.

Expected event types for category authentication:

start, end, info

configuration

Events in the configuration category have to deal with creating, modifying, or deleting the settings or parameters of an application, process, or system.

Example sources include security policy change logs, configuration auditing logging, and system integrity monitoring.

Expected event types for category configuration:

access, change, creation, deletion, info

database

The database category denotes events and metrics relating to a data storage and retrieval system. Note that use of this category is not limited to relational database systems. Examples include event logs from MS SQL, MySQL, Elasticsearch, MongoDB, etc. Use this category to visualize and analyze database activity such as accesses and changes.

Expected event types for category database:

access, change, info, error

driver

Events in the driver category have to do with operating system device drivers and similar software entities such as Windows drivers, kernel extensions, kernel modules, etc.

Use events and metrics in this category to visualize and analyze driver-related activity and status on hosts.

Expected event types for category driver:

change, end, info, start

file

Relating to a set of information that has been created on, or has existed on a filesystem. Use this category of events to visualize and analyze the creation, access, and deletions of files. Events in this category can come from both host-based and network-based sources. An example source of a network-based detection of a file transfer would be the Zeek file.log.

Expected event types for category file:

change, creation, deletion, info

host

Use this category to visualize and analyze information such as host inventory or host lifecycle events.

Most of the events in this category can usually be observed from the outside, such as from a hypervisor or a control plane’s point of view. Some can also be seen from within, such as "start" or "end".

Note that this category is for information about hosts themselves; it is not meant to capture activity "happening on a host".

Expected event types for category host:

access, change, end, info, start

iam

Identity and access management (IAM) events relating to users, groups, and administration. Use this category to visualize and analyze IAM-related logs and data from active directory, LDAP, Okta, Duo, and other IAM systems.

Expected event types for category iam:

admin, change, creation, deletion, group, info, user

intrusion_detection

Relating to intrusion detections from IDS/IPS systems and functions, both network and host-based. Use this category to visualize and analyze intrusion detection alerts from systems such as Snort, Suricata, and Palo Alto threat detections.

Expected event types for category intrusion_detection:

allowed, denied, info

malware

Malware detection events and alerts. Use this category to visualize and analyze malware detections from EDR/EPP systems such as Elastic Endpoint Security, Symantec Endpoint Protection, Crowdstrike, and network IDS/IPS systems such as Suricata, or other sources of malware-related events such as Palo Alto Networks threat logs and Wildfire logs.

Expected event types for category malware:

info

network

Relating to all network activity, including network connection lifecycle, network traffic, and essentially any event that includes an IP address. Many events containing decoded network protocol transactions fit into this category. Use events in this category to visualize or analyze counts of network ports, protocols, addresses, geolocation information, etc.

Expected event types for category network:

access, allowed, connection, denied, end, info, protocol, start

package

Relating to software packages installed on hosts. Use this category to visualize and analyze inventory of software installed on various hosts, or to determine host vulnerability in the absence of vulnerability scan data.

Expected event types for category package:

access, change, deletion, info, installation, start

process

Use this category of events to visualize and analyze process-specific information such as lifecycle events or process ancestry.

Expected event types for category process:

access, change, end, info, start

web

Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also be included in this category.

Expected event types for category web:

access, error, info

ECS Categorization Field: event.type

This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy.

event.type represents a categorization "sub-bucket" that, when used along with the event.category field values, enables filtering events down to a level appropriate for single visualization.

This field is an array. This will allow proper categorization of some events that fall in multiple event types.

Warning
After the beta period for categorization, only the allowed categorization values listed in the ECS repository and official ECS documentation should be considered official. Use of any other values may result in incompatible implementations that will require subsequent breaking changes.

Allowed Values

access

The access event type is used for the subset of events within a category that indicate that something was accessed. Common examples include event.category:database AND event.type:access, or event.category:file AND event.type:access. Note for file access, both directory listings and file opens should be included in this subcategory. You can further distinguish access operations using the ECS event.action field.

admin

The admin event type is used for the subset of events within a category that are related to admin objects. For example, administrative changes within an IAM framework that do not specifically affect a user or group (e.g., adding new applications to a federation solution or connecting discrete forests in Active Directory) would fall into this subcategory. Common example: event.category:iam AND event.type:change AND event.type:admin. You can further distinguish admin operations using the ECS event.action field.

allowed

The allowed event type is used for the subset of events within a category that indicate that something was allowed. Common examples include event.category:network AND event.type:connection AND event.type:allowed (to indicate a network firewall event for which the firewall disposition was to allow the connection to complete) and event.category:intrusion_detection AND event.type:allowed (to indicate a network intrusion prevention system event for which the IPS disposition was to allow the connection to complete). You can further distinguish allowed operations using the ECS event.action field, populating with values of your choosing, such as "allow", "detect", or "pass".

change

The change event type is used for the subset of events within a category that indicate that something has changed. If semantics best describe an event as modified, then include them in this subcategory. Common examples include event.category:process AND event.type:change, and event.category:file AND event.type:change. You can further distinguish change operations using the ECS event.action field.

connection

Used primarily with event.category:network this value is used for the subset of network traffic that includes sufficient information for the event to be included in flow or connection analysis. Events in this subcategory will contain at least source and destination IP addresses, source and destination TCP/UDP ports, and will usually contain counts of bytes and/or packets transferred. Events in this subcategory may contain unidirectional or bidirectional information, including summary information. Use this subcategory to visualize and analyze network connections. Flow analysis, including Netflow, IPFIX, and other flow-related events fit in this subcategory. Note that firewall events from many Next-Generation Firewall (NGFW) devices will also fit into this subcategory. A common filter for flow/connection information would be event.category:network AND event.type:connection AND event.type:end (to view or analyze all completed network connections, ignoring mid-flow reports). You can further distinguish connection events using the ECS event.action field, populating with values of your choosing, such as "timeout", or "reset".

creation

The "creation" event type is used for the subset of events within a category that indicate that something was created. A common example is event.category:file AND event.type:creation.

deletion

The deletion event type is used for the subset of events within a category that indicate that something was deleted. A common example is event.category:file AND event.type:deletion to indicate that a file has been deleted.

denied

The denied event type is used for the subset of events within a category that indicate that something was denied. Common examples include event.category:network AND event.type:denied (to indicate a network firewall event for which the firewall disposition was to deny the connection) and event.category:intrusion_detection AND event.type:denied (to indicate a network intrusion prevention system event for which the IPS disposition was to deny the connection to complete). You can further distinguish denied operations using the ECS event.action field, populating with values of your choosing, such as "blocked", "dropped", or "quarantined".

end

The end event type is used for the subset of events within a category that indicate something has ended. A common example is event.category:process AND event.type:end.

error

The error event type is used for the subset of events within a category that indicate or describe an error. A common example is event.category:database AND event.type:error. Note that pipeline errors that occur during the event ingestion process should not use this event.type value. Instead, they should use event.kind:pipeline_error.

group

The group event type is used for the subset of events within a category that are related to group objects. Common example: event.category:iam AND event.type:creation AND event.type:group. You can further distinguish group operations using the ECS event.action field.

info

The info event type is used for the subset of events within a category that indicate that they are purely informational, and don’t report a state change, or any type of action. For example, an initial run of a file integrity monitoring system (FIM), where an agent reports all files under management, would fall into the "info" subcategory. Similarly, an event containing a dump of all currently running processes (as opposed to reporting that a process started/ended) would fall into the "info" subcategory. An additional common examples is event.category:intrusion_detection AND event.type:info.

installation

The installation event type is used for the subset of events within a category that indicate that something was installed. A common example is event.category:package AND event.type:installation.

protocol

The protocol event type is used for the subset of events within a category that indicate that they contain protocol details or analysis, beyond simply identifying the protocol. Generally, network events that contain specific protocol details will fall into this subcategory. A common example is event.category:network AND event.type:protocol AND event.type:connection AND event.type:end (to indicate that the event is a network connection event sent at the end of a connection that also includes a protocol detail breakdown). Note that events that only indicate the name or id of the protocol should not use the protocol value. Further note that when the protocol subcategory is used, the identified protocol is populated in the ECS network.protocol field.

start

The start event type is used for the subset of events within a category that indicate something has started. A common example is event.category:process AND event.type:start.

user

The user event type is used for the subset of events within a category that are related to user objects. Common example: event.category:iam AND event.type:deletion AND event.type:user. You can further distinguish user operations using the ECS event.action field.

ECS Categorization Field: event.outcome

This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy.

event.outcome simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event.

Note that when a single transaction is described in multiple events, each event may populate different values of event.outcome, according to their perspective.

Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer.

Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with event.type:info, or any events for which an outcome does not make logical sense.

Warning
After the beta period for categorization, only the allowed categorization values listed in the ECS repository and official ECS documentation should be considered official. Use of any other values may result in incompatible implementations that will require subsequent breaking changes.

Allowed Values

failure

Indicates that this event describes a failed result. A common example is event.category:file AND event.type:access AND event.outcome:failure to indicate that a file access was attempted, but was not successful.

success

Indicates that this event describes a successful result. A common example is event.category:file AND event.type:create AND event.outcome:success to indicate that a file was successfully created.

unknown

Indicates that this event describes only an attempt for which the result is unknown from the perspective of the event producer. For example, if the event contains information only about the request side of a transaction that results in a response, populating event.outcome:unknown in the request event is appropriate. The unknown value should not be used when an outcome doesn’t make logical sense for the event. In such cases event.outcome should not be populated.

Migrating to ECS

There are multiple ways to reap the benefit of ECS. The simplest is to use [ecs-products-solutions].

If you have a custom pipeline or application you would like to convert to ECS, please have a look at [ecs-converting].

Products and Solutions that Support ECS

The following Elastic products support ECS out of the box, as of version 7.0:

Converting a Custom Implementation

A common schema helps you correlate and use data from various sources.

Fields for most Elastic modules and solutions (version 7.0 and later) are mapped to the Elastic Common Schema. You may want to map data from other implementations to ECS to help you correlate data across all of your products and solutions.

Before you start a conversion, be sure that you understand the basics below.

Core and extended levels

Make sure you understand the distinction between Core and Extended fields, as explained in the [ecs-guidelines].

An approach to mapping an existing implementation

Here’s the recommended approach for converting an existing implementation to ECS.

  1. Review each field in your original event and map it to the relevant ECS field.

    • Start by mapping your field to the relevant ECS Core field.

    • If a relevant ECS Core field does not exist, map your field to the relevant ECS Extended field.

    • If no relevant ECS Extended field exists, consider keeping your field with its original details, or possibly renaming it using ECS naming guidelines and attempt to map one or more of your original event fields to it.

  2. Review each ECS Core field, and attempt to populate it.

    • Review your original event data again

    • Consider populating the field based on additional meta-data such as static information (e.g. add event.type:syslog even if syslog events don’t mention this fact), or information gathered from the environment (e.g. host information).

  3. Review other extended fields from any field set you are already using, and attempt to populate it as well.

  4. Set ecs.version to the version of the schema you are conforming to. This will allow you to upgrade your sources, pipelines and content (like dashboards) smoothly in the future.

Using a spreadsheet to plan your migration

Using a spreadsheet to plan the migration from pre-existing source fields to ECS is a common practice. It’s a good way to address each of your fields methodically among colleagues.

If the data source is either a structured log, or if you already have a pipeline producing events with these non-ECS field names, the tool ECS Mapper may help you get started in performing all of these field renames.

After exporting your mapping spreadsheet to CSV, ECS Mapper will convert your field mapping to equivalent pipelines for Beats, Elasticsearch, and Logstash. Learn more at ECS Mapper.

Additional Information

Questions and Answers

What are the benefits of using ECS?

The benefits to a user adopting these fields and names in their clusters are:

  • Data correlation. Ability to easily correlate data from the same or different sources, including:

    • data from metrics, logs, and application performance management (apm) tools

    • data from the same machines/hosts

    • data from the same service

  • Ease of recall. Improved ability to remember commonly used field names (because there is a single set, not a set per data source)

  • Ease of deduction. Improved ability to deduce field names (because the field naming follows a small number of rules with few exceptions)

  • Reuse. Ability to re-use analysis content (searches, visualizations, dashboards, alerts, reports, and machine learning jobs) across multiple data sources

  • Future proofing. Ability to use any future Elastic-provided analysis content in your environment without modifications

What if I have fields that conflict with ECS?

The processor] can help you resolve field conflicts. For example, imagine that you already have a field called "user," but ECS employs user as an object. You can use the rename processor on ingest time to rename your field to the matching ECS field. If your field does not match ECS, you can rename your field to user.value instead.

What if my events have additional fields?

Events may contain fields in addition to ECS fields. These fields can follow the ECS naming and writing rules, but this is not a requirement.

Why does ECS use a dot notation instead of an underline notation?

There are two common key formats for ingesting data into Elasticsearch:

  • Dot notation: user.firstname: Nicolas, user.lastname: Ruflin

  • Underline notation: user_firstname: Nicolas, user_lastname: Ruflin

ECS uses the dot notation to represent nested objects.

What is the difference between the two notations?

Ingesting user.firstname: Nicolas and user.lastname: Ruflin is identical to ingesting the following JSON:

"user": {
  "firstname": "Nicolas",
  "lastname": "Ruflin"
}

datatype]. In the case of the underline notation, both are just

Advantages of dot notation

With dot notation, each prefix in Elasticsearch is an object. Each object can have that control how fields inside the object are treated. In the context of ECS, for example, these parameters would allow you to disable dynamic property creation for certain prefixes.

Individual objects give you more flexibility on both the ingest and the event sides. In Elasticsearch, for example, you can use the remove processor to drop complete objects instead of selecting each key inside. You don’t have to know ahead of time which keys will be in an object.

In Beats, you can simplify the creation of events. For example, you can treat each object as an object (or struct in Golang), which makes constructing and modifying each part of the final event easier.

Disadvantage of dot notation

In Elasticsearch, each key can have only one type. For example, if user is an object, you can’t use it as a keyword type in the same index, like {"user": "nicolas ruflin"}. This restriction can be an issue in certain datasets. For the ECS data itself, this is not an issue because all fields are predefined.

What if I already use the underline notation?

As long as there are no conflicts, underline notation and ECS dot notation can coexist in the same document.

Glossary of ECS Terms

ECS

Elastic Common Schema. The Elastic Common Schema (ECS) is a document schema for Elasticsearch, for use cases such as logging and metrics. ECS defines a common set of fields, their datatype, and gives guidance on their correct usage. ECS is used to improve uniformity of event data coming from different sources.

Contributing to ECS

All information related to ECS is versioned in the elastic/ecs repository. All changes to ECS happen through Pull Requests submitted through Git.

Downloads