diff --git a/CHANGELOG.md b/CHANGELOG.md index bc67a08e058..0eb3df17e74 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -174,6 +174,8 @@ release. - Clarify usage of log body for structured logs ([#3023](https://github.com/open-telemetry/opentelemetry-specification/pull/3023)) +- Move appendices from Data Model to new Data Model Appendix document + ([#3207](https://github.com/open-telemetry/opentelemetry-specification/pull/3207)) ### Resource diff --git a/specification/logs/README.md b/specification/logs/README.md index 8f3124d22da..e21654eb65e 100644 --- a/specification/logs/README.md +++ b/specification/logs/README.md @@ -138,7 +138,7 @@ Given the above state of the logging space we took the following approach: OpenTelemetry's log data model. More on this [later](#new-first-party-application-logs). - Existing log formats can be - [unambiguously mapped](data-model.md#appendix-a-example-mappings) to + [unambiguously mapped](data-model-appendix.md) to OpenTelemetry log data model. OpenTelemetry Collector can read such logs and translate them to OpenTelemetry log data model. diff --git a/specification/logs/data-model-appendix.md b/specification/logs/data-model-appendix.md new file mode 100644 index 00000000000..f0dd017d752 --- /dev/null +++ b/specification/logs/data-model-appendix.md @@ -0,0 +1,833 @@ +# Data Model Appendix + +Note: this document is NOT a spec, it is provided to support the Logs +[Data Model](./data-model.md) specification. These examples provided purely +for demonstrative purposes and are not exhaustive or canonical; please refer to +the respective exporter documentation if exact details are required. + + + +- [Appendix A. Example Mappings](#appendix-a-example-mappings) + * [RFC5424 Syslog](#rfc5424-syslog) + * [Windows Event Log](#windows-event-log) + * [SignalFx Events](#signalfx-events) + * [Splunk HEC](#splunk-hec) + * [Log4j](#log4j) + * [Zap](#zap) + * [Apache HTTP Server access log](#apache-http-server-access-log) + * [CloudTrail Log Event](#cloudtrail-log-event) + * [Google Cloud Logging](#google-cloud-logging) + * [Elastic Common Schema](#elastic-common-schema) +- [Appendix B: `SeverityNumber` example mappings](#appendix-b-severitynumber-example-mappings) + + + +## Appendix A. Example Mappings + +This section contains examples of mapping of other events and logs formats to +this data model. + +### RFC5424 Syslog + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PropertyTypeDescriptionMaps to Unified Model Field
TIMESTAMPTimestampTime when an event occurred measured by the origin clock.Timestamp
SEVERITYenumDefines the importance of the event. Example: `Debug`Severity
FACILITYenumDescribes where the event originated. A predefined list of Unix processes. Part of event source identity. Example: `mail system`Attributes["syslog.facility"]
VERSIONnumberMeta: protocol version, orthogonal to the event.Attributes["syslog.version"]
HOSTNAMEstringDescribes the location where the event originated. Possible values are FQDN, IP address, etc.Resource["host.hostname"]
APP-NAMEstringUser-defined app name. Part of event source identity.Resource["service.name"]
PROCIDstringNot well defined. May be used as a meta field for protocol operation purposes or may be part of event source identity.Attributes["syslog.procid"]
MSGIDstringDefines the type of the event. Part of event source identity. Example: "TCPIN"Attributes["syslog.msgid"]
STRUCTURED-DATAarray of maps of string to stringA variety of use cases depending on the SDID: +Can describe event source identity +Can include data that describes particular occurrence of the event. +Can be meta-information, e.g. quality of timestamp value.SDID origin.swVersion map to Resource["service.version"] + +SDID origin.ip map to attribute["net.sock.host.addr"] + +Rest of SDIDs -> Attributes["syslog.*"]
MSGstringFree-form text message about the event. Typically human readable.Body
+ +### Windows Event Log + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PropertyTypeDescriptionMaps to Unified Model Field
TimeCreatedTimestampThe time stamp that identifies when the event was logged.Timestamp
LevelenumContains the severity level of the event.Severity
ComputerstringThe name of the computer on which the event occurred.Resource["host.hostname"]
EventIDuintThe identifier that the provider used to identify the event.Attributes["winlog.event_id"]
MessagestringThe message string.Body
Rest of the fields.anyAll other fields in the event.Attributes["winlog.*"]
+ +### SignalFx Events + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
TimestampTimestampTime when the event occurred measured by the origin clock.Timestamp
EventTypestringShort machine understandable string describing the event type. SignalFx specific concept. Non-namespaced. Example: k8s Event Reason field.Attributes["com.splunk.signalfx.event_type"]
CategoryenumDescribes where the event originated and why. SignalFx specific concept. Example: AGENT. Attributes["com.splunk.signalfx.event_category"]
Dimensionsmap<string, string>Helps to define the identity of the event source together with EventType and Category. Multiple occurrences of events coming from the same event source can happen across time and they all have the value of Dimensions. Resource
Propertiesmap<string, any>Additional information about the specific event occurrence. Unlike Dimensions which are fixed for a particular event source, Properties can have different values for each occurrence of the event coming from the same event source.Attributes
+ +### Splunk HEC + +We apply this mapping from HEC to the unified model: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
timenumeric, stringThe event time in epoch time format, in seconds.Timestamp
hoststringThe host value to assign to the event data. This is typically the host name of the client that you are sending data from.Resource["host.name"]
sourcestringThe source value to assign to the event data. For example, if you are sending data from an app you are developing, you could set this key to the name of the app.Resource["com.splunk.source"]
sourcetypestringThe sourcetype value to assign to the event data.Resource["com.splunk.sourcetype"]
eventanyThe JSON representation of the raw body of the event. It can be a string, number, string array, number array, JSON object, or a JSON array.Body
fieldsmap<string, any>Specifies a JSON object that contains explicit custom fields.Attributes
indexstringThe name of the index by which the event data is to be indexed. The index you specify here must be within the list of allowed indexes if the token has the indexes parameter set.Attributes["com.splunk.index"]
+ +When mapping from the unified model to HEC, we apply this additional mapping: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Unified model elementTypeDescriptionMaps to HEC
SeverityTextstringThe severity of the event as a human-readable string.fields['otel.log.severity.text']
SeverityNumberstringThe severity of the event as a number.fields['otel.log.severity.number']
NamestringShort event identifier that does not contain varying parts.fields['otel.log.name']
TraceIdstringRequest trace id.fields['trace_id']
SpanIdstringRequest span id.fields['span_id']
TraceFlagsstringW3C trace flags.fields['trace_flags']
+ +### Log4j + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
InstantTimestampTime when an event occurred measured by the origin clock.Timestamp
LevelenumLog level.Severity
MessagestringHuman readable message.Body
All other fieldsanyStructured data.Attributes
+ +### Zap + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
tsTimestampTime when an event occurred measured by the origin clock.Timestamp
levelenumLogging level.Severity
callerstringCalling function's filename and line number. +Attributes, key=TBD
msgstringHuman readable message.Body
All other fieldsanyStructured data.Attributes
+ +### Apache HTTP Server access log + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
%tTimestampTime when an event occurred measured by the origin clock.Timestamp
%astringClient addressAttributes["net.sock.peer.addr"]
%AstringServer addressAttributes["net.sock.host.addr"]
%hstringClient hostname.Attributes["net.peer.name"]
%mstringThe request method.Attributes["http.method"]
%v,%p,%U,%qstringMultiple fields that can be composed into URL.Attributes["http.url"]
%>sstringResponse status.Attributes["http.status_code"]
All other fieldsanyStructured data.Attributes, key=TBD
+ +### CloudTrail Log Event + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
eventTimestringThe date and time the request was made, in coordinated universal time (UTC).Timestamp
eventSourcestringThe service that the request was made to. This name is typically a short form of the service name without spaces plus .amazonaws.com.Resource["service.name"]?
awsRegionstringThe AWS region that the request was made to, such as us-east-2.Resource["cloud.region"]
sourceIPAddressstringThe IP address that the request was made from.Attributes["net.sock.peer.addr"] or Attributes["net.sock.host.addr"]
errorCodestringThe AWS service error if the request returns an error.Attributes["cloudtrail.error_code"]
errorMessagestringIf the request returns an error, the description of the error.Body
All other fields*Attributes["cloudtrail.*"]
+ +### Google Cloud Logging + +Field | Type | Description | Maps to Unified Model Field +-----------------|--------------------| ------------------------------------------------------- | --------------------------- +timestamp | string | The time the event described by the log entry occurred. | Timestamp +resource | MonitoredResource | The monitored resource that produced this log entry. | Resource +log_name | string | The URL-encoded LOG_ID suffix of the log_name field identifies which log stream this entry belongs to. | Attributes["gcp.log_name"] +json_payload | google.protobuf.Struct | The log entry payload, represented as a structure that is expressed as a JSON object. | Body +proto_payload | google.protobuf.Any | The log entry payload, represented as a protocol buffer. | Body +text_payload | string | The log entry payload, represented as a Unicode string (UTF-8). | Body +severity | LogSeverity | The severity of the log entry. | Severity +trace | string | The trace associated with the log entry, if any. | TraceId +span_id | string | The span ID within the trace associated with the log entry. | SpanId +labels | map | A set of user-defined (key, value) data that provides additional information about the log entry. | Attributes +http_request | HttpRequest | The HTTP request associated with the log entry, if any. | Attributes["gcp.http_request"] +All other fields | | | Attributes["gcp.*"] + +### Elastic Common Schema + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeDescriptionMaps to Unified Model Field
@timestampdatetimeTime the event was recordedTimestamp
messagestringAny type of messageBody
labelskey/valueArbitrary labels related to the eventAttributes[*]
tagsarray of stringList of values related to the event?
trace.idstringTrace IDTraceId
span.id*stringSpan IDSpanId
agent.ephemeral_idstringEphemeral ID created by agent**Resource
agent.idstringUnique identifier of this agent**Resource
agent.namestringName given to the agentResource["telemetry.sdk.name"]
agent.typestringType of agentResource["telemetry.sdk.language"]
agent.versionstringVersion of agentResource["telemetry.sdk.version"]
source.ip, client.ipstringThe IP address that the request was made from.Attributes["net.sock.peer.addr"] or Attributes["net.sock.host.addr"]
cloud.account.idstringID of the account in the given cloudResource["cloud.account.id"]
cloud.availability_zonestringAvailability zone in which this host is running.Resource["cloud.zone"]
cloud.instance.idstringInstance ID of the host machine.**Resource
cloud.instance.namestringInstance name of the host machine.**Resource
cloud.machine.typestringMachine type of the host machine.**Resource
cloud.providerstringName of the cloud provider. Example values are aws, azure, gcp, or digitalocean.Resource["cloud.provider"]
cloud.regionstringRegion in which this host is running.Resource["cloud.region"]
cloud.image.id*stringResource["host.image.name"]
container.idstringUnique container idResource["container.id"]
container.image.namestringName of the image the container was built on.Resource["container.image.name"]
container.image.tagArray of stringContainer image tags.**Resource
container.labelskey/valueImage labels.Attributes[*]
container.namestringContainer name.Resource["container.name"]
container.runtimestringRuntime managing this container. Example: "docker"**Resource
destination.addressstringDestination address for the eventAttributes["destination.address"]
error.codestringError code describing the error.Attributes["error.code"]
error.idstringUnique identifier for the error.Attributes["error.id"]
error.messagestringError message.Attributes["error.message"]
error.stack_tracestringThe stack trace of this error in plain text.Attributes["error.stack_trace]
host.architecturestringOperating system architecture**Resource
host.domainstringName of the domain of which the host is a member. + +For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.**Resource
host.hostnamestringHostname of the host. + +It normally contains what the hostname command returns on the host machine.Resource["host.hostname"]
host.idstringUnique host id.Resource["host.id"]
host.ipArray of stringHost IPResource["host.ip"]
host.macarray of stringMAC addresses of the hostResource["host.mac"]
host.namestringName of the host. + +It may contain what hostname returns on Unix systems, the fully qualified, or a name specified by the user. Resource["host.name"]
host.typestringType of host.Resource["host.type"]
host.uptimestringSeconds the host has been up.?
service.ephemeral_id + +stringEphemeral identifier of this service**Resource
service.idstringUnique identifier of the running service. If the service is comprised of many nodes, the service.id should be the same for all nodes.**Resource
service.namestringName of the service data is collected from.Resource["service.name"]
service.node.namestringSpecific node serving that serviceResource["service.instance.id"]
service.statestringCurrent state of the service.Attributes["service.state"]
service.typestringThe type of the service data is collected from.**Resource
service.versionstringVersion of the service the data was collected from.Resource["service.version"]
+ +\* Not yet formalized into ECS. + +\*\* A resource that doesn’t exist in the +[OpenTelemetry resource semantic convention](../resource/semantic_conventions/README.md). + +This is a selection of the most relevant fields. See +[for the full reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) +for an exhaustive list. + +## Appendix B: `SeverityNumber` example mappings + +|Syslog |WinEvtLog |Log4j |Zap |java.util.logging|.NET (Microsoft.Extensions.Logging)|SeverityNumber| +|-------------|-----------|------|------|-----------------|-----------------------------------|--------------| +| | |TRACE | | FINEST |LogLevel.Trace |TRACE | +|Debug |Verbose |DEBUG |Debug | FINER |LogLevel.Debug |DEBUG | +| | | | | FINE | |DEBUG2 | +| | | | | CONFIG | |DEBUG3 | +|Informational|Information|INFO |Info | INFO |LogLevel.Information |INFO | +|Notice | | | | | |INFO2 | +|Warning |Warning |WARN |Warn | WARNING |LogLevel.Warning |WARN | +|Error |Error |ERROR |Error | SEVERE |LogLevel.Error |ERROR | +|Critical |Critical | |Dpanic| | |ERROR2 | +|Alert | | |Panic | | |ERROR3 | +|Emergency | |FATAL |Fatal | |LogLevel.Critical |FATAL | diff --git a/specification/logs/data-model.md b/specification/logs/data-model.md index 58faec0e78c..2c57f4915d7 100644 --- a/specification/logs/data-model.md +++ b/specification/logs/data-model.md @@ -34,18 +34,7 @@ * [Field: `Attributes`](#field-attributes) + [Errors and Exceptions](#errors-and-exceptions) - [Example Log Records](#example-log-records) -- [Appendix A. Example Mappings](#appendix-a-example-mappings) - * [RFC5424 Syslog](#rfc5424-syslog) - * [Windows Event Log](#windows-event-log) - * [SignalFx Events](#signalfx-events) - * [Splunk HEC](#splunk-hec) - * [Log4j](#log4j) - * [Zap](#zap) - * [Apache HTTP Server access log](#apache-http-server-access-log) - * [CloudTrail Log Event](#cloudtrail-log-event) - * [Google Cloud Logging](#google-cloud-logging) -- [Elastic Common Schema](#elastic-common-schema) -- [Appendix B: `SeverityNumber` example mappings](#appendix-b-severitynumber-example-mappings) +- [Example Mappings](#example-mappings) - [References](#references) @@ -147,7 +136,7 @@ fields: conventions for key names and possible values that allow all parties that work with the field to have the same interpretation of the data. See references to semantic conventions for `Resource` and `Attributes` fields and examples in - [Appendix A](#appendix-a-example-mappings). + [Appendix A](./data-model-appendix.md#appendix-a-example-mappings). The reasons for having these 2 kinds of fields are: @@ -178,7 +167,7 @@ top-level structure of the record. ## Log and Event Record Definition -[Appendix A](#appendix-a-example-mappings) contains many examples that show how +[Appendix A](./data-model-appendix.md#appendix-a-example-mappings) contains many examples that show how existing log formats map to the fields defined below. If there are questions about the meaning of the field reviewing the examples may be helpful. @@ -348,7 +337,7 @@ define a severity or log level concept then it is recommended to set record represents a non-erroneous event the `SeverityNumber` field may be omitted or may be set to any numeric value less than ERROR (numeric 17). The recommended value in this case is INFO (numeric 9). See -[Appendix B](#appendix-b-severitynumber-example-mappings) for more mapping +[Appendix B](./data-model-appendix.md#appendix-b-severitynumber-example-mappings) for more mapping examples. #### Displaying Severity @@ -478,815 +467,10 @@ If included, they MUST follow the OpenTelemetry For example log records see [JSON File serialization](../protocol/file-exporter.md#examples). -## Appendix A. Example Mappings - -This section contains examples of mapping of other events and logs formats to -this data model. - -### RFC5424 Syslog - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
PropertyTypeDescriptionMaps to Unified Model Field
TIMESTAMPTimestampTime when an event occurred measured by the origin clock.Timestamp
SEVERITYenumDefines the importance of the event. Example: `Debug`Severity
FACILITYenumDescribes where the event originated. A predefined list of Unix processes. Part of event source identity. Example: `mail system`Attributes["syslog.facility"]
VERSIONnumberMeta: protocol version, orthogonal to the event.Attributes["syslog.version"]
HOSTNAMEstringDescribes the location where the event originated. Possible values are FQDN, IP address, etc.Resource["host.hostname"]
APP-NAMEstringUser-defined app name. Part of event source identity.Resource["service.name"]
PROCIDstringNot well defined. May be used as a meta field for protocol operation purposes or may be part of event source identity.Attributes["syslog.procid"]
MSGIDstringDefines the type of the event. Part of event source identity. Example: "TCPIN"Attributes["syslog.msgid"]
STRUCTURED-DATAarray of maps of string to stringA variety of use cases depending on the SDID: -Can describe event source identity -Can include data that describes particular occurrence of the event. -Can be meta-information, e.g. quality of timestamp value.SDID origin.swVersion map to Resource["service.version"] - -SDID origin.ip map to attribute["net.sock.host.addr"] - -Rest of SDIDs -> Attributes["syslog.*"]
MSGstringFree-form text message about the event. Typically human readable.Body
- -### Windows Event Log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
PropertyTypeDescriptionMaps to Unified Model Field
TimeCreatedTimestampThe time stamp that identifies when the event was logged.Timestamp
LevelenumContains the severity level of the event.Severity
ComputerstringThe name of the computer on which the event occurred.Resource["host.hostname"]
EventIDuintThe identifier that the provider used to identify the event.Attributes["winlog.event_id"]
MessagestringThe message string.Body
Rest of the fields.anyAll other fields in the event.Attributes["winlog.*"]
- -### SignalFx Events - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
TimestampTimestampTime when the event occurred measured by the origin clock.Timestamp
EventTypestringShort machine understandable string describing the event type. SignalFx specific concept. Non-namespaced. Example: k8s Event Reason field.Attributes["com.splunk.signalfx.event_type"]
CategoryenumDescribes where the event originated and why. SignalFx specific concept. Example: AGENT. Attributes["com.splunk.signalfx.event_category"]
Dimensionsmap<string, string>Helps to define the identity of the event source together with EventType and Category. Multiple occurrences of events coming from the same event source can happen across time and they all have the value of Dimensions. Resource
Propertiesmap<string, any>Additional information about the specific event occurrence. Unlike Dimensions which are fixed for a particular event source, Properties can have different values for each occurrence of the event coming from the same event source.Attributes
- -### Splunk HEC - -We apply this mapping from HEC to the unified model: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
timenumeric, stringThe event time in epoch time format, in seconds.Timestamp
hoststringThe host value to assign to the event data. This is typically the host name of the client that you are sending data from.Resource["host.name"]
sourcestringThe source value to assign to the event data. For example, if you are sending data from an app you are developing, you could set this key to the name of the app.Resource["com.splunk.source"]
sourcetypestringThe sourcetype value to assign to the event data.Resource["com.splunk.sourcetype"]
eventanyThe JSON representation of the raw body of the event. It can be a string, number, string array, number array, JSON object, or a JSON array.Body
fieldsmap<string, any>Specifies a JSON object that contains explicit custom fields.Attributes
indexstringThe name of the index by which the event data is to be indexed. The index you specify here must be within the list of allowed indexes if the token has the indexes parameter set.Attributes["com.splunk.index"]
- -When mapping from the unified model to HEC, we apply this additional mapping: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Unified model elementTypeDescriptionMaps to HEC
SeverityTextstringThe severity of the event as a human-readable string.fields['otel.log.severity.text']
SeverityNumberstringThe severity of the event as a number.fields['otel.log.severity.number']
NamestringShort event identifier that does not contain varying parts.fields['otel.log.name']
TraceIdstringRequest trace id.fields['trace_id']
SpanIdstringRequest span id.fields['span_id']
TraceFlagsstringW3C trace flags.fields['trace_flags']
- -### Log4j - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
InstantTimestampTime when an event occurred measured by the origin clock.Timestamp
LevelenumLog level.Severity
MessagestringHuman readable message.Body
All other fieldsanyStructured data.Attributes
- -### Zap - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
tsTimestampTime when an event occurred measured by the origin clock.Timestamp
levelenumLogging level.Severity
callerstringCalling function's filename and line number. -Attributes, key=TBD
msgstringHuman readable message.Body
All other fieldsanyStructured data.Attributes
- -### Apache HTTP Server access log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
%tTimestampTime when an event occurred measured by the origin clock.Timestamp
%astringClient addressAttributes["net.sock.peer.addr"]
%AstringServer addressAttributes["net.sock.host.addr"]
%hstringClient hostname.Attributes["net.peer.name"]
%mstringThe request method.Attributes["http.method"]
%v,%p,%U,%qstringMultiple fields that can be composed into URL.Attributes["http.url"]
%>sstringResponse status.Attributes["http.status_code"]
All other fieldsanyStructured data.Attributes, key=TBD
- -### CloudTrail Log Event - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
eventTimestringThe date and time the request was made, in coordinated universal time (UTC).Timestamp
eventSourcestringThe service that the request was made to. This name is typically a short form of the service name without spaces plus .amazonaws.com.Resource["service.name"]?
awsRegionstringThe AWS region that the request was made to, such as us-east-2.Resource["cloud.region"]
sourceIPAddressstringThe IP address that the request was made from.Attributes["net.sock.peer.addr"] or Attributes["net.sock.host.addr"]
errorCodestringThe AWS service error if the request returns an error.Attributes["cloudtrail.error_code"]
errorMessagestringIf the request returns an error, the description of the error.Body
All other fields*Attributes["cloudtrail.*"]
- -### Google Cloud Logging - -Field | Type | Description | Maps to Unified Model Field ------------------|--------------------| ------------------------------------------------------- | --------------------------- -timestamp | string | The time the event described by the log entry occurred. | Timestamp -resource | MonitoredResource | The monitored resource that produced this log entry. | Resource -log_name | string | The URL-encoded LOG_ID suffix of the log_name field identifies which log stream this entry belongs to. | Attributes["gcp.log_name"] -json_payload | google.protobuf.Struct | The log entry payload, represented as a structure that is expressed as a JSON object. | Body -proto_payload | google.protobuf.Any | The log entry payload, represented as a protocol buffer. | Body -text_payload | string | The log entry payload, represented as a Unicode string (UTF-8). | Body -severity | LogSeverity | The severity of the log entry. | Severity -trace | string | The trace associated with the log entry, if any. | TraceId -span_id | string | The span ID within the trace associated with the log entry. | SpanId -labels | map | A set of user-defined (key, value) data that provides additional information about the log entry. | Attributes -http_request | HttpRequest | The HTTP request associated with the log entry, if any. | Attributes["gcp.http_request"] -All other fields | | | Attributes["gcp.*"] - -## Elastic Common Schema - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FieldTypeDescriptionMaps to Unified Model Field
@timestampdatetimeTime the event was recordedTimestamp
messagestringAny type of messageBody
labelskey/valueArbitrary labels related to the eventAttributes[*]
tagsarray of stringList of values related to the event?
trace.idstringTrace IDTraceId
span.id*stringSpan IDSpanId
agent.ephemeral_idstringEphemeral ID created by agent**Resource
agent.idstringUnique identifier of this agent**Resource
agent.namestringName given to the agentResource["telemetry.sdk.name"]
agent.typestringType of agentResource["telemetry.sdk.language"]
agent.versionstringVersion of agentResource["telemetry.sdk.version"]
source.ip, client.ipstringThe IP address that the request was made from.Attributes["net.sock.peer.addr"] or Attributes["net.sock.host.addr"]
cloud.account.idstringID of the account in the given cloudResource["cloud.account.id"]
cloud.availability_zonestringAvailability zone in which this host is running.Resource["cloud.zone"]
cloud.instance.idstringInstance ID of the host machine.**Resource
cloud.instance.namestringInstance name of the host machine.**Resource
cloud.machine.typestringMachine type of the host machine.**Resource
cloud.providerstringName of the cloud provider. Example values are aws, azure, gcp, or digitalocean.Resource["cloud.provider"]
cloud.regionstringRegion in which this host is running.Resource["cloud.region"]
cloud.image.id*stringResource["host.image.name"]
container.idstringUnique container idResource["container.id"]
container.image.namestringName of the image the container was built on.Resource["container.image.name"]
container.image.tagArray of stringContainer image tags.**Resource
container.labelskey/valueImage labels.Attributes[*]
container.namestringContainer name.Resource["container.name"]
container.runtimestringRuntime managing this container. Example: "docker"**Resource
destination.addressstringDestination address for the eventAttributes["destination.address"]
error.codestringError code describing the error.Attributes["error.code"]
error.idstringUnique identifier for the error.Attributes["error.id"]
error.messagestringError message.Attributes["error.message"]
error.stack_tracestringThe stack trace of this error in plain text.Attributes["error.stack_trace]
host.architecturestringOperating system architecture**Resource
host.domainstringName of the domain of which the host is a member. - -For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.**Resource
host.hostnamestringHostname of the host. - -It normally contains what the hostname command returns on the host machine.Resource["host.hostname"]
host.idstringUnique host id.Resource["host.id"]
host.ipArray of stringHost IPResource["host.ip"]
host.macarray of stringMAC addresses of the hostResource["host.mac"]
host.namestringName of the host. - -It may contain what hostname returns on Unix systems, the fully qualified, or a name specified by the user. Resource["host.name"]
host.typestringType of host.Resource["host.type"]
host.uptimestringSeconds the host has been up.?
service.ephemeral_id - -stringEphemeral identifier of this service**Resource
service.idstringUnique identifier of the running service. If the service is comprised of many nodes, the service.id should be the same for all nodes.**Resource
service.namestringName of the service data is collected from.Resource["service.name"]
service.node.namestringSpecific node serving that serviceResource["service.instance.id"]
service.statestringCurrent state of the service.Attributes["service.state"]
service.typestringThe type of the service data is collected from.**Resource
service.versionstringVersion of the service the data was collected from.Resource["service.version"]
- -\* Not yet formalized into ECS. - -\*\* A resource that doesn’t exist in the -[OpenTelemetry resource semantic convention](../resource/semantic_conventions/README.md). - -This is a selection of the most relevant fields. See -[for the full reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) -for an exhaustive list. - -## Appendix B: `SeverityNumber` example mappings - -|Syslog |WinEvtLog |Log4j |Zap |java.util.logging|.NET (Microsoft.Extensions.Logging)|SeverityNumber| -|-------------|-----------|------|------|-----------------|-----------------------------------|--------------| -| | |TRACE | | FINEST |LogLevel.Trace |TRACE | -|Debug |Verbose |DEBUG |Debug | FINER |LogLevel.Debug |DEBUG | -| | | | | FINE | |DEBUG2 | -| | | | | CONFIG | |DEBUG3 | -|Informational|Information|INFO |Info | INFO |LogLevel.Information |INFO | -|Notice | | | | | |INFO2 | -|Warning |Warning |WARN |Warn | WARNING |LogLevel.Warning |WARN | -|Error |Error |ERROR |Error | SEVERE |LogLevel.Error |ERROR | -|Critical |Critical | |Dpanic| | |ERROR2 | -|Alert | | |Panic | | |ERROR3 | -|Emergency | |FATAL |Fatal | |LogLevel.Critical |FATAL | +## Example Mappings + +For example log format mappings, see the +[Data Model Appendix](./data-model-appendix.md). ## References