diff --git a/rfcs/text/0008-email.md b/rfcs/text/0008-email.md new file mode 100644 index 0000000000..e7c15c8f7e --- /dev/null +++ b/rfcs/text/0008-email.md @@ -0,0 +1,166 @@ +# 0008: Email + + +- Stage: **1 (proposal)** +- Date: **2020-11-30** + +This RFC proposes a new top-level field to facilitate email use cases. + + + + + +## Fields + + + +Email specific fields: + +| field | type | description | +| --- | --- | --- | +| `email.bcc.addresses` | `wildcard[]` | Addresses of Bcc's | +| `email.cc.addresses` | `wildcard[]` | Addresses of Cc's | +| `email.attachments_count` | long | A field outside the flattened structure to control how many attachments are included in the email | +| `email.attachments` | flattened | A flattened field for anything related to attachments. This allows objects being stored with all information for each file when you have multiple attachments | +| `email.direction` | keyword | Direction of the message based on the sending and receving domains | +| `email.sender.address` | wildcard | Senders email address | +| `email.sender.domain` | wildcard | Domain of the sender | +| `email.sender.top_level_domain` | keyword | Top level domain of the sender | +| `email.sender.registered_domain` | wildcard | Registered domain of the sender | +| `email.sender.subdomain` | keyword | Subdomain of the sender | +| `email.message_id` | keyword | Internet message ID of the message | +| `email.reply_to.address` | wildcard | Reply-to address | +| `email.return_path.address` | wildcard | The return address for the message | +| `email.size` | long | Total size of the message, in bytes, including attachments | +| `email.subject` | wildcard | Subject of the message | +| `email.recipients.addresses` | `keyword[]` | Recipient addresses | +| `email.domains` | `keyword[]` | domains related to the email | + + +Other ECS fields used together with email usecases: +| field | description | +| --- | --- | +| `event.duration` | The duration related to the email event. Could be the total duration in Quarantine, how long the email took to send from source to destination etc | +| `event.start` | When the email event started +| `event.end` | When the email event ended +| `process.name` | When the event is related to a server or client. Does not take MTA into account which is part of a ongoing discussion | +| `network.protocol` | Type of email protocol used | +| `tls.*` | Used for TLS related information for the connection to for example a SMTP server over TLS | + + + +## Usage + + + +Email use cases stretch across all three Elastic solutions - Search, Observe, Protect. Whether it's searching for content within email, ensuring email infrastrucure is operational or detecting email based attacks, there are many possibilities for email fields within ECS. + +## Source data + + + +- **Email Analytics**: [Hubspot](https://legacydocs.hubspot.com/docs/methods/email/email_events_overview), Marketo, Salesforce Pardot +- **Email Server**: [O365 Message Tracing](https://docs.microsoft.com/en-us/exchange/monitoring/trace-an-email-message/run-a-message-trace-and-view-results), [Postfix](https://nxlog.co/documentation/nxlog-user-guide/postfix.html) +- **Email Security**: [Barracuda](https://campus.barracuda.com/product/emailsecuritygateway/doc/12193950/syslog-and-the-barracuda-email-security-gateway/), [Forcepoint](https://www.websense.com/content/support/library/email/v85/email_siem/siem_log_map.pdf), [Mimecast](https://www.mimecast.com/tech-connect/documentation/tutorials/understanding-siem-logs/), [Proofpoint](https://help.proofpoint.com/Threat_Insight_Dashboard/API_Documentation/SIEM_API) + + + + + +## Scope of impact + + + +## Concerns + + +Current concerns or topics still being discussed from stage 1: + +- Whether we want to add specific fields for email protocols, either as a root field or nested under email.* (SMTP, IMAP, POP etc). +- Need to make sure that the ECS fieldset for email catches all common usecases, for example spam, metrics and deliverables and logging. +- Whether we want to create a new event.category field (email) and which event.type it should be combined with. +- The email RFC will be the first ECS fieldset that uses the flattened datatype (for attachments), need to ensure that there will be major issues related to this. + + + + + + + +## Real-world implementations + + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @p1llus | Author +* @jamiehynds | Sponsor + + + + +## References + + + +- [Hubspot](https://legacydocs.hubspot.com/docs/methods/email/email_events_overview) +- [O365 Message Tracing](https://docs.microsoft.com/en-us/exchange/monitoring/trace-an-email-message/run-a-message-trace-and-view-results) +- [Postfix](https://nxlog.co/documentation/nxlog-user-guide/postfix.html) +- [Barracuda](https://campus.barracuda.com/product/emailsecuritygateway/doc/12193950/syslog-and-the-barracuda-email-security-gateway/) +- [Forcepoint](https://www.websense.com/content/support/library/email/v85/email_siem/siem_log_map.pdf) +- [Mimecast](https://www.mimecast.com/tech-connect/documentation/tutorials/understanding-siem-logs/) +- [Proofpoint](https://help.proofpoint.com/Threat_Insight_Dashboard/API_Documentation/SIEM_API) + +### RFC Pull Requests + + + +* Stage 1: https://github.com/elastic/ecs/pull/999 + +