[APM] Time range metadata API to drive available fields rather than individual logics to check for them

[achyutjhunjhunwala]

Currently we have the Time Range Metadata returning specifics about hasDurationSummaryField field as part of its response.

kibana/x-pack/plugins/apm/common/time_range_metadata.ts

Lines 10 to 15 in bb08db9

	export interface TimeRangeMetadata {
	isUsingServiceDestinationMetrics: boolean;
	sources: Array<
	ApmDataSource & { hasDocs: boolean; hasDurationSummaryField: boolean }
	>;
	}

This adds limitation given we might introduce more such fields in the future or new metrics might come up where such fields would add little to no value.

PS

I had an initial discussion with @sqren on this and this ticket is to brainstorm on the idea proposed below. Inspiration for this ticket came from here

Idea

Instead of returning such metadata fields in addition to Document types, the API can start returning better Document Types.

For example, transaction.duration.summary field is a new field added to transaction metrics documents. This means we have a new version of a Transaction Document Type which now supports an additional field transaction.duration.summary. Hence this can be treated as version 2 of this document.

Currently we identify documents types like this

type AnyApmDocumentType =
  | ApmDocumentType.ServiceTransactionMetric
  | ApmDocumentType.TransactionMetric
  | ApmDocumentType.TransactionEvent
  | ApmDocumentType.ServiceDestinationMetric
  | ApmDocumentType.ServiceSummaryMetric
  | ApmDocumentType.ErrorEvent

If we change them a bit to return this new versioned document, like

type AnyApmDocumentType =
  | ApmDocumentType.ServiceTransactionMetric
  | ApmDocumentType.TransactionMetric
  | ApmDocumentType.TransactionEvent
  | ApmDocumentType.ServiceDestinationMetric
  | ApmDocumentType.ServiceSummaryMetric
  | ApmDocumentType.ErrorEvent

  | ApmDocumentType.TransactionMetricV2

Now this is the document type which the Time Range Metadata API should return. With this we can encapsulate the whole logic and the client side now does not need to know about the availability of specific fields and what they need to pass while calling other APIs.

The whole flow would look like -

1. Get the document type from Time Range Metadata API.
2. Pass the Document Type as it is with RollUpIntervals to subsequent API calls
3. On the server side, we can may be enhance the apmEventClient to handle these specially created new Document Types and add filter queries accordingly.

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[smith]

Moving this back to backlog in favor of #180444.

[cauemarcondes]

@smith I'm missing the connection between this issue and this one #180444, can you elaborate?

[smith]

@smith I'm missing the connection between this issue and this one #180444, can you elaborate?

This issue was moved out of "Ready" and back to the backlog. The other issue was moved out of the backlog and in to Ready.