improve single trace query performance

[shichaoyuan]

Question

When query single trace by trace_id, we will scan all shards, which is a problem when the amount of data is large.

Reason

must scan all shard

how to improve

Customizing the shard routing rule by setting routing = trace_id, which we only need scan one shard.

https://www.cnblogs.com/Neeo/articles/10869556.html

related issue

#5023

[wu-sheng]

This should be able to improve the performance. If you want to implement this, propose what is your proposed way to add this.
SkyWalking doesn't have direct trace concept for storage level, it is a kind of record(no-aggregation) data only. I believe if you want to add this, you need some kinds of specific annotation for the traceId field on the ElasticSearch storage.

[shichaoyuan]

insert flow

1. specify routing through annotation

in org.apache.skywalking.oap.server.core.storage.annotation.ElasticSearch add

@interface Routing {
        String value();
}

in SegmentRecord add

@ElasticSearch.Routing(SegmentRecord.TRACE_ID)
public class SegmentRecord extends Record {

2. add routing info in Model

in org.apache.skywalking.oap.server.core.storage.model.Model add ElasticSearchModelExtension

public class ElasticSearchModelExtension {
    private String routingColumn;
}

parse Routing annotaion in org.apache.skywalking.oap.server.core.storage.model.StorageModels, add to ElasticSearchModelExtension

3. add routing info to the InsertRequest

in org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.RecordEsDAO, add routing info to the InsertRequest

4. codec routing field

org.apache.skywalking.library.elasticsearch.requests.factory.Codec

query flow

in org.apache.skywalking.oap.server.storage.plugin.elasticsearch.query.TraceQueryEsDAO

add queryByTraceId{routing=traceId}

[wu-sheng]

Generally, I think this should be good. Could you send a pull request? @kezhenxu94 Any concern or suggestion?

[wu-sheng]

One thing, we have log and zipkin traces, which has similar issues.
Adding this to zipkin span is easy, how this works if we add it to log? What happens in routing if routing field is null?(The log may not bind with trace context)

[shichaoyuan]

submitted a pr #10282 .

based on the code, we can discuss better