[Optimization] Profile and optimize https://api.outbreak.info/genomics/lineage-mutations

[flaneuse]

See #61, #63

[newgene]

Let's start with this one. Many of these heavy query handlers use create_nested_mutation_query function to construct an ES query:

outbreak.api/web/handlers/genomics/util.py

Lines 154 to 155 in 6a88be9

	def create_nested_mutation_query(location_id = None, lineages = [], mutations = []):
	# For multiple lineages and mutations: (Lineage 1 AND mutation 1 AND mutation 2..) OR (Lineage 2 AND mutation 1 AND mutation 2..) ...

Given this issue #59, this function needs to be changed to this:

def create_nested_mutation_query(location_id = None, lineages = [], mutations = [], genes=[]):

• Let's confirm the logic here for each filter when passed with multiple values:

  • lineages --> OR
  • mutations --> AND (based on the current implementation, but should it be an OR?)
  • genes --> Should it be an AND or OR here?

• Any other potential filtering field we might need from any handler?

[flaneuse]

It should be:

• lineages combined by OR
• mutations combined by AND
• genes combined by AND

For instance: the query:
lineages=["BA.1", "BA.5"], mutations=["S:E484K", "S:N501Y"], genes=["Orf1a", "S", N"]

would distribute the mutations to each of the lineages:

1. filter records with `(pangolin_lineage == "BA.1" & mutations.mutation == "S:E484K" & mutations.mutation == "S:N501Y") | (pangolin_lineage == "BA.5" & mutations.mutation == "S:E484K" & mutations.mutation == "S:N501Y")"
2. after identifying those records, collect the mutations objects and filter out gene == Orf1a | gene == S | gene == N
3. using only those records, count mutations.mutation.

@gkarthik had proposed changing the combination of lineages and mutations to be more generic, however: #55

It's more work, but it'd be useful to make it more generic.