Better names for API base URLs?

[jimallman]

Hi folks, regarding the ongoing tweaks to web-app config files (from Ansible templates), I'm reviewing the base URLs we define and their names, vs. what each actually provides. Can we make these consistent with the current architecture, esp. with an eye toward possibly hosting services on multiple servers?

This is more about making life easy for future maintainers, moreso than functionality. Ideally, we'd

• use names that reflect our current sub-projects (e.g. otindex vs. oti; something better than treemachine...)
• consolidate names where possible (if we're now getting more services from a single system)
• share base URLs for multiple services, if we're sure some services will always be on the same machine(?)

Anyway, here's the current list of API "domains" (base URLs), along with representative endpoints for each:

[domains]
treemachine = http://dev.opentreeoflife.org/treemachine:7474  [tree_of_life/about, tree_of_life/mrca]
taxomachine = http://dev.opentreeoflife.org/taxomachine:7476  [tnrs/match_names, tnrs/infer_context, taxonomy/mrca]
oti = http://dev.opentreeoflife.org/oti                       [studies/find_studies, studies/find_trees]
opentree_api = http://dev.opentreeoflife.org/phylesystem      [study/*, tree/*, phylesystem_config, render_markdown, trees_in_synth]
conflict_api = http://dev.opentreeoflife.org/conflict         [conflict/conflict-status]

[method_urls]
...

Here's what I have in mind so far:

[api_base_urls]
otcetera = http://dev.opentreeoflife.org/treemachine:7474     [WAS treemachine + taxomachine]
otindex = http://dev.opentreeoflife.org/oti                   [WAS oti]
phylesystem = http://dev.opentreeoflife.org/phylesystem       [WAS opentree_api]
conflict = http://dev.opentreeoflife.org/conflict             [WAS conflict_api]

[api_endpoints]
...

I think this does a nice job of making the base URLs more recognizable, though some names (otcetera) are not as descriptive of the services offered as, say, taxonomy or tnrs might be. Thoughts?

Any changes here will need a parallel pull request in the web-apps, but that's easy enough. I just want to make sure these changes are robust and correct first.

[snacktavish]

@bredelings and I chatted a bit on gitter. We were briefly thrown off bc the URLS themselves are not up to date. But the idea of simplifying the base url construction makes some sense... although I'm nut sure splitting them up is necessary, because they are all actually just devapi... and the subdomains are

• /tree_of_life (otcetera)
• /taxonomy (otcetera)
• /tnrs (otcetera)
• /conflict (otcetera)
• /studies (otindex)
• /study (phylesystem-api)

Even though we use a variety of variables here : https://github.com/OpenTreeOfLife/ot-ansible/blob/development/roles/webapp/templates/curatorful_config.j2

They basically all just get set to api.opentreeoflife.org or devapi.opentreeoflife.org!
e.g. https://github.com/OpenTreeOfLife/ot-ansible/blob/2d2e51ea307acf53c99af1dd492ec832c486dded/group_vars/devtree.yml

The one occasion where we do differentiate is on dev in the synth tree viewer, where /studies (otindex) queries should go to production (api.opentreeoflife.org) while everything else (tree of life, taxonomy, I guess) goes to dev.

[jimallman]

Apologies, the URLs listed are old and should be ignored.

They basically all just get set to api.opentreeoflife.org or devapi.opentreeoflife.org!

The one occasion where we do differentiate is on dev in the synth tree viewer, where /studies (otindex) queries should go to production (api.opentreeoflife.org) while everything else (tree of life, taxonomy, I guess) goes to dev.

Good catch regarding otindex going to production (all its methods? or only some?). And yes, maybe this is all cruft if we're passing all API calls through a common base URL (dev)api.opentreeoflife.org. Hmmm.

[jimallman]

A single common API base URL like https://(dev)api.opentreeoflife.org/ sort of moves all the answers to that server's Apache config, which makes more sense from a DRY perspective. So a maintainer with questions will need to keep digging, but ultimately that's probably less confusing than having all these base URLs defined as the same one or two URLs.

[snacktavish]

Makes sense to me! The one place I think we use the swap is here, where the "study_index_endpoint" is production, even though everything else is dev.:

ot-ansible/group_vars/devtree.yml

Line 21 in 2d2e51e

study_index_endpoint: https://otindex.opentreeoflife.org

And then we use that substitution here:
https://github.com/OpenTreeOfLife/ot-ansible/blob/development/roles/webapp/templates/curatorless_config.j2#L45

Even though "study_index_endpoint" doesn't get set in [domains] section of the config.

I think even that may muddy the water a bit, because using "https://otindex.opentreeoflife.org" or "https://api.opentreeoflife.org" would have the exact same effects (I think).

[jimallman]

The one place I think we use the swap is here, where the "study_index_endpoint" is production, even though everything else is dev.

This is already commented nicely in Ansible, and there are already sensible default values set in group_vars/all.yml. Nice!

[jimallman]

FYI - I'm restoring the odd curation-app URL in the "curatorless" synth-tree viewer in this tiny opentree feature branch.

[jimallman]

And then we use that substitution here: https://github.com/OpenTreeOfLife/ot-ansible/blob/development/roles/webapp/templates/curatorless_config.j2#L45

Ah, this should probably be used in all three of these methods, no?

Even though "study_index_endpoint" doesn't get set in [domains] section of the config.

Yeah, that makes it an exception, and a confusing one at that. I think defining this odd endpoint URL is reasonable here, as long as we also document it clearly as I've done for curator_url here.

I think even that may muddy the water a bit, because using "https://otindex.opentreeoflife.org" or "https://api.opentreeoflife.org" would have the exact same effects (I think).

I actually prefer "https://api.opentreeoflife.org" for this purpose, as in context it should jump out as different: "Oh, that's the production API server." 😮