iNaturalist Finetune application

[brunosan]

This discussion aims to scope how Clay could help iNaturalist (PoC @loarie).

iNaturalist is an "online social network of people sharing biodiversity information to help each other learn about nature". It's an app to make a picthre of a plant or animal and identify it. It's now an independent 501c3 entity. It has ~200M observations of living organisms around the world from 8M users.

One of their products is their Geomodel where they crunch the co-ocurrence of all the observation of 80k species with sufficient data (+ elevation). the model then infer new locations where a specie could be based on the presence of other species.

The trained model takes a location as input and returns the most likely species at that location as output.

Blue predicted, Red measured.

How can Clay help iNaturalist?

• Clay can provide an embedding for any location (and time) with the "semantics" learned from EO. One can imagine this as presence of forest types, roads, water, crops, ...

• This input could be part of the training for the model. That should drastically increase the capacity to correlate local semantics with enabling conditions for species presence.

Is this a sensible expectation, @loarie?
If we provide the embeddgins, could you do a proof of concept? Or would you need anything else to test this?

[loarie]

@brunosan, great talking today. I'm looping in @alexshepard on the iNat team and @macaodha who's been our key collaborator on the geomodel (see https://arxiv.org/abs/2306.02564)

It was great to hear about your vision for location embedding with Sentinel satellite images and all the applications that will have. I agree it would be very interesting to explore how your work could substitute in for the elevation layer the Geomodel is currently using to potentially provide a much richer environmental covariate surface to use in making species predictions.

Also thanks for sharing this paper https://arxiv.org/pdf/2311.17179.pdf

[macaodha]

Agreed. Would be very happy to discuss more.

[brunosan]

Should we want to test this, is there a specific region and time window whose embeddings (vectors representing the semantics of a 256x256 window at 10m/pixel resolution) could help us make a proof of concept most easily?
E.g. We create embeddings for Costa Rica for 2022, and then you can add those as co-variables in your model. Bigger extent? Smaller?

[macaodha]

I'll follow up via email.