Skip to content

Commit

Permalink
Merge branch 'release/2024.2.0' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
mmcfarland committed May 21, 2024
2 parents 93ca129 + 42698b3 commit 3199d21
Show file tree
Hide file tree
Showing 35 changed files with 46 additions and 955 deletions.
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@ First, copy `.env.sample` file to `.env`, and ensure the configuration values ar
|`REACT_APP_TILER_ROOT`| Optional | The root URL for the data tiler API, if not hosted from the domain of the STAC API.
|`REACT_APP_IMAGE_API_ROOT`| PC APIs pcfunc endpoint | The root URL for the image data API for animations.
|`REACT_APP_AZMAPS_CLIENT_ID`| Retrieve from Azure Portal | The Client ID used to authenticate against Azure Maps.
|`REACT_APP_HUB_URL`| Optional. URL to root Hub instance | Used to enable a request to launch the Hub with a specific git hosted file.
|`REACT_APP_ONEDS_TENANT_KEY`| Lookup at <https://1dswhitelisting.azurewebsites.net/> | Telemetry key (not needed for dev)
|`REACT_APP_AUTH_URL`| Optional. URL to root pc-session-api instance | Used to enable login work.

Expand Down
1 change: 0 additions & 1 deletion cypress/e2e/explorer/selector.cy.js
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,6 @@ describe("Explorer selector tests", () => {
cy.getBySel("item-snippet-button").click();
cy.contains("import planetary_computer");
cy.contains("Copy");
cy.contains("Open Hub").focus().type("{esc}");

// Typing escape has closed the dialog
cy.contains("import planetary_computer").should("not.exist");
Expand Down
42 changes: 2 additions & 40 deletions docs/concepts/computing.md
Original file line number Diff line number Diff line change
@@ -1,50 +1,14 @@
# Computing on the Planetary Computer
# Computing with the Planetary Computer

The core components of the Planetary Computer are the datasets and APIs for querying them. This document provides an overview of the various ways you can compute on data hosted by the Planetary Computer.

Regardless of how you compute on the data, to ensure maximum efficiency you should locate your compute as close to the data as possible. Most of the Planetary Computer Data Catalog is hosted in Azure's **West Europe** region, so your compute should be there too.

## Use our JupyterHub

The [Planetary Computer Hub](https://planetarycomputer.microsoft.com/compute) is a [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) deployment in the West Europe Azure region. This is the easiest way to get started with computing on the Planetary Computer.
That said, the Planetary Computer Hub is focused mainly on convenience. We recommend it for prototypes and exploration, but production workloads should use one of options using your own compute detailed below.


```{note} You'll need to [request access](https://planetarycomputer.microsoft.com/account/request) to use the Planetary Computer Hub.
```

Once approved, you can log into the JupyterHub with your credentials. You'll get a computing environment that includes standard scientific and geospatial packages from one of the [Pangeo Docker Images](https://github.com/pangeo-data/pangeo-docker-images#pangeo-docker-images).

For scalable computation, the JupyterHub is also configured with [Dask Gateway](https://gateway.dask.org/). To create a Dask Cluster:

```python
>>> from dask_gateway import GatewayCluster

>>> cluster = GatewayCluster() # Creates the Dask Scheduler. Might take a minute.
>>> client = cluster.get_client()
>>> cluster.adapt(minimum=1, maximum=100)
>>> cluster
GatewayCluster<prod.fbfed27704fc4d1da027dad20471993b, status=running>
```

With this setup, all of the computation happens on Azure, whether on a single node or on a cluster with Dask.

![Diagram showing compute components within Azure](images/jupyterhub-diagram.png)

See [Scaling with Dask](../quickstarts/scale-with-dask.md) for an introduction to Dask. This setup was pioneered by the [Pangeo Community](https://pangeo.io/). The [Pangeo Cloud](https://pangeo.io/cloud.html) documention provides additional background on how to use Dask-enabled JupyterHubs.

## Use VS Code to connect to a remote Jupyter Kernel

See [Using VS Code](../overview/ui-vscode) for how to use Visual Studio Code as a user interface for the Planetary Computer's Compute.

## Use your own compute

The previous methods relied on compute provided by the Planetary Computer, which is a great way to get started with the Planetary Computer's APIs and Data.
For production workloads, we recommend deploying your own compute, which gives you more control over the hardware and software environment.

### Using GitHub Codespaces

See [Use GitHub Codespaces](../overview/ui-codespaces) for how to use [GitHub Codespaces][codespaces] as a user interface and execution environment using data from the on the Planetary Computer catalog.
See [Scaling with Dask](../quickstarts/scale-with-dask.md) for an introduction to Dask. This setup was pioneered by the [Pangeo Community](https://pangeo.io/). The [Pangeo Cloud](https://pangeo.io/cloud.html) documention provides additional background on how to use Dask-enabled JupyterHubs.

### Using Azure Machine Learning

Expand Down Expand Up @@ -92,5 +56,3 @@ and connect to it
Like the previous setup, the Dask scheduler and workers are running in Azure near the data. The local client might be outside of Azure.

![Diagram showing compute with self-managed Dask cluster](images/cloudprovider-diagram.png)

[codespaces]: https://github.com/features/codespaces
11 changes: 6 additions & 5 deletions docs/concepts/hub-deployment.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
## Deploy your own Hub
## Deploy your own JupyterHub

The Planetary Computer Hub is a [JupyterHub] deployment configured with [Dask Gateway][gateway] for scalable computing.
Deploying your own Hub is a good option for a team of users looking to work with data from the Planetary Computer. You might choose this instead of the [Planetary Computer Hub][hub] if you need to specialize the environment, require additional computation resources, or want to tie a Hub into a broader Azure deployment while still using data from the Planetary Computer.
Deploying your own [JupyterHub] is a good option for a team of users looking to
work with data from the Planetary Computer who need a specialized environment,
require additional computation resources, or want to tie a compute environmnt into a broader
Azure deployment while still using data from the Planetary Computer.

In this guide you will:

* Deploy an [AKS] cluster using the Azure CLI
* Deploy JupyterHub and Dask Gateway using the [daskhub] Helm chart.

We describe two deployment scenarios, a [simple](docs/concepts/hub-deployment/#simple-deployment) and a [recommended](docs/concepts/hub-deployment/#recommended-deployment) deployment. If you're new to Azure, Kubernetes, or JupyterHub, then you should try the simple deployment to verify that the basics work, before moving on to the more advanced recommended deployment. Finally, the configuration for the Planetary Computer's Hub is available on [GitHub](https://github.com/microsoft/planetary-computer-hub), which provides a reference for a real-world deployment.
We describe two deployment scenarios, a [simple](docs/concepts/hub-deployment/#simple-deployment) and a [recommended](docs/concepts/hub-deployment/#recommended-deployment) deployment. If you're new to Azure, Kubernetes, or JupyterHub, then you should try the simple deployment to verify that the basics work, before moving on to the more advanced recommended deployment. Finally, the configuration for the Planetary Computer based JupyeteHub is available on [GitHub](https://github.com/microsoft/planetary-computer-hub), which provides a reference for a real-world deployment.

For background, we recommend reading the [Zero to JupyterHub with Kubernetes][z2jh] guide and the [Dask Gateway on Kubernetes][gateway-k8s] documentation.

Expand Down Expand Up @@ -393,4 +395,3 @@ Your AKS cluster and JupyterHub deployments can be customized in various ways. V
[JupyterHub]: https://jupyterhub.readthedocs.io/en/stable/
[prerequisites]: https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough#prerequisites
[z2jh]: https://zero-to-jupyterhub.readthedocs.io/en/latest/index.html
[hub]: ../overview/environment
31 changes: 3 additions & 28 deletions docs/concepts/sas.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,28 +47,7 @@ The `sign` endpoint makes it easy to convert an unsigned blob URL to a signed UR

The `href` field here contains the full, signed URL which may be used directly.

### When an account is needed

The STAC metatdata API is available to all users and does not require an account or a token to use. While all data assets require a token for accessing files on Azure Blob storage, some datasets also require an account key to be used when generating the token. This requirement is inidcated on the Data Catalog page of affected datasets. For these datasets, be sure to include your subscription key when requesting a token, as described below.

### Supplying a subscription key

When your Planetary Computer [account request](http://planetarycomputer.microsoft.com/account/request) was approved, a pair of subscription keys were automatically generated
for you. You can view your keys by singing in to the [developer portal](https://planetarycomputer.developer.azure-api.net/).

You can supply your subscription key in an HTTP request in two ways:

* Supply it in an `Ocp-Apim-Subscription-Key` on request header, for example:

```bash
curl -H "Ocp-Apim-Subscription-Key: 123456789" https://planetarycomputer.microsoft.com/api/sas/v1/token/naip
```

* Supply it in a `subscription-key` query parameter, for example:

```bash
curl https://planetarycomputer.microsoft.com/api/sas/v1/token/naip?subscription-key=123456789
```
The STAC metatdata API is available to all users and does not require an account or a token to use. All data assets require a token for accessing files on Azure Blob storage.

### Rate limits and access restrictions

Expand All @@ -78,12 +57,9 @@ Rate limiting and token expiry are dependent on two aspects of each requests:
* Whether or not the request is originating from within the same data center as the Planetary Computer service (West Europe)
* Whether or not a valid API subscription key has been supplied on the request

These two variables are used to determine the tier of rate limiting which is applied to requests, as well as the valid length of time for issued SAS tokens. For the most un-throttled access, we recommend utilizing a Planetary Computer subscription key and doing your work in the West Europe Azure region.

Most datasets in the Planetary Computer are anonymously accessible: you don't need to supply a subscription key to get a SAS token for downloading the data.
Some datasets do require a subscription key, and some datasets are only available to certain approved users even if a subscription key is provided. This will be noted in
the dataset detail page in the [data catalog](https://planetarycomputer.microsoft.com/catalog).
These two variables are used to determine the tier of rate limiting which is applied to requests, as well as the valid length of time for issued SAS tokens. For the most un-throttled access, we recommend doing your work in the West Europe Azure region.

Datasets in the Planetary Computer are anonymously accessible: you don't need to supply a subscription key to get a SAS token for downloading the data.

### `planetary-computer` Python package

Expand Down Expand Up @@ -130,4 +106,3 @@ planetarycomputer configure
```

Or you can set the environment variable `PC_SDK_SUBSCRIPTION_KEY` to your API subscription key.
Your subscription key is set automatically for you on the [Planetary Computer Hub](../overview/environment).
14 changes: 1 addition & 13 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,18 +49,6 @@
.. raw:: html
<div class="docs-launcher">
<a class="manual-ms-button manual-ms-primary"
href="https://pccompute.westeurope.cloudapp.azure.com/compute/hub/user-redirect/git-pull?repo=https://github.com/microsoft/PlanetaryComputerExamples&urlpath=lab/workspaces/{{ workspace }}/tree/PlanetaryComputerExamples/{{ docname }}&branch=main"
aria-label="Link will open in new tab" target="_blank" rel="noopener noreferrer"
title="This example can be launched in the Planetary Computer Hub"
target="_blank"
rel="noopener noreferrer"
>
<span class="flex-container">
<span>Launch in Hub</span>
</span>
</a>
<a class="manual-ms-button manual-ms-secondary"
href="https://github.com/microsoft/PlanetaryComputerExamples/blob/main/{{ docname }}"
aria-label="Link will open in new tab" target="_blank" rel="noopener noreferrer"
Expand All @@ -69,7 +57,7 @@
rel="noopener noreferrer"
>
<span class="flex-container">
<span>Edit</span>
<span>View</span>
</span>
</a>
</div>
Expand Down
1 change: 0 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
About <overview/about>
Explorer <overview/explorer>
The Hub <overview/environment>
Use VS Code <overview/ui-vscode>
Use GitHub Codespaces <overview/ui-codespaces>
Using QGIS <overview/qgis-plugin>
Expand Down
14 changes: 4 additions & 10 deletions docs/overview/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,25 @@ The Planetary Computer consists of four major components:

- The [Data Catalog](https://planetarycomputer.microsoft.com/catalog), which includes petabytes of data about Earth systems, hosted on Azure and made available to users for free.
- [APIs](../concepts/stac.md) that allow users to search for the data they need across space and time.
- The [Hub](./environment.md), a fully managed computing environment that allows scientists to process massive geospatial datasets.
- [Applications](https://planetarycomputer.microsoft.com/applications), built by our network of partners, that put the Planetary Computer platform to work for environmental sustainability.

## Built on Open

The Planetary Computer uses open source tools and supports open standards. In fact, the foundation of the Planetary Computer is the incredible ecosystem of tools being developed in the open by our partners and the much broader open source community. For example, our Hub builds on the work done by the [Pangeo](http://pangeo.io/) community to put the tools of data science to work for the Earth science community, and our API builds on the work done by the [STAC](https://stacspec.org/) community to streamline and standardize the cataloging and discovery of geospatial data.
The Planetary Computer uses open source tools and supports open standards. In fact, the foundation of the Planetary Computer is the incredible ecosystem of tools being developed in the open by our partners and the much broader open source community. For example, our API builds on the work done by the [STAC](https://stacspec.org/) community to streamline and standardize the cataloging and discovery of geospatial data.

Many of the Planetary Computer components are also open-source. These provide guidance on how to tie together open-source libraries on Azure for geospatial and environmental data analysis.

| GitHub repository | Purpose |
|-------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Microsoft/planetary-computer-apis](https://github.com/Microsoft/planetary-computer-apis) | Deploys the [STAC](https://planetarycomputer.microsoft.com/docs/reference/stac/) and [data](https://planetarycomputer.microsoft.com/docs/reference/data/) APIs |
| [Microsoft/planetary-computer-hub](https://github.com/Microsoft/planetary-computer-hub) | Deploys the [Planetary Computer Hub](https://planetarycomputer.microsoft.com/docs/overview/environment/) |
| [Microsoft/planetary-computer-containers](https://github.com/Microsoft/planetary-computer-containers) | Builds the container images with the [software environment](https://planetarycomputer.microsoft.com/docs/overview/environment/#understanding-the-software-environment) used on the Hub |
| [Microsoft/PlanetaryComputerExamples](https://github.com/microsoft/planetarycomputerexamples) | Contains notebooks with examples for each dataset, quickstarts, and tutorials for using the Planetary Computer |

## About the Preview

While the Planetary Computer data and APIs are publicly accessible, certain features of the Planetary Computer are in Preview and require access be granted. Use the [request access form](https://planetarycomputer.microsoft.com/account/request) to express your interest in becoming an early user.

In the meantime, the core components of the Planetary Computer are usable without an account:
The Planetary Computer data and APIs are publicly accessible and can be used without an account, including:

- The [STAC API](../reference/stac) is public and can be accessed anonymously.
- Most data can be downloaded anonymously, but will be throttled. See [Reading data from the STAC API](../quickstarts/reading-stac.ipynb) for an introduction and [Using Tokens for Data Access](../concepts/sas) for more background on accessing data. Some datasets require a Planetary Computer API subscription key.
- Most data can be downloaded anonymously, but will be throttled. See [Reading data from the STAC API](../quickstarts/reading-stac.ipynb) for an introduction and [Using Tokens for Data Access](../concepts/sas) for more background on accessing data.

We're just getting started. Check back for updated documentation and new features!

Expand All @@ -45,7 +40,6 @@ We're just getting started. Check back for updated documentation and new feature
- [Sentinel-2 L2A](https://planetarycomputer.microsoft.com/dataset/sentinel-2-l2a#Example-Notebook)
- [NAIP](https://planetarycomputer.microsoft.com/dataset/naip#Example-Notebook)
- [ASTER L1T](https://planetarycomputer.microsoft.com/dataset/aster-l1t#Example-Notebook)
- [Request a Planetary Computer account](https://planetarycomputer.microsoft.com/account/request)

## Beyond the Planetary Computer

Expand Down Expand Up @@ -74,4 +68,4 @@ If the Planetary Computer is useful for your work, please cite it using [this re
doi = {10.5281/zenodo.7261897},
url = {https://doi.org/10.5281/zenodo.7261897}
}
```
```
Loading

0 comments on commit 3199d21

Please sign in to comment.