Merge branch 'develop' into feature/PRD-586-ochestrator-urls

docs/book/component-guide/orchestrators/lightning.md

@@ -15,22 +15,16 @@ This component is only meant to be used within the context of a [remote ZenML de
 
 ## When to use it
 
-* You are looking for a fast and easy way to run your pipelines on GPU instances.
-
-* you're already using Lightning AI for your machine learning projects.
-
-* you want to leverage Lightning AI's managed infrastructure for running your pipelines.
-
-* you're looking for a solution that simplifies the deployment and scaling of your ML workflows.
-
-* you want to take advantage of Lightning AI's optimizations for machine learning workloads.
-
+* You are looking for a fast and easy way to run your pipelines on GPU instances
+* You're already using Lightning AI for your machine learning projects
+* You want to leverage Lightning AI's managed infrastructure for running your pipelines
+* You're looking for a solution that simplifies the deployment and scaling of your ML workflows
+* You want to take advantage of Lightning AI's optimizations for machine learning workloads
 
 ## How to deploy it
 
 To use the [Lightning AI Studio](https://lightning.ai/) orchestrator, you need to have a Lightning AI account and the necessary credentials. You don't need to deploy any additional infrastructure, as the orchestrator will use Lightning AI's managed resources.
 
-
 ## How it works
 
 The Lightning AI orchestrator is a ZenML orchestrator that runs your pipelines on Lightning AI's infrastructure. When you run a pipeline with the Lightning AI orchestrator, ZenML will archive your current ZenML repository and upload it to the Lightning AI studio. Once the code is archived, using `lightning-sdk`, ZenML will create a new stduio in Lightning AI and upload the code to it. Then ZenML runs list of commands via `studio.run()` to prepare for the pipeline run (e.g. installing dependencies, setting up the environment). Finally, ZenML will run the pipeline on Lightning AI's infrastructure.
@@ -46,49 +40,51 @@ To use the Lightning AI orchestrator, you need:
 
 *   The ZenML `lightning` integration installed. If you haven't done so, run
 
-    ```shell
-    pip install lightning-sdk
-    ```
+```shell
+zenml integration install lightning
+```
 
 * A [remote artifact store](../artifact-stores/artifact-stores.md) as part of your stack.
 
-
 * [Lightning AI credentials](#lightning-ai-credentials)
 
-
 ### Lightning AI credentials
 
-* `LIGHTNING_USER_ID`: Your Lightning AI user ID
+You will need the following credentials to use the Lightning AI orchestrator:
 
+* `LIGHTNING_USER_ID`: Your Lightning AI user ID
 * `LIGHTNING_API_KEY`: Your Lightning AI API key
-
 * `LIGHTNING_USERNAME`: Your Lightning AI username (optional)
-
 * `LIGHTNING_TEAMSPACE`: Your Lightning AI teamspace (optional)
-
 * `LIGHTNING_ORG`: Your Lightning AI organization (optional)
 
+To find these credentials, log in to your [Lightning AI](https://lightning.ai/)
+account and click on your avatar in the top right corner. Then click on "Global
+Settings". There are some tabs you can click on the left hand side. Click on the
+one that says "Keys" and you will see two ways to get your credentials. The
+'Login via CLI' will give you the `LIGHTNING_USER_ID` and `LIGHTNING_API_KEY`.
 
-Alternatively, you can configure these credentials when registering the orchestrator.
-
-
-We can then register the orchestrator and use it in our active stack:
+You can set these credentials as environment variables or you can set them
+when registering the orchestrator:
 
 ```shell
 zenml orchestrator register lightning_orchestrator \
     --flavor=lightning \
     --user_id=<YOUR_LIGHTNING_USER_ID> \
     --api_key=<YOUR_LIGHTNING_API_KEY> \
-    --username=<YOUR_LIGHTNING_USERNAME> \
-    --teamspace=<YOUR_LIGHTNING_TEAMSPACE> \
-    --organization=<YOUR_LIGHTNING_ORGANIZATION>
+    --username=<YOUR_LIGHTNING_USERNAME> \ # optional
+    --teamspace=<YOUR_LIGHTNING_TEAMSPACE> \ # optional
+    --organization=<YOUR_LIGHTNING_ORGANIZATION> # optional
+```
+
+We can then register the orchestrator and use it in our active stack:
 
+```bash
 # Register and activate a stack with the new orchestrator
 zenml stack register lightning_stack -o lightning_orchestrator ... --set
 ```
 
-You can also configure the orchestrator at pipeline level, using the `orchestrator` parameter.
-
+You can configure the orchestrator at pipeline level, using the `orchestrator` parameter.
 
 ```python
 from zenml.integrations.lightning.flavors.lightning_orchestrator_flavor import LightningOrchestratorSettings
@@ -112,8 +108,10 @@ def my_pipeline():
 
 
 {% hint style="info" %}
-ZenML will archive the current zenml repository (The code within the path where you run `zenml init`) and upload it to the Lightning AI studio.
-For this reason you need make sure that you are running `zenml init` in the same directory where you are running your pipeline.
+ZenML will archive the current zenml repository (the code within the path where
+you run `zenml init`) and upload it to the Lightning AI studio. For this reason
+you need make sure that you have run `zenml init` in the same repository root directory where
+you are running your pipeline.
 {% endhint %}
 
 ![Lightning AI studio VSCode](../../.gitbook/assets/lightning_studio_vscode.png)
@@ -129,12 +127,11 @@ You can now run any ZenML pipeline using the Lightning AI orchestrator:
 python file_that_runs_a_zenml_pipeline.py
 ```
 
-
 ### Lightning AI UI
 
 Lightning AI provides its own UI where you can monitor and manage your running applications, including the pipelines orchestrated by ZenML.
-![Lightning AI Studio](../../.gitbook/assets/lightning_studio_ui.png)
 
+![Lightning AI Studio](../../.gitbook/assets/lightning_studio_ui.png)
 
 For any runs executed on Lightning AI, you can get the URL to the Lightning AI UI in Python using the following code snippet:
 
@@ -145,7 +142,6 @@ pipeline_run = Client().get_pipeline_run("<PIPELINE_RUN_NAME>")
 orchestrator_url = pipeline_run.run_metadata["orchestrator_url"].value
 ```
 
-
 ### Additional configuration
 
 For additional configuration of the Lightning AI orchestrator, you can pass `LightningOrchestratorSettings` which allows you to configure various aspects of the Lightning AI execution environment:
@@ -161,10 +157,7 @@ lightning_settings = LightningOrchestratorSettings(
 )
 ```
 
-
-
-
-These settings can then be specified on either pipeline-level or step-level:
+These settings can then be specified on either a pipeline-level or step-level:
 
 ```python
 # Either specify on pipeline-level
@@ -193,12 +186,10 @@ To use GPUs with the Lightning AI orchestrator, you need to specify a GPU-enable
 
 ```python
 lightning_settings = LightningOrchestratorSettings(
-    machine_type="gpu", # or 
+    machine_type="gpu", # or `A10G` e.g.
 )
 ```
 
-
-Make sure to check Lightning AI's documentation for the available GPU-enabled machine types and their specifications.
-
+Make sure to check [Lightning AI's documentation](https://lightning.ai/docs/overview/studios/change-gpus) for the available GPU-enabled machine types and their specifications.
 
 <figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>