Neuron Device Plugin Addon #777

youngjeong46 · 2023-07-16T16:19:49Z

Issue #, if available:

Description of changes: This PR adds Neuron Device Plugin to be able to leverage Inferentia and Trainium nodes on EKS Blueprints. This includes:

Addon
Example showing how to deploy it with an Inferentia Nodegroup (inf1 & inf2)
Documentations

Other related changes include revisions to helper functions to load yaml files and additional unit tests.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

shapirov103

@youngjeong46 great work, I added a few tactical comments.
With respect to the usage of the manifests, can you check if there is any approach to test whether there is a new version of Neuron and how can we incorporate it for continuous maintenance, including patch releases, version upgrades, etc.

lib/addons/neuron/index.ts

docs/addons/neuron-plugin-addon.md

shapirov103 · 2023-07-17T19:18:18Z

lib/addons/neuron/index.ts

+import { ClusterAddOn, ClusterInfo } from "../../spi";
+import { loadYaml, readYamlDocument } from "../../utils/yaml-utils";
+
+export class NeuronPluginAddOn implements ClusterAddOn {


No options to deploy? Not even namespace? It is fine if ns should be kube-system, just want to check if anything is reasonable to expose for configuration.

On this one, it is straight forward. There may be some optional scheduler which I just saw, that I will do with options in a fast follow-up.

Do you mind creating an issue and assigning to you or Riccardo if you want to do a fast follow-up for options

Will do it in this PR actually, testing it right now.

freschri · 2023-09-12T11:29:03Z

@youngjeong46 great work, I added a few tactical comments. With respect to the usage of the manifests, can you check if there is any approach to test whether there is a new version of Neuron and how can we incorporate it for continuous maintenance, including patch releases, version upgrades, etc.

@shapirov103 @youngjeong46
can these 2 locations be an option?
https://awsdocs-neuron.readthedocs-hosted.com/en/latest/_downloads/f57f27621e52b305dba7d624c477977a/k8s-neuron-device-plugin.yml
https://awsdocs-neuron.readthedocs-hosted.com/en/latest/_downloads/46fb1da6e5e79c3310ebc0cbd6ad2353/k8s-neuron-device-plugin-rbac.yml

elamaran11

@youngjeong46 Nice work, thankyou for doing this. I have some comments on the PR. Also want mkdocs and Addon index update is missing for the new addon.

examples/young-construct/index.ts

elamaran11 · 2023-10-13T12:48:21Z

lib/addons/neuron/index.ts

+import { ClusterAddOn, ClusterInfo } from "../../spi";
+import { loadYaml, readYamlDocument } from "../../utils/yaml-utils";
+
+export class NeuronPluginAddOn implements ClusterAddOn {


Do you mind creating an issue and assigning to you or Riccardo if you want to do a fast follow-up for options

lib/utils/yaml-utils.ts

youngjeong46 · 2023-10-16T14:08:46Z

@youngjeong46 great work, I added a few tactical comments. With respect to the usage of the manifests, can you check if there is any approach to test whether there is a new version of Neuron and how can we incorporate it for continuous maintenance, including patch releases, version upgrades, etc.

@shapirov103 @youngjeong46 can these 2 locations be an option? https://awsdocs-neuron.readthedocs-hosted.com/en/latest/_downloads/f57f27621e52b305dba7d624c477977a/k8s-neuron-device-plugin.yml https://awsdocs-neuron.readthedocs-hosted.com/en/latest/_downloads/46fb1da6e5e79c3310ebc0cbd6ad2353/k8s-neuron-device-plugin-rbac.yml

@freschri I'm linking it to the following two links (as they are raw files that can be imported straight as part of the addon). For now, unless the Neuron team announces otherwise, they would update the manifests through this location on GitHub.
https://raw.githubusercontent.com/aws-neuron/aws-neuron-sdk/master/src/k8/k8s-neuron-device-plugin-rbac.yml
https://raw.githubusercontent.com/aws-neuron/aws-neuron-sdk/master/src/k8/k8s-neuron-device-plugin.yml

elamaran11 · 2023-12-15T20:35:39Z

@youngjeong46 @freschri Are you folks planning to make progress to this PR?

Neuron Device Plugin Addon Continuation

youngjeong46 · 2024-02-13T16:20:29Z

@shapirov103 It looks good to me, also just tested locally and can verify it works.

shapirov103 · 2024-02-13T23:12:42Z

/do-e2e-tests

aws-ia-ci

end to end tests failed. A maintainer can provide more details.

shapirov103 · 2024-02-14T14:42:09Z

Error:

| blueprint-construct-dev | 145/255 | 11:32:44 PM | CREATE_FAILED | Custom::AWSCDK-EKS-KubernetesResource | blueprint-construct-dev/adot-collector-sa/manifest-adot-collector-saServiceAccountResource/Resource/Default (blueprintconstructdevadotcollectorsamanifestadotcollectorsaServiceAccountResourceA34512D7) Received response status [FAILED] from custom resource. Message returned: Error: b'Error from server (NotFound): error when creating "/tmp/manifest.yaml": namespaces "adot" not found\n'

@elamaran11 - the issue seems to be specific to adot, has nothing to do with neuron.

elamaran11 · 2024-02-14T14:47:49Z

Error:

| blueprint-construct-dev | 145/255 | 11:32:44 PM | CREATE_FAILED | Custom::AWSCDK-EKS-KubernetesResource | blueprint-construct-dev/adot-collector-sa/manifest-adot-collector-saServiceAccountResource/Resource/Default (blueprintconstructdevadotcollectorsamanifestadotcollectorsaServiceAccountResourceA34512D7) Received response status [FAILED] from custom resource. Message returned: Error: b'Error from server (NotFound): error when creating "/tmp/manifest.yaml": namespaces "adot" not found\n'

@elamaran11 - the issue seems to be specific to adot, has nothing to do with neuron.

@shapirov103 I ran the e2e even yesterday on my local, didnt see any issues. I think the PR might not have pulled the latest changes from main we had for OTEL config. I recommend to rerun the E2E after merging from main. Also recommend a local run. Plus we are using ADOT addon consistently in CDK accelerator with no issues.

youngjeong46 · 2024-02-14T15:41:21Z

Error:

| blueprint-construct-dev | 145/255 | 11:32:44 PM | CREATE_FAILED | Custom::AWSCDK-EKS-KubernetesResource | blueprint-construct-dev/adot-collector-sa/manifest-adot-collector-saServiceAccountResource/Resource/Default (blueprintconstructdevadotcollectorsamanifestadotcollectorsaServiceAccountResourceA34512D7) Received response status [FAILED] from custom resource. Message returned: Error: b'Error from server (NotFound): error when creating "/tmp/manifest.yaml": namespaces "adot" not found\n'

@elamaran11 - the issue seems to be specific to adot, has nothing to do with neuron.

@shapirov103 I ran the e2e even yesterday on my local, didnt see any issues. I think the PR might not have pulled the latest changes from main we had for OTEL config. I recommend to rerun the E2E after merging from main. Also recommend a local run. Plus we are using ADOT addon consistently in CDK accelerator with no issues.

@elamaran11 This is with the main merged.

shapirov103 · 2024-02-14T15:55:21Z

@youngjeong46 I submitted a fix for it. The defect is very intermittent, race condition, but it is also non-determinstic. the same blueprint may fail one time, will succeed another time. See #931

shapirov103 · 2024-02-14T17:54:53Z

/do-e2e-tests

aws-ia-ci

end to end tests passed

first commit - adding the addons, manifests, and example with GPU nodes

3423025

youngjeong46 requested review from aws-ia-ci, a team, parkand1, shapirov103 and elamaran11 as code owners July 16, 2023 16:19

youngjeong46 added 2 commits July 16, 2023 11:53

add documentations

25a717b

slight fix to the docs

402eb10

shapirov103 reviewed Jul 17, 2023

View reviewed changes

elamaran11 mentioned this pull request Oct 11, 2023

Neuron observability aws-observability/cdk-aws-observability-accelerator#121

Merged

youngjeong46 added 6 commits October 12, 2023 15:19

merge main

0e878d6

Merge branch 'main' into feature/neuron-addon

6276978

test fix, lint fix, and removed local testing example blueprint

38b70f6

doc fix to remove nodegroup

5643240

added back GPU node group

7f1302a

removed local neuron yaml files in place of urls

51103c9

elamaran11 reviewed Oct 13, 2023

View reviewed changes

youngjeong46 added 2 commits October 13, 2023 07:59

removing local construct test

b272111

removed unneccessary yaml util, added jsdocs on helper functions

b4f9530

ariveroi mentioned this pull request Jan 30, 2024

Neuron Device Plugin Addon - Continuation #915

Closed

changed mkdocks and docs/index.md to point to neuron addon

c71979a

ariveroi mentioned this pull request Feb 12, 2024

Neuron Device Plugin Addon Continuation #923

Merged

shapirov103 and others added 2 commits February 12, 2024 10:23

Merge pull request #923 from ariveroi/feature/neuron-addon

b072506

Neuron Device Plugin Addon Continuation

Merge branch 'main' into feature/neuron-addon

e93145a

aws-ia-ci requested changes Feb 14, 2024

View reviewed changes

shapirov103 mentioned this pull request Feb 14, 2024

ADOT: intermittent failure namespace not found #930

Closed

aws-ia-ci approved these changes Feb 14, 2024

View reviewed changes

elamaran11 merged commit 7f6442a into main Feb 14, 2024
2 checks passed

elamaran11 deleted the feature/neuron-addon branch February 14, 2024 19:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Neuron Device Plugin Addon #777

Neuron Device Plugin Addon #777

youngjeong46 commented Jul 16, 2023 •

edited

Loading

shapirov103 left a comment

shapirov103 Jul 17, 2023

youngjeong46 Oct 12, 2023

elamaran11 Oct 13, 2023

youngjeong46 Oct 16, 2023

freschri commented Sep 12, 2023 •

edited

Loading

elamaran11 left a comment

elamaran11 Oct 13, 2023

youngjeong46 commented Oct 16, 2023

elamaran11 commented Dec 15, 2023

youngjeong46 commented Feb 13, 2024

shapirov103 commented Feb 13, 2024

aws-ia-ci left a comment

shapirov103 commented Feb 14, 2024

elamaran11 commented Feb 14, 2024 •

edited

Loading

youngjeong46 commented Feb 14, 2024

shapirov103 commented Feb 14, 2024

shapirov103 commented Feb 14, 2024

aws-ia-ci left a comment

Neuron Device Plugin Addon #777

Neuron Device Plugin Addon #777

Conversation

youngjeong46 commented Jul 16, 2023 • edited Loading

shapirov103 left a comment

Choose a reason for hiding this comment

shapirov103 Jul 17, 2023

Choose a reason for hiding this comment

youngjeong46 Oct 12, 2023

Choose a reason for hiding this comment

elamaran11 Oct 13, 2023

Choose a reason for hiding this comment

youngjeong46 Oct 16, 2023

Choose a reason for hiding this comment

freschri commented Sep 12, 2023 • edited Loading

elamaran11 left a comment

Choose a reason for hiding this comment

elamaran11 Oct 13, 2023

Choose a reason for hiding this comment

youngjeong46 commented Oct 16, 2023

elamaran11 commented Dec 15, 2023

youngjeong46 commented Feb 13, 2024

shapirov103 commented Feb 13, 2024

aws-ia-ci left a comment

Choose a reason for hiding this comment

shapirov103 commented Feb 14, 2024

elamaran11 commented Feb 14, 2024 • edited Loading

youngjeong46 commented Feb 14, 2024

shapirov103 commented Feb 14, 2024

shapirov103 commented Feb 14, 2024

aws-ia-ci left a comment

Choose a reason for hiding this comment

youngjeong46 commented Jul 16, 2023 •

edited

Loading

freschri commented Sep 12, 2023 •

edited

Loading

elamaran11 commented Feb 14, 2024 •

edited

Loading