Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide helpers to compile for AWS Inferentia #126

Open
vdesaintpern opened this issue Jul 19, 2024 · 0 comments
Open

Provide helpers to compile for AWS Inferentia #126

vdesaintpern opened this issue Jul 19, 2024 · 0 comments

Comments

@vdesaintpern
Copy link

I'm looking at compiling Stable Audio Open model for AWS Inferentia.

According to Neuronx documentation, you need to build examples and it doesn't seem so straightforward in the code what would be a good candidate for the trace method. As of today, I believe what is passed as parameter to K.sampling would be a great fit but still unsure.

Having a function similar to generate_diffusion_cond that would just generate valid inputs for the trace function would already be an improvement, even potentially facilitate HF Optimum integration.

I'm also considering doing the PR myself if someone from the team can provide some guidance.

Thanks,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant