support env vars for predict & run

[anotherjesse]

This PR adds support for setting an environment variable when running a model (not during building)

This is meant to augment the development process of "cog train" - where fine-tuned models are different from the original model only by the presence of a COG_WEIGHTS variable.

Currently there is no easy way of using cog predict or cog run to test models during development with the results of fine-tuning.

After this PR lands, if you are doing development of a cog model such as llama or sdxl, you can test that the fine-tuned code-path works before pushing to replicate by doing:

cog predict -e COG_WEIGHTS=https://replicate.delivery/pbxt/xyz/weights.tar -i prompt="a photo of TOK"

or if you want to test that it works for multiple predictions (and doesn't leak memory), you can:

cog run -e COG_WEIGHTS=https://replicate.delivery/pbxt/xyz/weights.tar -p 5000 python -m cog.http.server

I didn't add this to documentation yet as my understanding is cog train isn't yet finalized/documented

[anotherjesse]

cc: @joehoover @daanelson @technillogue

[anotherjesse]

I think this is a much smaller and different problem than https://github.com/replicate/replicate-web/issues/4289 which seems to be more about environment variables and how they relate to models you create

this is patch is being able to use the in-progress development models - built via cog build and using fine-tuned weights with them for testing purposes

[zeke]

This looks good to me. Thanks for the clear write-up. 👍🏼

I didn't add this to documentation yet as my understanding is cog train isn't yet finalized/documented

You're using this change for training purposes, but the change itself is not strictly tied to training, right? If that's the case it might be worth adding a blurb about this in https://github.com/replicate/cog/blob/main/docs/environment.md

[zeke]

@anotherjesse says:

This PR does not set env vars that get baked into the built image. That is a completely different env var feature that people have talked about. Rather it's just for when you're need to set env vars for when you run it locally. The only env var that it really makes sense to set right now is COG_WEIGHTS

[zeke]

Working on a PR to get this capability documented here: #1277