Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local Storage Card with local K8s #2139

Open
ch-enfuse opened this issue Nov 12, 2024 · 0 comments
Open

Local Storage Card with local K8s #2139

ch-enfuse opened this issue Nov 12, 2024 · 0 comments

Comments

@ch-enfuse
Copy link

ch-enfuse commented Nov 12, 2024

Hi, I'm currently running a k3s cluster locally, and I've been wanting to test a workflow that has a @card decorator to see some html and see it from the UI that is being served from K8s. The worfllow is ran locally. I am testing this without an s3 instance (that was giving me some issues with locating the /cards content when the UI was asking for them to retrieve from the ui service)

here is my ~/.metaflowconfig/config.json
{
"METAFLOW_DEFAULT_METADATA": "service",
"METAFLOW_SERVICE_URL": "http://localhost:8080/"
}
I've created a /.metaflow directory with no contents and chowned to my user to work

  • the first run i make (Im testing this workflow ) has no .metaflow directory.
    This saves metadata about the run on /.metadata (a root folder owned by my user - not ideal but i want to mount this onto the ui service pod so that it can serve the contents later, i can do it from any folder but I haven't figured out how to have a consistent location for this yet
  • second run: there is a .metaflow/ directory all of the sudden, and my data is split. in /.metaflow directory where i expect things to be has only a NBFlow sub directory, and my local ./.metaflow directory has the mf.cards I'm missing on the other one.

Now every subsequent run after this, metaflow defaults everything to my local ./.metaflow directory, but what i need is
I want to figure out a way to have a consistent way of setting my local storage directory and make sure the cards content also land here.
is there some combination of env variables or config i can add to my config.json?
also, if I were to instead just go back to my s3 setup, what configurations should i look at? I haven't had luck finding sources on this.

I must be something pretty simple here but I'm very new to metaflow, any help would be appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant