Issue running coinflip.py example workflow with Hera and Argo

[tavallaie]

I am trying to run a workflow using Hera, with the code available at https://github.com/argoproj-labs/hera/blob/main/examples/workflows/coinflip.py.

I have installed Hera and its dependencies using pip, and when I run the code using python coinflip.py, there is no error reported in the console. However, when I check the Argo list using argo list -n argo, I do not see the workflow listed.

I have verified that Argo is correctly installed, and I have also tried running other workflows directly on the Argo UI, which are successfully displayed. It seems that the issue is specific to the workflow created with Hera.

Here is my pip freeze output:

certifi==2022.12.7
charset-normalizer==3.1.0
dnspython==2.3.0
email-validator==1.3.1
hera==5.1.2
idna==3.4
pydantic==1.10.7
requests==2.28.2
typing_extensions==4.5.0
urllib3==1.26.15

[tavallaie]

I did some further research and found that curl -k --request GET --url https://localhost:2746/api/v1/workflows/argo/hello-world-sm789 works, indicating that the issue may be specific to Hera. Additionally, I found that Hera v4.4.2 works, but Hera v5 does not.

Could someone please help me troubleshoot this issue with Hera v5 and get the workflow to run on Argo?

[flaviuvadan]

Hi @tavallaie! As a double check, did you add the w.create() call in the coinflip example? We do not add that in the examples because we don't have AW running in CI at the moment, so when we test the examples we only import the w, generate the YAML, and compare to the upstream (official) Argo Workflows YAML examples to make sure Hera works correctly

[tavallaie]

Thank you for that. I forgot to put in "w. create()."

[tavallaie]

Problem:

When I tried to run w.create() in my code, I received an AssertionError with the message "workflow namespace not defined". When I added a namespace to my code, such as "argo", I received a new error message: requests.exceptions.MissingSchema: Invalid URL 'api/v1/workflows/argo': No scheme supplied. Perhaps you meant https://api/v1/workflows/argo?. I have attempted various solutions over the past few days, but I have been unable to successfully run my pipeline.

Additional information:

I am currently able to run my pipelines with Hera v4.4.2.

[flaviuvadan]

@tavallaie what is your global config? I think the above error suggests you did not set a host because only /api/v1/... shows up in the error 🤔 You can import the global config and set your host via this field:

from hera.shared import global_config

global_config.host = "localhost:2746"  # this can also be a custom domain like example.argo.co

... # do other stuff
w.create()

[tavallaie]

@flaviuvadan, I wanted to share a tip I learned the hard way when working with the requests library. In the following code:

resp = requests.post(
    url=urljoin(self.host, "api/v1/workflows/{namespace}").format(
        namespace=namespace if namespace is not None else self.namespace
    ),
    params=None,
    headers={"Authorization": f"Bearer {self.token}"},
    data=req.json(
        exclude_none=True, by_alias=True, skip_defaults=True, exclude_unset=True, exclude_defaults=True
    ),
    verify=self.verify_ssl,
)

If you don't include the protocol in the base URL, urljoin might not be able to determine the base URL properly, and only show the path part of the URL. For example, if you use localhost:2746 instead of https://localhost:2746, you might only see "api/v1/workflows/{namespace}" and experience unexpected behavior.

To avoid this, make sure to include the protocol in the base URL, like so: https://localhost:2746. I hope this helps someone avoid the same issue I faced!"

[flaviuvadan]

That's a great point, @tavallaie! What do you think about Hera having a check for this? 🤔 perhaps the service can check that the scheme is part of the host

[tavallaie]

@flaviuvadan, I am not very familiar with the code base and code of conduct to make contributions, but perhaps we could consider adding a "_host" function. This function could check if the host has a protocol scheme and add one if it does not.