Unable to authenticate using node role

[coufalja]

Cause and effect:
During the investigation of grafana/tempo#2743 I was able to track down the problem to specific commits in mino-go #1626 and #1682.
Those two together causes us not being able to authenticate, and subsequent Tempo error.

If that is expected behavior (as it seems according to this Cortex commit cortexproject/cortex#4897) the issue lies in the usage of the library.

[parshnt]

@coufalja - was there any progress on the issue?

I believe that the changes introduced in #1682 are causing the issue where if IMDSv2 is disabled, an error is raised. whereas earlier the same error was ignored. this is preventing the use of IMDSv1 as well - as the request to generate role creds is made afterwards.

https://github.com/minio/minio-go/pull/1682/files#diff-8a37275f64c96a9a71f11eb6008f45f1f764384f3ae02f685e3013af1d7f79b5L292

[harshavardhana]

I will check on the fix.

[coufalja]

I ended up doing the fix in Tempo itself grafana/tempo#2760 as I thought this was an intended behavior.

[parshnt]

@harshavardhana - incase of IMDSv2 is disabled. we are seeing an error thrown:
Put "http://169.254.169.254/latest/api/token": context deadline exceeded

I believe #1682 might've introduced this issue

[harshavardhana]

This is not clearly intended behavior, will fix it.

[parshnt]

@harshavardhana - let me know if I can help with the fix, will create a PR.

[harshavardhana]

@parshnt please look at the PR #1877 and see if this fixes your problem.

[parshnt]

@harshavardhana - LGTM