[http handler] add GET fallback if HEAD is not allowed or forbidden to get key size

[thomascoquet]

• Adding a fallback to get resource size if HEAD is not allowed.
• Similar behavior to GDAL vsicurl.
• Mainly used with URLs that are presigned for only one HTTP verb (AWS, GS).

[coveralls]

Pull Request Test Coverage Report for Build 1823560964

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 24 of 36 (66.67%) changed or added relevant lines in 1 file are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage increased (+0.8%) to 89.481%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
http.go	24	36	66.67%

Files with Coverage Reduction	New Missed Lines	%
http.go	1	64.29%

Totals
Change from base Build 1667244230:	0.8%
Covered Lines:	689
Relevant Lines:	770

---

💛 - Coveralls

[tbonfort]

Couldn't the HEAD request be skipped completely and instead always do a GET request for the first block that populates the cache and sets the size in a single request ?

[thomascoquet]

Do you mean parsing the Content-Range header instead of relying on the Content-Length one?

HTTP/1.1 206 Partial Content
Content-Range: bytes 0-1023/146515
Content-Length: 1024

Do you agree it requires modifying func (a *Adapter) Size(key string) (int64, error) ?
I can take a look ; it seems reasonable given we need the first block in most contexts (except in zips).

[tbonfort]

I'm not sure exactly which headers need to be parsed, your proposed solution seems reasonable. There is no need to modify the Adapter.Size function itself, all changes would happen in HTTPHandler.StreamAt with the parsing of Content-Range happening after having received the GET response.