Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix spark distribution url. #88

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

yoshihitoh
Copy link

@yoshihitoh yoshihitoh commented Jul 5, 2021

Issue #, if available:

Description of changes:
Fix spark distribution url.

  • replate file extension from tgz1 to tgz

Tested with commands below.

before this PR

HTTP/1.1 404 Not Found for both urls.

$ curl -I https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-1.0/spark-2.4.3-bin-hadoop2.8.tgz1
HTTP/1.1 404 Not Found
x-amz-request-id: QV5X09B338ZDSHQT
x-amz-id-2: M+qU+wPMd40bKAv34Q4zfkHJ0AyOu6pIshgw+yiaMJx7HOsSdpyJvj6On2jRVEJofeqbgIwB89I=
Content-Type: application/xml
Date: Mon, 05 Jul 2021 05:48:21 GMT
Server: AmazonS3

$ curl -I https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-2.0/spark-2.4.3-bin-hadoop2.8.tgz1
HTTP/1.1 404 Not Found
x-amz-request-id: GQJ1YTQTT9D1620P
x-amz-id-2: 2mEUf9Kf1IZvev9Ozhd/74nEim2wlglxax3SLePgxX90gmXlj9sa6eUxw7KfnD6flTNfigXmeM4=
Content-Type: application/xml
Date: Mon, 05 Jul 2021 05:49:01 GMT
Server: AmazonS3

after this PR

HTTP/1.1 200 OK for both urls.

$ curl -I https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-1.0/spark-2.4.3-bin-hadoop2.8.tgz
HTTP/1.1 200 OK
x-amz-id-2: gHENl0y5zmSVfj/2jyaYJGapXGDhwuZU7qnMcfuDoiAjGFHYnLAsPAOaODGFkAFtzZHfEN8lpsM=
x-amz-request-id: GYARPZEPJ8VR5TP9
Date: Mon, 05 Jul 2021 05:49:25 GMT
Last-Modified: Tue, 17 Sep 2019 17:47:19 GMT
ETag: "bff157160f1ae778baabb801d113915e-13"
x-amz-version-id: null
Accept-Ranges: bytes
Content-Type: application/x-gzip
Server: AmazonS3
Content-Length: 206432845

$ curl -I https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-2.0/spark-2.4.3-bin-hadoop2.8.tgz1
HTTP/1.1 200 OK
x-amz-id-2: Ji5ocVb9LQyr8s+C7OODGU6Aqk/sbz3Uj2CQIxDRLfDTvjung6pQ/KCQzwM23PSUm7WQ1gThTkE=
x-amz-request-id: JTZHVNMG477RYJEE
Date: Mon, 05 Jul 2021 05:49:48 GMT
Last-Modified: Tue, 08 Jun 2021 23:51:45 GMT
ETag: "3b600756fddc7c2733f777bda010484b-13"
x-amz-version-id: nAoVakG3r_B1JeMD8skkNUJ6MygX_74Q
Accept-Ranges: bytes
Content-Type: application/x-gzip
Server: AmazonS3
Content-Length: 206432845

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@tbugfinder
Copy link

HI @yoshihitoh , could you resolve the conflicts?

@yoshihitoh yoshihitoh force-pushed the master branch 2 times, most recently from 68b742b to 553eb78 Compare May 17, 2022 05:54
@yoshihitoh yoshihitoh changed the base branch from glue-0.9 to master May 17, 2022 05:57
@yoshihitoh
Copy link
Author

Hi @tbugfinder , thank you for checking this PR.

I resolved the conflicts by rebasing with the master branch, and changing the base branch from glue-0.9 to master. Maybe I made a mistake in choosing the branch when I opened this.

Confirmed those URLs work.

$ curl -I https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-1.0/spark-2.4.3-bin-hadoop2.8.tgz
HTTP/1.1 200 OK
x-amz-id-2: pTB3cZBwBOhU0DlYP0FUruGuZ/Eabm8EorLcjXFkoZpROtTrDQJJxmSmkBsFNv8Q4NXChuJ+BYo=
x-amz-request-id: 0BS6NMDKYXTYVNBZ
Date: Tue, 17 May 2022 06:01:53 GMT
Last-Modified: Tue, 17 Sep 2019 17:47:19 GMT
ETag: "bff157160f1ae778baabb801d113915e-13"
x-amz-version-id: null
Accept-Ranges: bytes
Content-Type: application/x-gzip
Server: AmazonS3
Content-Length: 206432845

$ curl -I https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-2.0/spark-2.4.3-bin-hadoop2.8.tgz
HTTP/1.1 200 OK
x-amz-id-2: pTB3cZBwBOhU0DlYP0FUruGuZ/Eabm8EorLcjXFkoZpROtTrDQJJxmSmkBsFNv8Q4NXChuJ+BYo=
x-amz-request-id: 0BS6NMDKYXTYVNBZ
Date: Tue, 17 May 2022 06:01:53 GMT
Last-Modified: Tue, 17 Sep 2019 17:47:19 GMT
ETag: "bff157160f1ae778baabb801d113915e-13"
x-amz-version-id: null
Accept-Ranges: bytes
Content-Type: application/x-gzip
Server: AmazonS3
Content-Length: 206432845

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants