Upgrade to sdk rc10 #807

smallhive · 2023-08-03T07:59:53Z

closes #806

codecov · 2023-08-03T08:02:59Z

Codecov Report

Merging #807 (cb7d1e5) into master (d013f71) will decrease coverage by 0.02%.
Report is 1 commits behind head on master.
The diff coverage is 4.39%.

@@            Coverage Diff             @@
##           master     #807      +/-   ##
==========================================
- Coverage   34.35%   34.33%   -0.02%     
==========================================
  Files          61       61              
  Lines       10483    10485       +2     
==========================================
- Hits         3601     3600       -1     
- Misses       6481     6484       +3     
  Partials      401      401

Files Changed	Coverage Δ
api/handler/multipart_upload.go	`19.42% <0.00%> (-0.16%)`	⬇️
api/handler/notifications.go	`30.82% <0.00%> (ø)`
authmate/authmate.go	`0.00% <0.00%> (ø)`
internal/neofs/neofs.go	`1.43% <0.00%> (-0.03%)`	⬇️
internal/neofs/tree.go	`4.59% <0.00%> (ø)`
api/layer/layer.go	`37.71% <75.00%> (+0.53%)`	⬆️
api/layer/neofs_mock.go	`58.67% <100.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

cmd/s3-gw/app.go

cmd/s3-gw/app_metrics.go

internal/neogo/sidechain/sidechain.go

api/resolver/resolver.go

roman-khimov · 2023-08-03T20:03:27Z

api/handler/handlers_test.go

@@ -85,7 +85,7 @@ func prepareHandlerContext(t *testing.T) *handlerContext {

 	layerCfg := &layer.Config{
 		Caches:      layer.DefaultCachesConfigs(zap.NewExample()),
-		AnonKey:     layer.AnonymousKey{Key: key},
+		GateKey:     key,


It's supposed to be a feature, see #271. Although I don't quite understand it.

On current master it doesn't work:

$ aws s3api create-bucket --bucket $BUCKET --endpoint http://localhost:9080 --no-sign-request An error occurred (InternalError) when calling the CreateBucket operation (reached max retries: 0): We encountered an internal error, please try again.

In the gate logs:

2023-08-04T07:42:20.324+0400 error handler/util.go:29 call method {"status": 500, "request_id": "dc3da4ba-b5b0-42aa-800f-2dbb71b80ddc", "method": "CreateBucket", "bucket": "heh3", "object": "", "description": "couldn't get bearer token signature key", "error": "couldn't get box data from context"}

In any case, we have to sign requests. Many client calls requires Signer as an explicit parameter. I tend to think it doesn't matter what key it should be either a gate key or an anonymous one.
But the gate key we have to have because all our sessions were created for the gate key. And we need to sign requests by it

I also agreed with comment in #271 because we can't create container with random key, because containerPut is not a free operation. Signer is required and must not be nil. The account corresponding to the specified Signer will be charged for the operation.

It can be used for reads, I guess.

api/layer/layer.go

cmd/s3-gw/app_metrics.go

api/handler/multipart_upload.go

smallhive · 2023-08-04T08:53:53Z

Added anonymous key using for the containerList operation

smallhive · 2023-08-07T04:35:25Z

Updated to the latest RC-10 SDK and neofs-contract versions

cmd/s3-gw/app.go

api/resolver/resolver.go

roman-khimov · 2023-08-07T20:42:43Z

api/layer/container.go

@@ -84,6 +84,11 @@ func (n *layer) containerList(ctx context.Context) ([]*data.BucketInfo, error) {
 		res []cid.ID
 		rid = api.GetRequestID(ctx)
 	)
+
+	if api.IsAnonymousRequest(ctx) {


Why only this request? I think we can have the same in other ones like object get.

Moved this check to the old place, inside Owner func. After that it will work like previous implementation

But what about keys? I suppose we should use a different key when we're doing anonymous requests. Maybe layer should return user.Signer now to provide both at the same time (gw normally and anon if no auth)?

I revised the code one more time.
We have the next component chain (gate component names):

layer -> neofs -> pool

layer - handles API requests, calls neofs component to prepare data and then it call pool to execute request in neofs node.

In fact, the layer even doesn't need to know something about a key or signer. We will drop the key from it.
The neofs component stores the same key (pool also stores it) and it is a good place to make magic with anonymous keys. We just need to pass isAnonymous flag to neofs component and anonymous signer will be used

Made changes in separate commits. At first look, I replaced all places when we cat override the signer. In other places, random keys are already in use by SDK client.

If these changes ok, even after few iterations, I will squash them to proper commit, if required

api/resolver/resolver.go

cmd/s3-gw/app.go

api/layer/anonymous.go

api/layer/container.go

roman-khimov · 2023-08-09T10:34:57Z

api/layer/container.go

@@ -84,6 +84,11 @@ func (n *layer) containerList(ctx context.Context) ([]*data.BucketInfo, error) {
 		res []cid.ID
 		rid = api.GetRequestID(ctx)
 	)
+
+	if api.IsAnonymousRequest(ctx) {


But what about keys? I suppose we should use a different key when we're doing anonymous requests. Maybe layer should return user.Signer now to provide both at the same time (gw normally and anon if no auth)?

cmd/s3-gw/app.go

internal/neofs/neofs.go

api/layer/layer.go

cmd/s3-gw/app.go

Signed-off-by: Evgenii Baidakov <[email protected]>

closes #806 Signed-off-by: Evgenii Baidakov <[email protected]>

Signed-off-by: Evgenii Baidakov <[email protected]>

The removing changes was made in 70957d7#diff-efcc85eef482b2cb183da5fd07ad20378963ee903b8876f59ae2b957ee684f67L393. The bug leads to almost infinite waiting for the waiting the result of setEACL operation. Fixing the bug may improve/fix #800 and nspcc-dev/neofs-testcases#591. Signed-off-by: Evgenii Baidakov <[email protected]>

In some cases in may be useful. To take data from public bucket or check the access rules for object Signed-off-by: Evgenii Baidakov <[email protected]>

Signed-off-by: Evgenii Baidakov <[email protected]>

smallhive commented Aug 3, 2023

View reviewed changes

cmd/s3-gw/app.go Outdated Show resolved Hide resolved

cmd/s3-gw/app_metrics.go Outdated Show resolved Hide resolved

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch 4 times, most recently from 250ad55 to 518fd38 Compare August 3, 2023 10:29

smallhive marked this pull request as ready for review August 3, 2023 11:03

smallhive requested a review from roman-khimov August 3, 2023 11:03

roman-khimov reviewed Aug 3, 2023

View reviewed changes

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch from 8cf4903 to 3778430 Compare August 4, 2023 03:53

smallhive requested a review from roman-khimov August 4, 2023 06:41

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch from 3778430 to 96e0d66 Compare August 4, 2023 07:11

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch 3 times, most recently from 36d2d04 to b777c3c Compare August 7, 2023 04:29

roman-khimov reviewed Aug 7, 2023

View reviewed changes

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch 2 times, most recently from 5deac8b to 5934df7 Compare August 8, 2023 04:13

smallhive requested a review from roman-khimov August 8, 2023 04:16

roman-khimov reviewed Aug 9, 2023

View reviewed changes

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch 2 times, most recently from 6c1b743 to db8db45 Compare August 10, 2023 10:10

smallhive requested a review from roman-khimov August 10, 2023 10:17

roman-khimov reviewed Aug 10, 2023

View reviewed changes

cmd/s3-gw/app.go Outdated Show resolved Hide resolved

internal/neofs/neofs.go Outdated Show resolved Hide resolved

internal/neofs/neofs.go Outdated Show resolved Hide resolved

api/layer/layer.go Show resolved Hide resolved

cmd/s3-gw/app.go Show resolved Hide resolved

smallhive added 3 commits August 11, 2023 11:16

resolver: Replace resolver with NNS contract bindings

eddc129

Signed-off-by: Evgenii Baidakov <[email protected]>

layer: Use gate private key instead of anonymous

5c4073a

Signed-off-by: Evgenii Baidakov <[email protected]>

*: Update SDK to the latest version

48b7ebc

closes #806 Signed-off-by: Evgenii Baidakov <[email protected]>

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch from db8db45 to 4b865a4 Compare August 11, 2023 08:24

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch from 4b865a4 to fe4f585 Compare August 11, 2023 09:03

smallhive requested a review from roman-khimov August 11, 2023 09:22

smallhive added 9 commits August 11, 2023 14:00

*: Remove pool's deprecated methods

b79745f

Signed-off-by: Evgenii Baidakov <[email protected]>

authmate: Init authmate with layer.NeoFS interface

7080a3d

Signed-off-by: Evgenii Baidakov <[email protected]>

api: Use anonymous key

65e7899

In some cases in may be useful. To take data from public bucket or check the access rules for object Signed-off-by: Evgenii Baidakov <[email protected]>

changelog: Updated changelog

cac5b70

Signed-off-by: Evgenii Baidakov <[email protected]>

layer: Remove unused gateKey field

0775e3b

Signed-off-by: Evgenii Baidakov <[email protected]>

layer: Remove gateSigner

4a66ae8

Signed-off-by: Evgenii Baidakov <[email protected]>

layer: User anonSigner

99525d8

Signed-off-by: Evgenii Baidakov <[email protected]>

neofs: Add maxObjectSize as parameter in constructor

cb7d1e5

Signed-off-by: Evgenii Baidakov <[email protected]>

smallhive force-pushed the 806-upgrade-to-sdk-rc10 branch from fe4f585 to cb7d1e5 Compare August 11, 2023 10:02

roman-khimov approved these changes Aug 15, 2023

View reviewed changes

roman-khimov merged commit 5c083a9 into master Aug 15, 2023
10 of 12 checks passed

roman-khimov deleted the 806-upgrade-to-sdk-rc10 branch August 15, 2023 13:45

roman-khimov mentioned this pull request Aug 25, 2023

Drop noop resolver #813

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to sdk rc10 #807

Upgrade to sdk rc10 #807

smallhive commented Aug 3, 2023

codecov bot commented Aug 3, 2023 •

edited

Loading

roman-khimov Aug 3, 2023

smallhive Aug 4, 2023 •

edited

Loading

roman-khimov Aug 4, 2023

smallhive commented Aug 4, 2023

smallhive commented Aug 7, 2023 •

edited

Loading

roman-khimov Aug 7, 2023

smallhive Aug 8, 2023

roman-khimov Aug 9, 2023

smallhive Aug 10, 2023

smallhive Aug 10, 2023

roman-khimov Aug 9, 2023

Upgrade to sdk rc10 #807

Upgrade to sdk rc10 #807

Conversation

smallhive commented Aug 3, 2023

codecov bot commented Aug 3, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

smallhive Aug 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smallhive commented Aug 4, 2023

smallhive commented Aug 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 3, 2023 •

edited

Loading

smallhive Aug 4, 2023 •

edited

Loading

smallhive commented Aug 7, 2023 •

edited

Loading