NSFS | NC | add option to set account supplemental groups #8552

nadavMiz · 2024-11-21T16:21:04Z

Explain the changes

supplemental groups are additional groups an account can be part of besides his main group (gid). add option to set a users supplemental groups through the nsfs cli. see Supplementary group IDs in https://man7.org/linux/man-pages/man7/credentials.7.html. having addition groups allows account to access files and directory that allow access to one of the accounts supplemental groups (similar to main group access ). supplementary group IDs are used mainly for adding permissions. for other purposes the accounts main GID will be used (for example determining the group of a file created by the account). note that this enables access only on file system level, s3 commands will still require bucket policy permissions. in the same manner account will still be block from accessing the file system even if it has bucket policy permissions

add option to set account supplemental groups:
supplemental groups can be added by nsfs cli either by passing a single positive number (202) or string of comma separated positive number ("200,101")
supplemental groups can be added using from_file as wither one of the previously mentioned types or as an array of positive numbers ([212,313])
supplemental groups will be saved as part of the account configuration in the section nsfs_account_config
set supplemental groups as part of setting the user id for both linux and darwin (see os_linux.cpp and os_darwin.cpp)
add tests for both adding and updating account configuration (including from_file, and anonymous access)
change documentation to include data about supplemental_groups option

Issues: Fixed #7274

Testing Instructions:

sudo npx jest test_nc_nsfs_account_cli.test.js
sudo npx jest test_nc_nsfs_anonymous_cli.test.js
sudo node ./node_modules/mocha/bin/mocha src/test/unit_tests/test_nsfs_access.js
sudo NC_CORETEST=true node ./node_modules/mocha/bin/mocha src/test/unit_tests/test_nsfs_integration.js

Doc added/updated
Tests added

src/sdk/nb.d.ts

src/manage_nsfs/manage_nsfs_help_utils.js

src/native/fs/fs_napi.cpp

src/native/util/os_linux.cpp

src/manage_nsfs/manage_nsfs_help_utils.js

src/manage_nsfs/manage_nsfs_constants.js

src/manage_nsfs/manage_nsfs_validations.js

src/server/system_services/schemas/nsfs_account_schema.js

src/test/unit_tests/test_nsfs_access.js

src/manage_nsfs/manage_nsfs_validations.js

src/cmd/manage_nsfs.js

src/manage_nsfs/manage_nsfs_validations.js

src/native/fs/fs_napi.cpp

shirady

I added my comments (without the native files).

I would list things that you can add in this PR:

Tests for the noobaa-cli that combines the supplemental groups with the --from_file flag.
Tests for the noobaa-cli that combines the supplemental groups with the --anonymous flag
Update the S3Ops.md about permission - currently only UID, GID is mentioned.

src/api/common_api.js

src/cmd/manage_nsfs.js

src/manage_nsfs/manage_nsfs_cli_utils.js

src/manage_nsfs/manage_nsfs_constants.js

src/native/fs/fs_napi.cpp

src/test/unit_tests/jest_tests/test_nc_nsfs_account_cli.test.js

src/test/unit_tests/test_nsfs_access.js

src/util/native_fs_utils.js

src/test/unit_tests/jest_tests/test_nc_nsfs_account_cli.test.js

src/manage_nsfs/manage_nsfs_cli_errors.js

docs/NooBaaNonContainerized/AccountsAndBuckets.md

docs/NooBaaNonContainerized/NooBaaCLI.md

src/manage_nsfs/manage_nsfs_cli_utils.js

src/test/unit_tests/test_nsfs_access.js

src/util/native_fs_utils.js

not relevant

shirady · 2025-01-08T14:50:17Z

docs/NooBaaNonContainerized/AccountsAndBuckets.md

-  - `uid/gid/user` - An account's access key is mapped to a file system uid/gid (or user). Before performing any file system operation, NooBaa switches to the account's UID/GID, ensuring that accounts access to buckets and objects is enforced by the file system.
+  - `uid/gid/user` - An account's access key is mapped to a file system uid/gid (or user). Before performing any file system operation, NooBaa switches to the account's UID/GID, ensuring that accounts access to buckets and objects is enforced by the file system.  
+
+  - `supplemental_groups` - In addition to the account main GID, an account can have supplementary group IDs  that are used to determine permissions for accessing files. These GIDs are validated against a files group (GID) permissions.


extra space

Suggested change

- `supplemental_groups` - In addition to the account main GID, an account can have supplementary group IDs that are used to determine permissions for accessing files. These GIDs are validated against a files group (GID) permissions.

- `supplemental_groups` - In addition to the account main GID, an account can have supplementary group IDs that are used to determine permissions for accessing files. These GIDs are validated against a files group (GID) permissions.

src/manage_nsfs/manage_nsfs_validations.js

shirady · 2025-01-08T15:04:50Z

src/native/util/os_darwin.cpp

+        if (!_groups.empty()) {
+            _groups.push_back(_gid);
+            std::swap(_groups.front(), _groups.back());
+            MUST_SYS(setgroups(_groups.size(), &_groups[0]));


Could you add a comment about this change?

shirady · 2025-01-08T15:08:49Z

src/test/unit_tests/test_nsfs_access.js

+        //TODO on mac new directories are created with the parents directory GID and not with the process GID. manually change the gid
+        fs.promises.chown(full_path_non_root1, NON_ROOT2_FS_CONFIG.uid, NON_ROOT2_FS_CONFIG.gid);


Why is it here and not after the nb_native().fs.mkdir?
Why the fs.promises.chown is on full_path_non_root1 and not full_path_non_root2?

Signed-off-by: nadav mizrahi <[email protected]>

nadavMiz requested a review from romayalon November 21, 2024 16:21

pull-request-size bot added the size/L label Nov 21, 2024

nadavMiz force-pushed the suplemental-groups branch 4 times, most recently from 702fd42 to 9abd913 Compare November 24, 2024 09:34

naveenpaul1 reviewed Dec 4, 2024

View reviewed changes

src/sdk/nb.d.ts Outdated Show resolved Hide resolved

src/manage_nsfs/manage_nsfs_help_utils.js Outdated Show resolved Hide resolved

src/native/fs/fs_napi.cpp Outdated Show resolved Hide resolved

src/native/util/os_linux.cpp Outdated Show resolved Hide resolved

romayalon previously requested changes Dec 4, 2024

View reviewed changes

nadavMiz force-pushed the suplemental-groups branch 11 times, most recently from f2ee5e0 to 6f7239b Compare December 11, 2024 16:53

nadavMiz requested review from romayalon and shirady December 11, 2024 16:53

shirady reviewed Dec 15, 2024

View reviewed changes

nadavMiz force-pushed the suplemental-groups branch 5 times, most recently from 08a66b9 to 406c2ed Compare December 15, 2024 17:51

shirady reviewed Dec 16, 2024

View reviewed changes

src/test/unit_tests/jest_tests/test_nc_nsfs_account_cli.test.js Show resolved Hide resolved

shirady reviewed Dec 16, 2024

View reviewed changes

src/manage_nsfs/manage_nsfs_cli_errors.js Outdated Show resolved Hide resolved

nadavMiz force-pushed the suplemental-groups branch from 406c2ed to b07ff70 Compare December 16, 2024 08:00

nadavMiz force-pushed the suplemental-groups branch 3 times, most recently from a0b9986 to 5ebbbc7 Compare December 19, 2024 08:04

pull-request-size bot added size/L and removed size/XL labels Dec 19, 2024

nadavMiz requested review from shirady and romayalon December 19, 2024 08:05

nadavMiz force-pushed the suplemental-groups branch from 5ebbbc7 to b306fb3 Compare December 19, 2024 09:12

shirady reviewed Dec 22, 2024

View reviewed changes

nadavMiz force-pushed the suplemental-groups branch 2 times, most recently from e0d6ef0 to 599225f Compare December 23, 2024 16:28

pull-request-size bot added size/XL and removed size/L labels Dec 23, 2024

nadavMiz force-pushed the suplemental-groups branch 5 times, most recently from b948756 to cbb9118 Compare December 23, 2024 19:36

nadavMiz force-pushed the suplemental-groups branch 3 times, most recently from 3ec6ddd to 1ef9a41 Compare January 8, 2025 10:41

nadavMiz requested a review from shirady January 8, 2025 10:49

nadavMiz force-pushed the suplemental-groups branch 3 times, most recently from 0e324bf to 87ed095 Compare January 8, 2025 12:11

nadavMiz self-assigned this Jan 8, 2025

shirady reviewed Jan 8, 2025

View reviewed changes

NSFS | NC | add option to set account supplemental groups

246a0e3

Signed-off-by: nadav mizrahi <[email protected]>

nadavMiz force-pushed the suplemental-groups branch from 87ed095 to 246a0e3 Compare January 8, 2025 16:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NSFS | NC | add option to set account supplemental groups #8552

NSFS | NC | add option to set account supplemental groups #8552

nadavMiz commented Nov 21, 2024 •

edited

Loading

shirady left a comment

shirady Jan 8, 2025

shirady Jan 8, 2025

shirady Jan 8, 2025

	- `supplemental_groups` - In addition to the account main GID, an account can have supplementary group IDs that are used to determine permissions for accessing files. These GIDs are validated against a files group (GID) permissions.
	- `supplemental_groups` - In addition to the account main GID, an account can have supplementary group IDs that are used to determine permissions for accessing files. These GIDs are validated against a files group (GID) permissions.

		//TODO on mac new directories are created with the parents directory GID and not with the process GID. manually change the gid
		fs.promises.chown(full_path_non_root1, NON_ROOT2_FS_CONFIG.uid, NON_ROOT2_FS_CONFIG.gid);

NSFS | NC | add option to set account supplemental groups #8552

Are you sure you want to change the base?

NSFS | NC | add option to set account supplemental groups #8552

Conversation

nadavMiz commented Nov 21, 2024 • edited Loading

Explain the changes

Issues: Fixed #7274

Testing Instructions:

shirady left a comment

Choose a reason for hiding this comment

shirady Jan 8, 2025

Choose a reason for hiding this comment

shirady Jan 8, 2025

Choose a reason for hiding this comment

shirady Jan 8, 2025

Choose a reason for hiding this comment

nadavMiz commented Nov 21, 2024 •

edited

Loading