Move min lock demand to allocation #1046
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sriep
requested review from
peterlimg,
dabasov,
guruhubb and
cnlangzi
as code owners
This was referenced Jun 16, 2023
|
Manual system tests [failure] with the following config
|
|
Manual system tests [failure] with the following config
|
|
Manual system tests [cancelled] with the following config
|
|
Manual system tests [failure] with the following config
|
|
Manual system tests [failure] with the following config
|
boddumanohar
pushed a commit
that referenced
this pull request
Jul 1, 2023
* min_lock_demand * mobile
dabasov
added a commit
that referenced
this pull request
Jul 4, 2023
* multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]>
boddumanohar
added a commit
that referenced
this pull request
Jul 4, 2023
* multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]>
boddumanohar
added a commit
that referenced
this pull request
Jul 4, 2023
* Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]>
Kishan-Dhakan
added a commit
that referenced
this pull request
Jul 16, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <[email protected]> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <[email protected]> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <[email protected]> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: Jayash Satolia <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: peterlimg <[email protected]>
dabasov
added a commit
that referenced
this pull request
Jul 18, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix * Fix * Fix update allocation * Fix create allocation * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <[email protected]> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <[email protected]> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <[email protected]> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Fix * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: peterlimg <[email protected]>
dabasov
added a commit
that referenced
this pull request
Jul 21, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * hotfix: fix `upload` and `download` in wasm (#1121) * Fix wasm download (#1117) * fix wasm download * fix syntax * fix reader * reverting reader changes --------- Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: din-mukhammed <[email protected]> * add wasm download changes * fix merge * fix typo * fix repair cb * fix unit test * rmv log * check if env is wasm * cleanup * defer lock --------- Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: Jayash Satolia <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: din-mukhammed <[email protected]>
YarikRevich
pushed a commit
that referenced
this pull request
Jul 22, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix * Fix * Fix update allocation * Fix create allocation * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <[email protected]> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <[email protected]> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <[email protected]> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Fix * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: peterlimg <[email protected]>
YarikRevich
pushed a commit
that referenced
this pull request
Jul 22, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * hotfix: fix `upload` and `download` in wasm (#1121) * Fix wasm download (#1117) * fix wasm download * fix syntax * fix reader * reverting reader changes --------- Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: din-mukhammed <[email protected]> * add wasm download changes * fix merge * fix typo * fix repair cb * fix unit test * rmv log * check if env is wasm * cleanup * defer lock --------- Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: Jayash Satolia <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: din-mukhammed <[email protected]>
cnlangzi
added a commit
that referenced
this pull request
Jul 28, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix * Fix * Fix update allocation * Fix create allocation * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <[email protected]> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <[email protected]> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <[email protected]> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Fix * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: peterlimg <[email protected]>
cnlangzi
added a commit
that referenced
this pull request
Jul 28, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <[email protected]> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * fixed comments --------- Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <[email protected]> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <[email protected]> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> --------- Co-authored-by: Lz <[email protected]> Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Hitenjain14 <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Yury <[email protected]> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Yury <[email protected]> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <[email protected]> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <[email protected]> * fix encrypt in multi op (#1097) Co-authored-by: Yury <[email protected]> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * hotfix: fix `upload` and `download` in wasm (#1121) * Fix wasm download (#1117) * fix wasm download * fix syntax * fix reader * reverting reader changes --------- Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: din-mukhammed <[email protected]> * add wasm download changes * fix merge * fix typo * fix repair cb * fix unit test * rmv log * check if env is wasm * cleanup * defer lock --------- Co-authored-by: boddumanohar <[email protected]> Co-authored-by: Piers Shepperson <[email protected]> Co-authored-by: Yury <[email protected]> Co-authored-by: sarvesh raut <[email protected]> Co-authored-by: Manali-Jain-Squareops <[email protected]> Co-authored-by: Kishan Dhakan <[email protected]> Co-authored-by: Dinmukhammed <[email protected]> Co-authored-by: Roshan Kumar <[email protected]> Co-authored-by: Lz <[email protected]> Co-authored-by: shahnawaz-creator <[email protected]> Co-authored-by: Sunil Kumar <[email protected]> Co-authored-by: Jayash Satolia <[email protected]> Co-authored-by: stewartie4 <[email protected]> Co-authored-by: din-mukhammed <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Fixes
Tests
Tasks to complete before merging PR:
Associated PRs (Link as appropriate):