Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
build: merge dev/4.3.x into main (#4177)
* feat(external table): adapt external tables (#3195) * modify according to comments * finish syncing column for external table * remove redundant code * Modify external table judgment logic * external table does not support indexes and constraints * add sort test for repository * finish tests of TableService and DBIdentitiesService * modify error of test * modify format * add test for mulitple database sources * revert list-basic-schema-table-columns * modify according to comments * modify according to comments * fix(dlm): task status is wrong (#3491) * fix the issue of wrong task status * format * fix the issue of throw not found exception when database is deleted. * simplify code * fix(schedule): risk level mismatch when operating a schedule (#3529) * fix select risklevel failed * fix description * fix(sqlplan): generate task failed (#3535) * fix execute sqlplan * fix execute sqlplan * feat(osc): let osc task resumable (#3516) * dispatch osc away from flow task, make osc resumable * fix integrate test case * fix comment * fix comment * correct status and progress compute logic * correct status and progress compute logic * correct status and progress compute logic * feat(osc): check ghost table not exist before dispatch flow task (#3605) * fix(osc): osc task result not update when oms step info changed (#3678) * fix osc task result not update when oms step changed * fix osc task result not update when oms step changed * update submodule * feat(task): separate resource state from job (#3700) * task module refactor:separate resource state from job entity * task module refactor:separate resource state from job entity * fix(global-search): failed to sync database which get 'PENDING' status and 'object_last_sync_time' is null (#3731) * fix failed to sync database * update submodule * build: upgrade odc version from 4.3.2 to 4.3.3 #3732 * update submodule * resp to comments * feat(sqlcheck): add sql affect rows support for oracle and ob oracle (#3735) * feat: add support for OBOracleAffectedRowsExceedLimit * Solve the unit test error * support Oracle * modify according to commend * modify according to commend * modify according to commend * modify according to commend * feat(pl): support editing pl sql for ob mysql (#3661) * feat(databaseChange): support editing pl sql for ob mysql * add IF EXISTS for drop procedure in ob mysql * make stopOnError true in editing ob mysql pl * use stringUtil rather than visitor to obtain temp pl * minimize length of lock key * support trigger and function * modify code format * Test two cases: the normal case and the case where the temporary pl already existed * modify code format * modify code format * modify code format * created DBPLModifyHelper to manage related methods * remove redundant code * modify code format * modify code format * add controller and service for modify pl * remove redundant code * remove redundant code * remove redundant code * remove redundant code * add integration test * removed @lazy * DBPLModifyHelper aggregate ConnectSessionService to execute the wrapped edit pl sql * modify according to commend * modify according to commend * modify according to commend * modify according to comments * modify code format * feat(task): refactor task getStatus interface return TaskStatus instead of JobStatus (#3766) * feat(ob-sql-parser): make parser adapt for oceanbase 4.3.3 (#3762) * adapt for oceanbase mysql * make ob-oracle adapt for ob433 * add more test cases * rename class * fix array out of bound err * secure(framework): enable secure cookie for http session (#3781) * secure(framework): offer cookie secure option in system configurations * fix secure cookie not works * mrege 4.3.2 to 4.3.x * mrege 4.3.2 to 4.3.x * fix(external table):get table and external table together without exception (#3793) * can get table and external table together without exception * Refactor code * Add exception handling for external table support check * Refactor method isExternalTableSupported * feat(task): refactor base task, let it not be force depended any more (#3796) * feat(ticket): allow project members to view and owners to update all tickets in the project (#3739) * add project in detail flow instance * listFlowInstances refactor * refactor * delete unused code * fix pmd * remove not implemented code * fix schedule * fix schedule * fix ut * fix ut * fix apply project * fix * feat(web): session cookie secure default false (#3798) * feat(task): refactor task interface, introduce task runtime component (#3807) * refactor task interface, introduce task runtime component * change agent.runtime package class visiable scope to default * change agent.runtime package class visiable scope to default * change agent.runtime package class visiable scope to default * fix TaskApplicationTest * feat(task): remove DefaultTaskResult (#3827) * remove DefaultTaskResult, fix TaskResult json deserialize failed * remove DefaultTaskResult, fix TaskResult json deserialize failed * remove DefaultTaskResult, fix TaskResult json deserialize failed * feat(external tables): supports syncing external table files (#3802) * finish framework of syncing external table files * Implement external table file sync in OBMySQL and OBOracle * Refactor external table file sync response handling * Refactor external table file sync to return boolean * feat(sql-console-rule): sql console adapt sql type of call ,comment on and set session (#3764) * feat(external table): external table support sensitive column (#3821) * enhancement(external table):add swagger annotation for sync external table files (#3840) * build: recover submodule * feat(logicaldatabase): add missed code for data.sql when merge main to dev/4.3.x (#3882) * feat(dlm): data archive support generating dynamic target table name (#3883) * feat:data archive support generating dynamic target table name. * response comment. * builds: merge from main into dev/4.3.x (#3884) * add missed code when merge main to dev/4.3.x * add missed code when merge main to dev/4.3.x * add missed code when merge main to dev/4.3.x * feat(regulation): sql check and sql console rules support for Oracle (#3877) * support sql check for oracle mode * fix duplicated metadata * fix(sql-check): cannot get oracle affect sql rows (#3892) * fix: adjust OracleAffectedRowsExceedLimit loop start index * Refactor affected rows estimation in SQL check rules * security: exclude sshd-common from spring-cloud-context (#3901) * docs: replace idea-run-configuration-start-odc-server-2 with correct image (#3890) * doc: replace idea-run-configuration-start-odc-server-2 with correct image * doc: replace idea-run-configuration-start-odc-server-2 with correct image * feat(session): support Oracle kill session (#3898) * support for oracle kill session * delete unused code * fix code format * delete unused code * fix ut * response to comments * fix ut * add @notempty * fix(pl-edit): procedure name in drop procedure statements are recognized as the table name in DBSchemaExtractor (#3894) * fix pl name are recognized as the table name * fix typo in test_db_schema_extractor.yaml comment * Add drop stmt handling in DBSchemaExtractor and tests * drop view statement should be extracted * feat(subpartition): finish sub partitions (#3905) * finish subpartitions for ob mysql * modify according to comments * Enhance OBMySQLGetDBTableByParser for subpartition support * Refactor OBMySQLGetDBTableByParser for subpartition naming clarity * modify format * Enhance OBOracleGetDBTableByParser for subpartition support * Refactor fillSubPartitionValue method * Add tests for ob oracle * Refactor fillSubPartitionValue method for clarity * feat(sub partition) : add the associated partition information for sub partition (#3926) * Adds the associated partition information for subpartition * Add @ApiOperation to DBTableController getTable endpoint * Refactor partition parsing in OBMySQL and OBOracle plugins * fix(task): rollback DestroyExecutorJob to do destroy job action * fix(task): add index to resource_resource table * feat(sso): support saml integration (#3783) * add saml integration * add saml integration * fill secret when testLogin * fill secret when testLogin * fill secret when testLogin * amend test * amend test * fix * fix * fix * fix(migrate): rename V_4_3_3_1 to V_4_3_3_2 (#3974) * feat(permission): add view permission control (#3946) * Add view support and optimize table sync in TableService * Add support for VIEW and EXTERNAL_TABLE in ApplyTablePermissionPreprocessor * Optimize view support in TableService * modify code format * add unit tests * rename method * modify code format * Remove redundant assertions in TableServiceTest * pass unit tests * Refactor DBObjectRepository and enhance permission handling * Grant SYSDBA role to test user in TestDBConfigurations * feat(project): support delete projects (#3948) * support batch delete projects * list unfinished tickets * bugfix * fix archive project name * delete unstaged code * add unit tests * add TODO * feat(session): SQL console connection keep alive (#3993) * support client keep alive * add ut * response to comments * do not reconstruct exectuor service when reset connection * make robust * optimize code * only keep alive in console connection * feat(pl debug): pl debugging adapt odp directional routing (#3938) * finish pl debugging adapt odp directional routing * Add ODP version check for CLOUD_OB_ORACLE in DBPLOperators * Add ODP version check for CLOUD_OB_ORACLE in DBPLOperators * Add PLDebugODPSpecifiedRoute to GetPLErrorCallBack and DebuggerSession * Fix log message and SQL syntax in OraclePLOperator and PLUtils * Optimize PLDebugODPSpecifiedRoute usage in OBOracleCallFunctionCallBack * Refactor OBProxy version check and routing in DBPLOperators * modify code format * Refactor OBProxy version handling and add ODP version support * Enhance ODP routing logic in AbstractDebugSession * modify code format * Refactor CallProcedureCallBack to handle null PLDebugODPSpecifiedRoute * Refactor OBOracleCallFunctionCallBack to handle null route * Refactor ODP routing logic and remove unused imports * modify code format * Enhance ODP routing with PLUtils in PLDebugSession * Add `@NonNull` annotations to PLDebugODPSpecifiedRoute constructor * Refactor ODP routing logic and enhance error handling in AbstractDebugSession * Remove unused getODPVersion methods and update ODP routing logic * Remove trailing whitespace in Postgres and Oracle extensions * fix(db-browser): failed to get table due to create extended stats in column in oracle mode (#4026) * fix failed to get table in oracle mode when there is a extended stats in column * fix generate incorrect column ddl when column type is bit * feat(permission): support for global project roles (#3820) * ComposedPermission implementation * add ProjectPermission implementation * add actions in @PreAuthenticate * fix * refactor * add ut cases * delete deprecated code * refactor check project roles * refactor ResourceRoleService * bugfix * delete unused code * fix ut * response to comments * fix ut * build: change code owners (#4033) * change code owners * change code owners * change * Update IAM role & permission migration files (#4041) * fix(permission):time is missing when initiating apply database task again (#4046) * Add expireTimeValue to ApplyDatabaseParameter for frontend task initiation * Refactor ApplyDatabaseParameter: rename expireTimeValue to validDuration * fix(osc): remove distinct from osc query user sql * opt offset unit (#4075) * rollback for Exception (#4069) * disable log rolling policy (#4068) * fix change log detail (#4070) * merge main into 4.3.x (#4079) * feat(dlm): update dlm sdk version to 1.1.4.bp1 (#3251) * fix(statefulRoute): batchCompilations and getRecyclebinSettings will failed if statefulroute enabled (#3257) * fix(resultset-export): the exported file is empty if use lower table name for oracle mode (#3254) * convert to upper case for oracle * fix typo * use lower case for mysql mode * list directory instead of looking up by name * fix(statefulRoute): sensitive column may can't get result (#3261) * fix statefulroute * fix statefulroute * fix(manual execution): manual execution button should not be displayed after clicking manual execution in odc431 (#3279) * Turn off the manual execution button display * add enum PRE_EXECUTION for FlowStatus * remove enum PRE_EXECUTION for FlowStatus * feat(dlm): support configure sharding strategy (#3275) * support configure sharding strategy. * upgrade version to 1.1.4.bp2 * fix(taskframework): cherry-pick the bug fix from obcloud_202409bp (#3909) * build: update 4.3.2 submodule (#3924) * merge 24v4bp into main (#3925) * fix(schedule): list task by creator invalid (#3752) * fix(schedule): optimize the logic for listing schedules and scheduling task (#3769) * opt list schedule & schedue task * bugfix * opt list schedule task * opt list schedule task * opt list schedule task * fix list is empty * support filter by json filed * simply code * simply code * rsp comments * rsp comments * feat(object-storage): use s3 sdk for google cloud storage (#3785) * security(object-storage): add user id into object key (#3786) * fix(dlm): do not print logs if the data source is uninitialized (#3819) * fix(schedule): customize description generator according to different environments (#3815) * Customize description generator according to different environments * code format * fix(dlm): opt error message when editing the limit configuration (#3817) * opt error message * add i18 * rsp comments * feat(schedule):support list schedule filter by datasource name (#3855) * fix(schedule): allowed to filter by datasource name #3859 * fix(schedule): list datasources with attributes (#3862) * support list schedule filter by datasource name. * list datasource with attributes * fix(schedule): schedule can be edited while subtasks are still in progress (#3891) * bugfix * bugfix * fix(database-change): delete with check schedule task status (#3899) * fix:delete schedule with check schedule task status * response comment. * code format --------- Co-authored-by: kiko <[email protected]> Co-authored-by: LuckyLeo <[email protected]> * fix (#3934) * fix(osc): add version whitelist to enable lock table feature * fix(session): add svrIp in session list (#3961) * fix duplicate sessionId * fix * fix(session): add non support kill version (#3979) * fix * fix * fix * fix * fix * change to data.sql * change to data.sql * change to data.sql * fix(actuator): diasble actuator by default (#3991) * default disable actuator * default disable actuator * default disable actuator * fix the issue of table not exist (#4003) * feat(common): reduce log of JsonUtils * update 432bp2 submodule (#4007) * fix(actuator): disable actuator autoconfiguration in client mode (#4047) * default disable actuator * default disable actuator * rsp comments --------- Co-authored-by: Ang <[email protected]> Co-authored-by: LuckyLeo <[email protected]> Co-authored-by: zijia.cj <[email protected]> Co-authored-by: IL MARE <[email protected]> Co-authored-by: kiko <[email protected]> Co-authored-by: LioRoger <[email protected]> Co-authored-by: pynzzZ <[email protected]> * fix(changelog): opt the changelog content (#4083) * fix change log * fix change log * code format * opt error message (#4086) * fix(flow):view export function can be initiated without permission #4090 * fix(project): archiving projects will fail because of wrong check of tickets references #4089 * fix(approval): approval is not needed in individual organizations #4067 * fix(saml): saml may blocked tomcat thread * fix(task): correct region key name for resourceID * fix(db): cannot replace original pl name where editing pl (#4088) * Enhance regex escaping in DBPLModifyHelper * Enhance regex escaping in DBPLModifyHelper * Refactor test method name for clarity in StringUtilsTest * fix(dbbrowser): cant get correct ddl of procedure (#4084) * Add show create procedure for MySQL schema accessor * Add test for MySQL procedure with DEFINER clause * Refactor MySQL procedure parsing and add tests * Remove redundant MySQL procedure parameter query * fix(flow): add currentUserResourceRole in flow APIs #4096 * fix(notification): support send account name in message (#4085) * do not validate webhook * support sending approver account name * support sending creator account name * fix(execution-plan): avoid invalid number #4087 Open * fix(notification): fail to enqueue schedule event (#4091) * fix failed to enqueue event * fix incorrect task type * avoid NPE * fix NPE (#4106) fix(integration): external approval integration will cause NPE if hyperlinkExpression is null #4106 * fix(integration): it doesn't sync internal schemas in the project in the bastion mode #4109 * fix(session):drop pl require database change permission #4112 * fix(db): The method of determining whether opening the global client session is incorrect * Refactor global client session check in DefaultDBSessionManage * Add log for failed column 'time' lookup in DefaultDBSessionManage * fix(migrate): complete data for connect_database.connect_type (#4113) * complete connect_database.connect_type if is null * update sql * fix(permission): global project role cannot see approvable tickets #4116 * fix(db):miss quote of pl name which contains @ causes pl edit failed #4115 * fix(function):The function does not display properly if the return value type is year (#4093) * Add YEAR support in OBMysqlCallFunctionCallBackTest and JdbcDataTypeUtil * Enhance YEAR type handling in OBMysqlCallFunctionCallBackTest and JdbcDataTypeUtil * Refactor OBMysqlCallFunctionCallBack and move tests * Optimize error handling in OBMysqlCallFunctionCallBack * Optimize row handling and import CollectionUtils in OBMysqlCallFunctionCallBack * Refactor OBMysqlCallFunctionCallBack, optimize imports and error handling * Add logging for function call failures in OBMysqlCallFunctionCallBack * fix(permission): global project roles cannot view/execute/approve tickets #4117 * fix(structure-compare):the structure synchronization task cannot be initiated when the structure comparison task is not created by yourself #4122 * fix(permission): global project roles cannot operate database/table permission apply and schedule tickets #4124 * fix(session): kill session may lead npe #4114 * fix(task): log4j set failed for task * build: update 4.3.3 submodule #4126 * fix(permission): external approval integration doesn't take effect #4128 * fix(session): oracle mode effected by kill-query-or-session.max-supported-ob-version #4134 * oracle mode effected by kill-query-or-session.max-supported-ob-version * oracle mode effected by kill-query-or-session.max-supported-ob-version * fix(audit): update i18n resources and add archive/delete projects audit events #4141 * fix audit * fix i18n * response to comments * feat(resource): add double check when destroy resource * fix(schedule):check has running task when delete (#4143) * fix:check has running task when delete * resp comments * client mode exclude saml (#4142) * fix(logicaldatabase): it may NPE when the logical database task starts * fix(schedule): remove slow sql #4130 * fix(session): kill oracle session may cause sockettimeout #4148 * fix(schedule): cherry pick describe database (#4150) * do not request api when describe database (#4147) * fix describe database (#4149) * fix(session): kill session may happen invalid server ip address #4144 * IP REGEX outdate * remove useless code * fix(sql check):add two sql check rules for CREATE LIKE/AS statement * Add checks for LIKE table in MySQLNoTableCommentExists and NoPrimaryKeyExists rules * Add SQL check rules for CREATE LIKE/AS table statements * Optimize CREATE LIKE statement checks and rule descriptions * Update CREATE LIKE/AS rule descriptions and names * Refactor SQL check rules for CREATE LIKE/AS conditions * Fix punctuation in SQL check messages and disable CREATE LIKE/AS rules * Enable SQL check rules in regulation-rule-applying.yaml * Fix SQL check rule descriptions in regulation metadata * Fix SQL check rule descriptions in regulation metadata * Fix SQL check rule descriptions in regulation metadata * security: add @SkipAuthorize in public method #4153 * fix(schedule):delete without running task #4157 * remove duplicated annotation * fix(sql-parser): failed to parse table ddl when create fulltext key with parser #4154 * fix failed to parse table ddl when create fulltext index with parser * with parser adaption for native mysql * fix mysql g4 create fulltext index with parser * fix ut test * security: add @SkipAuthorize for IntegrationService #4160 * build: upgrade db-browser to 1.2.1 #4159 * build: update 4.3.3 submodule #4163 * feat(dlm): support filesystem (#4151) * supports create and test file system datasource * add i18 * fix access denied * fix access denied * fix access denied * fix delete obj failed * support sync database * set default schema * fix * support archive to file system * upgrade hadoop-common from 3.3.4 to 3.3.6 * bugfix * bugfix * bugfix * bugfix * pull mode * revert * cannot sync oss datasource * code format * code format * opt dlm task implements * fix npe * store partition range in disk * bugfix * code format * opt error message * opt task log * create table for all type * enable save point * opt log info * bugfix * bugfix * modify oss configuration * modify oss configuration * fix object key * use file url as schema name * opt oracle username * rollback commit * enable ob sync table structure * adapt check point * fix region is null * update status when task failed * bugfix * merge 4.3.x * record global range in statistic * record global range in statistic * upgrade sdk version to 1.2.0 * rsp comment * rsp comment * rsp comments * fix(session): killing sessions failed in OSC tasks #4166 * doc: init 4.3.3 changelog (#4162) * upgrade changelog * rsp comments * rsp comments * rsp comments * rsp comments * rsp comments * update changelog * rsp comments * rsp comments * rsp comments * exclusion aws-java-sdk-bundle (#4167) * fix(session): killing session may fail due to a wrong regex #4169 * fix wrong regex * add ut case * remove regex * fix(session): get wrong server port #4171 * build: upgrade db-browser from 1.2.1 to 1.2.2 #4172 * fix(datatransfer): invalid batchSize when importing external csv file * fix invalid batchSize * add default batch size * fix(session): DBSession does not involve svrIp when OB version less than 4.0 #4174 * fix list sessions * fix * fix * fix ut * response to comments * rollback comments * refactor ut * fix(session): kill console query may failed cause session occupied #4175 * new connect session * new connect session * skip permission * refactor --------- Co-authored-by: MarkPotato777 <[email protected]> * fix(session): add retry logic when get session list #4176 * add retry * change timeout to 2000L --------- Co-authored-by: zijia.cj <[email protected]> Co-authored-by: guowl3 <[email protected]> Co-authored-by: LioRoger <[email protected]> Co-authored-by: IL MARE <[email protected]> Co-authored-by: yizhou <[email protected]> Co-authored-by: kiko <[email protected]> Co-authored-by: Yanze Li <[email protected]> Co-authored-by: Ang <[email protected]> Co-authored-by: zhangxiao <[email protected]> Co-authored-by: LuckyLeo <[email protected]>
- Loading branch information
11 people
authored
Jan 17, 2025
1 parent
ef6a197
commit cfa4d9c