Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#4714] feat(paimon-spark-connector): Add tests for partitionManagement of paimon table in paimon spark connector #5860

Merged
Merged
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
ed8e5b7
basic ddl
Dec 1, 2024
e4a11d1
basic ddl
Dec 1, 2024
a16e949
basic dml
Dec 1, 2024
e28d323
support partition
Dec 1, 2024
6de61c2
basic dml
Dec 1, 2024
17690df
basic schema ddl
Dec 1, 2024
dae7ccf
Merge branch 'support-paimon-connector-ddl' of github.com:caican00/gr…
Dec 2, 2024
7c6013c
fix
Dec 2, 2024
0915488
fix
Dec 2, 2024
4190fd9
Merge branch 'support-paimon-connector-ddl' into support-paimon-conne…
Dec 2, 2024
01bf325
Merge branch 'main' of github.com:apache/gravitino into support-paimo…
Dec 2, 2024
a83a406
fix
Dec 2, 2024
8846dc9
support partition management
Dec 2, 2024
c114d1c
support partition management
Dec 2, 2024
af0b32a
fix
Dec 3, 2024
79403e2
fix
Dec 3, 2024
1aa58cd
fix
Dec 3, 2024
33c6956
Merge branch 'support-paimon-connector-dml' of github.com:caican00/gr…
Dec 3, 2024
659338f
fix
Dec 2, 2024
069b748
fix
Dec 6, 2024
d354e2c
Merge branch 'support-paimon-connector-ddl' of github.com:caican00/gr…
Dec 9, 2024
658f687
support paimon connector dml together
Dec 9, 2024
03f1a2d
Merge branch 'main' into support-paimon-connector-ddl
caican00 Dec 9, 2024
f0bfd60
fix
Dec 9, 2024
22de28e
fix
Dec 9, 2024
3195e8c
Merge branch 'support-paimon-connector-ddl' of github.com:caican00/gr…
Dec 9, 2024
e337024
fix
Dec 9, 2024
ce0d591
fix
Dec 9, 2024
c77b2f4
Merge branch 'support-paimon-connector-ddl' of github.com:caican00/gr…
Dec 9, 2024
1ce5e4e
fix
Dec 9, 2024
a5413d1
Merge branch 'support-paimon-connector-partition-management' of githu…
Dec 9, 2024
bee5314
fix
Dec 9, 2024
f8220e5
fix
Dec 9, 2024
e2eed28
Merge branch 'support-paimon-connector-ddl' of github.com:caican00/gr…
Dec 15, 2024
51f7b64
fix
Dec 15, 2024
1844efc
fix
Dec 15, 2024
3f75e07
Merge branch 'main' of github.com:apache/gravitino into support-paimo…
Dec 24, 2024
88fe087
fix
Dec 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,37 @@ void testPaimonPartitions() {
checkDirExists(partitionPath);
}

@Test
void testPaimonPartitionManagement() {
testPaimonListAndDropPartition();
// TODO: replace, add and load partition operations are unsupported now.
}

private void testPaimonListAndDropPartition() {
String tableName = "test_paimon_drop_partition";
dropTableIfExists(tableName);
String createTableSQL = getCreatePaimonSimpleTableString(tableName);
createTableSQL = createTableSQL + " PARTITIONED BY (name);";
sql(createTableSQL);

String insertData =
String.format(
"INSERT into %s values(1,'a','beijing'), (2,'b','beijing'), (3,'c','beijing');",
tableName);
sql(insertData);
List<String> queryResult = getTableData(tableName);
Assertions.assertEquals(3, queryResult.size());

List<String> partitions = getQueryData(String.format("show partitions %s", tableName));
Assertions.assertEquals(3, partitions.size());
Assertions.assertEquals("name=a;name=b;name=c", String.join(";", partitions));

sql(String.format("ALTER TABLE %s DROP PARTITION (`name`='a')", tableName));
partitions = getQueryData(String.format("show partitions %s", tableName));
Assertions.assertEquals(2, partitions.size());
Assertions.assertEquals("name=b;name=c", String.join(";", partitions));
}

private String getCreatePaimonSimpleTableString(String tableName) {
return String.format(
"CREATE TABLE %s (id INT COMMENT 'id comment', name STRING COMMENT '', address STRING COMMENT '') USING paimon",
Expand Down
Loading