Skip to content

Commit

Permalink
REST: Docker file for Rest catalog adapter image
Browse files Browse the repository at this point in the history
  • Loading branch information
@ajantha-bhat
ajantha-bhat committed Nov 6, 2024
1 parent 2ffd3b0 commit d8b98e3
Show file tree
Hide file tree
Showing 3 changed files with 139 additions and 0 deletions.
9 changes: 9 additions & 0 deletions build.gradle
View file
Original file line number Diff line number Diff line change
Expand Up @@ -985,6 +985,15 @@ project(':iceberg-open-api') {
exclude group: 'org.apache.commons', module: 'commons-configuration2'
exclude group: 'org.apache.hadoop.thirdparty', module: 'hadoop-shaded-protobuf_3_7'
exclude group: 'org.eclipse.jetty'
exclude group: 'com.google.re2j', module: 're2j'
exclude group: 'com.google.code.gson', module: 'gson'
exclude group: 'com.jcraft', module: 'jsch'
exclude group: 'com.google.code.findbugs', module: 'jsr305'
exclude group: 'io.dropwizard.metrics', module: 'metrics-core'
exclude group: 'dnsjava', module: 'dnsjava'
exclude group: 'org.xerial.snappy', module: 'snappy-java'
exclude group: 'commons-cli', module: 'commons-cli'
exclude group: 'com.github.pjfanning', module: 'jersey-json'
}
testFixturesImplementation project(path: ':iceberg-bundled-guava', configuration: 'shadow')
testFixturesImplementation libs.junit.jupiter
Expand Down
43 changes: 43 additions & 0 deletions docker/iceberg-rest-adapter-image/Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

FROM azul/zulu-openjdk:17-jre-headless

# Set up the user and group
RUN \
set -xeu && \
groupadd iceberg --gid 1000 && \
useradd iceberg --uid 1000 --gid 1000 --create-home

# Working directory for the application
WORKDIR /usr/lib/iceberg-rest

# Copy the JAR file directly to the target location
COPY --chown=iceberg:iceberg open-api/build/libs/iceberg-open-api-test-fixtures-runtime-*.jar /usr/lib/iceberg-rest/iceberg-rest-adapter.jar

ENV CATALOG_CATALOG__IMPL=org.apache.iceberg.jdbc.JdbcCatalog
ENV CATALOG_URI=jdbc:sqlite::memory:
ENV CATALOG_JDBC_USER=user
ENV CATALOG_JDBC_PASSWORD=password
ENV REST_PORT=8181

EXPOSE $REST_PORT
USER iceberg:iceberg
ENV LANG=en_US.UTF-8
CMD ["java", "-jar", "iceberg-rest-adapter.jar"]
87 changes: 87 additions & 0 deletions docker/iceberg-rest-adapter-image/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-->

# Iceberg rest adapter image

For converting different catalog implementations into a rest one.

## Building the Docker Image locally

If you want to make changes to the local files, and test them out, you can build the image locally and use that instead:

```bash
# Navigate to the iceberg root directory
cd ../../

# Build the project
./gradlew :iceberg-open-api:shadowJar

# Rebuild the docker image
docker image rm -f apache/iceberg-rest-adapter && docker build -t apache/iceberg-rest-adapter -f docker/iceberg-rest-adapter-image/Dockerfile .
```

## Browse

To browse the catalog, you can use `pyiceberg`:

```
➜ ~ pyiceberg --uri http://localhost:8181 list
default
nyc
ride_sharing_dataset
➜ ~ pyiceberg --uri http://localhost:8181 list nyc
nyc.taxis
nyc.taxis3
nyc.taxis4
nyc.taxis_copy_maybe
nyc.taxis_skeleton
nyc.taxis_skeleton2
➜ ~ pyiceberg --uri http://localhost:8181 describe --entity=table tpcds_iceberg.customer
Table format version 2
Metadata location s3://iceberg-test-data/tpc/tpc-ds/3.2.0/1000/iceberg/customer/metadata/00001-1bccfcc4-69f6-4505-8df5-4de78356e327.metadata.json
Table UUID dce215f7-6301-4a73-acc4-6e12db016abb
Last Updated 1653550004061
Partition spec []
Sort order []
Schema Schema
├── 1: c_customer_sk: optional int
├── 2: c_customer_id: optional string
├── 3: c_current_cdemo_sk: optional int
├── 4: c_current_hdemo_sk: optional int
├── 5: c_current_addr_sk: optional int
├── 6: c_first_shipto_date_sk: optional int
├── 7: c_first_sales_date_sk: optional int
├── 8: c_salutation: optional string
├── 9: c_first_name: optional string
├── 10: c_last_name: optional string
├── 11: c_preferred_cust_flag: optional string
├── 12: c_birth_day: optional int
├── 13: c_birth_month: optional int
├── 14: c_birth_year: optional int
├── 15: c_birth_country: optional string
├── 16: c_login: optional string
├── 17: c_email_address: optional string
└── 18: c_last_review_date: optional string
Snapshots Snapshots
└── Snapshot 0: s3://iceberg-test-data/tpc/tpc-ds/3.2.0/1000/iceberg/customer/metadata/snap-643656366840285027-1-5ce13497-7330-4d02-8206-7e313e43209c.avro
Properties write.object-storage.enabled true
write.object-storage.path s3://iceberg-test-data/tpc/tpc-ds/3.2.0/1000/iceberg/customer/data
```


0 comments on commit d8b98e3

Please sign in to comment.