You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current GlueCatalog implementation does not allow for the reuse of the FileIO object, leading to inefficient usage of manifest cache implemented in ManifestFiles class.
Apache Iceberg version
1.7.1 (latest release)
Query engine
None
Please describe the bug 🐞
The current GlueCatalog implementation does not allow for the reuse of the FileIO object, leading to inefficient usage of manifest cache implemented in
ManifestFiles
class.Problematic Code
The
GlueTableOperations
class creates a newFileIO
object for each instance:https://github.com/apache/iceberg/blob/apache-iceberg-1.7.1/aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java#L113
This prevents the
ManifestFiles
class from using the cache :https://github.com/apache/iceberg/blob/apache-iceberg-1.7.1/core/src/main/java/org/apache/iceberg/ManifestFiles.java#L75
Proposed Solution
Add a constructor or method to the
GlueCatalog
class that accepts aFileIO
object or a function that builds aFileIO
object, similar toJdbcCatalog
:https://github.com/apache/iceberg/blob/apache-iceberg-1.7.1/core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java#L99
Willingness to contribute
The text was updated successfully, but these errors were encountered: