You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Support JDBC catalog in PyPaimon. This adds a Python JDBC catalog implementation that uses the same catalog metadata tables as Java Paimon JDBC catalog: paimon_tables, paimon_database_properties, and paimon_table_properties.
The implementation supports SQLite with the Python standard library and dynamically supports MySQL/PostgreSQL when a corresponding Python DB-API driver is installed. Table data and schema files continue to use existing PyPaimon FileIO and SchemaManager behavior.
What changed
Register metastore=jdbc in CatalogFactory
Add JdbcCatalog and JdbcCatalogLoader
Add catalog-key and sync-all-properties catalog options
Cover database and table create/list/get/alter/rename/drop behavior with SQLite-backed tests
Document JDBC catalog creation in PyPaimon Python API docs
Nice! I have a similar change locally that uses SQLAlchemy - but this looks great as it adds fewer dependencies.
Is it worth calling it something other than JDBC? It may be confusing to folks who think it uses the JVM underneath for the database connections.
Nice! I have a similar change locally that uses SQLAlchemy - but this looks great as it adds fewer dependencies. Is it worth calling it something other than JDBC? It may be confusing to folks who think it uses the JVM underneath for the database connections.
Good point. I kept the public catalog type as jdbc because it matches Paimon's existing JDBC catalog configuration and lets users reuse the same metastore=jdbc / jdbc: URI options across engines.
To avoid implying that PyPaimon uses JVM JDBC drivers, I updated the implementation and docs to clarify that PyPaimon uses native Python DB-API drivers under the hood. I also renamed the internal connection helper to _DbApiConnection and adjusted the driver error messages accordingly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Support JDBC catalog in PyPaimon. This adds a Python JDBC catalog implementation that uses the same catalog metadata tables as Java Paimon JDBC catalog:
paimon_tables,paimon_database_properties, andpaimon_table_properties.The implementation supports SQLite with the Python standard library and dynamically supports MySQL/PostgreSQL when a corresponding Python DB-API driver is installed. Table data and schema files continue to use existing PyPaimon
FileIOandSchemaManagerbehavior.What changed
metastore=jdbcinCatalogFactoryJdbcCatalogandJdbcCatalogLoadercatalog-keyandsync-all-propertiescatalog optionsTests
python3 -m py_compile pypaimon/catalog/jdbc_catalog.py pypaimon/catalog/jdbc_catalog_loader.py pypaimon/catalog/catalog_factory.py pypaimon/common/options/config.py pypaimon/tests/jdbc_catalog_test.pyPYTHONPATH=/tmp/paimon-python-test-deps POLARS_SKIP_CPU_CHECK=1 python3 -m unittest pypaimon.tests.jdbc_catalog_test pypaimon.tests.filesystem_catalog_test