Delta Lake Table tools

Creating Delta Lake Tables

Using the deltalake Python library

You can use the deltalake Python library to create Delta Tables and write to the bucket.

Using the lakehouse-loader utility

You can also use the lakehouse-loader utility that EDB created for this task to export data from an arbitrary Postgres instance to Lakehouse tables in a storage bucket.

For example, with the lakehouse-loader utility:

export PGPASSWORD="..."
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
# export other AWS envvars

./lakehouse-loader postgres-to-delta postgres://test-user@localhost:5432/test-db -q "SELECT * FROM some_table" s3://my-bucket/my_schema/my_table

This code exports the data from the some_table table in the test-db database to a Delta Table in the my_schema/my_table path in the my-bucket bucket.

You can then query this table in the Lakehouse node by creating an external table that references the Delta Table in the my_schema/my_table path. See Querying Delta Lake Tables for details on how to do this.