diff --git a/src/content/docs/snowflake/features/dynamic-tables.md b/src/content/docs/snowflake/features/dynamic-tables.md index baf6d846..68de6ef0 100644 --- a/src/content/docs/snowflake/features/dynamic-tables.md +++ b/src/content/docs/snowflake/features/dynamic-tables.md @@ -88,3 +88,124 @@ The output should be: | 2 | bar | +----+------+ ``` + +## Dynamic Iceberg Tables + +Dynamic Iceberg Tables combine the auto-refresh capabilities of Dynamic Tables with the Apache Iceberg open table format. This allows you to create materialized views that automatically update from source queries while storing data in Iceberg format on external object storage (such as S3). + +A Dynamic Iceberg Table consists of three key concepts: +- **Dynamic table**: A materialized view that auto-refreshes from a source query on a schedule (controlled by `TARGET_LAG` and `WAREHOUSE`). +- **Iceberg table**: A table stored in Apache Iceberg format on external object storage (configured via `EXTERNAL_VOLUME`, `CATALOG`, and `BASE_LOCATION`). +- **External volume**: A Snowflake object that references an S3 bucket as the backing storage for Iceberg data files. + +### Create an S3 bucket + +Create a local S3 bucket using the `mb` command with the `awslocal` CLI: + +```bash +awslocal s3 mb s3://test-bucket +``` + +### Create an external volume + +Create an external volume to define the storage location for the Iceberg data files: + +```sql showLineNumbers +CREATE OR REPLACE EXTERNAL VOLUME test_volume + STORAGE_LOCATIONS = ( + ( + NAME = 'aws-s3-test' + STORAGE_PROVIDER = 'S3' + STORAGE_BASE_URL = 's3://test-bucket/' + STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/s3-role' + ) +) +``` + +### Create a source table + +Create a source table that the Dynamic Iceberg Table will refresh from: + +```sql +CREATE TABLE source_table (id INT, name TEXT); +INSERT INTO source_table(id, name) VALUES (1, 'foo'), (2, 'bar'); +``` + +### Create a Dynamic Iceberg Table + +Create a Dynamic Iceberg Table using the `CREATE DYNAMIC ICEBERG TABLE` statement: + +```sql showLineNumbers +CREATE DYNAMIC ICEBERG TABLE my_dynamic_iceberg_table + TARGET_LAG = '2 minutes' + WAREHOUSE = test + REFRESH_MODE = INCREMENTAL + INITIALIZE = on_create + EXTERNAL_VOLUME = 'test_volume' + CATALOG = 'SNOWFLAKE' + BASE_LOCATION = 'my_table_data' +AS SELECT id, name FROM source_table; +``` + +The output should be: + +```sql ++----------------------------------------------------------+ +| result | +|----------------------------------------------------------+ +| Dynamic table MY_DYNAMIC_ICEBERG_TABLE successfully created. | ++----------------------------------------------------------+ +``` + +### Query the Dynamic Iceberg Table + +You can query the Dynamic Iceberg Table using a standard `SELECT` statement: + +```sql +SELECT * FROM my_dynamic_iceberg_table ORDER BY id; +``` + +The output should be: + +```sql ++----+------+ +| ID | NAME | +|----+------+ +| 1 | foo | +| 2 | bar | ++----+------+ +``` + +### Show Dynamic Tables + +You can view the metadata of your Dynamic Iceberg Table using the `SHOW DYNAMIC TABLES` command. The `is_iceberg` column indicates whether the table is a Dynamic Iceberg Table: + +```sql +SHOW DYNAMIC TABLES LIKE 'my_dynamic_iceberg_table'; +``` + +### Rename a Dynamic Iceberg Table + +You can rename a Dynamic Iceberg Table using the `ALTER DYNAMIC TABLE` statement: + +```sql +ALTER DYNAMIC TABLE my_dynamic_iceberg_table RENAME TO my_renamed_table; +``` + +### Drop a Dynamic Iceberg Table + +You can drop a Dynamic Iceberg Table using the `DROP DYNAMIC TABLE` statement: + +```sql +DROP DYNAMIC TABLE my_renamed_table; +``` + +The output should be: + +```sql ++------------------------------------------+ +| status | +|------------------------------------------+ +| MY_RENAMED_TABLE successfully dropped. | ++------------------------------------------+ +```