Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions src/content/docs/snowflake/features/dynamic-tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,124 @@ The output should be:
| 2 | bar |
+----+------+
```

## Dynamic Iceberg Tables

Dynamic Iceberg Tables combine the auto-refresh capabilities of Dynamic Tables with the Apache Iceberg open table format. This allows you to create materialized views that automatically update from source queries while storing data in Iceberg format on external object storage (such as S3).

A Dynamic Iceberg Table consists of three key concepts:
- **Dynamic table**: A materialized view that auto-refreshes from a source query on a schedule (controlled by `TARGET_LAG` and `WAREHOUSE`).
- **Iceberg table**: A table stored in Apache Iceberg format on external object storage (configured via `EXTERNAL_VOLUME`, `CATALOG`, and `BASE_LOCATION`).
- **External volume**: A Snowflake object that references an S3 bucket as the backing storage for Iceberg data files.

### Create an S3 bucket

Create a local S3 bucket using the `mb` command with the `awslocal` CLI:

```bash
awslocal s3 mb s3://test-bucket
```

### Create an external volume

Create an external volume to define the storage location for the Iceberg data files:

```sql showLineNumbers
CREATE OR REPLACE EXTERNAL VOLUME test_volume
STORAGE_LOCATIONS = (
(
NAME = 'aws-s3-test'
STORAGE_PROVIDER = 'S3'
STORAGE_BASE_URL = 's3://test-bucket/'
STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/s3-role'
)
)
```

### Create a source table

Create a source table that the Dynamic Iceberg Table will refresh from:

```sql
CREATE TABLE source_table (id INT, name TEXT);
INSERT INTO source_table(id, name) VALUES (1, 'foo'), (2, 'bar');
```

### Create a Dynamic Iceberg Table

Create a Dynamic Iceberg Table using the `CREATE DYNAMIC ICEBERG TABLE` statement:

```sql showLineNumbers
CREATE DYNAMIC ICEBERG TABLE my_dynamic_iceberg_table
TARGET_LAG = '2 minutes'
WAREHOUSE = test
REFRESH_MODE = INCREMENTAL
INITIALIZE = on_create
EXTERNAL_VOLUME = 'test_volume'
CATALOG = 'SNOWFLAKE'
BASE_LOCATION = 'my_table_data'
AS SELECT id, name FROM source_table;
```

The output should be:

```sql
+----------------------------------------------------------+
| result |
|----------------------------------------------------------+
| Dynamic table MY_DYNAMIC_ICEBERG_TABLE successfully created. |
+----------------------------------------------------------+
```

### Query the Dynamic Iceberg Table

You can query the Dynamic Iceberg Table using a standard `SELECT` statement:

```sql
SELECT * FROM my_dynamic_iceberg_table ORDER BY id;
```

The output should be:

```sql
+----+------+
| ID | NAME |
|----+------+
| 1 | foo |
| 2 | bar |
+----+------+
```

### Show Dynamic Tables

You can view the metadata of your Dynamic Iceberg Table using the `SHOW DYNAMIC TABLES` command. The `is_iceberg` column indicates whether the table is a Dynamic Iceberg Table:

```sql
SHOW DYNAMIC TABLES LIKE 'my_dynamic_iceberg_table';
```

### Rename a Dynamic Iceberg Table

You can rename a Dynamic Iceberg Table using the `ALTER DYNAMIC TABLE` statement:

```sql
ALTER DYNAMIC TABLE my_dynamic_iceberg_table RENAME TO my_renamed_table;
```

### Drop a Dynamic Iceberg Table

You can drop a Dynamic Iceberg Table using the `DROP DYNAMIC TABLE` statement:

```sql
DROP DYNAMIC TABLE my_renamed_table;
```

The output should be:

```sql
+------------------------------------------+
| status |
|------------------------------------------+
| MY_RENAMED_TABLE successfully dropped. |
+------------------------------------------+
```