Unity-Technologies · UnityZappy · Jun 3, 2026 · Jun 3, 2026
@@ -0,0 +1,104 @@
+---
+name: memory-db-sql
+description: Write and manage SQL queries against an exported Unity memory-snapshot database (DuckDB/SQLite), and keep the schema, views, version, and docs consistent. Use when writing/editing queries over native_objects, native_allocations, native_roots, memory_regions, system_memory_regions, etc.; investigating memory regions; adding or changing tables/views/macros; or checking database version compatibility.
+---
+
+# Memory snapshot database SQL
+
+Use this when working with the database the tool exports from a `.snap` (the `export` command), as
+opposed to running the export/report pipeline itself (use `memory-snapshot-report` for that).
+
+**Canonical schema reference:** [`docs/database-schema.md`](../../../docs/database-schema.md). It
+lists every table, column, view, macro, join key, and the version policy. Read it before writing
+non-trivial queries — this skill is the workflow; that doc is the source of truth.
+
+## Before you query: check the schema version
+
+The database stamps a **major.minor** version in `schema_meta`:
+
+```sql
+SELECT schema_version_major, schema_version_minor, msdt_version FROM schema_meta;
+```
+
+- **No `schema_meta` table** → a pre-versioning export (treated as 0.0): re-export the `.snap`.
+- **major < `DatabaseSchemaInfo.SchemaMajor`** → table/column structure changed; **re-export required**
+  (`MemorySnapshotDataTools export "<snap>" "<db>"`).
+- **same major, minor < `DatabaseSchemaInfo.SchemaMinor`** → only views/indexes changed; **upgrade in
+  place** with `MemorySnapshotDataTools upgrade "<db>"` (no re-export).
+- In code, classify with `DatabaseSchemaInfo.Evaluate(major, minor)` → `SchemaAction`
+  (`None`/`UpgradeInPlace`/`ReExport`/`ToolOutdated`); the CLI `SchemaGate` already does this before
+  `report`/`summary`/`validate`, and `DatabaseMaintenance.{Inspect,UpgradeInPlace}` are the entry
+  points (see [`Core/Models/DatabaseSchemaInfo.cs`](../../../Core/Models/DatabaseSchemaInfo.cs)).
+
+## Querying: the things that bite
+
+These are spelled out in the schema doc, but the high-frequency traps:
+
+- **Two region tables, no FK between them.** `memory_regions` = Unity allocator buckets
+  (`native_allocations.memory_region_index` points here). `system_memory_regions` = OS/VM regions
+  (the RAM truth). Bridge an allocation to its OS region **by address range**, not a key.
+- **Prefer the views/macros** over rebuilding joins: `v_allocation_enriched` (allocation + Unity
+  region + OS region + root + object), `v_system_region_summary` (committed/resident/Unity-tracked
+  per OS region), `v_region_owner_breakdown`, `v_connection_edges` (reference graph with both
+  endpoints typed — filter it, don't `SELECT *`), `v_assetbundle_utilization` (per-bundle: does it
+  reference other loaded assets, and how much). DuckDB also has `region_allocations(name)` and
+  `region_page_density(name)` (SQLite has the views only).
+- **Connections: don't hand-count "own" edges with magic numbers.** A native object's outbound edges
+  include its self-reference and its managed wrappers (`native_gc_handle_bridge`/`native_connection`
+  to managed). For "references to *other* loaded objects," count `native_object→native_object`
+  `native_connection` edges where `to_index <> from_index` — exactly what `v_assetbundle_utilization`
+  does.
+- **`native_object_address` ≠ `native_allocations.address`** — bridge objects and allocations
+  through a shared `root_reference_id` → `native_roots.root_id`, never by address.
+- **Don't divide by `memory_regions.address_size`** (0 for `ALLOC_DEFAULT` et al.).
+- **`system_memory_regions.type` is uniformly 0 on iOS** — group by `name`.
+- **Resident data and `page_size` require `snap_format_version` ≥ 17** (else NULL).
+- `region_page_density` is for **small-allocation zones** (NANO/TINY/SMALL); `avg_fill_pct` > 100%
+  means the region holds page-spanning allocations and the metric doesn't apply.
+
+## Writing query code safely
+
+This repo's first-class rule is SQL safety — see [`CLAUDE.md`](../../../CLAUDE.md) and
+[`docs/sql-safety.md`](../../../docs/sql-safety.md):
+
+- **Parameterize every value.** DuckDB: positional `?`. SQLite: named `$name`.
+- **Identifiers can't be parameters** — validate against the catalog (`information_schema.columns`
+  for DuckDB, `pragma_table_info($t)` for SQLite); see the `HasColumn` helpers.
+- **Open read-only** for analysis (`ACCESS_MODE=READ_ONLY` / `Mode=ReadOnly`).
+- The report `ExecuteQuery(string)` sink takes only internally-constructed SQL (constants in
+  `ReportSql`); never pass it external input.
+
+Ad-hoc CLI querying (no SQL command exists in the tool): use the `duckdb` CLI on a `.duckdb` file
+(open with `-readonly`; if a lock error mentions DataGrip, ask the user to close that connection).
+
+## Upkeep: when you change the schema
+
+A schema change is not done until **all** of these are consistent:
+
+1. **Both backends** — mirror the change in
+   [`DuckDbExportDestination.cs`](../../../Core/ExportDestination/DuckDbExportDestination.cs) **and**
+   [`SqliteWriter.cs`](../../../Core/ExportDestination/SqliteWriter.cs) (tables, indexes, and the
+   `CreateViewsScript`). Remember: DuckDB has `ASOF` joins and table macros; SQLite does not (use a
+   correlated subquery; macros are DuckDB-only). Keep index/view DDL **re-runnable** (`CREATE INDEX
+   IF NOT EXISTS`, DuckDB `CREATE OR REPLACE VIEW` / SQLite drop-then-create) so `UpgradeSchema` works.
+2. **The doc** — update [`docs/database-schema.md`](../../../docs/database-schema.md): table/column
+   tables, view/macro list, join keys, and the version table. New tables, views, columns, and
+   identifiers must appear here.
+3. **The version** — in
+   [`Core/Models/DatabaseSchemaInfo.cs`](../../../Core/Models/DatabaseSchemaInfo.cs):
+   - **View/index change only** (add/change a view or index): bump **`SchemaMinor`**. Existing
+     databases can be upgraded in place (`MemorySnapshotDataTools upgrade`), so no re-export.
+   - **Table/column change** (add/rename/remove a table or column, or change a column's
+     meaning/units): bump **`SchemaMajor`** and reset `SchemaMinor` to 0. Old databases require a
+     re-export. Make sure both writers' table DDL and `schema_meta` insert stay in sync.
+   - Add a row to the doc's version table either way.
+4. **Readers** — `snapshot_info` is read with `SELECT *` by column name
+   (`SummaryReportRunner.ReadSnapshotInfo`), so new columns are safe; if you add a column some
+   reader needs, wire it there.
+5. **Tests** — extend `DatabaseSchemaInfoTests` and the export round-trip tests so the new
+   schema/view/version is asserted.
+
+## See also
+
+- `memory-snapshot-report` skill — export a `.snap` to a database and generate reports.
+- [`docs/database-schema.md`](../../../docs/database-schema.md) — canonical schema.
@@ -85,9 +85,13 @@ dotnet run --project Cli/MemorySnapshotDataTools.Cli.csproj -- report <path/to/o
 ### 4. Optional
 
 - Open the generated HTML file or DB in the user’s preferred viewer.
-- For ad-hoc SQL, use the same DB path; tables are `snapshot_info`, `native_objects`,
+- For ad-hoc SQL, use the same DB path; tables are `schema_meta`, `snapshot_info`, `native_objects`,
   `managed_objects`, `connections`, `native_roots`, `memory_regions`, `native_allocations`,
-  `system_memory_regions`, and `summary_metrics` (MemoryProfiler Summary-page breakdown).
+  `system_memory_regions`, and `summary_metrics` (MemoryProfiler Summary-page breakdown). Analysis
+  views (`v_allocation_enriched`, `v_system_region_summary`, `v_region_owner_breakdown`) and DuckDB
+  macros (`region_allocations`, `region_page_density`) simplify native-memory/region queries. For
+  schema details, join keys, and version compatibility, use the **`memory-db-sql`** skill and
+  [`docs/database-schema.md`](../../../docs/database-schema.md).
 
 ## Domain
 

@@ -1,4 +1,4 @@
-#Validation script for cse-memory-snapshot-data-tool
+#Validation script for MemorySnapshotDataTools
 #This is used to validate your catalog-info.yaml 
 
 name: Validate Catalog-info.yaml

@@ -10,6 +10,7 @@ internal enum CommandKind
     MultiReport,
     ValidateGolden,
     Summary,
+    Upgrade,
 }
 
 /// <summary>
@@ -32,6 +33,7 @@ internal sealed class CliOptions
     public string GoldenPath { get; set; } = string.Empty;
     public string? ValidationOutputPath { get; set; }
     public string SummaryInputPath { get; set; } = string.Empty;
+    public string UpgradeDbPath { get; set; } = string.Empty;
     public int BatchSize { get; set; } = 2048;
     public int QueueCapacity { get; set; } = 256;
     public ValidationMode Validate { get; set; } = ValidationMode.Minimal;

@@ -14,7 +14,8 @@ public static RootCommand Build(
         Func<CliOptions, int> runReport,
         Func<CliOptions, int> runMultiReport,
         Func<CliOptions, int> runValidateGolden,
-        Func<CliOptions, int> runSummary)
+        Func<CliOptions, int> runSummary,
+        Func<CliOptions, int> runUpgrade)
     {
         var root = new RootCommand("Export Unity memory snapshots to DuckDB or SQLite and generate HTML reports.");
 
@@ -342,12 +343,40 @@ public static RootCommand Build(
             return runSummary(options);
         });
 
+        // ---- upgrade ----
+        var upgradeCmd = new Command(
+            "upgrade",
+            "Upgrade an exported database's analysis views/indexes to the current minor schema version (in place; no re-export).");
+        var upgradeDatabaseArg = new Argument<string>("database")
+        {
+            Description = "Path to the exported database (.duckdb or .db) to upgrade.",
+            Arity = ArgumentArity.ExactlyOne,
+        };
+        upgradeCmd.Add(upgradeDatabaseArg);
+
+        upgradeCmd.SetAction((ParseResult parseResult) =>
+        {
+            var dbPath = ExpandPath(parseResult.GetValue(upgradeDatabaseArg)!);
+            if (!File.Exists(dbPath))
+            {
+                Console.Error.WriteLine($"Database file not found: {dbPath}");
+                return 1;
+            }
+            var options = new CliOptions
+            {
+                Command = CommandKind.Upgrade,
+                UpgradeDbPath = dbPath,
+            };
+            return runUpgrade(options);
+        });
+
         root.Add(exportCmd);
         root.Add(batchExportCmd);
         root.Add(reportCmd);
         root.Add(multiReportCmd);
         root.Add(validateCmd);
         root.Add(summaryCmd);
+        root.Add(upgradeCmd);
         return root;
     }
 

@@ -1,5 +1,6 @@
 using MemorySnapshotDataTools;
 using MemorySnapshotDataTools.Export;
+using MemorySnapshotDataTools.ExportDestination;
 using MemorySnapshotDataTools.Report;
 using MemorySnapshotDataTools.Report.MultiSnapshotReport;
 using MemorySnapshotDataTools.Validation;
@@ -10,7 +11,7 @@ internal static class Program
 {
     private static int Main(string[] args)
     {
-        var root = CommandLineBuilder.Build(RunExport, RunBatchExport, RunReport, RunMultiReport, RunValidateGolden, RunSummary);
+        var root = CommandLineBuilder.Build(RunExport, RunBatchExport, RunReport, RunMultiReport, RunValidateGolden, RunSummary, RunUpgrade);
         return root.Parse(args).Invoke();
     }
 
@@ -72,6 +73,7 @@ private static int RunBatchExport(CliOptions options)
 
     private static int RunReport(CliOptions options)
     {
+        SchemaGate.Check(options.ReportDbPath);
         var reportOptions = new ReportRunOptions
         {
             ReportDbPath = options.ReportDbPath,
@@ -97,6 +99,7 @@ private static int RunMultiReport(CliOptions options)
 
     private static int RunValidateGolden(CliOptions options)
     {
+        SchemaGate.Check(options.ReportDbPath);
         try
         {
             return GoldenValidationRunner.ValidateAndWriteResult(
@@ -114,6 +117,8 @@ private static int RunValidateGolden(CliOptions options)
 
     private static int RunSummary(CliOptions options)
     {
+        // Summary accepts either a .snap or an exported database; only databases have a schema to check.
+        SchemaGate.Check(options.SummaryInputPath);
         var progress = new ConsoleProgress(options.Verbose);
         using var cts = CreateCancellationSource();
 
@@ -135,6 +140,48 @@ private static int RunSummary(CliOptions options)
         }
     }
 
+    private static int RunUpgrade(CliOptions options)
+    {
+        try
+        {
+            var before = DatabaseMaintenance.Inspect(options.UpgradeDbPath);
+            var current = $"v{DatabaseSchemaInfo.SchemaMajor}.{DatabaseSchemaInfo.SchemaMinor}";
+
+            switch (before.Action)
+            {
+                case SchemaAction.None:
+                    Console.WriteLine($"Database is already at the current schema {current}. Nothing to do.");
+                    return 0;
+
+                case SchemaAction.ToolOutdated:
+                    Console.Error.WriteLine(
+                        $"Database schema v{before.Major}.{before.Minor} is newer than this build ({current}). " +
+                        $"Update {DatabaseSchemaInfo.ToolName} instead of downgrading.");
+                    return 1;
+
+                case SchemaAction.ReExport:
+                    Console.Error.WriteLine(
+                        $"Database major version (v{before.Major}) is behind v{DatabaseSchemaInfo.SchemaMajor}; an in-place upgrade is not possible. " +
+                        "Re-export from the original snapshot:");
+                    Console.Error.WriteLine($"  {before.ReExportCommand ?? $"{DatabaseSchemaInfo.ToolName} export <snapshot.snap> \"{options.UpgradeDbPath}\""}");
+                    return 1;
+
+                case SchemaAction.UpgradeInPlace:
+                    DatabaseMaintenance.UpgradeInPlace(options.UpgradeDbPath);
+                    Console.WriteLine($"Upgraded database schema from v{before.Major}.{before.Minor} to {current}.");
+                    return 0;
+            }
+
+            return 0;
+        }
+        catch (Exception ex)
+        {
+            Console.Error.WriteLine("Schema upgrade failed.");
+            Console.Error.WriteLine(ex.Message);
+            return 1;
+        }
+    }
+
     private static CancellationTokenSource CreateCancellationSource()
     {
         var cts = new CancellationTokenSource();