[MarcAntoineGmd]

Closes #17

Overview

The goal of this ticket is to produce a clear diagram and short documentation of the full database schema, ready to present to Nafissa at the end of the sprint.

Diagram: docs/database/schema.dbml describes all 6 tables (roles, users, donors, food_items, receipts, receipt_items) with every column, key, index, check and link, plus the four roles. The tables are grouped into Accounts, Donations and Tax receipts, so the dbdiagram.io export reads by part.

Documentation: docs/database/schema.md is written for non-developers. It covers:

• what each part and table holds;
• how a receipt is valued, with an example;
• the rules the database enforces, in everyday words;
• how dates and time zones work;
• which planned features will add tables later, with their issues.

A "For developers" section at the end lists the migrations, SQL scripts and detailed pages.

The diagram is tested against the migrations. New tests build an empty PostgreSQL 17 database from the migrations, read its real structure and compare it with every .dbml file in docs/database/. The comparison covers:

• tables, columns, types with their exact precision, nullability and identity;
• defaults and check expressions, compared after PostgreSQL rewrites the diagram's version in its own spelling;
• keys and unique indexes, including the condition of a partial index and the expression of an expression index, which a diagram cannot show and are therefore reported;
• foreign keys with their delete rules;
• the roles data.

Any difference fails the test with a readable message, so the diagram cannot drift from the migrations unnoticed.

Changes:

• docs/database/schema.dbml, docs/database/schema.md, docs/database/images/schema.png
• Tests in tests/MoissonMontreal.Tests/Docs/:
  • SchemaDiagramTests: every diagram against the migrated database
  • SchemaDriftTests: 18 kinds of drift must be reported, the reader rejects DBML it cannot check, and Windows line endings give the same result
  • DatabaseDocsTests:
    • every link and image in the docs exists;
    • every table has a section in schema.md;
    • no em dash;
    • no SQL script starts with a BOM;
    • one SQL script per migration, in order;
    • every migration name in the docs exists.

Review feedback from #14 and #16, applied to the older tables:

• UTC on every date and time: UtcDateTimeOffsetConverter is now registered once in MoissonDbContext for every DateTimeOffset column. This covers users.created_at, users.updated_at, donors.created_at and food_items.created_at. Before, writing DateTimeOffset.Now to users.updated_at threw. The per-property conversion added to ReceiptConfiguration in [Issue]: Database schema: receipts with price #16 is replaced by this convention. No migration is needed.
• BOM: removed from 0001_create_users_and_roles.sql, like 0002 and 0003 in [Issue]: Database schema: receipts with price #16.

Review feedback addressed

• CRLF on Windows: the tests now normalize line endings when they read schema.dbml and schema.md. The full suite also passes on a copy of the repo where every .md, .dbml, .sql and .cs file was converted to CRLF.
• Check expressions: checks are now compared by their expression, not just their name. Defaults are compared the same way.
• Partial and expression indexes: the reader now reads indpred, and uses pg_get_indexdef for every key column, so expression columns are no longer dropped. A partial or expression index that the diagram cannot show is reported as a difference. Integration tests create both kinds in a rolled-back transaction.
• Lossy normalization: removed. PostgreSQL rewrites the diagram's defaults and checks itself: they are added to empty temporary copies of the real tables, and read back with pg_get_expr and pg_get_constraintdef. Both sides are then in PostgreSQL's own spelling and are compared exactly. A test checks that (total_amount + 1) * 2 and total_amount + 1 * 2 stay different.
• Intent of the records test: a MigrateAsync() helper now runs the migrations, and ReadDatabaseAsync() uses it.

Testing

Unit and integration tests: dotnet test (Docker must be running). All 141 tests pass.

• All issue 17 tests (82): dotnet test --filter "FullyQualifiedName~MoissonMontreal.Tests.Docs|FullyQualifiedName~UtcDateTimeTests"
• Only the tests added for this review (11): dotnet test --filter "FullyQualifiedName~Diagram_ReportsADifference_WhenAnExpressionDiffers|FullyQualifiedName~Diagram_KeepsParenthesesThatChangeTheMeaning|FullyQualifiedName~Diagram_ReportsAPartialOrExpressionIndex|FullyQualifiedName~Read_GivesTheSameSchema_WithWindowsLineEndings|FullyQualifiedName~Read_KeepsTheDefaultAndCheckExactlyAsWritten"

What the tests check:

• schema.dbml, users-roles.dbml and receipts.dbml all match the migrated database exactly, including every check expression and default.
• Against the real database, a changed check expression (IN ('A', 'B'), >= 0, BETWEEN 0 AND 75), an invalid one, a changed time zone default and a changed boolean default are each reported.
• A partial unique index (WHERE quantity > 0) and an expression index (lower(email)) are each reported.
• schema.dbml lists every table and every foreign key of the database.
• Breaking the diagram on purpose is reported in the comparison. The drift tests cover:
  • wrong precision, wrong length;
  • missing or extra column, wrong nullability;
  • wrong default, wrong time zone default, missing identity;
  • renamed index, unique index made non-unique, index columns in another order;
  • missing check, wrong check expression;
  • wrong delete rule, donor left out of a composite link, missing link, renamed table.
• UtcDateTimeTests:
  • users, donors and food items saved with -05:00, -12:00 and +14:00 offsets are read back as the same moment in UTC;
  • an empty updated_at stays empty;
  • a guard fails if any future DateTimeOffset column misses the conversion.
• With the convention switched off on purpose, 7 of those tests fail.

Screenshots / Screencasts

No frontend changes. Full schema diagram from dbdiagram.io:

Checklist

• Code is neat, readable, and works
• Code is commented where appropriate and well-documented
• Commit messages follow our guidelines
• Issue number is linked
• Branch is linked
• Reviewers are assigned (one of your tech leads)

Notes

• The docs are in English, like the rest of docs/.
• docs/database/ still has no separate page for donors and food items. schema.md describes both tables, and the "For developers" table points there.

[lauriechammah]

See requested changes

[lauriechammah]

Perfect!