feat(service-mailbox): store an ingested message's attachments, and each forwarded message as an .eml, through file-attachments

.changeset/23307-mail-attachments.md

@@ -0,0 +1,12 @@
+---
+'@objectstack/service-mailbox': minor
+---
+
+Inbound mail stores its attachments: each attachment of an ingested message, and each message it forwards as an attachment as an `.eml` file, is stored through the `file-attachments` service as a file of the message's `sys_mail_message` row
+
+Clause-②: no
+
+- **What is stored, and where.** Every part of a stored message besides its text and HTML bodies (inline parts included, and each forwarded `message/rfc822` or `message/global` part as the `.eml` file it travels as) becomes one `sys_file` and one `sys_attachment` whose parent is the message's own `sys_mail_message` row, written by `IFileAttachmentService.attachFile` in the mailbox's organization after the row, and only for a message the call stored, so a re-delivered copy adds no attachment.
+- **Without the storage capability.** A kernel that composes no `file-attachments` service (registered by `@objectstack/service-storage`) stores each message without its attachments, and says so once in the log.
+- **Refused attachments.** A part the service refuses, such as one over the `max_upload_mb` upload size limit, is logged at `error` with its file name and the refusal, and the message and its other attachments are still stored.
+- **For a host that constructs `MailIngestService` itself.** It takes the service as the optional `attachments` resolver (`MailboxServicePlugin` supplies it), and its `MailIngestLogger` now needs `error`, as `MailSyncLogger` already does.

packages/services/service-mailbox/README.md

@@ -4,7 +4,7 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record
 
 > ⚠️ **Mail arrives only where an IMAP account source is composed.** The capability boot (`requires: ['mailbox']`) constructs the plugin with no options, so it mounts the objects, their read gates and the ingest seam, and syncs no mailbox — it says so once at boot. Compose the plugin yourself with `imap.accountFor` (below) to poll mailboxes.
 >
-> ⚠️ **Attachments are not stored.** Nothing on the server writes `sys_attachment` rows today, so a message's attachments — and the body of a message forwarded as an attachment — are not kept; only its participants are read.
+> ⚠️ **Attachments are stored only where the storage capability is composed.** A message's attachments are written through the `file-attachments` service that `@objectstack/service-storage` registers (the `storage` capability, on the always-on slate). A kernel that does not compose it stores each message without its attachments, and says so once.
 
 ## What this package provides
 
@@ -18,15 +18,15 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record
 - **`manage_mailboxes`** — an organization capability declared in `@objectstack/spec`. Platform administrators and organization administrators hold it by default; every other position only by explicit grant. Object grants and field-level security still apply to a holder.
 - **The ingest seam** — `MailIngestService`, registered as the kernel service `mailbox` (`MAILBOX_SERVICE`):
   - **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one raw RFC 822 message and writes at most one `sys_mail_message` row, answering `stored`, `duplicate` or `suppressed`:
-    - **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part, or its internationalized twin `message/global`, RFC 6532). At most `MAX_FORWARDED_PARTS` (20) forwarded parts of one message are read; the rest, and any part the MIME parser refuses, take no part in matching, and the message is stored without them (a warning names how many). The mailbox's own address is left out. A forward quoted inline in the body is read too, for an organization user's forward only: the "Forwarded message" block Gmail, Outlook (on the web and Microsoft 365) or Apple Mail writes into the text/plain body, with its English labels, adds its From, Reply-To, To and Cc — when the message's From is a user (`sys_user`) who is a member (`sys_member`) of the mailbox's organization. A block the parser does not recognise is not guessed at, and a forward from anyone else is not read: the message is matched on its other participants. The participants are stored on the row in `match_addresses`, each with its source (`outer`, `attachment` or `inline`);
+    - **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part, or its internationalized twin `message/global`, RFC 6532). At most `MAX_FORWARDED_PARTS` (20) forwarded parts of one message are read; the rest, and any part the MIME parser refuses, take no part in matching (a warning names how many), and are still stored as attachments. The mailbox's own address is left out. A forward quoted inline in the body is read too, for an organization user's forward only: the "Forwarded message" block Gmail, Outlook (on the web and Microsoft 365) or Apple Mail writes into the text/plain body, with its English labels, adds its From, Reply-To, To and Cc — when the message's From is a user (`sys_user`) who is a member (`sys_member`) of the mailbox's organization. A block the parser does not recognise is not guessed at, and a forward from anyone else is not read: the message is matched on its other participants. The participants are stored on the row in `match_addresses`, each with its source (`outer`, `attachment` or `inline`);
     - **association** — every record of every object whose email field declares `mailAssociation: true` and holds one of those addresses (ASCII case-insensitive) is linked to the message, inside the mailbox's organization only, with one `sys_mail_association` row each (`matched_by: 'ingest'`). A message no record matches waits in the pool (`association_state: 'unassociated'`);
     - **dedupe** — one row per message per mailbox, keyed on the Message-ID (or, for a message without one, on its header block as received), so a second fetch answers `duplicate` and writes nothing;
     - **threading** — `message_id`, `in_reply_to` and `reference_ids` hold the ids of RFC 5322 §3.6.4, angle brackets stripped;
     - **suppression** — an auto-reply (`Auto-Submitted` other than `no`, `Precedence: bulk|junk|list`, `X-Autoreply`, `X-Autorespond`) or a bounce (`multipart/report; report-type=delivery-status`) is not stored at all.
 
     Only the system context may ingest: a message enters a mailbox from its mail source, never from a user.
   - **`rematch({ mailboxId, messageIds? }, context)`** replays each message's stored participant set against the records that exist now and adds the links that are missing (`matched_by: 'rematch'`), moving a pool message to `associated` once it has one. It never removes a link. The system context may run it, and so may the mailbox's owner and a `manage_mailboxes` holder of the mailbox's organization; anyone else is refused (`PERMISSION_DENIED`), and another organization's mailbox is not found (`RECORD_NOT_FOUND`).
-  - Attachments are not stored yet, so a message forwarded as an attachment is stored with the forwarder's own text, and the forwarded message's body is not kept.
+  - **attachments** — each part of a stored message besides its text and HTML bodies is stored as a file of its `sys_mail_message` row: one `sys_file` and one `sys_attachment` (parent `sys_mail_message`, the row's id), through `IFileAttachmentService.attachFile`, the kernel service `file-attachments` (`FILE_ATTACHMENT_SERVICE`). Both rows land in the mailbox's organization, and the deployment's upload size limit (`max_upload_mb`) applies. That covers every file the message carries, a part marked `Content-Disposition: inline` included (Apple Mail marks attached files that way, and an image the HTML body shows by `cid:` reference is part of the message), and every message forwarded as an attachment (`message/rfc822` or `message/global`), stored as the `.eml` file it travels as: named by the part's own file name, else by its Subject, else `forwarded.eml`. A part with no file name is `attachment-<n>`, its position among the message's parts. Parts are stored only for a message this call stored, after its row: a duplicate or a suppressed message stores none, so a re-delivery adds no attachment, and a re-match never reads a part. A part the service refuses (over the upload size limit, for instance) is logged at `error` with its name and the refusal, and the message and its other parts are still stored. No column of the row names a part, and no byte is kept in a column;
 - **The IMAP sync** — `MailboxSyncService`, composed with `MailboxServicePlugin({ imap })` and run as the interval job `mailbox:imap-sync` on the composed job service (`requires: ['job']`):
   - for every mailbox your `accountFor` answers, it opens a **read-only** session on its IMAP account through `@objectstack/connector-imap` — the folder is opened with `EXAMINE` and bodies are fetched with `BODY.PEEK[]`, so the mailbox is never altered — and hands each message above the stored cursor, in UID order, to `ingest` under the system context. Dedupe, threading, association, the pool and suppression are the seam's, unchanged;
   - **the cursor** is kept on the mailbox row — `imap_folder`, `imap_uid_validity`, `imap_last_uid`, `imap_synced_at` — and written by the sync only. A cursor taken under another UIDVALIDITY, or in another folder, is discarded and the folder is read from its start; every message already stored then answers `duplicate`;

packages/services/service-mailbox/package.json

@@ -35,6 +35,7 @@
     "@objectstack/driver-sql": "workspace:*",
     "@objectstack/lint": "workspace:*",
     "@objectstack/objectql": "workspace:*",
+    "@objectstack/service-storage": "workspace:*",
     "@types/node": "^26.6.3",
     "typescript": "^6.0.3",
     "vitest": "^4.1.11"