From 9bfd20189df2976a37668e7fe9ec58cb11b2eada Mon Sep 17 00:00:00 2001 From: Koen van der Veen Date: Fri, 18 Sep 2026 17:14:07 +0200 Subject: [PATCH 1/2] feat: a double-blind eval demo on a Tinfoil enclave Two Colab notebooks, one per party, that rebuild the tinfoilsh/double-blind-eval flow over PySyft with no dbe involved. A benchmark owner and a model owner each upload half of an evaluation, both attest the enclave against a pinned image digest, both approve the job, and only the benchmark owner reads the results. The benchmark owner submits the job rather than the model owner, because distribute_results always forwards output to the submitter and there is no per-recipient output policy. The model owner approves the run and its copy of the job never reaches done. The prompt set is fetched from OpenMined/double-blind-eval-bench. The model is a PEFT LoRA adapter fetched from HuggingFace; the enclave has no GPU, so the demo runs TinyLlama-1.1B and carries a commented line for the gemma-4-31B adapter the dbe demo uses. SETUP.md covers the operator side: resetting each party's state before deploying, deploying the published v0.1.14 release, and republishing after a config or image change. Co-Authored-By: Claude Opus 5 (1M context) --- .gitignore | 4 + .../1. DO-benchmark-owner-dbe.ipynb | 633 ++++++++++++++++++ .../2. DO-model-owner-dbe.ipynb | 461 +++++++++++++ notebooks/enclave/double-blind-eval/SETUP.md | 130 ++++ 4 files changed, 1228 insertions(+) create mode 100644 notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb create mode 100644 notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb create mode 100644 notebooks/enclave/double-blind-eval/SETUP.md diff --git a/.gitignore b/.gitignore index 778a663445f..9546495436a 100644 --- a/.gitignore +++ b/.gitignore @@ -196,6 +196,10 @@ test_suite_output.log *ailuminate_prompts.csv *safety_prompts*.csv !notebooks/enclave/gemma/data/safety_prompts*.csv +# double-blind-eval notebooks: prompts come from OpenMined/double-blind-eval-bench +# and the adapter from HuggingFace, so nothing canonical is checked in here. +*dbe_prompts*.csv +dbe-adapter/ notebooks/e2e/sales_mock.csv notebooks/e2e/sales_private.csv notebooks/e2e/readme.mdpackages/syft-bg/docs/sync-service.md diff --git a/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb b/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb new file mode 100644 index 00000000000..1a2935fd31b --- /dev/null +++ b/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb @@ -0,0 +1,633 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Double-blind evaluation — Benchmark Owner\n", + "\n", + "| Actor | Email | Role |\n", + "|-------|-------|------|\n", + "| **Enclave** | `ENCLAVE_EMAIL` | Tinfoil Container that runs the evaluation |\n", + "| **Benchmark owner** | `BENCHMARK_OWNER_EMAIL` | Owns the private prompt set |\n", + "| **Model owner** | `MODEL_OWNER_EMAIL` | Owns the private LoRA adapter |\n", + "\n", + "**The setting.** A benchmark owner wants to know how a model behaves on prompts it has never\n", + "seen. A model owner is willing to be measured but will not hand over the weights. Neither will show\n", + "the other what they hold, and neither trusts the machine in the middle.\n", + "\n", + "Both connect to an [enclave](https://github.com/OpenMined/PySyft/tree/dev/packages/syft-enclave) — a\n", + "sealed environment whose contents nobody, including whoever runs the hardware, can read. Each\n", + "uploads their half. Nothing runs until **both** owners approve the exact code, and the enclave\n", + "proves what it is before either of them uploads anything.\n", + "\n", + "This is the flow from [tinfoilsh/double-blind-eval](https://github.com/tinfoilsh/double-blind-eval),\n", + "carried out entirely over PySyft. The enclave is a [Tinfoil\n", + "Container](https://docs.tinfoil.sh/containers/overview), and the attestation is checked against the\n", + "measurement published for a signed release of its config.\n", + "\n", + "We hold the prompts, and we submit the job. The model owner has to approve it before it runs, and\n", + "the results come back only to us." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!uv pip install -Uq \"syft-enclave[tinfoil] @ git+https://github.com/OpenMined/PySyft.git@dev#subdirectory=packages/syft-enclave\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import csv\n", + "import json\n", + "import random\n", + "import tempfile\n", + "from pathlib import Path\n", + "\n", + "from syft_enclaves import login_do\n", + "from syft_enclaves.attestation.tinfoil import TinfoilAppraisalPolicy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ENCLAVE_EMAIL = \"enclave@openmined.org\"\n", + "BENCHMARK_OWNER_EMAIL = \"bench@openmined.org\"\n", + "MODEL_OWNER_EMAIL = \"model@openmined.org\"\n", + "\n", + "# What the enclave is checked against. TINFOIL_TAG and IMAGE_DIGEST come from\n", + "# whoever deployed it: `just tinfoil-release` prints the digest, and the tag is\n", + "# the config release it was deployed from.\n", + "TINFOIL_REPO = \"OpenMined/syft-enclave-tinfoil\"\n", + "TINFOIL_TAG = \"v0.1.14\"\n", + "IMAGE_DIGEST = \"sha256:d0bd57f22af80b9dcd0dc151fb68d89cca65b65fcbbd1d2e4586cdfa9d7daebc\"\n", + "\n", + "print(f\" Enclave: {ENCLAVE_EMAIL}\")\n", + "print(f\" Benchmark owner: {BENCHMARK_OWNER_EMAIL} | Model owner: {MODEL_OWNER_EMAIL}\")\n", + "print(f\" Verifying against {TINFOIL_REPO} {TINFOIL_TAG}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 0 — Log in as Benchmark Owner\n", + "\n", + "Colab authenticates us to Google Drive, which is how every party in this demo reaches every other\n", + "one. Nothing here talks to the enclave yet." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "CRYPTO_KEYS_PATH = None\n", + "# NOTE: Re-storing Private Key from local_file, if you have it, uncomment the following line and upload the file.\n", + "# from google.colab import files; files.upload()\n", + "# CRYPTO_KEYS_PATH = \"crypto_keys.json\"\n", + "\n", + "benchmark_owner = login_do(encryption=True, crypto_keys_path=CRYPTO_KEYS_PATH)\n", + "print(f\" Benchmark owner : {benchmark_owner.email}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Optionally, to start from a clean SyftBox\n", + "# benchmark_owner.delete_syftbox()\n", + "# benchmark_owner._rds.peer_manager.write_own_version()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 1 — Connect to the enclave and to the model owner\n", + "\n", + "We ask the enclave to peer with us, which is what lets us send it data and jobs. We also peer with\n", + "the model owner directly, so we can read their public model card before deciding to run against\n", + "their model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "benchmark_owner.add_peer(ENCLAVE_EMAIL)\n", + "benchmark_owner.add_peer(MODEL_OWNER_EMAIL)\n", + "print(\" Peer requests sent to the enclave and the model owner\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 1.1 — Wait until both accept\n", + "\n", + "The enclave accepts automatically; the model owner approves us from their notebook. Re-run the cell\n", + "below until both show as accepted." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "benchmark_owner.peers" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Wait" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 2 — Attest the enclave\n", + "\n", + "Before uploading anything, we check what we are uploading it to. The enclave holds a key that never\n", + "leaves the hardware, and it serves a report signed under that key over a connection pinned to it. We\n", + "compare that report against the measurement published for `TINFOIL_TAG` — a signed GitHub release of\n", + "the enclave's config — and against the facts we expect: the container image digest, which datasite\n", + "the enclave runs as, and which two owners have to approve a job.\n", + "\n", + "Every one of those has to be pinned. Unpinned, the attestation would prove a genuine enclave booted\n", + "a signed config, but not that the config was the one we reviewed. Tinfoil is not in the trust path,\n", + "and neither are we to each other." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "policy = TinfoilAppraisalPolicy(\n", + " repo=TINFOIL_REPO,\n", + " release_tag=TINFOIL_TAG,\n", + " expected_image_digest=IMAGE_DIGEST,\n", + " expected_data_owners=[BENCHMARK_OWNER_EMAIL, MODEL_OWNER_EMAIL],\n", + " expected_email=ENCLAVE_EMAIL,\n", + " # The measured config pins no SYFT_VERSION, so there is nothing to compare.\n", + " expected_syft_version=None,\n", + ")\n", + "\n", + "result = benchmark_owner.attest_peer(ENCLAVE_EMAIL, policy=policy)\n", + "\n", + "if result is None:\n", + " print(\" 🟠 The enclave published no attestation yet. Wait a moment and re-run this cell.\")\n", + "else:\n", + " result.print_checklist()\n", + " print()\n", + " print(f\" Bound to the report: {result.verified_key_bundle is not None}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3 — Upload the benchmark\n", + "\n", + "We download the prompt set and upload it into the enclave. The **mock** half is public — five\n", + "prompts the model owner may read, so they can see the shape of what they are being measured on. The\n", + "**private** half is five different prompts, shared only with the enclave.\n", + "\n", + "Uploading grants nothing on its own. The prompts are used once, by a job both of us approve, and the\n", + "model owner never sees them.\n", + "\n", + "> **Provenance.** Both files are a deterministic sample of the [MLCommons\n", + "> AILuminate](https://github.com/mlcommons/ailuminate) demo prompt set — the first prompt of each\n", + "> hazard, split in half — with columns renamed to match the real AILuminate *reserve* prompt set.\n", + "> They live in\n", + "> [OpenMined/double-blind-eval-bench](https://github.com/OpenMined/double-blind-eval-bench); that\n", + "> repo's README has the details. Any CSV with a `prompt_text` column works, so replace the download\n", + "> with your own if you have one.\n", + "\n", + "> **Quoting.** The real AILuminate *reserve* set leaves a field containing a comma unquoted, which\n", + "> `csv` splits across columns — silently truncating that prompt. The files above are well-formed, so\n", + "> `read_prompt_csv` only checks; if you swap in your own CSV, that check is what catches it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "BENCH_URL = \"https://raw.githubusercontent.com/OpenMined/double-blind-eval-bench/main/bench\"\n", + "MOCK_CSV = \"dbe_prompts_mock.csv\"\n", + "PRIVATE_CSV = \"dbe_prompts.csv\"\n", + "\n", + "# Column names/order of the real AILuminate reserve prompt set (see the note above).\n", + "EXPECTED_COLUMNS = [\"prompt_uid\", \"hazard\", \"locale\", \"prompt_text\"]\n", + "\n", + "!mkdir -p prompts/mock prompts/private\n", + "\n", + "\n", + "def read_prompt_csv(path: Path) -> list[dict]:\n", + " \"\"\"Read a prompt CSV, checking it has the columns the job expects.\"\"\"\n", + " with open(path, newline=\"\", encoding=\"utf-8-sig\") as f:\n", + " reader = csv.DictReader(f)\n", + " assert reader.fieldnames == EXPECTED_COLUMNS, reader.fieldnames\n", + " rows = list(reader)\n", + " # A field containing an unquoted comma would spill into extra columns and\n", + " # silently truncate the prompt, so fail here rather than upload that.\n", + " for i, row in enumerate(rows, start=2):\n", + " missing = [c for c, v in row.items() if not v]\n", + " assert not missing, f\"line {i}: empty or missing columns {missing}\"\n", + " assert None not in row, f\"line {i}: extra columns — is a field unquoted?\"\n", + " return rows" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Download the mock (public) benchmark\n", + "!curl -sfL \"{BENCH_URL}/{MOCK_CSV}\" -o \"prompts/mock/{MOCK_CSV}\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Download the private benchmark\n", + "# NOTE: to use your own prompts, comment out this line and upload your CSV to\n", + "# prompts/private/ under the name `dbe_prompts.csv`.\n", + "!curl -sfL \"{BENCH_URL}/{PRIVATE_CSV}\" -o \"prompts/private/{PRIVATE_CSV}\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "prompt_mock = Path(\"prompts/mock\") / MOCK_CSV\n", + "prompt_private = Path(\"prompts/private\") / PRIVATE_CSV\n", + "\n", + "mock_rows = read_prompt_csv(prompt_mock)\n", + "private_rows = read_prompt_csv(prompt_private)\n", + "\n", + "print(f\"Mock prompts : {len(mock_rows)}\")\n", + "print(f\"Private prompts : {len(private_rows)}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "benchmark_owner.create_dataset(\n", + " name=\"dbe_prompts\",\n", + " mock_path=prompt_mock,\n", + " private_path=prompt_private,\n", + " summary=\"MLCommons AILuminate safety evaluation prompts — one per hazard\",\n", + " users=[MODEL_OWNER_EMAIL, ENCLAVE_EMAIL],\n", + ")\n", + "print(f\" Created 'dbe_prompts' ({len(private_rows)} private prompts)\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Send the private half to the enclave, and only to the enclave.\n", + "benchmark_owner.share_private_dataset(\"dbe_prompts\", ENCLAVE_EMAIL)\n", + "print(\" Private benchmark shared with the enclave\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Wait" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 4 — Read the model owner's model card\n", + "\n", + "What we can see of their model: a card naming the base model, the rank and the size. Not the\n", + "weights. This is the mirror of what they can see of our benchmark — the mock prompts and nothing\n", + "more." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model_dataset = benchmark_owner.datasets.get(\"dbe_adapter\", datasite=MODEL_OWNER_EMAIL)\n", + "print(model_dataset.mock_files[0].read_text())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 5 — Submit the evaluation job\n", + "\n", + "The job runs the model owner's adapter on our prompts, inside the enclave, and writes one row per\n", + "prompt: the completion, time to first token, and decode tokens per second.\n", + "\n", + "`datasets` names whose data the job reads. `share_results_with_do=False` is what makes this\n", + "double-blind in our favour — results go to whoever submitted the job, which is us. The model owner\n", + "approves the run and receives nothing.\n", + "\n", + "Both parties read the same code before approving, so it is written to be read: the two\n", + "`resolve_dataset_*` calls are the only places it touches private data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Keep this modest: a job is killed at 600 seconds, and the enclave has 2 CPUs\n", + "# and no GPU, so the base model runs in software.\n", + "MAX_NEW_TOKENS = 32\n", + "\n", + "JOB_CODE = f'''\n", + "import csv\n", + "import json\n", + "import os\n", + "import time\n", + "\n", + "import syft as sy\n", + "import torch\n", + "from peft import PeftModel\n", + "from transformers import AutoModelForCausalLM, AutoTokenizer\n", + "\n", + "# The model owner's private adapter. Inside the enclave this resolves to the real\n", + "# weights; anywhere else it would resolve to their public model card.\n", + "adapter_files = sy.resolve_dataset_files_path(\n", + " \"dbe_adapter\", owner_email=\"{MODEL_OWNER_EMAIL}\"\n", + ")\n", + "adapter_dir = str(adapter_files[0].parent)\n", + "config_path = [p for p in adapter_files if p.name == \"adapter_config.json\"][0]\n", + "adapter_config = json.loads(config_path.read_text())\n", + "base_model_name = adapter_config[\"base_model_name_or_path\"]\n", + "print(f\"Loading {{base_model_name}} with a rank-{{adapter_config['r']}} adapter...\")\n", + "\n", + "tokenizer = AutoTokenizer.from_pretrained(base_model_name)\n", + "base_model = AutoModelForCausalLM.from_pretrained(base_model_name, dtype=torch.float32)\n", + "model = PeftModel.from_pretrained(base_model, adapter_dir)\n", + "model.eval()\n", + "print(\"Model loaded\")\n", + "\n", + "# Our private benchmark.\n", + "prompt_path = sy.resolve_dataset_file_path(\n", + " \"dbe_prompts\", owner_email=\"{benchmark_owner.email}\"\n", + ")\n", + "with open(prompt_path, newline=\"\") as f:\n", + " prompt_rows = list(csv.DictReader(f))\n", + "print(f\"Loaded {{len(prompt_rows)}} evaluation prompts\")\n", + "\n", + "\n", + "def generate(prompt):\n", + " \"\"\"One completion, with the two timings a benchmark reports.\"\"\"\n", + " chat = tokenizer.apply_chat_template(\n", + " [{{\"role\": \"user\", \"content\": prompt}}], tokenize=False, add_generation_prompt=True\n", + " )\n", + " inputs = tokenizer(chat, return_tensors=\"pt\")\n", + " prompt_len = inputs[\"input_ids\"].shape[-1]\n", + "\n", + " start = time.time()\n", + " with torch.no_grad():\n", + " model.generate(**inputs, max_new_tokens=1, do_sample=False,\n", + " pad_token_id=tokenizer.eos_token_id)\n", + " ttft = time.time() - start\n", + "\n", + " start = time.time()\n", + " with torch.no_grad():\n", + " out = model.generate(**inputs, max_new_tokens={MAX_NEW_TOKENS}, do_sample=True,\n", + " temperature=0.8, top_k=40,\n", + " pad_token_id=tokenizer.eos_token_id)\n", + " elapsed = time.time() - start\n", + " completion = tokenizer.decode(out[0][prompt_len:], skip_special_tokens=True)\n", + " return completion, ttft, (out.shape[-1] - prompt_len) / elapsed\n", + "\n", + "\n", + "results = []\n", + "for i, row in enumerate(prompt_rows):\n", + " print(f\" [{{i + 1}}/{{len(prompt_rows)}}] {{row['prompt_uid']}}\")\n", + " completion, ttft, decode_tps = generate(row[\"prompt_text\"])\n", + " results.append({{\n", + " \"prompt_uid\": row[\"prompt_uid\"],\n", + " \"hazard\": row[\"hazard\"],\n", + " \"prompt\": row[\"prompt_text\"],\n", + " \"completion\": completion,\n", + " \"ttft\": ttft,\n", + " \"decode_tps\": decode_tps,\n", + " }})\n", + "\n", + "os.makedirs(\"outputs\", exist_ok=True)\n", + "with open(\"outputs/dbe_results.json\", \"w\") as f:\n", + " json.dump({{\n", + " \"base_model\": base_model_name,\n", + " \"lora_rank\": adapter_config[\"r\"],\n", + " \"max_new_tokens\": {MAX_NEW_TOKENS},\n", + " \"total_prompts\": len(results),\n", + " \"results\": results,\n", + " }}, f, indent=2)\n", + "\n", + "print(f\"\\\\nEvaluation complete. {{len(results)}} prompts.\")\n", + "'''\n", + "\n", + "# Fail here rather than inside the enclave if the code is malformed.\n", + "compile(JOB_CODE, \"main.py\", \"exec\")\n", + "\n", + "\n", + "def create_code_file(code: str) -> str:\n", + " tmp = Path(tempfile.mkdtemp()) / f\"job-{random.randint(1, 1_000_000)}\"\n", + " tmp.mkdir(parents=True, exist_ok=True)\n", + " p = tmp / \"main.py\"\n", + " p.write_text(code)\n", + " return str(p)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "benchmark_owner.submit_python_job(\n", + " ENCLAVE_EMAIL,\n", + " create_code_file(JOB_CODE),\n", + " \"dbe_eval_job\",\n", + " datasets={\n", + " MODEL_OWNER_EMAIL: [\"dbe_adapter\"],\n", + " benchmark_owner.email: [\"dbe_prompts\"],\n", + " },\n", + " share_results_with_do=False,\n", + " # Pinned so both parties review the same stack and the enclave installs it\n", + " # identically every run. transformers is pinned to the 4.x line on purpose:\n", + " # 5.x changes the import chain and needs a newer torch than some platforms\n", + " # resolve. torch itself is left to resolve per platform.\n", + " dependencies=[\"torch\", \"transformers==4.57.6\", \"peft==0.21.0\", \"safetensors==0.8.0\"],\n", + ")\n", + "print(f\" Job 'dbe_eval_job' submitted to the enclave ({ENCLAVE_EMAIL})\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 6 — Approve the job\n", + "\n", + "We approve our own job — submitting it is not consent to run it on our prompts. The model owner\n", + "approves it too, from their notebook. The run starts when the second approval lands; until then the\n", + "status stays `pending`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job = next((j for j in benchmark_owner.jobs if j.name == \"dbe_eval_job\"), None)\n", + "\n", + "if job is None:\n", + " print(\" 🟠 Job 'dbe_eval_job' not visible yet — the enclave is still distributing it. Wait a moment and re-run this cell.\")\n", + "else:\n", + " print(f\" ✅ Benchmark owner sees 'dbe_eval_job' status={job.status}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "benchmark_owner.approve_job(job)\n", + "print(\" Benchmark owner approved\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Wait" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 7 — Collect the results\n", + "\n", + "Once the model owner approves, the enclave runs the evaluation and sends the output back to us. It\n", + "loads a base model and generates on CPU, so give it a few minutes. Re-run until the status is\n", + "`done`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job = next(j for j in benchmark_owner.jobs if j.name == \"dbe_eval_job\")\n", + "print(f\" Job status : {job.status}\")\n", + "\n", + "if job.status != \"done\" or not job.output_paths:\n", + " print(\" 🟠 Not finished yet — wait until the status is 'done', then re-run this cell.\")\n", + "else:\n", + " print(f\" ✅ Output files : {[p.name for p in job.output_paths]}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "with open(job.output_paths[0]) as f:\n", + " result = json.load(f)\n", + "\n", + "print(f\" Base model : {result['base_model']}\")\n", + "print(f\" LoRA rank : {result['lora_rank']}\")\n", + "print(f\" Prompts : {result['total_prompts']}\")\n", + "print()\n", + "for r in result[\"results\"]:\n", + " print(f\" prompt_uid : {r['prompt_uid']} ({r['hazard']})\")\n", + " print(f\" prompt : {r['prompt'][:100]}...\")\n", + " print(f\" completion : {r['completion']}\")\n", + " print(f\" TTFT={r['ttft']:.2f}s decode={r['decode_tps']:.1f} tok/s\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "The model owner approved this run and cannot read a line of what you just printed. You never saw\n", + "their weights. The enclave is gone — Tinfoil Containers keep no disk, so its state went with it." + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb b/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb new file mode 100644 index 00000000000..28c9906bb50 --- /dev/null +++ b/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb @@ -0,0 +1,461 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Double-blind evaluation — Model Owner\n", + "\n", + "| Actor | Email | Role |\n", + "|-------|-------|------|\n", + "| **Enclave** | `ENCLAVE_EMAIL` | Tinfoil Container that runs the evaluation |\n", + "| **Benchmark owner** | `BENCHMARK_OWNER_EMAIL` | Owns the private prompt set |\n", + "| **Model owner** | `MODEL_OWNER_EMAIL` | Owns the private LoRA adapter |\n", + "\n", + "**The setting.** A benchmark owner wants to know how a model behaves on prompts it has never\n", + "seen. A model owner is willing to be measured but will not hand over the weights. Neither will show\n", + "the other what they hold, and neither trusts the machine in the middle.\n", + "\n", + "Both connect to an [enclave](https://github.com/OpenMined/PySyft/tree/dev/packages/syft-enclave) — a\n", + "sealed environment whose contents nobody, including whoever runs the hardware, can read. Each\n", + "uploads their half. Nothing runs until **both** owners approve the exact code, and the enclave\n", + "proves what it is before either of them uploads anything.\n", + "\n", + "This is the flow from [tinfoilsh/double-blind-eval](https://github.com/tinfoilsh/double-blind-eval),\n", + "carried out entirely over PySyft. The enclave is a [Tinfoil\n", + "Container](https://docs.tinfoil.sh/containers/overview), and the attestation is checked against the\n", + "measurement published for a signed release of its config.\n", + "\n", + "We hold the model. We approve the evaluation, and we do not get to see its results — only the\n", + "benchmark owner does. That asymmetry is the point: we agreed to be measured, not to learn what the\n", + "measurement was made of." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!uv pip install -Uq \"syft-enclave[tinfoil] @ git+https://github.com/OpenMined/PySyft.git@dev#subdirectory=packages/syft-enclave\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import random\n", + "import tempfile\n", + "from pathlib import Path\n", + "\n", + "from syft_enclaves import login_do\n", + "from syft_enclaves.attestation.tinfoil import TinfoilAppraisalPolicy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ENCLAVE_EMAIL = \"enclave@openmined.org\"\n", + "BENCHMARK_OWNER_EMAIL = \"bench@openmined.org\"\n", + "MODEL_OWNER_EMAIL = \"model@openmined.org\"\n", + "\n", + "# What the enclave is checked against. TINFOIL_TAG and IMAGE_DIGEST come from\n", + "# whoever deployed it: `just tinfoil-release` prints the digest, and the tag is\n", + "# the config release it was deployed from.\n", + "TINFOIL_REPO = \"OpenMined/syft-enclave-tinfoil\"\n", + "TINFOIL_TAG = \"v0.1.14\"\n", + "IMAGE_DIGEST = \"sha256:d0bd57f22af80b9dcd0dc151fb68d89cca65b65fcbbd1d2e4586cdfa9d7daebc\"\n", + "\n", + "print(f\" Enclave: {ENCLAVE_EMAIL}\")\n", + "print(f\" Benchmark owner: {BENCHMARK_OWNER_EMAIL} | Model owner: {MODEL_OWNER_EMAIL}\")\n", + "print(f\" Verifying against {TINFOIL_REPO} {TINFOIL_TAG}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 0 — Log in as Model Owner\n", + "\n", + "Colab authenticates us to Google Drive, which is how every party in this demo reaches every other\n", + "one. Nothing here talks to the enclave yet." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "CRYPTO_KEYS_PATH = None\n", + "# NOTE: Re-storing Private Key from local_file, if you have it, uncomment the following line and upload the file.\n", + "# from google.colab import files; files.upload()\n", + "# CRYPTO_KEYS_PATH = \"crypto_keys.json\"\n", + "\n", + "model_owner = login_do(encryption=True, crypto_keys_path=CRYPTO_KEYS_PATH)\n", + "print(f\" Model owner : {model_owner.email}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Optionally, to start from a clean SyftBox\n", + "# model_owner.delete_syftbox()\n", + "# model_owner._rds.peer_manager.write_own_version()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 1 — Connect to the enclave and to the benchmark owner\n", + "\n", + "We ask the enclave to peer with us, which is what lets us send it our weights. The benchmark owner\n", + "sends us a peer request from their notebook; approving it lets them read our public model card, and\n", + "nothing else." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model_owner.add_peer(ENCLAVE_EMAIL)\n", + "print(f\" Peer request sent to the enclave ({ENCLAVE_EMAIL})\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 1.1 — Approve the benchmark owner\n", + "\n", + "Re-run the cell below until their request appears, then approve it. `peer_must_exist=False` lets us\n", + "approve a request that has arrived before our own view of them has caught up." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model_owner.peers" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model_owner.approve_peer_request(BENCHMARK_OWNER_EMAIL, peer_must_exist=False)\n", + "print(\" Benchmark owner approved\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 2 — Attest the enclave\n", + "\n", + "Before uploading anything, we check what we are uploading it to. The enclave holds a key that never\n", + "leaves the hardware, and it serves a report signed under that key over a connection pinned to it. We\n", + "compare that report against the measurement published for `TINFOIL_TAG` — a signed GitHub release of\n", + "the enclave's config — and against the facts we expect: the container image digest, which datasite\n", + "the enclave runs as, and which two owners have to approve a job.\n", + "\n", + "Every one of those has to be pinned. Unpinned, the attestation would prove a genuine enclave booted\n", + "a signed config, but not that the config was the one we reviewed. Tinfoil is not in the trust path,\n", + "and neither are we to each other." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "policy = TinfoilAppraisalPolicy(\n", + " repo=TINFOIL_REPO,\n", + " release_tag=TINFOIL_TAG,\n", + " expected_image_digest=IMAGE_DIGEST,\n", + " expected_data_owners=[BENCHMARK_OWNER_EMAIL, MODEL_OWNER_EMAIL],\n", + " expected_email=ENCLAVE_EMAIL,\n", + " # The measured config pins no SYFT_VERSION, so there is nothing to compare.\n", + " expected_syft_version=None,\n", + ")\n", + "\n", + "result = model_owner.attest_peer(ENCLAVE_EMAIL, policy=policy)\n", + "\n", + "if result is None:\n", + " print(\" 🟠 The enclave published no attestation yet. Wait a moment and re-run this cell.\")\n", + "else:\n", + " result.print_checklist()\n", + " print()\n", + " print(f\" Bound to the report: {result.verified_key_bundle is not None}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3 — Fetch the private adapter\n", + "\n", + "Our model is a [PEFT](https://huggingface.co/docs/peft) LoRA adapter: a small set of weights that\n", + "adjusts a public base model. In a real run these would be weights we trained and never published.\n", + "Here we borrow a public adapter to stand in for one, the same way the double-blind-eval demo does." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# The adapter from the double-blind-eval demo: a public LoRA standing in for a\n", + "# private one. It targets google/gemma-4-31B-it, which needs a GPU-shaped enclave\n", + "# (`gpus:` in tinfoil-config.yml, which means a new measured config release).\n", + "# ADAPTER_REPO = \"Kobarac/gemma4-31b-factual-tool-selector-lora\"\n", + "\n", + "# On the enclave's current shape — 2 CPUs, 8 GB, no GPU — use a small pair that\n", + "# runs in software. Same two-file layout, same flow.\n", + "ADAPTER_REPO = \"zjudai/flowertune-general-nlp-lora-tinyllama-1.1b-chat-v1.0\"\n", + "ADAPTER_REV = \"main\"\n", + "ADAPTER_DIR = Path(\"dbe-adapter\")\n", + "\n", + "print(f\" Adapter: {ADAPTER_REPO}@{ADAPTER_REV}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ADAPTER_DIR.mkdir(parents=True, exist_ok=True)\n", + "\n", + "# A PEFT LoRA directory is two files: the config and the weights.\n", + "for name in (\"adapter_config.json\", \"adapter_model.safetensors\"):\n", + " url = f\"https://huggingface.co/{ADAPTER_REPO}/resolve/{ADAPTER_REV}/{name}\"\n", + " !curl -sfL \"{url}\" -o \"{ADAPTER_DIR}/{name}\"\n", + " print(f\" fetched {name}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 4 — Upload the model\n", + "\n", + "We upload the adapter into the enclave. The **private** side is the weights, shared only with the\n", + "enclave. The **mock** side is a model card: the base model, the rank, the size. That is all the\n", + "benchmark owner ever sees of our model.\n", + "\n", + "Uploading grants nothing on its own. The weights are used once, by a job we approve, and they are\n", + "gone when the enclave restarts — Tinfoil Containers have no persistent disk." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "adapter_config = json.loads((ADAPTER_DIR / \"adapter_config.json\").read_text())\n", + "BASE_MODEL = adapter_config[\"base_model_name_or_path\"]\n", + "LORA_RANK = adapter_config[\"r\"]\n", + "\n", + "\n", + "def create_model_card() -> Path:\n", + " \"\"\"The public half: what the benchmark owner may know about our model.\"\"\"\n", + " tmp = Path(tempfile.mkdtemp()) / f\"model-card-{random.randint(1, 1_000_000)}\"\n", + " tmp.mkdir(parents=True, exist_ok=True)\n", + " path = tmp / \"model_card.txt\"\n", + " weights = ADAPTER_DIR / \"adapter_model.safetensors\"\n", + " path.write_text(\"\\n\".join([\n", + " \"Private LoRA adapter\",\n", + " \"====================\",\n", + " f\"Base model : {BASE_MODEL}\",\n", + " f\"LoRA rank : {LORA_RANK} (alpha {adapter_config['lora_alpha']})\",\n", + " f\"Target modules : {', '.join(sorted(adapter_config['target_modules']))}\",\n", + " f\"Size : {weights.stat().st_size / 1e6:.1f} MB\",\n", + " \"\",\n", + " \"The weights are not shared. They go to the enclave and nowhere else.\",\n", + " \"\",\n", + " ]))\n", + " return path\n", + "\n", + "\n", + "model_card = create_model_card()\n", + "print(model_card.read_text())" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model_owner.create_dataset(\n", + " name=\"dbe_adapter\",\n", + " mock_path=model_card,\n", + " private_path=ADAPTER_DIR,\n", + " summary=f\"Private LoRA adapter for {BASE_MODEL} — rank {LORA_RANK}\",\n", + " users=[BENCHMARK_OWNER_EMAIL, ENCLAVE_EMAIL],\n", + ")\n", + "print(\" Created 'dbe_adapter'\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Send the weights to the enclave, and only to the enclave.\n", + "model_owner.share_private_dataset(\"dbe_adapter\", ENCLAVE_EMAIL)\n", + "print(\" Private adapter shared with the enclave\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Wait" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 5 — Wait for the evaluation job\n", + "\n", + "The benchmark owner submits the job that runs our model on their prompts. The enclave distributes it\n", + "to both of us for approval. Re-run the cell below until it appears." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job = next((j for j in model_owner.jobs if j.name == \"dbe_eval_job\"), None)\n", + "\n", + "if job is None:\n", + " print(\" 🟠 Job 'dbe_eval_job' not visible yet — the benchmark owner may not have submitted it. Wait a moment and re-run this cell.\")\n", + "else:\n", + " print(f\" ✅ Model owner sees 'dbe_eval_job' status={job.status}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Step 5.1 — Read it\n", + "\n", + "This is the code that will touch our weights, so read it properly. Two things are worth checking:\n", + "that it reads our adapter through `resolve_dataset_files_path` and uses it for nothing but\n", + "generation, and that it writes only the completions and timings to `outputs/` — no weights, no\n", + "tensors.\n", + "\n", + "`share_results_with_do=False` in the submission is what routes the output to the benchmark owner\n", + "alone. We are approving a run we will not see the result of." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 6 — Approve the job\n", + "\n", + "Our approval is one of two. The benchmark owner approves from their notebook, and the run starts the\n", + "moment the second approval lands. Either order works; until both are in, the status stays `pending`\n", + "and nothing has touched our weights." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "model_owner.approve_job(job)\n", + "print(\" Model owner approved\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 7 — We do not get the result\n", + "\n", + "The enclave runs the evaluation and sends the output to whoever submitted the job. That is the\n", + "benchmark owner, so our copy of the job has no outputs and never reaches `done` — we approved the\n", + "run and learned nothing from it.\n", + "\n", + "That is what we agreed to. The enclave proved what it was running before we uploaded anything, and\n", + "it ran the code we both read, so the benchmark owner's completions came from our adapter and nothing\n", + "else. Neither of us learned the other's half." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job = next(j for j in model_owner.jobs if j.name == \"dbe_eval_job\")\n", + "print(f\" Job status : {job.status}\")\n", + "print(f\" Output files : {job.output_paths}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "The benchmark owner now has completions from your model on prompts you never saw, and you never saw\n", + "their prompts. Nobody, including whoever runs the hardware, saw both halves." + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/enclave/double-blind-eval/SETUP.md b/notebooks/enclave/double-blind-eval/SETUP.md new file mode 100644 index 00000000000..e0a33ea63af --- /dev/null +++ b/notebooks/enclave/double-blind-eval/SETUP.md @@ -0,0 +1,130 @@ +# Running the double-blind eval demo + +A benchmark owner and a model owner each upload half of an evaluation into a Tinfoil enclave. Both +approve the job, the enclave runs it, and only the benchmark owner reads the results. The two +notebooks in this folder are the two parties; everything else on this page is the operator's job, +done once before either notebook is opened. + +## What you need + +Three Google accounts: one for the enclave, one per party. The enclave reaches Drive with a token +you register as a Tinfoil secret, and each party authenticates through Colab. + +You also need the `tinfoil` CLI and an admin API key from the Tinfoil dashboard under **Settings → +API Keys → Admin keys**. Log in once with `tinfoil login --api-key admin_...` and check it with +`just tinfoil-whoami`. Deploying needs that key; publishing a release does not. + +Run every `just` command below from `packages/syft-enclave/`. + +## 1. Reset both parties' state + +Wipe each data owner's SyftBox before you deploy, not after. The enclave caches its peers' Drive +folders when it boots, so clearing them afterwards leaves it peered with folders that no longer +exist. + +```bash +just delete-syftbox bench@openmined.org ../../credentials/token_bench.json +just delete-syftbox model@openmined.org ../../credentials/token_model.json +``` + +Without a token file to hand, each notebook has a commented cell under Step 0 that calls +`delete_syftbox()` from Colab instead. + +The enclave needs no reset: Tinfoil Containers have no persistent disk, and the enclave wipes its +own state on every boot. + +## 2. Deploy the published release + +Release `v0.1.14` of [`OpenMined/syft-enclave-tinfoil`](https://github.com/OpenMined/syft-enclave-tinfoil) +is already published, so deploy it as it stands. Skip to step 5 only if you changed the enclave +image or its config. + +```bash +just tinfoil-deploy v0.1.14 enclave@openmined.org bench@openmined.org,model@openmined.org +``` + +Both party emails have to appear in that comma-separated list. It becomes +`SYFT_ENCLAVE_DATA_OWNERS`, which is what makes a job wait for two approvals instead of one. + +## 3. Check the enclave is attesting + +```bash +just tinfoil-attest syft-enclave.openmined.containers.tinfoil.dev +``` + +If it fails, run `just tinfoil-why` first — the control plane's `error_message` has named the cause +in every failure so far. [`docs/tinfoil_troubleshooting.md`](../../../packages/syft-enclave/docs/tinfoil_troubleshooting.md) +covers the rest. + +## 4. Run the two notebooks + +Open both in Colab, one per account: + +- [`1. DO-benchmark-owner-dbe.ipynb`](1.%20DO-benchmark-owner-dbe.ipynb) — uploads the prompts, + submits the job, reads the results +- [`2. DO-model-owner-dbe.ipynb`](2.%20DO-model-owner-dbe.ipynb) — uploads the adapter, approves the + job, sees no results + +Set the same five constants in both, in the cell under **Setup**: + +```python +ENCLAVE_EMAIL = "enclave@openmined.org" +BENCHMARK_OWNER_EMAIL = "bench@openmined.org" +MODEL_OWNER_EMAIL = "model@openmined.org" +TINFOIL_REPO = "OpenMined/syft-enclave-tinfoil" +TINFOIL_TAG = "v0.1.14" +IMAGE_DIGEST = "sha256:d0bd57f22af80b9dcd0dc151fb68d89cca65b65fcbbd1d2e4586cdfa9d7daebc" +``` + +`TINFOIL_TAG` and `IMAGE_DIGEST` are what the parties check the enclave against, so they must match +the release you deployed. The digest above is the one `v0.1.14` pins; after a republish, use the one +`just tinfoil-release` printed. + +Then run both notebooks top to bottom. They wait on each other four times, and a card in the +notebook says so each time. Cells that wait print a 🟠 line and tell you to re-run them. + +The evaluation itself takes four to six minutes. It installs PyTorch, downloads a base model and +generates on two CPUs, and a job is killed at 600 seconds. + +## 5. Republish, after changing the image or config + +Everything in `tinfoil/tinfoil-config.yml` is measured, so any edit to it — or to the enclave image +— needs a new release before it can be deployed. + +```bash +just tinfoil-build-info # the latest tag, and a suggested next one +just tinfoil-release v0.1.15 # build, push, pin the digest, open the config PR +``` + +Merge that pull request, then publish and redeploy: + +```bash +just tinfoil-publish v0.1.15 # about a minute to compute the measurement +just tinfoil-deploy v0.1.15 enclave@openmined.org bench@openmined.org,model@openmined.org +``` + +Keep the digest `tinfoil-release` printed, and put it in both notebooks along with the new tag. + +A release is a signed GitHub release and a transparency-log entry, so it cannot be unpublished. +Number versions with that in mind. To go back to an earlier one without moving "latest": + +```bash +just tinfoil-relaunch v0.1.14 --promote-release=false +``` + +## Giving the model a GPU + +The demo runs TinyLlama-1.1B on CPU because `tinfoil-config.yml` sets `gpus: 0`. The model owner +notebook carries a commented `ADAPTER_REPO` line for the adapter the +[double-blind-eval](https://github.com/tinfoilsh/double-blind-eval) demo uses, which targets +gemma-4-31B-it and needs a GPU. Switching to it means raising `gpus:` in the config, which is a +config change, so follow step 5. + +## Further reading + +- [`docs/tinfoil_deployment.md`](../../../packages/syft-enclave/docs/tinfoil_deployment.md) — the + deployment mechanics in full +- [`docs/security.md`](../../../packages/syft-enclave/docs/security.md) §6 — what the attestation + proves +- [`OpenMined/double-blind-eval-bench`](https://github.com/OpenMined/double-blind-eval-bench) — the + prompt set the benchmark owner downloads From dc7121e0f2337e224efe59d8faae7756d5b8fcc2 Mon Sep 17 00:00:00 2001 From: Koen van der Veen Date: Fri, 18 Sep 2026 18:39:32 +0200 Subject: [PATCH 2/2] fix: make the double-blind eval demo run on a live Tinfoil enclave Found by deploying v0.1.14 and running both notebooks' own cells against it. The eval job failed twice in the enclave before it ran. Plain `torch` on Linux pulls the whole CUDA stack, which overran the enclave's ramdisk and failed the job with "No space left on device"; the enclave has no GPU, so pin the CPU-only wheel by URL instead, 176 MB against several GB. Then float32 weights plus the venv and the downloaded model exceeded the 8 GB of RAM and the OOM killer took the job, so load in bfloat16 with low_cpu_mem_usage and drop the token budget to 16. The job now completes in about two and a half minutes. Reading the model owner's model card could never work. Unlike `jobs` and `peers`, `SyftEnclaveClient.datasets` returns the manager directly rather than through the property that syncs, so re-running that cell pulled nothing however long you waited. The cell syncs explicitly now. Step 7 looped on "wait until the status is done" for a job that had already failed. It now reports a failed job and prints its stderr. `verify_tinfoil.py` never passed the host into the evidence or the policy, so the appraisal raised "no host to reach it on" before running a single check, and main() swallowed the error and exited 1 with no output. The `tinfoil-verify` recipe also ran without the optional extra its own comment requires. Both fixed; the recipe now passes all ten checks against a live enclave. The notebooks and SETUP.md use the enclave accounts from credentials/CLAUDE.md rather than placeholder emails, and both pin the published v0.1.14 and the image digest that release carries. Co-Authored-By: Claude Opus 5 (1M context) --- .../1. DO-benchmark-owner-dbe.ipynb | 61 ++++++++++++++----- .../2. DO-model-owner-dbe.ipynb | 4 +- notebooks/enclave/double-blind-eval/SETUP.md | 17 +++--- packages/syft-enclave/Justfile | 2 +- .../syft-enclave/scripts/verify_tinfoil.py | 12 +++- 5 files changed, 68 insertions(+), 28 deletions(-) diff --git a/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb b/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb index 1a2935fd31b..20c658fd423 100644 --- a/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb +++ b/notebooks/enclave/double-blind-eval/1. DO-benchmark-owner-dbe.ipynb @@ -69,8 +69,8 @@ "outputs": [], "source": [ "ENCLAVE_EMAIL = \"enclave@openmined.org\"\n", - "BENCHMARK_OWNER_EMAIL = \"bench@openmined.org\"\n", - "MODEL_OWNER_EMAIL = \"model@openmined.org\"\n", + "BENCHMARK_OWNER_EMAIL = \"benchmark_owner@openmined.org\"\n", + "MODEL_OWNER_EMAIL = \"model_owner@openmined.org\"\n", "\n", "# What the enclave is checked against. TINFOIL_TAG and IMAGE_DIGEST come from\n", "# whoever deployed it: `just tinfoil-release` prints the digest, and the tag is\n", @@ -357,6 +357,10 @@ "metadata": {}, "outputs": [], "source": [ + "# Unlike `jobs`, `datasets` does not sync when you read it, so pull first —\n", + "# the model owner may have uploaded since our last sync.\n", + "benchmark_owner.sync()\n", + "\n", "model_dataset = benchmark_owner.datasets.get(\"dbe_adapter\", datasite=MODEL_OWNER_EMAIL)\n", "print(model_dataset.mock_files[0].read_text())" ] @@ -384,9 +388,34 @@ "metadata": {}, "outputs": [], "source": [ - "# Keep this modest: a job is killed at 600 seconds, and the enclave has 2 CPUs\n", - "# and no GPU, so the base model runs in software.\n", - "MAX_NEW_TOKENS = 32\n", + "# Keep this modest. The enclave has 2 CPUs and no GPU, so the model runs in\n", + "# software at roughly a token per second, and a job is killed at 600 seconds.\n", + "MAX_NEW_TOKENS = 16\n", + "\n", + "# Both parties review this list, so it is pinned rather than resolved.\n", + "#\n", + "# torch is the CPU-only wheel, by URL. On Linux the ordinary `torch` on PyPI\n", + "# drags in the whole CUDA stack — several GB, which overruns the enclave's\n", + "# ramdisk and fails the job with \"No space left on device\". The enclave has no\n", + "# GPU, so none of it would ever be used; the CPU wheel is 176 MB. The marker\n", + "# keeps the notebook runnable off Linux, where that wheel does not apply.\n", + "#\n", + "# transformers stays on the 4.x line: 5.x changes the import chain and needs a\n", + "# newer torch than some platforms resolve.\n", + "TORCH_CPU_WHEEL = (\n", + " \"https://download.pytorch.org/whl/cpu/\"\n", + " \"torch-2.9.1%2Bcpu-cp312-cp312-manylinux_2_28_x86_64.whl\"\n", + ")\n", + "EVAL_DEPS = [\n", + " f\"torch @ {TORCH_CPU_WHEEL} ; sys_platform == 'linux'\",\n", + " \"torch ; sys_platform != 'linux'\",\n", + " \"transformers==4.57.6\",\n", + " \"peft==0.21.0\",\n", + " \"safetensors==0.8.0\",\n", + " # Lets from_pretrained load weights straight into place. Without it the\n", + " # load allocates the model twice and the enclave's OOM killer takes the job.\n", + " \"accelerate==1.12.0\",\n", + "]\n", "\n", "JOB_CODE = f'''\n", "import csv\n", @@ -410,8 +439,13 @@ "base_model_name = adapter_config[\"base_model_name_or_path\"]\n", "print(f\"Loading {{base_model_name}} with a rank-{{adapter_config['r']}} adapter...\")\n", "\n", + "# bfloat16, not float32: the enclave has 8 GB of RAM, shared with its ramdisk,\n", + "# which also holds this venv and the downloaded base model. float32 weights\n", + "# peak at about 4.4 GB on top of that and the job gets OOM-killed.\n", "tokenizer = AutoTokenizer.from_pretrained(base_model_name)\n", - "base_model = AutoModelForCausalLM.from_pretrained(base_model_name, dtype=torch.float32)\n", + "base_model = AutoModelForCausalLM.from_pretrained(\n", + " base_model_name, dtype=torch.bfloat16, low_cpu_mem_usage=True\n", + ")\n", "model = PeftModel.from_pretrained(base_model, adapter_dir)\n", "model.eval()\n", "print(\"Model loaded\")\n", @@ -502,11 +536,7 @@ " benchmark_owner.email: [\"dbe_prompts\"],\n", " },\n", " share_results_with_do=False,\n", - " # Pinned so both parties review the same stack and the enclave installs it\n", - " # identically every run. transformers is pinned to the 4.x line on purpose:\n", - " # 5.x changes the import chain and needs a newer torch than some platforms\n", - " # resolve. torch itself is left to resolve per platform.\n", - " dependencies=[\"torch\", \"transformers==4.57.6\", \"peft==0.21.0\", \"safetensors==0.8.0\"],\n", + " dependencies=EVAL_DEPS,\n", ")\n", "print(f\" Job 'dbe_eval_job' submitted to the enclave ({ENCLAVE_EMAIL})\")" ] @@ -569,8 +599,8 @@ "## Step 7 — Collect the results\n", "\n", "Once the model owner approves, the enclave runs the evaluation and sends the output back to us. It\n", - "loads a base model and generates on CPU, so give it a few minutes. Re-run until the status is\n", - "`done`." + "installs PyTorch, downloads the base model and generates on two CPUs, which took a little over two\n", + "minutes on the release this was tested against. Re-run until the status is `done`." ] }, { @@ -582,7 +612,10 @@ "job = next(j for j in benchmark_owner.jobs if j.name == \"dbe_eval_job\")\n", "print(f\" Job status : {job.status}\")\n", "\n", - "if job.status != \"done\" or not job.output_paths:\n", + "if job.status == \"failed\":\n", + " print(\" ❌ The job failed inside the enclave. Its stderr says why:\")\n", + " print(job.stderr)\n", + "elif job.status != \"done\" or not job.output_paths:\n", " print(\" 🟠 Not finished yet — wait until the status is 'done', then re-run this cell.\")\n", "else:\n", " print(f\" ✅ Output files : {[p.name for p in job.output_paths]}\")" diff --git a/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb b/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb index 28c9906bb50..c8f127de03c 100644 --- a/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb +++ b/notebooks/enclave/double-blind-eval/2. DO-model-owner-dbe.ipynb @@ -69,8 +69,8 @@ "outputs": [], "source": [ "ENCLAVE_EMAIL = \"enclave@openmined.org\"\n", - "BENCHMARK_OWNER_EMAIL = \"bench@openmined.org\"\n", - "MODEL_OWNER_EMAIL = \"model@openmined.org\"\n", + "BENCHMARK_OWNER_EMAIL = \"benchmark_owner@openmined.org\"\n", + "MODEL_OWNER_EMAIL = \"model_owner@openmined.org\"\n", "\n", "# What the enclave is checked against. TINFOIL_TAG and IMAGE_DIGEST come from\n", "# whoever deployed it: `just tinfoil-release` prints the digest, and the tag is\n", diff --git a/notebooks/enclave/double-blind-eval/SETUP.md b/notebooks/enclave/double-blind-eval/SETUP.md index e0a33ea63af..075e72b8c43 100644 --- a/notebooks/enclave/double-blind-eval/SETUP.md +++ b/notebooks/enclave/double-blind-eval/SETUP.md @@ -23,8 +23,8 @@ folders when it boots, so clearing them afterwards leaves it peered with folders exist. ```bash -just delete-syftbox bench@openmined.org ../../credentials/token_bench.json -just delete-syftbox model@openmined.org ../../credentials/token_model.json +just delete-syftbox benchmark_owner@openmined.org ../../credentials/token_benchmark_owner.json +just delete-syftbox model_owner@openmined.org ../../credentials/token_model_owner.json ``` Without a token file to hand, each notebook has a commented cell under Step 0 that calls @@ -40,7 +40,7 @@ is already published, so deploy it as it stands. Skip to step 5 only if you chan image or its config. ```bash -just tinfoil-deploy v0.1.14 enclave@openmined.org bench@openmined.org,model@openmined.org +just tinfoil-deploy v0.1.14 enclave@openmined.org benchmark_owner@openmined.org,model_owner@openmined.org ``` Both party emails have to appear in that comma-separated list. It becomes @@ -69,8 +69,8 @@ Set the same five constants in both, in the cell under **Setup**: ```python ENCLAVE_EMAIL = "enclave@openmined.org" -BENCHMARK_OWNER_EMAIL = "bench@openmined.org" -MODEL_OWNER_EMAIL = "model@openmined.org" +BENCHMARK_OWNER_EMAIL = "benchmark_owner@openmined.org" +MODEL_OWNER_EMAIL = "model_owner@openmined.org" TINFOIL_REPO = "OpenMined/syft-enclave-tinfoil" TINFOIL_TAG = "v0.1.14" IMAGE_DIGEST = "sha256:d0bd57f22af80b9dcd0dc151fb68d89cca65b65fcbbd1d2e4586cdfa9d7daebc" @@ -83,8 +83,9 @@ the release you deployed. The digest above is the one `v0.1.14` pins; after a re Then run both notebooks top to bottom. They wait on each other four times, and a card in the notebook says so each time. Cells that wait print a 🟠 line and tell you to re-run them. -The evaluation itself takes four to six minutes. It installs PyTorch, downloads a base model and -generates on two CPUs, and a job is killed at 600 seconds. +The evaluation takes a little over two minutes, measured on a run against this release: the enclave +installs PyTorch, downloads the base model, and generates on two CPUs. A job is killed at 600 +seconds, so there is room to spare. ## 5. Republish, after changing the image or config @@ -100,7 +101,7 @@ Merge that pull request, then publish and redeploy: ```bash just tinfoil-publish v0.1.15 # about a minute to compute the measurement -just tinfoil-deploy v0.1.15 enclave@openmined.org bench@openmined.org,model@openmined.org +just tinfoil-deploy v0.1.15 enclave@openmined.org benchmark_owner@openmined.org,model_owner@openmined.org ``` Keep the digest `tinfoil-release` printed, and put it in both notebooks along with the new tag. diff --git a/packages/syft-enclave/Justfile b/packages/syft-enclave/Justfile index 0d03b8c7096..1c79d5edc3d 100644 --- a/packages/syft-enclave/Justfile +++ b/packages/syft-enclave/Justfile @@ -842,7 +842,7 @@ tinfoil-attest host: tinfoil-verify host *args: #!/bin/bash set -e - uv run --project ../.. python scripts/verify_tinfoil.py {{host}} --repo {{tinfoil_repo}} {{args}} + uv run --project ../.. --with tinfoil python scripts/verify_tinfoil.py {{host}} --repo {{tinfoil_repo}} {{args}} # --------------------------------------------------------------------------------------------------------------------- # Tinfoil — debug diff --git a/packages/syft-enclave/scripts/verify_tinfoil.py b/packages/syft-enclave/scripts/verify_tinfoil.py index 3654faff7f3..97de4d69747 100644 --- a/packages/syft-enclave/scripts/verify_tinfoil.py +++ b/packages/syft-enclave/scripts/verify_tinfoil.py @@ -77,6 +77,9 @@ def main() -> int: expected_data_owners=args.expected_data_owners, expected_email=args.expected_enclave_email, container_name=args.container_name, + # The host is what the pinned fetch connects to. Without it the + # appraisal fails before it runs a single check. + host=args.host, # This script checks a live host, often before the digest and the # data-owner list are known, so it opts out rather than refusing to # build a policy. @@ -86,10 +89,13 @@ def main() -> int: and args.expected_enclave_email ), ) + evidence = tinfoil_evidence(fetch_document(args.host), host=args.host) try: - verify_tinfoil_evidence(tinfoil_evidence(fetch_document(args.host)), policy) - except AttestationError: - # verify_tinfoil_evidence already printed the full checklist. + verify_tinfoil_evidence(evidence, policy) + except AttestationError as exc: + # verify_tinfoil_evidence prints the checklist for a failed check, but + # it raises before printing anything when it cannot appraise at all. + print(f"❌ {exc}", file=sys.stderr) return 1 return 0