Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion dashboard/topic-tldrs.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,5 +42,11 @@
"0450-Beyond-Declared-Dependencies-The-Limits-of-Hermetic-Build-Tools.tex": "Measure the hermetic gap: what builds really consume beyond their declared dependencies, comparing Hermeto with Nix's stronger closure model.",
"0460-Counterfactual-Causal-Graphs-for-Coding-Agent-Runs.tex": "Which step made the coding agent break things? Replay recorded agent runs, intervene (drop a tool call, revert a file) and re-run to build true cause-and-effect graphs.",
"0470-Zero-Knowledge-Proof-of-eth-call-Execution.tex": "eth_call answers from an RPC node must be taken on trust. Build a system that attaches a zero-knowledge proof that the result is correct for a given block.",
"0480-Evaluation-of-Safety-Measures-for-Coding-Agents.tex": "Benchmark the safety guards of coding agents (permission lists, sandboxes, LLM classifiers) on dangerous vs. benign tasks: what do they block, and what does it cost in usefulness?"
"0480-Evaluation-of-Safety-Measures-for-Coding-Agents.tex": "Benchmark the safety guards of coding agents (permission lists, sandboxes, LLM classifiers) on dangerous vs. benign tasks: what do they block, and what does it cost in usefulness?",
"0490-Latent-Communication-for-Coding-Agents.tex": "Let coding agents talk in hidden states instead of text. Build a multi-agent pipeline with latent messages and measure pass rate, token cost and latency on SWE benchmarks.",
"0500-Benchmarking-Long-Term-Software-Maintainability.tex": "Passing tests today is not enough. Score a solution by replaying years of a product's roadmap on top of it with a coding agent: how costly is evolving it later?",
"0510-AI-Agent-Trajectories-Faithfulness-Verification.tex": "An agent's trajectory log is a self-report. Watch what it really does with eBPF and flag fabricated, omitted or misdescribed actions in Claude Code, Codex and Gemini CLI runs.",
"0520-Agentic-Bill-of-Materials-Generation.tex": "Design an Agentic Bill of Materials (dependencies, MCP servers, sub-agents, models and their lineage) and a generator that checks configured components against what is actually served.",
"0530-Verifying-Declared-Derivations-of-Open-Weight-Models.tex": "\"LoRA fine-tune of Llama\" is never checked. Compare weights against the trusted base model to verify declared derivations and spot hidden backdoors or removed safety alignment.",
"0540-Which-Model-Wrote-This-Robust-Attribution-of-LLM-Generated-Text.tex": "Combine watermarks and model fingerprints to tell which LLM wrote a text or code, even after paraphrasing, watermark scrubbing or spoofing."
}
12 changes: 6 additions & 6 deletions dashboard/topics.json
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,12 @@
},
{
"title": "Latent Communication for Coding Agents",
"tldr": "",
"tldr": "Let coding agents talk in hidden states instead of text. Build a multi-agent pipeline with latent messages and measure pass rate, token cost and latency on SWE benchmarks.",
"description": "Multi-agent coding systems rely on agents communicating with each other in natural language. Every message is decoded into tokens by one model and re-encoded by the next, which is slow, costly, and loses information. Recent work shows that LLM agents can instead communicate directly in latent space, by exchanging activations, hidden states, or a shared latent memory, with higher accuracy and a fraction of the tokens. So far this has only been evaluated on simple math and question-answering benchmarks, not on complex software engineering tasks. In this thesis, you will build a multi-agent coding pipeline with open-weight models in which agent-to-agent messages are (partly) latent instead of textual. You will evaluate it on SWE benchmarks (e.g., DeepSWE, FrontierCode, SWE-Bench-Pro), measuring pass rates, token cost, and latency against a text-based baseline. If time permits, you will also train models to natively communicate in the latent space."
},
{
"title": "Benchmarking Long-Term Software Maintainability",
"tldr": "",
"tldr": "Passing tests today is not enough. Score a solution by replaying years of a product's roadmap on top of it with a coding agent: how costly is evolving it later?",
"description": "Coding benchmarks judge a solution by whether it passes functional tests today. Real software lives for years, and a change that passes its tests can still make the next hundred changes harder. Static maintainability metrics (complexity, duplication, code smells) are weak proxies for this. The question that matters is how costly it is to evolve the code later. Recent benchmarks have started to evaluate agents over chains of tasks, but they span weeks or months of history, not the multi-year evolution of a product. In this thesis, you will build a benchmark that measures maintainability by its consequences. You will collect long-term product roadmaps (e.g., five years of an application's evolution, reconstructed from release notes, milestones, and issue trackers of open-source projects) or create them synthetically. A solution to a task at $t=0$ is then scored by replaying the rest of the roadmap on top of it with a fixed coding agent, measuring how many future tasks succeed and at what cost (tokens, edits, and how much of the $t=0$ code has to be rewritten). With this benchmark, you will compare human-written and agent-written solutions, and check whether classical maintainability metrics predict actual future evolution cost."
}
]
Expand Down Expand Up @@ -141,22 +141,22 @@
},
{
"title": "AI Agent Trajectories Faithfulness Verification",
"tldr": "",
"tldr": "An agent's trajectory log is a self-report. Watch what it really does with eBPF and flag fabricated, omitted or misdescribed actions in Claude Code, Codex and Gemini CLI runs.",
"description": "AI agents such as Claude Code, Codex and Gemini CLI record their own work as a trajectory. Users, auditors and safety monitors trust this log, but it is a self-report. A compromised, deceived or buggy agent can leave out actions, claim actions it never took, or misdescribe what a command did. In this Master's thesis, you will design and implement a trajectory faithfulness verifier. An observer outside the agent, built with eBPF in the Linux kernel, records file, process and network activity. The verifier then checks each claim in the trajectory against this independent ground truth and reports fabricated, omitted and substituted actions. You will then use the verifier for an empirical study. It will measure how often real trajectories from different agents are unfaithful, how much activity hides beneath the commands agents report, and how reliably manipulated trajectories are detected, including by adaptive attackers."
},
{
"title": "Agentic Bill of Materials Generation and Verification",
"tldr": "",
"tldr": "Design an Agentic Bill of Materials (dependencies, MCP servers, sub-agents, models and their lineage) and a generator that checks configured components against what is actually served.",
"description": "Agentic AI systems are built from software dependencies, models, tools, MCP servers, sub-agents, memory components and external AI services. Current SBOM and AIBOM generators list these components by scanning source code. They do not capture how the components are connected, which model is actually served behind an endpoint, or where a model came from. In this thesis, you will design an Agentic Bill of Materials (ABOM) with three layers: software dependencies, a typed composition graph of agentic components, and the lineage of AI artifacts. You will also implement a generator for it. The generator combines static analysis of manifests, MCP registrations, A2A configurations and model routing definitions with dynamic probing of deployed interfaces. It records both configured and observed component identities and flags any inconsistency between them. You will build a benchmark of agentic applications with known ground-truth composition, and use it to compare your generator against existing ones on completeness and accuracy."
},
{
"title": "Verifying Declared Derivations of Open-Weight Models",
"tldr": "",
"tldr": "\"LoRA fine-tune of Llama\" is never checked. Compare weights against the trusted base model to verify declared derivations and spot hidden backdoors or removed safety alignment.",
"description": "Open-weight models on Hugging Face declare their lineage in model cards and AIBOMs, for example \"LoRA fine-tune of Llama-3.1-8B\" or \"4-bit quantization of Qwen3\". These declarations are never verified. Existing lineage tests can tell whether two models are related, but not whether the declared derivation is complete. For example, a model truthfully declared as a fine-tune may also contain an undeclared backdoor or have its safety alignment removed. In this thesis, you will design and implement a verifier that takes a trusted base model as the reference point. It will check whether the weight differences of a derived model match the declared transformation and localize modifications that no declared step explains. You will run the verifier on a large sample of Hugging Face models to measure how often declarations are wrong or incomplete. You will also evaluate it on a benchmark of models with planted undeclared modifications, including adaptive evasion attempts."
},
{
"title": "Which Model Wrote This? Robust Attribution of LLM-Generated Text",
"tldr": "",
"tldr": "Combine watermarks and model fingerprints to tell which LLM wrote a text or code, even after paraphrasing, watermark scrubbing or spoofing.",
"description": "Text written by large language models is everywhere: in news, reviews, essays, commit messages and source code. Knowing which model produced a given text matters for accountability, for enforcing terms of use, for detecting unauthorized distillation, and for regulation. With the transparency obligations of the EU AI Act taking effect in 2026, providers such as Google and Anthropic have started to embed statistical watermarks in the text their models generate. Watermarks alone do not answer the question, however. Mainstream open-weight models ship without a watermark, and whoever runs one decides whether to add it. Checking a watermark requires the provider's secret key, and providers open their detectors at best to vetted organizations, not to the public. A thorough rewrite by another model can erase a watermark, as providers themselves acknowledge, and free tools now automate such rewriting. Worse, some watermarking schemes can be learned from a model's outputs and then forged, so that a text appears to come from a model that never generated it. At the same time, models leave unintentional fingerprints, such as word choices, formatting habits and code structure, that a classifier can often use to tell them apart, even after the text has been rewritten. In this thesis, you will design and implement an attribution system that, given a text, identifies which model produced it by combining both kinds of evidence. Since the detectors of deployed watermarks are not public, you will watermark open-weight models yourself with open-source schemes, such as the SynthID-Text implementation in Hugging Face Transformers, and study fingerprints on both open-weight and proprietary models. You will build a benchmark of texts from current proprietary and open-weight models, covering natural-language prose as well as code and commit messages written by coding agents. You will then evaluate attribution under adversarial conditions: watermark scrubbing, paraphrasing by a second model, human editing, mixed authorship and watermark spoofing. Research questions include: Does scrubbing a watermark also remove the model fingerprint, or does the laundered text carry the fingerprint of the rewriting model instead? Can a mismatch between watermark and fingerprint expose a spoofed watermark? How does attribution degrade for models of the same family, for fine-tuned and distilled models, and for short texts?"
}
]
Expand Down
Loading