Catch forgotten definitions and catch up with the BEAM servers

CHANGELOG.md

@@ -5,6 +5,9 @@
 ### Changes
 
 - [#22](https://github.com/nrepl/proof/pull/22): The bundled profiles have short names, with versions from their `version` snippets, so the servers in the compatibility matrix get columns of the same width.
+- [#24](https://github.com/nrepl/proof/pull/24): `session.persistent` evaluates something else between a definition and its use, so it catches a server that keeps only what each eval uses.
+- [#24](https://github.com/nrepl/proof/pull/24): The profile of repartee checks that each expression in an eval gets its own value.
+- [#24](https://github.com/nrepl/proof/pull/24): `proof serve -like dialtone` and `-like repartee` act like nREPL, as neither server has the quirks of its scenarios anymore.
 
 ## 0.2.0 (2026-11-03)
 

README.md

@@ -111,8 +111,8 @@ $ proof serve -listen 127.0.0.1:7888 last-value no-err
 ```
 
 Here it sends only the value of the last form (like Basilisp, jank and
-dialtone) and drops what the code prints to stderr (like Basilisp,
-dialtone and repartee). `proof list` shows all the scenarios, and every
+let-go) and drops what the code prints to stderr (like Basilisp, nbb
+and let-go). `proof list` shows all the scenarios, and every
 one of them is something a real server does (or something TCP can do to
 the replies). `-like jank` turns on all of jank's scenarios at once, and
 `proof list` shows the other servers it can behave like.

cmd/proof/main.go

@@ -426,6 +426,9 @@ func list(w io.Writer) {
 			}
 			line += " " + s
 		}
+		if len(p.Scenarios) == 0 {
+			line += " none"
+		}
 		fmt.Fprintln(w, line)
 	}
 }

doc/design.md

@@ -187,7 +187,7 @@ different problem, which `proof serve` takes care of.
 own it behaves like nREPL, and scenarios make it behave like other
 servers where they differ from nREPL. The scenarios aren't made up.
 Every one of them is something a server in the compatibility matrix does
-(e.g. `last-value` is what Basilisp, jank and dialtone do), or something
+(e.g. `last-value` is what Basilisp, jank and let-go do), or something
 TCP can do to any server's replies (e.g. deliver a message in pieces).
 That keeps the list down to the differences a client will actually run
 into, and the tests make sure it stays that way. proof runs its own

doc/spec-changes.md

@@ -220,16 +220,16 @@ failed evaluation doesn't include it either.
 
 ### `close`
 
-nREPL, Babashka, nbb, ClojureCLR and let-go respond with
-`session-closed`, while dialtone, Basilisp and jank respond with just
-`done` (so does the spec's example). vim-fireplace forgets a session only
+nREPL, Babashka, nbb, ClojureCLR, let-go and dialtone respond with
+`session-closed`, while Basilisp and jank respond with just `done` (so
+does the spec's example). vim-fireplace forgets a session only
 when it sees `session-closed`, and nREPL's own documentation tells
 clients to check for it, so it should be required.
 
 ### Closing an Unknown Session
 
-nREPL responds with `unknown-session`, dialtone with a plain `done`, and
-Babashka, nbb, ClojureCLR and let-go with `session-closed`. No client
+nREPL responds with `unknown-session`, and Babashka, nbb, ClojureCLR,
+let-go and dialtone with `session-closed`. No client
 cares about this, so we just need to pick one of them.
 
 ### Unknown Sessions

doc/usage.md

@@ -345,29 +345,29 @@ Running proof serve (last-value, no-err) on 127.0.0.1:7888. Connect your client
 ```
 
 With these two scenarios it sends only the value of the last form (like
-Basilisp, jank and dialtone) and drops whatever the code prints to
-stderr (like Basilisp, dialtone and repartee). `proof list` shows all
+Basilisp, jank and let-go) and drops whatever the code prints to
+stderr (like Basilisp, nbb and let-go). `proof list` shows all
 the scenarios, and every one of them is something a real server does
 (or something TCP can do to the replies):
 
 | Scenario | What changes | Who does it |
 |---|---|---|
 | `split-output` | Output comes one character per message | nREPL splits long output, Basilisp sends `println`'s newline on its own |
 | `empty-messages` | Replies to `eval` include messages with nothing but `id` and `session` | jank |
-| `last-value` | Only the value of the last form is sent | Basilisp, jank, let-go, dialtone |
-| `no-err` | What the code prints to stderr never reaches the client | Basilisp, nbb, let-go, dialtone, repartee |
+| `last-value` | Only the value of the last form is sent | Basilisp, jank, let-go |
+| `no-err` | What the code prints to stderr never reaches the client | Basilisp, nbb, let-go |
 | `error-with-done` | `eval-error` comes in the same message as `done` | jank |
 | `no-eval-error` | An error comes with `err` and `ex`, but without the `eval-error` status | nbb, let-go |
 | `ansi-errors` | Error messages are colored with ANSI escape codes | let-go |
-| `no-op-echo` | Replies to unknown ops don't say which op it was | ClojureCLR, Basilisp, jank, nbb, let-go, dialtone, repartee |
+| `no-op-echo` | Replies to unknown ops don't say which op it was | ClojureCLR, Basilisp, jank, nbb, let-go |
 | `no-close-op` | `describe` doesn't list `close`, even though `close` works | jank |
 | `no-interrupt` | There's no `interrupt` op | ClojureCLR, Basilisp, jank, nbb, and let-go doesn't list it, nor stop an eval with it |
 | `no-stdin` | There's no `stdin` op, and reading input gets `nil` right away | ClojureCLR and Basilisp started in the background, as they read their own stdin, where Basilisp gets an empty string |
 | `read-line-throws` | There's no `stdin` op, and reading input throws | jank, and nbb, which has no `read-line` |
 | `read-line-waits` | There's no `stdin` op, and reading input waits until the eval is interrupted or its session closed | let-go, which reads the terminal it runs in, where its own REPL takes the lines |
 | `string-versions` | `versions.proof` is a plain string rather than a dict | Babashka for `versions.babashka`, ClojureCLR for `versions.clojure.tools.nrepl` |
 | `no-version-string` | `versions` has `proof` and `nrepl` with the numbers of their versions as strings, but no `version-string` | let-go for `versions.let-go` and `versions.nrepl` |
-| `no-session-closed` | `close` replies with `done` alone | Basilisp, jank, dialtone, repartee |
+| `no-session-closed` | `close` replies with `done` alone | Basilisp, jank |
 | `shared-state` | Sessions on the same connection share `*1`, `*e` and the current namespace | ClojureCLR, Basilisp, jank, nbb, and let-go for the current namespace |
 | `socket-sessions` | A session only exists on the connection that cloned it | ClojureCLR, Basilisp, jank |
 | `any-session` | Requests for sessions that don't exist run in a new session | ClojureCLR, Basilisp, jank, nbb, let-go |

internal/checks/checks_test.go

@@ -119,6 +119,7 @@ func TestChecksCatchMisbehaviour(t *testing.T) {
 			"interrupt.other-connection": S, "robust.disconnect": S}},
 		{"no ephemeral sessions", quirks{noEphemeral: true}, map[string]check.Verdict{"session.ephemeral": F, "fireplace.connect": F}},
 		{"last value only", quirks{lastValueOnly: true}, map[string]check.Verdict{"eval.multiple-forms": F, "fireplace.connect": F}},
+		{"unused definitions forgotten", quirks{forgetUnused: true}, map[string]check.Verdict{"session.persistent": F}},
 		{"stderr dropped", quirks{dropErr: true}, map[string]check.Verdict{"eval.stderr": F}},
 		{"output after value", quirks{outAfterValue: true}, map[string]check.Verdict{"eval.stdout-order": W}},
 		{"no eval-error", quirks{noEvalError: true}, map[string]check.Verdict{"eval.error-status": F, "load-file.error": F}},

internal/checks/fake_test.go

@@ -27,6 +27,7 @@ type quirks struct {
 	socketSessions   bool // sessions only exist on the connection that made them
 	noEphemeral      bool // requests without a session get unknown-session
 	lastValueOnly    bool
+	forgetUnused     bool // an eval keeps only the definitions it uses, like repartee did
 	dropErr          bool
 	outAfterValue    bool
 	noEvalError      bool // failed evals get no eval-error status
@@ -566,6 +567,7 @@ func (s *fakeServer) eval(c net.Conn, req nrepl.Message, local map[string]bool)
 		}
 	}
 	var values []string
+	used := false
 	for _, form := range forms(code) {
 		value, out := "nil", ""
 		switch form {
@@ -610,8 +612,10 @@ func (s *fakeServer) eval(c net.Conn, req nrepl.Message, local map[string]bool)
 		case "define":
 			sess.vars["answer"] = "42"
 			value = "#'user/answer"
+			used = true
 		case "use":
 			value = sess.vars["answer"]
+			used = true
 		case "read":
 			if !q.noNeedInput {
 				sess.pending = req
@@ -666,5 +670,8 @@ func (s *fakeServer) eval(c net.Conn, req nrepl.Message, local map[string]bool)
 	if q.lastValueOnly && len(values) > 0 {
 		s.send(c, req, map[string]any{"value": values[len(values)-1], "ns": "user"})
 	}
+	if q.forgetUnused && !used {
+		sess.vars = map[string]string{}
+	}
 	finish()
 }

internal/checks/session.go

@@ -34,16 +34,20 @@ func sessionChecks() []*check.Check {
 			Severity: check.Fail,
 			Why:      "Every REPL workflow depends on later evals seeing earlier definitions.",
 			Refs:     []check.Ref{specClone},
-			Snippets: []string{"define"},
+			Snippets: []string{"define", "value"},
 			Requires: []string{"session.clone"},
 			Run: func(t *check.T) {
 				s := t.Snippet("define")
+				other := t.Snippet("value")
 				c := t.Connect()
 				session := t.Session(c)
 				t.Eval(c, s.Code, session)
+				// An eval in between that doesn't use the definition, as a
+				// server that keeps only what each eval touches loses it there.
+				t.Eval(c, other.Code, session)
 				resp := t.Eval(c, s.Use, session)
 				if vs := resp.Values(); !slices.Equal(vs, []string{s.Value}) {
-					t.Violatef("after evaluating %q, %q gave %q, want [%q]", s.Code, s.Use, vs, s.Value)
+					t.Violatef("after evaluating %q and then %q, %q gave %q, want [%q]", s.Code, other.Code, s.Use, vs, s.Value)
 				}
 			},
 		},

internal/serve/matrix_test.go

@@ -166,10 +166,8 @@ func TestPresetsGetTheColumnsOfTheirServers(t *testing.T) {
 			"op.unknown-echo": W, "session.isolated": F, "session.unknown": F, "session.closed": F, "eval.stderr": F,
 			"eval.error-status": F, "eval.error-report": W, "eval.multiple-forms": F, "eval.unknown-ns": F,
 			"load-file.error": F}),
-		"dialtone": checktest.Merged(noJava, noLanguage, map[string]check.Verdict{"op.unknown-echo": W, "session.close": F,
-			"eval.stderr": F, "eval.multiple-forms": F}),
-		"repartee": checktest.Merged(noJava, noLanguage, map[string]check.Verdict{"op.unknown-echo": W, "session.close": F,
-			"eval.stderr": F}),
+		"dialtone": checktest.Merged(noJava, noLanguage),
+		"repartee": checktest.Merged(noJava, noLanguage),
 	}
 	for name, want := range columns {
 		t.Run(name, func(t *testing.T) {

internal/serve/scenarios.go

@@ -57,19 +57,19 @@ var catalog = []scenario{
 		func(b *behavior) { b.splitOutput = true }, []string{"basilisp"}},
 	{Scenario{"empty-messages", "Replies to eval include messages with nothing but id and session", "jank"},
 		func(b *behavior) { b.emptyMessages = true }, []string{"jank"}},
-	{Scenario{"last-value", "Only the value of the last form is sent", "Basilisp, jank, let-go, dialtone"},
-		func(b *behavior) { b.lastValue = true }, []string{"basilisp", "jank", "let-go", "dialtone"}},
-	{Scenario{"no-err", "What the code prints to stderr never reaches the client", "Basilisp, nbb, let-go, dialtone, repartee"},
-		func(b *behavior) { b.noErr = true }, []string{"basilisp", "nbb", "let-go", "dialtone", "repartee"}},
+	{Scenario{"last-value", "Only the value of the last form is sent", "Basilisp, jank, let-go"},
+		func(b *behavior) { b.lastValue = true }, []string{"basilisp", "jank", "let-go"}},
+	{Scenario{"no-err", "What the code prints to stderr never reaches the client", "Basilisp, nbb, let-go"},
+		func(b *behavior) { b.noErr = true }, []string{"basilisp", "nbb", "let-go"}},
 	{Scenario{"error-with-done", "eval-error comes in the same message as done", "jank"},
 		func(b *behavior) { b.errorWithDone = true }, []string{"jank"}},
 	{Scenario{"no-eval-error", "An error comes with err and ex, but without the eval-error status", "nbb, let-go"},
 		func(b *behavior) { b.noEvalError = true }, []string{"nbb", "let-go"}},
 	{Scenario{"ansi-errors", "Error messages are colored with ANSI escape codes", "let-go"},
 		func(b *behavior) { b.ansiErrors = true }, []string{"let-go"}},
 	{Scenario{"no-op-echo", "Replies to unknown ops don't say which op it was",
-		"ClojureCLR, Basilisp, jank, nbb, let-go, dialtone, repartee"},
-		func(b *behavior) { b.noOpEcho = true }, []string{"clojure-clr", "basilisp", "jank", "nbb", "let-go", "dialtone", "repartee"}},
+		"ClojureCLR, Basilisp, jank, nbb, let-go"},
+		func(b *behavior) { b.noOpEcho = true }, []string{"clojure-clr", "basilisp", "jank", "nbb", "let-go"}},
 	{Scenario{"no-close-op", "describe doesn't list close, even though close works", "jank"},
 		func(b *behavior) { b.noCloseOp = true }, []string{"jank"}},
 	{Scenario{"no-interrupt", "There's no interrupt op",
@@ -89,8 +89,8 @@ var catalog = []scenario{
 	{Scenario{"no-version-string", "versions has proof and nrepl with the numbers of their versions as strings, but no version-string",
 		"let-go for versions.let-go and versions.nrepl"},
 		func(b *behavior) { b.noVersionString = true }, []string{"let-go"}},
-	{Scenario{"no-session-closed", "close replies with done alone, without session-closed", "Basilisp, jank, dialtone, repartee"},
-		func(b *behavior) { b.noSessionClosed = true }, []string{"basilisp", "jank", "dialtone", "repartee"}},
+	{Scenario{"no-session-closed", "close replies with done alone, without session-closed", "Basilisp, jank"},
+		func(b *behavior) { b.noSessionClosed = true }, []string{"basilisp", "jank"}},
 	{Scenario{"shared-state", "Sessions on the same connection share *1, *e and the current namespace",
 		"ClojureCLR, Basilisp, jank, nbb, and let-go for the current namespace"},
 		func(b *behavior) { b.sharedState = true }, []string{"clojure-clr", "basilisp", "jank", "nbb", "let-go"}},

profiles/repartee.toml

@@ -34,6 +34,10 @@ code = "proof_marker = :proof_marker"
 use = "proof_marker"
 value = ":proof_marker"
 
+[snippets.multiple]
+code = "1\n2"
+values = ["1", "2"]
+
 # Reads a line from stdin (newline included).
 [snippets.read-line]
 code = 'IO.gets("")'