TESTS.md update

docs/LEARNING.md

@@ -2,48 +2,49 @@
 
 There are many great resources for learning Factor.
 
-* [Andrea Ferretti's Factor tutorial][ferretti-tutorial] is a thorough walk-through of the language, progressing from stack basics through metaprogramming, distributed computing, and web development with Furnace. <br /> <https://andreaferretti.github.io/factor-tutorial/>
+* [Andrea Ferretti's Factor tutorial][ferretti-tutorial] is a thorough walk-through of the language, progressing from stack basics through metaprogramming, distributed computing, and web development with Furnace.
 
-* "[Factor in 2022][factor-in-2022]" is a 2022 talk by John Benediktsson and Doug Coleman that gives a tour of the language and what it's like to use it today. <br /> <https://www.youtube.com/watch?v=OLh61q4c4XE>
+* [Factor in 2022][factor-in-2022] is a 2022 talk by John Benediktsson and Doug Coleman that gives a tour of the language and what it's like to use it today.
 
-* *[Thinking Forth][thinking-forth]* by Leo Brodie is a classic on the design philosophy and habits of mind behind Forth. Factor is not Forth, but it inherits the concatenative, stack-oriented mindset, and Brodie's book is the best introduction to that way of thinking.
+* [*Thinking Forth*][thinking-forth] by Leo Brodie is a classic on the design philosophy and habits of mind behind Forth. Factor is not Forth, but it inherits the concatenative, stack-oriented mindset, and Brodie's book is the best introduction to that way of thinking.
 
-* Documentation is an important part of every good Factor vocabulary. Consequently, Factor's amazing and extensive docs are available offline, searchable right from the GUI Listener, or on the command line. These same docs, equally searchable, can be found online. <br /> <https://docs.factorcode.org>
+* [Factor's documentation][factor-docs] is amazing and extensive. It is available offline, searchable right from the GUI Listener or on the command line, and the same searchable docs are available online.
 
-* Björn Lindqvist, one of Factor's developers, has a repo full of interesting tips and tricks. It's not updated very often and is WIP (and much of its content is being moved to the GitHub Wiki), but it's still very cool stuff. <br /> <https://github.com/bjourne/playground-factor>
+* [Björn Lindqvist's playground-factor][playground-factor] is a repo full of interesting tips and tricks. One of Factor's developers, Björn doesn't update it very often and it's a WIP (much of its content is being moved to the GitHub Wiki), but it's still very cool stuff.
 
-* [Learn X in Y Minutes][learn-x-in-y-minutes] is a good resource for many languages, and Factor is no exception. Its tutorial is not *extensive* by any means, but it is a good reference and enough to get you started. <br /> <https://learnxinyminutes.com/docs/factor>
+* [Learn X in Y Minutes][learn-x-in-y-minutes] has a Factor tutorial. It is not *extensive* by any means, but it is a good reference and enough to get you started.
 
-* [Rosetta Code][rosetta-code] is a [programming chrestomathy ][chrestomathy] site. On it, among the hundreds of other languages, is [Rosetta Code:Factor][rosetta-code-factor], where you can find solutions to many problems, and compare with languages you know. It also serves as a good reference and starting point for solutions. <br /> <https://rosettacode.org/wiki/Category:Factor>
+* [Rosetta Code: Factor][rosetta-code-factor] is part of a [programming chrestomathy][chrestomathy] site. Among hundreds of other languages, it has solutions to many problems in Factor, which you can compare with languages you already know — a good reference and starting point for solutions.
 
-Lastly and maybe leastly,
+Lastly, and maybe leastly:
 
-* [Stack Overflow][stack-overflow]. There's not a large community (okay, about three users including yours truly) but ask there if you're really, truly stuck and someone will surely help you out.
+* [Stack Overflow][stack-overflow]. There's not a large community (okay, about three users including yours truly), but ask there if you're really, truly stuck and someone will surely help you out.
 
-* The [Mailing List][mailing-list]. Here, you can ask about anything Factor-related and a collaborator will answer helpfully.
+* The [mailing list][mailing-list]. Here, you can ask about anything Factor-related and a collaborator will answer helpfully.
 
 * [Factor is on IRC][irc-channel]! Come join us; we're happy to help.
 
-* Learn and read about [Joy][joy], [Forth][forth] and concatenative / stack-based programming in general.
+* Learn and read about [Joy][joy], [Forth][forth], and concatenative / stack-based programming in general.
 
 * Learn and read about the [Common Lisp Object System][clos]. The CLOS is widely regarded as the most advanced and innovative object model in the world, and Factor's object model is heavily based on and inspired by it.
 
 * In a lot of ways, Factor is a Lisp. It's a postfix, point-free, functional, inside-out-and-backwards funny-looking one, but many ideologies are the same. Because of this, it may be beneficial to learn a Lisp, preferably a Lisp-1 in which functions and variables share a namespace (because functions are values in functional programming). Yours truly humbly recommends [Scheme][scheme], or, for something more modern and usable, [Racket][racket], a descendant of Scheme.
 
 Finally, Factor is written almost entirely in Factor. So, read the source code, and search the docs for words you don't know.
 
-[learn-x-in-y-minutes]: https://learnxinyminutes.com/
-[rosetta-code]: https://rosettacode.org/wiki/Rosetta_Code
 [chrestomathy]: https://en.wikipedia.org/wiki/Chrestomathy
-[rosetta-code-factor]: https://rosettacode.org/wiki/Category:Factor
-[stack-overflow]: https://stackoverflow.com/questions/tagged/factor-lang
-[mailing-list]: https://concatenative.org/wiki/view/Factor/Mailing%20list
+[clos]: https://enwp.org/Common_Lisp_Object_System
+[factor-docs]: https://docs.factorcode.org
+[factor-in-2022]: https://www.youtube.com/watch?v=OLh61q4c4XE
+[ferretti-tutorial]: https://andreaferretti.github.io/factor-tutorial/
+[forth]: https://enwp.org/Forth_%28programming_language%29
 [irc-channel]: https://concatenative.org/wiki/view/Concatenative%20IRC%20channel
 [joy]: https://enwp.org/Joy_%28programming_language%29
-[forth]: https://enwp.org/Forth_%28programming_language%29
-[clos]: https://enwp.org/Common_Lisp_Object_System
-[scheme]: https://www.scheme.org/schemers/
+[learn-x-in-y-minutes]: https://learnxinyminutes.com/docs/factor
+[mailing-list]: https://concatenative.org/wiki/view/Factor/Mailing%20list
+[playground-factor]: https://github.com/bjourne/playground-factor
 [racket]: https://racket-lang.org/
-[ferretti-tutorial]: https://andreaferretti.github.io/factor-tutorial/
-[factor-in-2022]: https://www.youtube.com/watch?v=OLh61q4c4XE
+[rosetta-code-factor]: https://rosettacode.org/wiki/Category:Factor
+[scheme]: https://www.scheme.org/schemers/
+[stack-overflow]: https://stackoverflow.com/questions/tagged/factor-lang
 [thinking-forth]: https://www.forth.com/wp-content/uploads/2018/11/thinking-forth-color.pdf

docs/TESTS.md

@@ -1,33 +1,86 @@
-# [The `tools.test` framework](https://docs.factorcode.org/content/vocab-tools.test.html)
+# Running the tests
 
-Factor's built-in testing framework, rather like Factor itself, marries simplicity with unbridled power.
+When you download an exercise, Exercism saves it as a self-contained folder, for example `~/exercism/factor/hello-world`. That folder holds a sub-folder with your solution and another with the test runner:
 
 ```
-{ 1 } [ 1 ]             unit-test
-{   } [ "Hello" print ] unit-test ! print doesn't leave anything on the stack
-{ 3 } [ 1 2 + ]         unit-test
+hello-world/
+├── hello-world/
+│   ├── hello-world.factor        <- your solution
+│   └── hello-world-tests.factor  <- the tests
+└── exercism-tools/               <- the bundled test runner
 ```
 
-Assuming you've learned a little Factor by now, you will see that the `unit-test` word (which is actually a special syntax element) takes an array of how the stack should look after running a given quotation.
+`cd` into the top of that folder and run the tests with:
 
-Word definitions should be concise and simplified. They should not be more than 5 or 10 lines long in most cases, and their inputs and outputs should be simple and clearly understandable. Importantly, a given word should do one thing and do it well.
+```
+factor -roots=. -run=exercism-tools <exercise>
+```
+
+Replace `<exercise>` with the exercise's name (the same name as the folder). For example, to test `hello-world`:
+
+```
+cd ~/exercism/factor/hello-world
+factor -roots=. -run=exercism-tools hello-world
+```
+
+Run the command from the top of the exercise folder — the directory that holds both the `<exercise>` and `exercism-tools` sub-folders.
+
+These are the same tests Exercism runs when you submit, so once they all pass locally you are ready to submit.
+
+## One test at a time
+
+Each exercise starts with only its first test active. The remaining tests are held back by a `STOP-HERE` line in the `<exercise>-tests.factor` file; the runner ignores everything below it.
+
+```
+"year not divisible by 4 in common year" description
+{ f } [ 2015 leap-year? ] unit-test
 
-Words written in this way will implicitly be easily `unit-test`able.
+STOP-HERE
 
-Unit tests (usually a bunch of assertions like above) go in a file called `vocab-name-tests.factor` beside your implementation `vocab-name.factor`. This file is already created for you by Exercism, but would normally need to be created by hand, or by `"exercise" scaffold-tests`.
+"year divisible by 2, not divisible by 4 in common year" description
+{ f } [ 1970 leap-year? ] unit-test
+```
+
+Get the first test passing, then move `STOP-HERE` down past the next test (or delete it to enable them all) and run the tests again. Work through the file this way until every test passes.
+
+When you submit, Exercism's runner ignores `STOP-HERE` and runs every test, so make sure they all pass before submitting.
+
+## When tests pass
 
-The Factor track uses **Factor 0.101**. Each exercise ships with a small bundled `exercism-tools` vocabulary that defines `STOP-HERE` and `TASK:` parsing words and a test runner. From the exercise's directory, run:
+The runner prints each test it ran and exits quietly:
 
 ```
-factor -roots=. -run=exercism-tools <exercise-slug>
+Unit Test: { { f } [ 2015 leap-year? ] }
 ```
 
-For example, to run the `hello-world` tests:
+## When a test fails
+
+The runner shows the test, then the value it expected versus the value your code produced:
 
 ```
-factor -roots=. -run=exercism-tools hello-world
+Unit Test: { { f } [ 2015 leap-year? ] }
+--> test failed!
+###FAIL_BEGIN###
+leap/leap-tests.factor: 5
+=== Expected:
+f
+=== Got:
+t
+###FAIL_END###
 ```
 
-The runner exits with status 0 when all tests pass, and non-zero with diagnostic output when any test fails.
+Here the test ran `2015 leap-year?` and expected `f` on the stack, but got `t`.
+
+## Reading a test
+
+Tests are written with `unit-test`, which takes an array describing what the stack should look like after running a quotation:
+
+```
+{ 3 } [ 1 2 + ] unit-test
+```
+
+This runs `[ 1 2 + ]` and checks that it leaves `3` on the stack. An empty array, `{ }`, means the quotation should leave the stack empty. The optional `description` line above a test simply gives it a readable name.
+
+For more detail, see the Factor documentation on [unit testing][unit-testing].
 
-For more information, see the Factor documentation on [Unit testing](https://docs.factorcode.org/content/article-tools.test.html).
+[unit-testing]: https://docs.factorcode.org/content/article-tools.test.html

exercises/concept/annalyns-infiltration/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/backyard-birdcount/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/backyard-birdwatcher/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/belgian-boxcars/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/bering-bearings/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/boardwalk-games/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/boatswains-bilge/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?

exercises/concept/bosuns-briefing/exercism-tools/exercism-tools.factor

@@ -9,15 +9,11 @@ SYNTAX: STOP-HERE
 SYNTAX: TASK:
     lexer get next-line ;
 
-! Label the test that follows with its description. The marker lets the
-! wrapper strip this line from captured output and attach it to the next
-! test as a name, rather than leaving it in the previous test's output.
+! Label the test that follows with its description.
 : description ( str -- )
     "###DESC### " write print ;
 
-! Print one failure block in a stable, parser-friendly form. Bracketed by
-! markers so a wrapper can split the stream reliably and avoid Factor's
-! noisy callstack output (which is interleaved with subsequent failures).
+! Print one failure block in a stable, parser-friendly form.
 :: print-failure ( failure -- )
     "###FAIL_BEGIN###" print
     failure error-location print
@@ -29,6 +25,7 @@ SYNTAX: TASK:
     test-failures get [ print-failure ] each ;
 
 : run-exercism-tests ( -- )
+    vocab-roots [ "." prefix ] change-global
     command-line get first
     [ require ] [ test ] bi
     test-failures get empty?