diff --git a/.github/race-weights b/.github/race-weights index c2f6ec6a..0ba8b1a2 100644 --- a/.github/race-weights +++ b/.github/race-weights @@ -26,6 +26,7 @@ unlisted 0.58 67.9 TestADeclaredDefaultIsTheOneTheFormatUses 64.3 TestNoTrackedFileCarriesPrivateContent 63.8 TestNoGeneratorHoldsTheWholeFileInMemory +60.8 TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat 57.3 TestAFilmSaysItCarriesItsLabelOnlyWhenAPictureShowsItWhole 55.3 TestAFilmOfANamedSizeKeepsEveryPictureInsideItsReserve 53.4 TestTheArchiveScanLooksIntoTheArchivesInside diff --git a/CHANGELOG.md b/CHANGELOG.md index 58e84e11..2749c07b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,14 @@ because it turns other people's test suites red. ### Added +- **Two more ways to break a file on purpose: `truncate` and `append-junk`.** + `--damage truncate` cuts the end off a file, the way an interrupted upload + leaves it, and `--damage append-junk` writes random bytes after its end. + Both keep exactly the size you asked for and work with every format. Where + readers differ - a text cut short is still text - the manifest says the + outcome is `unspecified`, offers both answers in the new `policy_options` + field and says why. `tfg damage ` lists those formats. + - **tfg is on npm, for tests in Node.** `npm install --save-dev testing-files-generator` gives Playwright, Vitest and `node:test` a generated set of files, and says what the system under @@ -65,6 +73,12 @@ because it turns other people's test suites red. ### Changed +- **`expected: accept` beside a damage is allowed where the outcome depends + on your system.** It used to be refused beside every damage. It is still + refused where the damage breaks the rules of the format, because nothing + that follows them accepts the file, and both the command line and a recipe + answer the same. + - **The Windows archive of the window carries Mesa 26.2.4.** The software OpenGL renderer in the `opengl` folder, which the window loads only after the graphics toolkit could not open a window, or when it is started with diff --git a/README.md b/README.md index 9ca979da..779652bf 100644 --- a/README.md +++ b/README.md @@ -482,29 +482,48 @@ the tool holds in memory has a ceiling whatever the archive holds. ``` tfg damage [--json] every damage, with the smallest file it takes and its settings -tfg damage what one damage does to the bytes, and what it takes +tfg damage what one damage does to the bytes, what it takes, and where the + outcome depends on your system ``` -A damaged file comes out **exactly** the size you asked for and is one a reader -refuses, so the manifest records it as expected to be rejected. That is the third +A damaged file comes out **exactly** the size you asked for. That is the third question an upload validator asks - does it open - and until damage arrived every -file this tool wrote was well formed by definition. +file this tool wrote was well formed by definition. Three damages work with every +format: + +| damage | what it does | +|---|---| +| `zero-head` | overwrites the first bytes with zeros (`bytes`, 8 unless you say otherwise) | +| `truncate` | cuts the end off, the way an interrupted upload leaves a file (`missing`, the percent of the whole file that is gone, 50 unless you say otherwise) | +| `append-junk` | writes random bytes after the end (`bytes`, 64 unless you say otherwise) | + +Where the damage breaks the rules of the format, the manifest records the file as +expected to be rejected, with certainty. Where readers differ - a text cut short is +still text, and most viewers skip bytes after the end of a PNG - it says the outcome +is `unspecified` and depends on your system, offers both answers in +`policy_options`, and says why in `detail`. `tfg damage ` lists those formats, +and `tfg damage --json` carries them as `policy_dependent`. Use them with `tfg generate --damage `, repeatable and applied in the order given, or with the `damage` key of a target in a recipe: ``` tfg generate --format png --size 20kb --count 10 --damage zero-head:bytes=16 +tfg generate --format zip --size 5mb --damage truncate:missing=30 ``` The smallest file a damage can be given follows its settings, so the column is measured with the defaults. Ask for less and the run is refused before anything is written, naming a size that would work. -Asking for `--expected accept` beside a damage is refused too, because nothing -could meet it. Write `sanitize` if the system under test is meant to repair the -file, or `unspecified` if that is the question you are asking - both of those, -and `reject`, work as they always did. +Asking for `--expected accept` beside a damage that breaks the rules of the +format is refused too, because nothing that follows them accepts the file. Where +the outcome depends on your system, `accept` is allowed and records your policy. +Write `sanitize` if the system under test is meant to repair the file, or +`unspecified` if that is the question you are asking - both of those, and +`reject`, work as they always did. `truncate` placed after `append-junk` is +refused, because it would cut away the bytes that were added, and so is a damage +that changes the length of an archive that takes its size from what it holds. ## 📜 Recipes @@ -948,8 +967,10 @@ rounding are the two things this tool will not do. Yes. Add `--damage zero-head` and the file comes out at exactly the size you asked for with its first bytes overwritten by zeros, so a reader turns it away, -and the manifest says your system should reject it. [`tfg damage`](#tfg-damage) -lists what this build can do and what each damage takes. +and the manifest says your system should reject it. `--damage truncate` cuts the +end off and `--damage append-junk` writes bytes after it, still at the size you +asked for. [`tfg damage`](#tfg-damage) lists what this build can do, what each +damage takes, and where the outcome depends on your system. ``` tfg generate --format png --size 2mb --damage zero-head --out ./out diff --git a/internal/cli/damagecmd.go b/internal/cli/damagecmd.go index bf628358..87227d4a 100644 --- a/internal/cli/damagecmd.go +++ b/internal/cli/damagecmd.go @@ -5,6 +5,7 @@ import ( "flag" "fmt" "io" + "slices" "strings" "github.com/donislawdev/TestingFilesGenerator/internal/core" @@ -32,6 +33,24 @@ type damageEntry struct { // took 3286. SmallestFileBytes int64 `json:"smallest_file_bytes"` Parameters []propertyEntry `json:"parameters,omitempty"` + // PolicyDependent is every format where the manifest says the outcome + // of this damage depends on the reader, with why. + // + // Always present, empty for a damage that breaks the rules of every + // format, so a script reads one key for every damage rather than asking + // whether it is there. Owner's call on 2026-11-10: every pair makes a + // file, and this is how a script learns before generating which files the + // manifest will be certain about. + PolicyDependent []policyEntry `json:"policy_dependent"` +} + +// policyEntry is one format where a damage leaves the outcome to the reader. +type policyEntry struct { + Format string `json:"format"` + // Settings narrows the entry to files whose setting holds one of these + // values. Absent when it covers every file of the format. + Settings map[string][]string `json:"settings,omitempty"` + Reason string `json:"reason"` } // damageEntryFor is one damage as a script sees it. @@ -43,10 +62,43 @@ func damageEntryFor(d damage.Descriptor) damageEntry { Unit: p.Unit, Choices: p.Choices, Default: p.Default, Detail: p.Detail, }) } + policy := make([]policyEntry, 0, len(d.Policy)) + for _, c := range d.Policy { + policy = append(policy, policyEntry{Format: c.Format, Settings: c.When, Reason: c.Why}) + } return damageEntry{ ID: d.ID, Detail: d.Detail, - SmallestFileBytes: defaultFloor(d), Parameters: params, + SmallestFileBytes: defaultFloor(d), Parameters: params, PolicyDependent: policy, + } +} + +// policyFormats is the formats where a damage leaves the outcome to the +// reader, as a person reads them in a list: a format narrowed to some of its +// settings says which. +func policyFormats(d damage.Descriptor) string { + names := make([]string, 0, len(d.Policy)) + for _, c := range d.Policy { + names = append(names, c.Format+settingsNarrowing(c.When)) + } + return strings.Join(names, ", ") +} + +// settingsNarrowing is " (encoding utf-16be or utf-16le)" for a case narrowed +// to some values of a setting, and nothing for one that covers the format. +func settingsNarrowing(when map[string][]string) string { + if len(when) == 0 { + return "" + } + names := make([]string, 0, len(when)) + for name := range when { + names = append(names, name) + } + slices.Sort(names) + parts := make([]string, 0, len(names)) + for _, name := range names { + parts = append(parts, name+" "+strings.Join(when[name], " or ")) } + return " (" + strings.Join(parts, ", ") + ")" } // defaultFloor is the smallest file this damage takes when nobody states a @@ -87,6 +139,21 @@ func describeOneDamage(d damage.Descriptor, out io.Writer) { d.ID, core.ExactBytes(defaultFloor(d))) fmt.Fprintf(out, " %s\n", d.Detail) + describeDamageSettings(d, out) + + if len(d.Policy) == 0 { + fmt.Fprint(out, "\nIn every format the manifest expects the damaged file to be rejected.\n") + return + } + fmt.Fprint(out, "\nThe manifest expects the damaged file to be rejected, except in these formats,\nwhere it says the outcome depends on your system:\n") + for _, c := range d.Policy { + fmt.Fprintf(out, " %s\n %s\n", c.Format+settingsNarrowing(c.When), c.Why) + } +} + +// describeDamageSettings prints what one damage takes, with the flag a person +// would type. +func describeDamageSettings(d damage.Descriptor, out io.Writer) { if len(d.Parameters) == 0 { fmt.Fprint(out, "\nThis damage takes no settings.\n") return @@ -122,17 +189,22 @@ func listDamages(out io.Writer) { // reads "zero-head" and knows what comes out - so a list of names // would say no more than the --damage flag help already says. fmt.Fprintf(out, " %s\n", d.Detail) + if len(d.Policy) > 0 { + fmt.Fprintf(out, " Depends on your system in: %s\n", policyFormats(d)) + } } - fmt.Fprint(out, "\nRun \"tfg damage \" for what one damage takes.\n") - fmt.Fprint(out, "The smallest file is what the default settings need.\n") + fmt.Fprint(out, "\nRun \"tfg damage \" for what one damage takes and why the outcome depends on your system where it does.\n") + fmt.Fprint(out, "The smallest file is what the default settings need. A format can need more, because some damages start from a larger file.\n") } func damageUsage(w io.Writer, fs *flag.FlagSet, errOut io.Writer) { fmt.Fprint(w, `tfg damage - list the ways this build can break a file on purpose. -A damaged file comes out exactly the size you asked for and is one a reader -refuses, so the manifest records it as expected to be rejected. Use them with -"tfg generate --damage " or with the damage key of a recipe. +A damaged file comes out exactly the size you asked for. Where the damage +breaks the rules of the format, the manifest expects the file to be rejected. +Where readers differ - a text cut short is still text - it says the outcome +depends on your system, and why. Use them with "tfg generate --damage " +or with the damage key of a recipe. Usage: tfg damage diff --git a/internal/cli/errors.go b/internal/cli/errors.go index c9afdf89..660226d4 100644 --- a/internal/cli/errors.go +++ b/internal/cli/errors.go @@ -276,6 +276,18 @@ func classifyRequest(err error) (int, bool) { if errors.As(err, &impossible) { return ExitUsage, true } + // The same class, for the same reason: a list of damages that cuts away + // what it added, and a damage that changes the length of an archive sized + // by what it holds, are wrong whatever format or size is asked for. The fix + // is in the invocation - the order of the flags, or a size. + var eaten *damage.EatenError + if errors.As(err, &eaten) { + return ExitUsage, true + } + var unsized *damage.NeedsSizeError + if errors.As(err, &unsized) { + return ExitUsage, true + } if code, ok := classifyFormat(err); ok { return code, true } diff --git a/internal/core/seed.go b/internal/core/seed.go index 20244b0f..d18d2766 100644 --- a/internal/core/seed.go +++ b/internal/core/seed.go @@ -74,6 +74,24 @@ func SizeSeed(targetSeed uint64, index int) uint64 { return binary.BigEndian.Uint64(h.Sum(nil)[:8]) } +// DamageSeed derives the seed one damage uses for one file, from the file's +// seed and the damage's place in its chain. +// +// Its own separator byte, for the reason SizeSeed has one: the bytes a damage +// writes and the bytes the generator wrote are two questions about the same +// file, and drawing both from one state would tie the junk appended to a file +// to the content before it. The place in the chain is what gives two of the +// same damage on one file two different results. +func DamageSeed(fileSeed uint64, place int) uint64 { + h := sha256.New() + var buf [8]byte + binary.BigEndian.PutUint64(buf[:], fileSeed) + h.Write(buf[:]) + h.Write([]byte{2}) + h.Write([]byte(strconv.Itoa(place))) + return binary.BigEndian.Uint64(h.Sum(nil)[:8]) +} + // NewRand returns the random source for one file. // // It is deterministic given the seed and it is not shared between files. A diff --git a/internal/damage/appendjunk.go b/internal/damage/appendjunk.go new file mode 100644 index 00000000..ed896fa3 --- /dev/null +++ b/internal/damage/appendjunk.go @@ -0,0 +1,140 @@ +package damage + +import ( + "encoding/binary" + "io" + "strconv" + + "github.com/donislawdev/TestingFilesGenerator/internal/core" + "github.com/donislawdev/TestingFilesGenerator/internal/format" +) + +// AppendJunk is the damage that writes bytes after the end of a file. A public +// name under untouchable rule 10. Its setting is bytes, the word zero-head +// already uses for how many bytes a damage touches. +const AppendJunk = "append-junk" + +const ( + // smallestJunk is the fewest bytes worth appending, and it is a + // measurement. + // + // Measured 2026-11-09 across 31 formats: one random byte is noticed by 21, + // sixteen by 25, and from 64 up the set stays the same at every size tried + // up to 1 MiB (docs/DAMAGE-ETAP-2-2026-11-09.md section 2.1). What the + // bytes ARE matters as much: zeros are taken for padding by PDF, AVIF and + // JPEG XL, so the bytes are random. + smallestJunk = 64 + + // largestJunk is where the setting stops. Chosen rather than measured: a + // range with no top draws a field anything can be typed into, and 64 KiB + // is far past any length a format reads its end markers from. + largestJunk = 65536 + + defaultJunk = smallestJunk + + // junkChunk is how much junk is made and written at a time, so 64 KiB of + // it is never held at once. A multiple of eight, because the bytes come + // from 64 bit draws and a chunk boundary inside one would change them. + junkChunk = 8192 +) + +func init() { + Register(Descriptor{ + ID: AppendJunk, + Detail: "Writes random bytes after the end of the file and keeps exactly the size you asked for, " + + "the way a file with something stuck to its end arrives.", + Parameters: []format.Property{ + { + Name: SettingBytes, Kind: format.PropertyInt, + Min: smallestJunk, Max: largestJunk, Unit: "bytes", + Default: strconv.Itoa(defaultJunk), + Detail: "How many random bytes come after the end. " + + "Below 64, some formats come out with damage no reader complains about.", + }, + }, + Floor: func(v Values) int64 { return int64(junkBytes(v)) }, + Length: func(v Values, out int64) int64 { return out - int64(junkBytes(v)) }, + AddsTail: true, + Policy: appendPolicy, + Open: func(v Values, f File, out io.Writer) (Stream, error) { + return &junkTail{out: out, seed: f.Seed, n: junkBytes(v)}, nil + }, + }) +} + +// appendPolicy is where bytes after the end of a file leave its fate to the +// reader. +// +// Measured on 2026-11-10 with every setting of every format, with the reason +// of each refusal read (docs/DAMAGE-ETAP-2-2026-11-09.md sections 12.2 and +// 12.3). Two kinds stand here. Bytes past the format's own end - after a +// PNG's IEND, a ZIP's last record, an RTF's closing brace - break a rule most +// readers do not enforce. Bytes the format allows - a TIFF or an icon is read +// through offsets, so bytes nothing points at are legal - break nothing. +// Everywhere else the bytes make something the file says about itself false: +// a length in its header, the encoding of its text, the boxes it is made of. +var appendPolicy = []PolicyCase{ + {Format: "png", Why: "A PNG ends at its IEND chunk and most viewers skip what follows it."}, + {Format: "gif", Why: "A GIF ends at its trailer and most viewers skip what follows it."}, + {Format: "jpg", Why: "A JPEG ends at its end of image marker, and cameras and editors often leave data after it."}, + {Format: "pdf", Why: "Readers look for the end of a PDF near the end of the file and skip what follows it."}, + {Format: "rtf", Why: "An RTF document ends at its closing brace and most readers skip what follows it."}, + {Format: "zip", Why: "Some ZIP readers skip bytes after the last record of an archive and others warn about them."}, + {Format: "docx", Why: "A Word document is a ZIP package, and some readers skip bytes after its last record while others refuse the file."}, + {Format: "xlsx", Why: "An Excel workbook is a ZIP package, and some readers skip bytes after its last record while others refuse the file."}, + {Format: "pptx", Why: "A PowerPoint presentation is a ZIP package, and some readers skip bytes after its last record while others refuse the file."}, + {Format: "targz", Why: "Some gzip readers skip bytes after the end of the stream with a warning and others refuse the archive."}, + {Format: "ico", Why: "An icon is read through the offsets in its directory, so bytes nothing points at are allowed."}, + {Format: "tiff", Why: "A TIFF is read through the offsets in its directories, so bytes nothing points at are allowed."}, + {Format: "eml", Why: "Whether bytes after the end of a message break it depends on how its body is encoded, and mail readers skip a part they cannot place."}, + {Format: "txt", When: map[string][]string{"encoding": {"utf-16be", "utf-16le"}}, + Why: "In UTF-16 most random bytes are valid characters, so the file can still be valid text."}, + {Format: "md", When: map[string][]string{"encoding": {"utf-16be", "utf-16le"}}, + Why: "In UTF-16 most random bytes are valid characters, so the document can still be valid text."}, +} + +// junkBytes is how many bytes this run appends, falling back for the reason +// headBytes gives. +func junkBytes(v Values) int { + n, err := strconv.Atoi(v[SettingBytes]) + if err != nil || n < smallestJunk || n > largestJunk { + return defaultJunk + } + return n +} + +// junkTail passes the file through untouched and writes the junk once it has +// ended. +type junkTail struct { + out io.Writer + seed uint64 + n int + // wrote records that the junk went out. Junk is never empty, so a junk + // tail that wrote is one that changed the file. + wrote bool +} + +func (j *junkTail) Write(p []byte) (int, error) { return j.out.Write(p) } + +func (j *junkTail) Touched() bool { return j.wrote } + +// Finish writes the junk: the 64 bit draws of this damage's own source, eight +// bytes each, least significant first. Frozen from the first release by a +// golden hash (D11), because changing how the bytes are drawn changes every +// file this damage ever wrote. +func (j *junkTail) Finish() error { + r := core.NewRand(j.seed) + buf := make([]byte, min(j.n, junkChunk)+8) + for left := j.n; left > 0; { + size := min(left, junkChunk) + for i := 0; i < size; i += 8 { + binary.LittleEndian.PutUint64(buf[i:], r.Uint64()) + } + if _, err := j.out.Write(buf[:size]); err != nil { + return err + } + left -= size + j.wrote = true + } + return nil +} diff --git a/internal/damage/apply.go b/internal/damage/apply.go index 42875eb2..5ee6f871 100644 --- a/internal/damage/apply.go +++ b/internal/damage/apply.go @@ -5,6 +5,7 @@ import ( "io" "sort" + "github.com/donislawdev/TestingFilesGenerator/internal/core" "github.com/donislawdev/TestingFilesGenerator/internal/format" ) @@ -111,58 +112,208 @@ func (d Descriptor) Check(v Values) error { // those bytes do not move. type Chain []Spec -// Floor is the smallest file every damage in the chain can be applied to. -// -// The largest of the floors rather than their sum: each damage is applied to -// the whole file that reaches it, so a chain is bounded by its most demanding -// member. That is true for length preserving damage, which is all this build -// has - a length CHANGING damage makes the later members see a different file, -// and that arithmetic arrives with truncate rather than being guessed at here. -func (c Chain) Floor() (int64, string, error) { - var floor int64 - var owner string +// step is one entry of a chain with its declaration and its settings resolved, +// which every question below needs and none should ask the registry twice for. +type step struct { + d Descriptor + values Values +} + +func (c Chain) steps() ([]step, error) { + out := make([]step, 0, len(c)) for _, s := range c { d, err := Get(s.ID) if err != nil { - return 0, "", err + return nil, err + } + out = append(out, step{d: d, values: withDefaults(d, s.Values)}) + } + return out, nil +} + +// lengths is how many bytes leave each damage for a file of size bytes on +// disk, in the order of the chain, and how many the generator writes. ok is +// false when some damage would be left fewer bytes than its floor, or handed +// fewer than none. +// +// Worked from the END of the chain, because the size somebody asked for is +// what leaves the last damage, and each damage says how much it needs to be +// given for what it hands on. +func lengths(steps []step, size int64) (outs []int64, generator int64, ok bool) { + outs = make([]int64, len(steps)) + at := size + ok = true + for i := len(steps) - 1; i >= 0; i-- { + outs[i] = at + if at < steps[i].d.Floor(steps[i].values) { + ok = false + } + if steps[i].d.Length != nil { + at = steps[i].d.Length(steps[i].values, at) + } + if at < 0 { + ok = false + at = 0 + } + } + return outs, at, ok +} + +// lengthCeiling is where the search for a floor stops. Far past any size a +// run accepts, and far enough below math.MaxInt64 that doubling towards it +// cannot wrap. +const lengthCeiling = int64(1) << 60 + +// Floor is the smallest file on disk every damage in the chain can be applied +// to, and the damage that sets it. +// +// For a chain whose damages keep the length this is the largest of their +// floors, which is what it was before lengths could change. A damage that +// changes the length makes the ones before it see a different file, so the +// answer is searched for rather than added up: what leaves each damage grows +// with the size asked for, so the smallest size that leaves every damage its +// floor is one number, found by halving. +func (c Chain) Floor() (int64, string, error) { + steps, err := c.steps() + if err != nil || len(steps) == 0 { + return 0, "", err + } + if _, _, ok := lengths(steps, 0); ok { + return 0, "", nil + } + lo, hi := int64(0), int64(1) + for { + if _, _, ok := lengths(steps, hi); ok { + break + } + if hi >= lengthCeiling { + return lengthCeiling, steps[len(steps)-1].d.ID, nil + } + lo, hi = hi, hi*2 + } + // lo fails and hi passes, so the floor is in (lo, hi]. + for hi-lo > 1 { + mid := lo + (hi-lo)/2 + if _, _, ok := lengths(steps, mid); ok { + hi = mid + } else { + lo = mid + } + } + return hi, ownerOfFloor(steps, lo), nil +} + +// ownerOfFloor is the damage that refuses a file of size bytes - the one a +// refusal names. The last one that does, because a refusal speaks about the +// size somebody wrote, and the last damage is the one that sees it. +func ownerOfFloor(steps []step, size int64) string { + outs, _, _ := lengths(steps, size) + owner := steps[len(steps)-1].d.ID + for i := len(steps) - 1; i >= 0; i-- { + if outs[i] < steps[i].d.Floor(steps[i].values) { + return steps[i].d.ID + } + } + return owner +} + +// Before is how many bytes the generator writes for a file of size bytes on +// disk. It refuses a size below the chain's floor, in the same words the plan +// does. +func (c Chain) Before(size int64) (int64, error) { + steps, err := c.steps() + if err != nil { + return 0, err + } + _, generator, ok := lengths(steps, size) + if !ok { + floor, owner, err := c.Floor() + if err != nil { + return 0, err } - if f := d.Floor(withDefaults(d, s.Values)); f > floor { - floor, owner = f, s.ID + return 0, &TooSmallError{Damage: owner, Requested: size, Floor: floor} + } + return generator, nil +} + +// ChangesLength says whether the file the generator writes is a different +// length from the one on disk. +func (c Chain) ChangesLength() bool { + steps, err := c.steps() + if err != nil { + return false + } + for _, s := range steps { + if s.d.Length != nil { + return true } } - return floor, owner, nil + return false } -// Open builds the chain as one writer, with the first damage of the list -// outermost so that the bytes meet them in the order written. +// Stretches says the generator may write MORE than Before asks for and the +// file on disk still comes out the size asked for. +// +// True when a damage cuts the tail and nothing before it changes the length: +// whatever the generator writes past the cut is dropped, so a size the format +// cannot write exactly - a band a PNG cannot reach, an odd number in UTF-16 - +// can be moved up to one it can. That is what makes truncate's missing an +// "at least" (owner's call, 2026-11-10). +func (c Chain) Stretches() bool { + steps, err := c.steps() + if err != nil { + return false + } + for _, s := range steps { + if s.d.CutsTail { + return true + } + if s.d.Length != nil { + return false + } + } + return false +} + +// Open builds the chain as one writer for a file of size bytes on disk, with +// the first damage of the list outermost so that the bytes meet them in the +// order written. seed is the file's own seed, and each damage is handed one +// derived from it and its place in the list. // // Returns the streams as well, because whether each one changed anything is a // question asked per damage rather than once at the end - two damages can // cancel out, and a chain whose second member was idle would otherwise pass. -func (c Chain) Open(out io.Writer) (io.Writer, []Stream, error) { +func (c Chain) Open(out io.Writer, size int64, seed uint64) (io.Writer, []Stream, error) { + steps, err := c.steps() + if err != nil { + return nil, nil, err + } + outs, _, _ := lengths(steps, size) w := out - streams := make([]Stream, 0, len(c)) + streams := make([]Stream, len(steps)) // Built back to front so that the first entry ends up outermost. - for i := len(c) - 1; i >= 0; i-- { - d, err := Get(c[i].ID) - if err != nil { - return nil, nil, err - } - s, err := d.Open(withDefaults(d, c[i].Values), w) + for i := len(steps) - 1; i >= 0; i-- { + s, err := steps[i].d.Open(steps[i].values, File{Seed: core.DamageSeed(seed, i), Bytes: outs[i]}, w) if err != nil { return nil, nil, err } - streams = append(streams, s) + streams[i] = s w = s } - // Back into the order of the list, so a caller reporting on stream i is - // reporting on entry i. - for l, r := 0, len(streams)-1; l < r; l, r = l+1, r-1 { - streams[l], streams[r] = streams[r], streams[l] - } return w, streams, nil } +// Finish ends every damage of the chain in the order of the list, after the +// generator's last write. +func (c Chain) Finish(streams []Stream) error { + for _, s := range streams { + if err := s.Finish(); err != nil { + return err + } + } + return nil +} + // Idle is the first damage of the chain that changed nothing, by name, or an // empty string when every one of them moved a byte. func (c Chain) Idle(streams []Stream) string { diff --git a/internal/damage/damage.go b/internal/damage/damage.go index 7ea1fe81..0122088a 100644 --- a/internal/damage/damage.go +++ b/internal/damage/damage.go @@ -16,20 +16,31 @@ // format agnostic damages produce 86 testable pairs with zero lines of code // per format. docs/CORRUPTION-ARCHITECTURE-2026-09-08.md carries the numbers. // -// What this package does NOT do is decide whether a damaged file is worth -// writing. That is the witness rule - a pair with no judge able to refuse the -// result is a pair this tool does not offer - and it belongs with the oracles, -// because it is a measurement rather than a declaration. The guard -// TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat takes that -// measurement for every format and every damage registered here. +// Every damage is applied to every format. What the witness rule decides is +// what the manifest may SAY about the result, not whether the file is written: +// a file that breaks its format's own rules is one a reader must refuse, and +// the manifest says reject with certainty. A file whose fate depends on the +// reader - a text cut short is still text, bytes after a PNG's end are skipped +// by most viewers - is a fair question to put to a system under test, and the +// manifest says so instead of guessing. Owner's call on 2026-11-10, after +// measuring both: refusing such a file took away exactly the test somebody +// came here to run (docs/DAMAGE-ETAP-2-2026-11-09.md section 12.9). +// +// Which pairs are which is declared here, per damage (Descriptor.Policy), and +// measured by the guard TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat +// for every format and every damage registered here: a pair declared certain +// that the structural check accepts is red, and so is a pair declared +// policy-dependent that the check refuses for a reason nobody has looked at. package damage import ( + "errors" "fmt" "io" "sort" "sync" + "github.com/donislawdev/TestingFilesGenerator/internal/core" "github.com/donislawdev/TestingFilesGenerator/internal/format" ) @@ -63,6 +74,62 @@ type Stream interface { // overwrites them, so the answer is a handful of byte compares. That is // why this sits on the damage rather than on the engine. Touched() bool + + // Finish is called once, after the generator's last write, in the order of + // the chain. A damage that writes after the end of the file writes there, + // and every other one returns nil. + // + // After the last write rather than on a close, because the bytes it adds + // have to pass through the damages that come after it in the chain - and + // the first of the list is outermost, so finishing in list order is what + // sends them through the rest. + Finish() error +} + +// ErrKeptEnough is what a damage that cuts the file short returns once it has +// passed on every byte it keeps. +// +// The generator stops writing on it, which is the whole point: cutting 90% of +// a file would otherwise cost generating all of it while the progress bar +// stood at the end. The engine reads it as the file being complete. The bytes +// kept do not depend on whether the rest was ever generated, because a +// generator writes from the front, so stopping early changes no byte that +// reaches the disk - the guard TestATruncatedFileIsTheStartOfTheWholeOne holds +// that byte for byte. +// +// A defect if it ever reaches a person: the engine reads it as success, so +// only a fault in the program could let it out. +var ErrKeptEnough = core.Defect(errors.New("damage: the file has every byte it keeps")) + +// File is what one damage is told about the file it is breaking. +type File struct { + // Seed is this damage's own seed for this file - derived from the file's + // seed and the damage's place in the chain, never drawn from a stream, so + // a damage that writes bytes of its own writes the same bytes for the same + // recipe (RC4, D11). + Seed uint64 + // Bytes is how many bytes leave this damage. + Bytes int64 +} + +// PolicyCase is a format where this damage leaves a file whose fate depends on +// the reader, so the manifest says unspecified rather than reject. +// +// Three shapes of it were measured, and each case says which in Why: the +// damage breaks nothing the format's rules forbid (a text cut short is still +// text), the only thing broken is the shape this tool writes (a CSV with no +// line break after the last row, which RFC 4180 allows), or the bytes sit +// after the format's own end, where the rules forbid them and readers skip +// them (data after a PNG's IEND). +type PolicyCase struct { + // Format is the id of the format. + Format string + // When narrows the case to files whose setting holds one of the values, + // by setting name. Empty means every file of the format. + When map[string][]string + // Why is the sentence the manifest carries as the detail of its + // expectation, and tfg damage --json as the reason. + Why string } // Descriptor is one kind of damage. @@ -89,7 +156,7 @@ type Descriptor struct { // here. preset.Preset.Parameters is the same type for the same reason. Parameters []format.Property - // Floor is the smallest file this damage can be given, for the values + // Floor is the fewest bytes this damage can leave behind, for the values // stated. // // It exists because a file can be smaller than its own damage. Measured on @@ -98,13 +165,43 @@ type Descriptor struct { // has to be a refusal in the PLAN rather than a failure part way through, // because an invalid recipe writes no files at all. // + // Counted on what LEAVES the damage, because that is the one length every + // damage can state about itself: zero-head needs as many bytes as it + // zeroes, truncate needs a file long enough that its cut is at least four + // bytes, and append-junk needs room for what it appends. + // // It is also what the witness matrix is measured at, which is the whole // reason one number does two jobs: a witness measured at 20 kB does not // answer for a file of 300 B, and that was measured too. Floor func(Values) int64 + // Length is how many bytes have to come into this damage for out bytes to + // leave it. Nil for a damage that keeps the length, which is what makes a + // chain of those exactly what it was before lengths could change. + // + // Saturating rather than failing: a result past what an int64 holds comes + // back as math.MaxInt64, and the format it is asked of refuses that size + // in its own words. + Length func(v Values, out int64) int64 + + // CutsTail says this damage keeps only the first bytes of what reaches it, + // so it takes ANY input at least Length(out) long. That is what lets a + // size the generator cannot write exactly be moved up to one it can. + CutsTail bool + + // AddsTail says this damage writes bytes after everything that reached it. + // A damage that cuts the tail must not come after one, or the cut takes + // away what was added and the manifest lists a damage the file does not + // have. + AddsTail bool + + // Policy is where this damage leaves a file whose fate depends on the + // reader. Everywhere else the result breaks its format's rules and the + // manifest says reject with certainty. + Policy []PolicyCase + // Open builds the transformation for one file, writing on to out. - Open func(Values, io.Writer) (Stream, error) + Open func(Values, File, io.Writer) (Stream, error) } // Spec is one entry of a damage list: which damage, and what was said to it. @@ -140,6 +237,14 @@ func Register(d Descriptor) { if d.Detail == "" { panic(fmt.Sprintf("damage: %q has no sentence describing it", d.ID)) } + if d.CutsTail && d.Length == nil { + panic(fmt.Sprintf("damage: %q cuts the tail and does not say how long a file it needs", d.ID)) + } + for _, c := range d.Policy { + if c.Format == "" || c.Why == "" { + panic(fmt.Sprintf("damage: %q declares a policy case with no format or no reason", d.ID)) + } + } for i := range d.Parameters { format.SortChoices(d.Parameters[i].Choices) } diff --git a/internal/damage/policy.go b/internal/damage/policy.go new file mode 100644 index 00000000..f7b92616 --- /dev/null +++ b/internal/damage/policy.go @@ -0,0 +1,84 @@ +package damage + +import ( + "slices" + + "github.com/donislawdev/TestingFilesGenerator/internal/format" +) + +// Kind is the format of a file and every one of its settings, defaults filled +// in - what a chain is asked about when the manifest needs to know whether the +// result breaks its format's rules. +// +// Defaults filled in rather than the settings as stated, because a policy case +// narrowed to one value has to match a file that got that value by default as +// surely as one that asked for it. +type Kind struct { + Format string + Settings map[string]string +} + +// KindOf is a format with the settings stated for it, defaults filled in. +func KindOf(d format.Descriptor, stated map[string]string) Kind { + settings := make(map[string]string, len(d.Properties)) + for _, p := range d.Properties { + if p.Default != "" { + settings[p.Name] = p.Default + } + } + for name, value := range stated { + if value != "" { + settings[name] = value + } + } + return Kind{Format: d.ID, Settings: settings} +} + +// matches says whether this case covers a file of this kind. +func (c PolicyCase) matches(k Kind) bool { + if c.Format != k.Format { + return false + } + for name, values := range c.When { + if !slices.Contains(values, k.Settings[name]) { + return false + } + } + return true +} + +// PolicyFor is the case covering this damage on a file of this kind, or false +// when the damage breaks the format's rules there. +func (d Descriptor) PolicyFor(k Kind) (PolicyCase, bool) { + for _, c := range d.Policy { + if c.matches(k) { + return c, true + } + } + return PolicyCase{}, false +} + +// Certain says whether the chain leaves a file of this kind that its format's +// rules forbid. When it does not, why is the sentence the manifest carries. +// +// Certain as soon as ONE damage is, because what it broke stays broken after +// the next: a file cut short after its head was zeroed still has no +// signature. Measured per pair rather than per chain - the witness guard runs +// each damage on its own - and that limit is written down in +// docs/DAMAGE-ETAP-2-2026-11-09.md section 12.9. +func (c Chain) Certain(k Kind) (certain bool, why string, err error) { + steps, err := c.steps() + if err != nil { + return false, "", err + } + for _, s := range steps { + pc, depends := s.d.PolicyFor(k) + if !depends { + return true, "", nil + } + if why == "" { + why = pc.Why + } + } + return false, why, nil +} diff --git a/internal/damage/refusals.go b/internal/damage/refusals.go index 84f26838..26148559 100644 --- a/internal/damage/refusals.go +++ b/internal/damage/refusals.go @@ -174,8 +174,14 @@ const RuledOutExpectation = "accept" // four case program - "reject", "sanitize", "" and "accept" all came back // err != nil - so the engine would have refused EVERY target, damaged or not, // and the guard beside this one asserts exactly that it does not. -func (c Chain) ConflictsWithExpectation(expected string) error { - if bad := c.ExpectationConflict(expected); bad != nil { +// +// Only where the chain breaks the rules of this kind of file. Where the +// manifest says the outcome depends on the reader - a text cut short, bytes +// after a PNG's end - accepting the file is one of the two answers it offers, +// so a recipe stating it is writing down its own policy. Owner's call on +// 2026-11-10, by the rule of 2026-09-09: refuse only what cannot be true. +func (c Chain) ConflictsWithExpectation(expected string, k Kind) error { + if bad := c.ExpectationConflict(expected, k); bad != nil { return bad } return nil @@ -183,13 +189,150 @@ func (c Chain) ConflictsWithExpectation(expected string) error { // ExpectationConflict is the same question answered with the refusal itself, // or nil. For a caller that needs the parts rather than an error. -func (c Chain) ExpectationConflict(expected string) *ExpectationConflictError { +func (c Chain) ExpectationConflict(expected string, k Kind) *ExpectationConflictError { if len(c) == 0 || expected != RuledOutExpectation { return nil } + // An error here is a damage the registry does not know, and that is + // refused on its own in both surfaces, in its own words. + if certain, _, err := c.Certain(k); err != nil || !certain { + return nil + } return &ExpectationConflictError{Outcome: expected} } +// Eaten is the refusal a chain earns by cutting the tail after a damage that +// added one, or nil. +// +// Asked in the plan because nothing later can see it. Each damage did move +// bytes - the junk was written, and the cut dropped some - so the check that a +// damage changed something passes both, and the manifest would list a damage +// whose bytes the file no longer holds. Order is significant on purpose +// (Chain), so the same two the other way round are allowed: cut, then append. +func (c Chain) Eaten() *EatenError { + steps, err := c.steps() + if err != nil { + return nil + } + added := "" + for _, s := range steps { + if s.d.AddsTail && added == "" { + added = s.d.ID + } + if s.d.CutsTail && added != "" { + return &EatenError{Cut: s.d.ID, Added: added} + } + } + return nil +} + +// EatenError is a damage that cuts the tail placed after one that added it. +type EatenError struct { + Cut string + Added string +} + +// AboutSetting puts this on the damage, because the order of the list is the +// thing to change. +func (e *EatenError) AboutSetting() string { return "damage" } + +func (e *EatenError) What() string { return e.what().String() } + +func (e *EatenError) what() core.Said { + return core.Says("damage.EatenWhat", "%s comes after %s and cuts away what it added", + core.A("Cut", e.Cut), core.A("Added", e.Added)) +} + +func (e *EatenError) Why() string { return e.why().String() } + +func (e *EatenError) why() core.Said { + return core.Says("damage.EatenWhy", "the manifest would list a damage whose bytes the file no longer holds") +} + +func (e *EatenError) Instead() string { return e.instead().String() } + +func (e *EatenError) instead() core.Said { + return core.Says("damage.EatenInstead", "Put %s before %s.", core.A("Cut", e.Cut), core.A("Added", e.Added)) +} + +// Parts is what happened, why and what to do instead, for a reader that lays +// them out apart and in its own language. +func (e *EatenError) Parts() (what, why, instead core.Said) { + return e.what(), e.why(), e.instead() +} + +func (e *EatenError) Error() string { return e.Said().String() } + +// Said is the whole refusal, for a window that says it in its own language. +func (e *EatenError) Said() core.Said { + return core.Says("damage.Eaten", "%s. %s", core.A("What", e.what()), core.A("Instead", e.instead())) +} + +// SizeConflict is the refusal a chain earns by changing the length of a file +// whose size comes from what it holds, or nil. +func (c Chain) SizeConflict(sizeFromContents bool) *NeedsSizeError { + if !sizeFromContents { + return nil + } + steps, err := c.steps() + if err != nil { + return nil + } + for _, s := range steps { + if s.d.Length != nil { + return &NeedsSizeError{Damage: s.d.ID} + } + } + return nil +} + +// NeedsSizeError is a damage that changes the length, on a container whose +// size comes from what it holds. +// +// The two are two definitions of one number. The size of such an archive is +// only known once it is written, so there is no length to cut to, and the +// damage would have to choose one the person never stated (docs/DAMAGE-ETAP-2-2026-11-09.md +// section 4.8). +type NeedsSizeError struct { + Damage string +} + +// AboutSetting puts this on the size box, because naming a size is the half +// a person can change without giving up the damage. +func (e *NeedsSizeError) AboutSetting() string { return "size" } + +func (e *NeedsSizeError) What() string { return e.what().String() } + +func (e *NeedsSizeError) what() core.Said { + return core.Says("damage.NeedsSizeWhat", "%s changes the length of the file, and this target takes its size from what it holds", + core.A("Damage", e.Damage)) +} + +func (e *NeedsSizeError) Why() string { return e.why().String() } + +func (e *NeedsSizeError) why() core.Said { + return core.Says("damage.NeedsSizeWhy", "that size is only known once the file is written, so there is no length to work from") +} + +func (e *NeedsSizeError) Instead() string { return e.instead().String() } + +func (e *NeedsSizeError) instead() core.Said { + return core.Says("damage.NeedsSizeInstead", "Give the target a size, or use a damage that keeps the length, such as zero-head.") +} + +// Parts is what happened, why and what to do instead, for a reader that lays +// them out apart and in its own language. +func (e *NeedsSizeError) Parts() (what, why, instead core.Said) { + return e.what(), e.why(), e.instead() +} + +func (e *NeedsSizeError) Error() string { return e.Said().String() } + +// Said is the whole refusal, for a window that says it in its own language. +func (e *NeedsSizeError) Said() core.Said { + return core.Says("damage.NeedsSize", "%s. %s", core.A("What", e.what()), core.A("Instead", e.instead())) +} + // ExpectationConflictError is a target that damages a file and expects it to // be accepted. // diff --git a/internal/damage/truncate.go b/internal/damage/truncate.go new file mode 100644 index 00000000..c0f38d77 --- /dev/null +++ b/internal/damage/truncate.go @@ -0,0 +1,167 @@ +package damage + +import ( + "io" + "math" + "strconv" + + "github.com/donislawdev/TestingFilesGenerator/internal/format" +) + +// Truncate is the damage that cuts the end off a file. A public name under +// untouchable rule 10, and so is its setting. +const ( + Truncate = "truncate" + + SettingMissing = "missing" +) + +const ( + // smallestCut is the fewest bytes worth cutting, and it is a measurement. + // + // Measured 2026-11-09 across 31 formats at 20 kB and 1 MiB: cutting one + // byte leaves pdf, svg and xml intact, because each ends with a line + // break, and from four bytes up the set of formats that notice stays the + // same at every share tried, from 1% to 90% (docs/DAMAGE-ETAP-2-2026-11-09.md + // section 2.1). Below four the cut lands in whitespace the format allows. + smallestCut = 4 + + // fewestMissing and mostMissing bound the share, in percent. + // + // The top is chosen rather than measured: at 90% the generator is asked + // for ten times the file, and past it the plan grows for no new test. The + // bottom is the smallest whole share. + fewestMissing = 1 + mostMissing = 90 + defaultMissing = 50 +) + +func init() { + Register(Descriptor{ + ID: Truncate, + Detail: "Cuts the end off the file and keeps exactly the size you asked for, " + + "the way an interrupted upload or copy leaves it.", + Parameters: []format.Property{ + { + Name: SettingMissing, Kind: format.PropertyInt, + // No unit: "whole number of percent from 1 to 90" is not how + // anyone says it, and the sentence below names the percent. + Min: fewestMissing, Max: mostMissing, + Default: strconv.Itoa(defaultMissing), + Detail: "How much of the whole file is gone, as a percentage. " + + "At least this much is cut - a few bytes more where the format cannot end exactly there.", + }, + }, + Floor: func(v Values) int64 { return smallestKept(missingShare(v)) }, + Length: func(v Values, out int64) int64 { return withCut(out, missingShare(v)) }, + CutsTail: true, + Policy: truncatePolicy, + Open: func(_ Values, f File, out io.Writer) (Stream, error) { + return &truncation{out: out, keep: f.Bytes}, nil + }, + }) +} + +// truncatePolicy is where a file cut short is still a valid file of its +// format, or is refused only for the shape this tool writes. +// +// Measured on 2026-11-10 with every setting of every format, reading the +// reason of each refusal rather than counting them +// (docs/DAMAGE-ETAP-2-2026-11-09.md section 12.1). Everywhere else the cut +// leaves something the format's rules forbid: a box or chunk longer than the +// file, a missing end marker, an element never closed. +var truncatePolicy = []PolicyCase{ + {Format: "txt", Why: "A text cut short is still text, and nothing in the file says it was cut."}, + {Format: "md", Why: "A Markdown document cut short is still a valid document."}, + {Format: "eml", Why: "A message cut short can still be a valid message, because its body is optional."}, + {Format: "csv", Why: "A table cut short ends in a row with no line break, which RFC 4180 allows."}, + {Format: "log", Why: "A log cut short ends part way through an entry, which most readers keep or skip."}, + {Format: "yaml", Why: "A YAML document cut short can still be valid YAML with fewer or shorter values."}, + {Format: "toml", Why: "A TOML document cut short can still be valid TOML, for example when the cut falls between two lines or inside a number."}, + {Format: "html", Why: "HTML readers are required to finish a page that is cut short."}, +} + +// missingShare is the share this run cuts, in percent. +// +// The value has been past the declaration by the time it arrives. It falls +// back rather than failing for the reason headBytes gives: this is callable +// directly, and code that trusts its input is one registry change away from +// writing a file nobody ordered. +func missingShare(v Values) int64 { + n, err := strconv.ParseInt(v[SettingMissing], 10, 64) + if err != nil || n < fewestMissing || n > mostMissing { + return defaultMissing + } + return n +} + +// cutFor is how many bytes go missing from a file that keeps out bytes, so +// that they are the stated share of the whole: out * m / (100 - m), rounded +// up so the share is never less than stated. +// +// Divided before it is multiplied, so the remainder cannot wrap whatever the +// size. The product still can, once the cut is larger than what is kept: nine +// tenths missing is nine times the size, which passes the top of an int64 +// above about 1e18 B. It saturates there rather than wrapping to a smaller +// number, which a format would have been asked for and a refusal would have +// named. +func cutFor(out, m int64) int64 { + rest := 100 - m + q, r := out/rest, out%rest + if q > (math.MaxInt64-m)/m { + return math.MaxInt64 + } + return q*m + (r*m+rest-1)/rest +} + +// withCut is the whole file for out bytes kept, saturating rather than +// wrapping - the format refuses the size then, in its own words. +func withCut(out, m int64) int64 { + cut := cutFor(out, m) + if out > math.MaxInt64-cut { + return math.MaxInt64 + } + return out + cut +} + +// smallestKept is the fewest bytes a file can keep so that at least +// smallestCut bytes are cut: out * m / (100 - m) has to pass smallestCut - 1. +func smallestKept(m int64) int64 { + return (smallestCut-1)*(100-m)/m + 1 +} + +// truncation passes on the first bytes of a file and drops the rest. +// +// Streaming like zero-head, so the regression line "a generator does not hold +// the whole file in memory" is untouched: it holds a count, never the file. +type truncation struct { + out io.Writer + // keep is how many bytes are still to be passed on. + keep int64 + // cut records whether any byte was dropped. A file the generator ended at + // exactly the bytes kept was not cut, and is a damage that did nothing. + cut bool +} + +func (t *truncation) Write(p []byte) (int, error) { + if int64(len(p)) <= t.keep { + n, err := t.out.Write(p) + t.keep -= int64(n) + return n, err + } + t.cut = true + head := p[:t.keep] + n, err := t.out.Write(head) + t.keep -= int64(n) + if err != nil { + return n, err + } + // Fewer bytes taken than offered, with the reason - which is what the + // io.Writer contract asks of a short write, and what stops the generator. + return n, ErrKeptEnough +} + +func (t *truncation) Touched() bool { return t.cut } + +// Finish has nothing to add. What was cut is gone. +func (t *truncation) Finish() error { return nil } diff --git a/internal/damage/zerohead.go b/internal/damage/zerohead.go index 82918c63..327bc045 100644 --- a/internal/damage/zerohead.go +++ b/internal/damage/zerohead.go @@ -62,7 +62,10 @@ func init() { }, }, Floor: func(v Values) int64 { return int64(headBytes(v)) }, - Open: func(v Values, out io.Writer) (Stream, error) { + // No policy cases: measured 2026-11-09 and again with every setting + // of every format on 2026-11-10, a zeroed head breaks the rules of + // all 31 formats (docs/DAMAGE-ETAP-2-2026-11-09.md section 2). + Open: func(v Values, _ File, out io.Writer) (Stream, error) { return &zeroHead{out: out, left: headBytes(v)}, nil }, }) @@ -139,3 +142,6 @@ func (z *zeroHead) Write(p []byte) (int, error) { } func (z *zeroHead) Touched() bool { return z.touched } + +// Finish has nothing to add. The head is behind it by the time the file ends. +func (z *zeroHead) Finish() error { return nil } diff --git a/internal/engine/damage.go b/internal/engine/damage.go index f7d25c81..e6aceef6 100644 --- a/internal/engine/damage.go +++ b/internal/engine/damage.go @@ -9,7 +9,12 @@ package engine import ( + "context" + "errors" + "io" + "github.com/donislawdev/TestingFilesGenerator/internal/damage" + "github.com/donislawdev/TestingFilesGenerator/internal/format" "github.com/donislawdev/TestingFilesGenerator/internal/manifest" ) @@ -24,11 +29,35 @@ import ( // // The floor first, because it names a number a person can act on. A target // earning both refusals gets that one. -func checkDamage(t *Target) error { +func checkDamage(t *Target, desc format.Descriptor) error { if err := checkDamageFloor(t); err != nil { return err } - return checkDamageExpectation(t) + return checkDamageExpectation(t, desc) +} + +// prepareDamage is what a damaged target is refused for before a size is +// drawn, and the format its files are planned through. +// +// Before the draw, because a chain that cuts away what it added and a damage +// with no size to work from are wrong whatever size comes out of the seed. +// And the format it hands back is the one the draw has to ask: a damage that +// changes the length needs the generator asked for a different size than the +// one on disk (damagedformat.go). +func prepareDamage(t *Target, desc format.Descriptor) (format.Descriptor, error) { + if len(t.Damage) == 0 { + return desc, nil + } + if eaten := t.Damage.Eaten(); eaten != nil { + return desc, eaten + } + if sized := t.Damage.SizeConflict(t.SizeFromContents); sized != nil { + return desc, sized + } + if !t.Damage.ChangesLength() { + return desc, nil + } + return damagedFormat(desc, t.Damage), nil } // checkDamageFloor refuses a file smaller than the damage it was given. @@ -48,8 +77,13 @@ func checkDamage(t *Target) error { // against anything would be comparing against a number nobody set. The damage // still applies to the finished archive, and the check that it moved a byte // still runs - what is missing is only the early refusal. +// +// A chain that changes the length is left to the format it plans through +// (damagedformat.go), which knows the format's floor as well as the damage's +// and names the larger. Asked here, a PNG with junk appended would be told 64 +// B is enough and then refused at 64 B for needing 138. func checkDamageFloor(t *Target) error { - if len(t.Damage) == 0 || t.SizeFromContents { + if len(t.Damage) == 0 || t.SizeFromContents || t.Damage.ChangesLength() { return nil } floor, owner, err := t.Damage.Floor() @@ -78,8 +112,65 @@ func checkDamageFloor(t *Target) error { // The window cannot reach this today - damage sits on the generate screen and // the expectation on the recipe screen - and being here rather than in the // reader is what covers it on the day those two meet. -func checkDamageExpectation(t *Target) error { - return t.Damage.ConflictsWithExpectation(t.Expected) +func checkDamageExpectation(t *Target, desc format.Descriptor) error { + return t.Damage.ConflictsWithExpectation(t.Expected, damage.KindOf(desc, t.Properties)) +} + +// writeDamaged runs the generator into the writer it is given and, for a +// damaged file, ends the damages once the generator has ended. +// +// A damage that cut the file short stops the generator with +// damage.ErrKeptEnough, and that is the file being complete rather than a +// failure: every byte it keeps has passed, and the rest was never going to +// reach the disk. Only a damaged file can stop that way, so the error is read +// as success for nothing else. A stop of the run arriving at the same moment +// comes back as the context's own error, and the file fails and is removed +// like any other file stopped part way. +func writeDamaged(ctx context.Context, f PlannedFile, w io.Writer, streams []damage.Stream) error { + err := writeWithoutCrashing(ctx, f, w) + if !f.Damaged() { + return err + } + if errors.Is(err, damage.ErrKeptEnough) { + err = nil + } + if err != nil { + return err + } + return f.Target.Damage.Finish(streams) +} + +// damagedExpectation is what the manifest says a system under test should do +// with a damaged file nobody declared an expectation for. +// +// Reject, with certainty, where the damage breaks the rules of the format: the +// guard TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat measures +// that a judge refuses every such pair, so untouchable rule 5 is kept - the +// expectation is not invented. Where the damage leaves the file's fate to the +// reader, the manifest says so rather than guessing, in the shape MF5 gives +// that case: unspecified, policy dependent, with both answers offered and the +// reason in a sentence (owner's call, 2026-11-10). +// +// The settings are the ones the target stated, defaults filled in - the +// same question the plan asked when it decided whether accept was allowed. +func damagedExpectation(f PlannedFile) manifest.Expected { + certain, why, err := f.Target.Damage.Certain(damage.KindOf(f.Desc, f.Target.Properties)) + if err != nil || certain { + // An error is a damage the registry does not know, which planning + // refused already - the file could not have been written. + return manifest.Expected{ + Outcome: manifest.OutcomeReject, + Reason: "content_malformed", + Confidence: "certain", + } + } + return manifest.Expected{ + Outcome: manifest.OutcomeUnspecified, + Reason: "content_malformed", + Detail: why, + Confidence: "policy_dependent", + PolicyOptions: []string{manifest.OutcomeAccept, manifest.OutcomeReject}, + } } // damageFor records what was broken about this file, with the settings diff --git a/internal/engine/damagedformat.go b/internal/engine/damagedformat.go new file mode 100644 index 00000000..c32d2eeb --- /dev/null +++ b/internal/engine/damagedformat.go @@ -0,0 +1,293 @@ +package engine + +import ( + "context" + "errors" + "io" + "strings" + + "github.com/donislawdev/TestingFilesGenerator/internal/core" + "github.com/donislawdev/TestingFilesGenerator/internal/damage" + "github.com/donislawdev/TestingFilesGenerator/internal/format" +) + +// A target whose damage changes the length of its files plans them through +// this: a format that takes the size the file will have ON DISK, asks the real +// generator for the size the damage needs, and answers in the size on disk. +// +// A wrapper rather than a second size carried beside the plan, and that is a +// measurement: on 2026-11-10 six places in the engine and the command line +// read the plan's byte count as the size of the file - the manifest, the +// length check after writing, the progress bar, the run's total, the free +// space check and --dry-run - and a seventh added later would have been the +// one nobody updated. Here every one of them reads the size on disk without a +// line of theirs changing, and so do drawing a size from a range and +// SmallestAccepted, because the refusals they jump along come back in that +// size too (docs/DAMAGE-ETAP-2-2026-11-09.md section 12.5). +// +// A chain that keeps the length is never wrapped, so zero-head plans, writes +// and refuses exactly as it did before lengths could change. + +// SmallestDamaged is the smallest file on disk a target of this format with +// this damage is given, for a request shaped like r. +// +// For a caller that has to know before planning - a guard measuring each pair +// at its smallest, or a list announcing the number. Asked of the same format a +// run plans through, so the number cannot be one the run refuses. +func SmallestDamaged(d format.Descriptor, chain damage.Chain, r format.Request) int64 { + floor, _, err := chain.Floor() + if err != nil { + return d.SmallestAccepted(r) + } + if chain.ChangesLength() { + d = damagedFormat(d, chain) + } + return max(floor, d.SmallestAccepted(r)) +} + +// damagedFormat is d, planning through chain. +func damagedFormat(d format.Descriptor, chain damage.Chain) format.Descriptor { + d.Generator = &damagedGenerator{inner: d.Generator, format: d.ID, chain: chain} + return d +} + +type damagedGenerator struct { + inner format.Generator + format string + chain damage.Chain +} + +// damagedMemo carries the real generator's plan from planning to writing. +type damagedMemo struct { + plan format.Plan +} + +func (g *damagedGenerator) Plan(r format.Request) (format.Plan, error) { + return g.PlanContext(context.Background(), r) +} + +// PlanContext is Plan, stoppable, so a container wrapped here still stops +// between the files inside it. +func (g *damagedGenerator) PlanContext(ctx context.Context, r format.Request) (format.Plan, error) { + size := r.Bytes + want, err := g.chain.Before(size) + var small *damage.TooSmallError + if errors.As(err, &small) { + return format.Plan{}, g.belowTheDamage(ctx, r, small) + } + if err != nil { + return format.Plan{}, err + } + r.Bytes = want + p, err := g.plan(ctx, r) + if err != nil && g.chain.Stretches() { + p, err = g.stretch(ctx, r, err) + } + if err != nil { + return format.Plan{}, g.inSizeOnDisk(ctx, r, size, err) + } + return format.Plan{ + Bytes: size, + Exact: p.Exact, + Properties: p.Properties, + Determinism: p.Determinism, + Notes: p.Notes, + Memo: damagedMemo{plan: p}, + Work: p.Work, + Holds: p.Holds, + }, nil +} + +// plan asks the real generator, through the wrapper everything the engine asks +// of a generator goes through. +// +// The engine reaches this generator from under that wrapper already, so a +// crash in the real one never ended a run. What going through it again buys is +// the callers with no wrapper in hand - SmallestDamaged is asked by guards - +// and a rule with no exception to remember. +// +// One thing it changes, named rather than implied. A crash at a size this is +// only TRYING - the damage's floor, or the next size a refusal named - is read +// by the searches below as "not this size". The person is then told about the +// size they asked for, which was refused in words before anything crashed. A +// crash at the size they asked for is still reported as the crash it is. +func (g *damagedGenerator) plan(ctx context.Context, r format.Request) (format.Plan, error) { + return planWithoutCrashing(ctx, format.Descriptor{ID: g.format, Generator: g.inner}, r) +} + +// belowTheDamage is a size under the damage's own floor, refused in the shape a +// format refuses a size with, so that drawing a size from a range jumps past +// it instead of stopping. A fixed size of a chain that keeps the length never +// gets here: checkDamage refuses it first, in the damage's own words. +// +// The format may need more than the damage does - junk appended to a PNG +// needs the PNG under it as well - so the floor is asked in turn, and the +// larger answer is the one named. +func (g *damagedGenerator) belowTheDamage(ctx context.Context, r format.Request, small *damage.TooSmallError) error { + size := r.Bytes + r.Bytes = small.Floor + _, deeper := g.PlanContext(ctx, r) + var below *format.BelowMinimumError + if errors.As(deeper, &below) && below.Minimum > small.Floor { + below.Requested = size + return below + } + return &format.BelowMinimumError{ + Format: g.format, Requested: size, Minimum: small.Floor, + Reason: core.Says("engine.DamagedFloorWhy", "%s needs at least %s", + core.A("Damage", small.Damage), core.A("Floor", core.ExactBytes(small.Floor))), + Hint: g.askFor(small.Floor, true)} +} + +// Write hands the real plan to the real generator. The damage itself is not +// applied here: the engine opens the chain around the writer it passes in, as +// it does for every damaged file. +func (g *damagedGenerator) Write(ctx context.Context, w io.Writer, p format.Plan) error { + memo, ok := p.Memo.(damagedMemo) + if !ok { + return core.Defect(errors.New("a damaged file was written from a plan its damage did not make")) + } + return g.inner.Write(ctx, w, memo.plan) +} + +// stretch moves a size the format cannot write exactly up to the first one it +// can, for a chain that cuts the file back to its size anyway. +// +// Only above the format's own floor. A size inside a band the format cannot +// reach - eleven byte counts above a PNG's picture, an odd count in UTF-16 - +// costs a few more bytes cut, which is what makes truncate's missing an "at +// least" (owner's call, 2026-11-10). A size under the floor would cost half +// the file and more, so it is refused instead, naming the smallest size that +// works. +func (g *damagedGenerator) stretch(ctx context.Context, r format.Request, first error) (format.Plan, error) { + var below *format.BelowMinimumError + if !errors.As(first, &below) || below.Minimum <= r.Bytes { + return format.Plan{}, first + } + want := r.Bytes + floor, err := g.firstWritable(ctx, r, 0) + if err != nil || want < floor { + return format.Plan{}, first + } + r.Bytes, err = g.firstWritable(ctx, r, want) + if err != nil { + return format.Plan{}, first + } + return g.plan(ctx, r) +} + +// firstWritable is the smallest size at or above from that the real generator +// takes, jumping along the sizes its refusals name, as drawSizes does. +// +// Not the function drawSizes jumps with, although the jump is the same one. +// That one stops at the top of a range, and there is no range here: what ends +// a format that keeps naming larger sizes is a count of rounds, sixty four for +// the reason SmallestAccepted gives. +func (g *damagedGenerator) firstWritable(ctx context.Context, r format.Request, from int64) (int64, error) { + at := from + for round := 0; round < 64; round++ { + r.Bytes = at + _, err := g.plan(ctx, r) + if err == nil { + return at, nil + } + var below *format.BelowMinimumError + if !errors.As(err, &below) || below.Minimum <= at { + return 0, err + } + at = below.Minimum + } + return 0, core.Defect(errors.New("a format kept naming larger sizes it could not write")) +} + +// inSizeOnDisk is a refusal of the real generator, said in the size the +// person asked for rather than the one the damage needed. +// +// Without it a person asking for a 30 B png cut in half would read "png +// cannot be smaller than 74 B. Requested: 60 B" - a number they never wrote, +// and a way out that does not work. +// +// The size a format's refusal names is the next one worth trying, not always +// one it writes - measured 2026-11-10, wav refuses 0 B naming 96 and then 96 +// naming 98 - so the way out is walked to a size it really writes first. +// Translated after one step, the refusal named a size that was refused in +// turn. +func (g *damagedGenerator) inSizeOnDisk(ctx context.Context, r format.Request, size int64, err error) error { + var below *format.BelowMinimumError + if errors.As(err, &below) { + first := below.Minimum + if writable, werr := g.firstWritable(ctx, r, first); werr == nil { + first = writable + } + least := g.smallestReaching(first) + return &format.BelowMinimumError{ + Format: g.format, Requested: size, Minimum: least, + Reason: core.Says("engine.DamagedBelowWhy", "%s has to start from a %s file of at least %s", + core.A("Damage", g.names()), core.A("Format", g.format), core.A("Minimum", core.ExactBytes(first))), + Hint: g.askFor(least, true)} + } + var above *format.AboveMaximumError + if errors.As(err, &above) { + most := g.largestWithin(above.Maximum) + return &format.AboveMaximumError{ + Format: g.format, Requested: size, Maximum: most, + Reason: core.Says("engine.DamagedAboveWhy", "%s has to start from a %s file of at most %s", + core.A("Damage", g.names()), core.A("Format", g.format), core.A("Maximum", core.ExactBytes(above.Maximum))), + Hint: g.askFor(most, false)} + } + return err +} + +// smallestReaching is the smallest size on disk whose damage needs at least +// written bytes from the generator. What the damage needs grows with the size +// asked for, so this is found by halving. +func (g *damagedGenerator) smallestReaching(written int64) int64 { + needs := func(size int64) bool { + w, err := g.chain.Before(size) + return err == nil && w >= written + } + lo, hi := int64(0), int64(1) + for !needs(hi) && hi < lengthLimit { + lo, hi = hi, hi*2 + } + for hi-lo > 1 { + mid := lo + (hi-lo)/2 + if needs(mid) { + hi = mid + } else { + lo = mid + } + } + return hi +} + +// largestWithin is the largest size on disk whose damage needs no more than +// written bytes from the generator. +func (g *damagedGenerator) largestWithin(written int64) int64 { + return g.smallestReaching(written+1) - 1 +} + +// lengthLimit is where the search above stops - far past any size a run +// accepts, and far enough below the top of an int64 that doubling cannot wrap. +const lengthLimit = int64(1) << 60 + +// names is the chain as a person writes it on the command line. +// +// The format's own reason is NOT carried into the refusal. It speaks about the +// size it refused - "a 1x1 picture already encodes to that much" - and beside +// a size the person did not write, "that much" points at the wrong number. +func (g *damagedGenerator) names() string { + names := make([]string, 0, len(g.chain)) + for _, s := range g.chain { + names = append(names, s.String()) + } + return strings.Join(names, ", ") +} + +// askFor is the way out, in the size on disk. +func (g *damagedGenerator) askFor(size int64, atLeast bool) core.Said { + if atLeast { + return core.Says("engine.DamagedSizeAtLeast", "Ask for %s or more.", core.A("Size", core.ExactBytes(size))) + } + return core.Says("engine.DamagedSizeAtMost", "Ask for %s or less.", core.A("Size", core.ExactBytes(size))) +} diff --git a/internal/engine/engine.go b/internal/engine/engine.go index 6042a69c..65aea1eb 100644 --- a/internal/engine/engine.go +++ b/internal/engine/engine.go @@ -290,6 +290,11 @@ func settleTarget(ctx context.Context, t *Target, opt Options, seen map[string]b return format.Descriptor{}, &format.NotAContainerError{Format: t.Format, Containers: format.Containers()} } + desc, err = prepareDamage(t, desc) + if err != nil { + return format.Descriptor{}, err + } + // A size below the minimum is refused by the generator itself, on the first // size that cannot be delivered, and planning writes nothing - so every size // of a boundary set is judged before any file exists. @@ -308,7 +313,7 @@ func settleTarget(ctx context.Context, t *Target, opt Options, seen map[string]b // After the draw, because a range arrives here carrying only a count and a // damage floor has to be judged against the sizes that will really be // written. - if err := checkDamage(t); err != nil { + if err := checkDamage(t, desc); err != nil { return format.Descriptor{}, err } return desc, nil @@ -793,23 +798,12 @@ func (res *Result) note(code string, said core.Said) manifest.Note { func expectationFor(f PlannedFile) manifest.Expected { switch f.Target.Expected { case "": - // A damaged file is one a judge was measured to refuse, so reject is - // what it means rather than a guess - which is why this is allowed to - // state it with certainty where an ordinary file gets unspecified. - // Untouchable rule 5 forbids inventing an expectation, and this one is - // not invented: the witness rule says a damage with nothing able to - // refuse its result is a damage this tool does not offer. - // // Only where nothing was declared. A recipe that states an expectation // reaches the manifest unchanged, and the one statement that cannot be - // true beside damage - accept - is refused while reading the recipe - // rather than quietly replaced here. + // true beside damage that breaks a format's rules - accept - is + // refused while reading the recipe rather than quietly replaced here. if f.Damaged() { - return manifest.Expected{ - Outcome: manifest.OutcomeReject, - Reason: "content_malformed", - Confidence: "certain", - } + return damagedExpectation(f) } return manifest.Expected{ Outcome: manifest.OutcomeUnspecified, diff --git a/internal/engine/parallel.go b/internal/engine/parallel.go index ed6af43b..015782d9 100644 --- a/internal/engine/parallel.go +++ b/internal/engine/parallel.go @@ -573,7 +573,7 @@ func writeOne(ctx context.Context, f PlannedFile, outDir string, p *fileProgress sink := io.Writer(counter) var streams []damage.Stream if f.Damaged() { - damaged, opened, err := f.Target.Damage.Open(counter) + damaged, opened, err := f.Target.Damage.Open(counter, f.Plan.Bytes, f.Seed) if err != nil { _ = fh.Close() _ = os.Remove(tmp) @@ -582,7 +582,7 @@ func writeOne(ctx context.Context, f PlannedFile, outDir string, p *fileProgress sink, streams = damaged, opened } - writeErr := writeWithoutCrashing(ctx, f, sink) + writeErr := writeDamaged(ctx, f, sink, streams) if writeErr == nil { writeErr = buffered.Flush() } diff --git a/internal/guard/contextroots_test.go b/internal/guard/contextroots_test.go index c590906d..4084266a 100644 --- a/internal/guard/contextroots_test.go +++ b/internal/guard/contextroots_test.go @@ -26,20 +26,21 @@ import ( // Keyed by file and by the function the call sits in, never by line, so moving // code inside a function does not make the list lie (pitfall 11). var contextRoots = map[string]string{ - "cmd/tfg/main.go main": "the process's own context, the one the signal cancels", - "internal/gui/window/work.go begin": "a preview, a run or a tool started in the window begins here, and Cancel or closing the window is what stops it", - "internal/oracle/oracle.go Checker.Check": "the oracle runs somebody else's reader under a time limit of its own - a measuring instrument for the guards, never on the path of a run", - "internal/oracle/oracle.go PictureChanges": "the same instrument, a reference reader under a time limit", - "internal/oracle/oracle.go Strict": "the same instrument, a reference reader under a time limit", - "internal/engine/engine.go Plan": "the version without a context, for callers that have none - it is PlanContext, so it cannot disagree with it", - "internal/manifest/manifest.go Load": "reads a document of at most sixteen megabytes that is in memory already, so there is nothing long to stop - verify and cleanup read through ReadClaims, which takes the caller's", - "internal/format/eml/eml.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", - "internal/format/htmlfile/html.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", - "internal/format/rtf/rtf.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", - "internal/format/targz/targz.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", - "internal/format/zip/zip.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", - "internal/format/opc/opc.go size": "measures a package with the padding left out, so there is nothing long to stop", - "internal/format/zip/zip.go archiveSize": "measures an archive with the contents left out, so there is nothing long to stop", + "cmd/tfg/main.go main": "the process's own context, the one the signal cancels", + "internal/gui/window/work.go begin": "a preview, a run or a tool started in the window begins here, and Cancel or closing the window is what stops it", + "internal/oracle/oracle.go Checker.Check": "the oracle runs somebody else's reader under a time limit of its own - a measuring instrument for the guards, never on the path of a run", + "internal/oracle/oracle.go PictureChanges": "the same instrument, a reference reader under a time limit", + "internal/oracle/oracle.go Strict": "the same instrument, a reference reader under a time limit", + "internal/engine/engine.go Plan": "the version without a context, for callers that have none - it is PlanContext, so it cannot disagree with it", + "internal/manifest/manifest.go Load": "reads a document of at most sixteen megabytes that is in memory already, so there is nothing long to stop - verify and cleanup read through ReadClaims, which takes the caller's", + "internal/format/eml/eml.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", + "internal/format/htmlfile/html.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", + "internal/format/rtf/rtf.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", + "internal/format/targz/targz.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", + "internal/format/zip/zip.go generator.Plan": "the Generator interface's Plan for callers without a context - the engine asks PlanContext", + "internal/engine/damagedformat.go damagedGenerator.Plan": "the Generator interface's Plan for callers without a context, such as SmallestAccepted - the engine asks PlanContext", + "internal/format/opc/opc.go size": "measures a package with the padding left out, so there is nothing long to stop", + "internal/format/zip/zip.go archiveSize": "measures an archive with the contents left out, so there is nothing long to stop", } func TestAContextIsMadeFromNothingOnlyWhereItIsMeantToStart(t *testing.T) { diff --git a/internal/guard/damagebytes_test.go b/internal/guard/damagebytes_test.go index 7ef2275c..85a03027 100644 --- a/internal/guard/damagebytes_test.go +++ b/internal/guard/damagebytes_test.go @@ -2,6 +2,8 @@ package guard import ( "bytes" + "errors" + "math" "strconv" "testing" @@ -36,7 +38,7 @@ func TestZeroHeadZeroesTheBytesItSaysAndNoOthers(t *testing.T) { ID: damage.ZeroHead, Values: damage.Values{damage.SettingBytes: strconv.Itoa(head)}, }} - w, streams, err := chain.Open(&out) + w, streams, err := chain.Open(&out, size, 1) if err != nil { t.Fatalf("opening the chain: %v", err) } @@ -82,7 +84,7 @@ func TestADamageThatMovedNothingSaysSo(t *testing.T) { var out bytes.Buffer chain := damage.Chain{{ID: damage.ZeroHead}} - w, streams, err := chain.Open(&out) + w, streams, err := chain.Open(&out, int64(len(source)), 1) if err != nil { t.Fatalf("opening the chain: %v", err) } @@ -151,7 +153,7 @@ func TestAChainAppliesItsDamagesInTheOrderItListsThem(t *testing.T) { {ID: damage.ZeroHead, Values: damage.Values{damage.SettingBytes: "4"}}, {ID: damage.ZeroHead, Values: damage.Values{damage.SettingBytes: "16"}}, } - w, streams, err := chain.Open(&out) + w, streams, err := chain.Open(&out, int64(len(source)), 1) if err != nil { t.Fatalf("opening the chain: %v", err) } @@ -182,3 +184,152 @@ func TestAChainAppliesItsDamagesInTheOrderItListsThem(t *testing.T) { t.Errorf("the chain names %q as idle and neither damage was", idle) } } + +// A cut keeps exactly the bytes it was told to, and tells the generator to +// stop on the write that crosses the line. +// +// The stop is what saves cutting 90% of a file from generating all of it, and +// nothing outside the damage can see whether it happened - the file comes out +// the same either way. So it is asked here, of the stream itself: the write +// that crosses the line takes only the bytes kept and answers ErrKeptEnough, +// and so does every write after it. +func TestACutKeepsItsBytesAndStopsTheGenerator(t *testing.T) { + source := make([]byte, 100) + for i := range source { + source[i] = byte(i + 1) + } + var out bytes.Buffer + chain := damage.Chain{{ID: damage.Truncate}} + w, streams, err := chain.Open(&out, 30, 1) + if err != nil { + t.Fatal(err) + } + if n, err := w.Write(source[:20]); n != 20 || err != nil { + t.Fatalf("a write inside the bytes kept gave %d, %v", n, err) + } + if streams[0].Touched() { + t.Error("nothing was cut yet and the cut says it touched the file") + } + n, err := w.Write(source[20:]) + if n != 10 || !errors.Is(err, damage.ErrKeptEnough) { + t.Errorf("the write crossing the line gave %d, %v - it should take the 10 B kept and tell the generator to stop", n, err) + } + if n, err := w.Write(source[:5]); n != 0 || !errors.Is(err, damage.ErrKeptEnough) { + t.Errorf("a write after the line gave %d, %v", n, err) + } + if !bytes.Equal(out.Bytes(), source[:30]) { + t.Errorf("the cut passed on %d B, and should have passed the first 30 B unchanged", out.Len()) + } + if !streams[0].Touched() { + t.Error("70 B were dropped and the cut says it touched nothing") + } +} + +// Junk is written after the file, once, and nothing before it moves. +func TestJunkComesAfterTheFileAndOnlyThere(t *testing.T) { + source := []byte("a whole file, ending here") + var out bytes.Buffer + chain := damage.Chain{{ID: damage.AppendJunk, Values: damage.Values{damage.SettingBytes: "100"}}} + size := int64(len(source) + 100) + w, streams, err := chain.Open(&out, size, 7) + if err != nil { + t.Fatal(err) + } + if _, err := w.Write(source); err != nil { + t.Fatal(err) + } + if out.Len() != len(source) || streams[0].Touched() { + t.Fatal("junk arrived, or was claimed, before the file had ended") + } + if err := chain.Finish(streams); err != nil { + t.Fatal(err) + } + if int64(out.Len()) != size || !bytes.Equal(out.Bytes()[:len(source)], source) { + t.Errorf("after the junk the file is %d B, and should be the %d B before it and 100 B more", out.Len(), len(source)) + } + if !streams[0].Touched() { + t.Error("100 B were appended and the damage says it touched nothing") + } + // The same seed gives the same junk, a different one different junk. + again := func(seed uint64) []byte { + var b bytes.Buffer + w, s, err := chain.Open(&b, size, seed) + if err != nil { + t.Fatal(err) + } + if _, err := w.Write(source); err != nil { + t.Fatal(err) + } + if err := chain.Finish(s); err != nil { + t.Fatal(err) + } + return b.Bytes() + } + if !bytes.Equal(again(7), out.Bytes()) || bytes.Equal(again(8), out.Bytes()) { + t.Error("the junk does not follow the seed: the same seed has to give the same bytes and another seed others") + } +} + +// The floor of a cut follows its share, and a chain works its floor out from +// what each damage needs to leave. +// +// A cut has to drop at least four bytes, so the smallest file it can leave +// grows as the share shrinks: half gone needs 4 B left, one percent gone +// needs 298 B left before 1% of the whole is four bytes. In a chain, junk +// after a cut needs its own 64 B on top of what the cut leaves. +func TestTheFloorOfACutFollowsItsShareAndAChainAddsUp(t *testing.T) { + for _, c := range []struct { + chain damage.Chain + want int64 + owner string + }{ + {damage.Chain{{ID: damage.Truncate}}, 4, damage.Truncate}, + {damage.Chain{{ID: damage.Truncate, Values: damage.Values{damage.SettingMissing: "1"}}}, 298, damage.Truncate}, + {damage.Chain{{ID: damage.Truncate, Values: damage.Values{damage.SettingMissing: "90"}}}, 1, damage.Truncate}, + {damage.Chain{{ID: damage.AppendJunk}}, 64, damage.AppendJunk}, + {damage.Chain{{ID: damage.Truncate}, {ID: damage.AppendJunk}}, 68, damage.Truncate}, + {damage.Chain{{ID: damage.ZeroHead}, {ID: damage.Truncate}}, 4, damage.Truncate}, + } { + got, owner, err := c.chain.Floor() + if err != nil { + t.Fatal(err) + } + if got != c.want || owner != c.owner { + t.Errorf("%s has a floor of %d B set by %q, and should be %d B set by %q", c.chain, got, owner, c.want, c.owner) + } + if _, err := c.chain.Before(got); err != nil { + t.Errorf("%s refuses its own floor of %d B: %v", c.chain, got, err) + } + if got > 0 { + if _, err := c.chain.Before(got - 1); err == nil { + t.Errorf("%s takes %d B, one under its floor", c.chain, got-1) + } + } + } + + // And the file a cut starts from: the share of the whole is never less + // than stated, so the cut is rounded up - at its floor, exactly the four + // bytes the floor is there for. + // + // The last row is a size whose cut does not fit in an int64: nine tenths + // missing is nine times what is kept. It has to come back as the largest + // number there is, which every format refuses in its own words. Wrapped, + // it was a smaller number that a format was asked for instead. + for _, c := range []struct { + missing string + kept, from int64 + }{ + {"50", 4, 8}, + {"50", 7, 14}, + {"1", 298, 302}, + {"90", 1, 10}, + {"33", 7, 11}, + {"90", 2_100_000_000_000_000_000, math.MaxInt64}, + } { + chain := damage.Chain{{ID: damage.Truncate, Values: damage.Values{damage.SettingMissing: c.missing}}} + if got, err := chain.Before(c.kept); err != nil || got != c.from { + t.Errorf("missing=%s keeping %d B starts from %d B (%v), and should start from %d B", + c.missing, c.kept, got, err, c.from) + } + } +} diff --git a/internal/guard/damagecommand_test.go b/internal/guard/damagecommand_test.go index 3ba6c1f8..755cdb5c 100644 --- a/internal/guard/damagecommand_test.go +++ b/internal/guard/damagecommand_test.go @@ -4,6 +4,8 @@ import ( "bytes" "context" "encoding/json" + "maps" + "slices" "strconv" "strings" "testing" @@ -201,6 +203,13 @@ func TestTheMachineReadableDamageListCarriesTheWholeDeclaration(t *testing.T) { Default string `json:"default"` Detail string `json:"detail"` } `json:"parameters"` + // A pointer, so a key that is missing is told apart from an empty + // list: the key is promised for every damage. + PolicyDependent *[]struct { + Format string `json:"format"` + Settings map[string][]string `json:"settings"` + Reason string `json:"reason"` + } `json:"policy_dependent"` } if err := json.Unmarshal([]byte(stdout), &entries); err != nil { t.Fatalf("the list does not parse: %v\n%s", err, stdout) @@ -210,7 +219,10 @@ func TestTheMachineReadableDamageListCarriesTheWholeDeclaration(t *testing.T) { len(damage.All()), len(entries)) } - checked := 0 + // Counted apart. Settings alone used to satisfy one shared count, so a + // registry declaring no policy case anywhere left that comparison unrun + // and this test green. + checked, policies := 0, 0 for _, e := range entries { d, err := damage.Get(e.ID) if err != nil { @@ -224,6 +236,19 @@ func TestTheMachineReadableDamageListCarriesTheWholeDeclaration(t *testing.T) { t.Errorf("%s: the list says %d B and the declaration says %d B", e.ID, e.SmallestFileBytes, d.Floor(d.Defaults())) } + if e.PolicyDependent == nil { + t.Errorf("%s: policy_dependent is missing, and it is promised for every damage so a script reads one key", e.ID) + } else if got := *e.PolicyDependent; len(got) != len(d.Policy) { + t.Errorf("%s declares %d format(s) where the outcome depends on the reader and the list carries %d", + e.ID, len(d.Policy), len(got)) + } else { + for i, c := range d.Policy { + if got[i].Format != c.Format || got[i].Reason != c.Why || !maps.EqualFunc(got[i].Settings, c.When, slices.Equal) { + t.Errorf("%s: policy case %d arrives as %+v and the declaration is %+v", e.ID, i, got[i], c) + } + policies++ + } + } if len(e.Parameters) != len(d.Parameters) { t.Errorf("%s declares %d setting(s) and the list carries %d", e.ID, len(d.Parameters), len(e.Parameters)) @@ -252,6 +277,97 @@ func TestTheMachineReadableDamageListCarriesTheWholeDeclaration(t *testing.T) { if checked == 0 { t.Fatal("no setting was compared, so this proved nothing") } + if policies == 0 { + t.Fatal("no policy case was compared, so policy_dependent was not proved") + } +} + +// The damage command says in words, not only in JSON, where the outcome of a +// damaged file depends on the reader. +// +// The machine readable list is compared field by field above. What a person +// reads had no guard: the list could stop naming the formats, or the +// description of one damage could drop the reason, and every test stayed +// green. Asked for by the review of 2026-11-10. +// +// Rebuilt from the declarations, like the guard on settings. A damage with no +// case to declare is the control: it has to stay silent on the subject, or a +// build printing the same lines under every damage would pass. +func TestTheDamageCommandSaysInWordsWhereTheOutcomeDependsOnTheReader(t *testing.T) { + code, listing, errOut := run(t, "damage") + if code != cli.ExitOK { + t.Fatalf("listing the damages ended with %d: %s", code, errOut) + } + var reasons []string + for _, d := range damage.All() { + for _, c := range d.Policy { + reasons = append(reasons, c.Why) + } + } + + dependent, certain := 0, 0 + for _, d := range damage.All() { + code, described, errOut := run(t, "damage", d.ID) + if code != cli.ExitOK { + t.Fatalf("describing %q ended with %d: %s", d.ID, code, errOut) + } + under := strings.Join(listedUnder(listing, d.ID), "\n") + if len(d.Policy) == 0 { + certain++ + if strings.TrimSpace(under) != d.Detail { + t.Errorf("%s declares no format where readers differ, and the list prints more than its sentence under it:\n%s", d.ID, under) + } + if slices.ContainsFunc(reasons, func(why string) bool { return strings.Contains(described, why) }) { + t.Errorf("%s declares no format where readers differ, and its description carries another damage's reason:\n%s", d.ID, described) + } + continue + } + dependent++ + for _, c := range d.Policy { + for _, want := range policyWords(c) { + if !strings.Contains(under, want) { + t.Errorf("%s leaves %s to the reader and the list does not say %q under it:\n%s", d.ID, c.Format, want, under) + } + if !strings.Contains(described, want) { + t.Errorf("%s leaves %s to the reader and its description does not say %q:\n%s", d.ID, c.Format, want, described) + } + } + if !strings.Contains(described, c.Why) { + t.Errorf("%s leaves %s to the reader and its description does not say why:\n%s", d.ID, c.Format, described) + } + } + } + if dependent == 0 || certain == 0 { + t.Fatalf("%d damage(s) leave an outcome to the reader and %d leave none - both kinds are needed, or half of this proved nothing", + dependent, certain) + } +} + +// policyWords is what a person has to be able to read about one case: the +// format, and every value of a setting the case is narrowed to. +func policyWords(c damage.PolicyCase) []string { + words := []string{c.Format} + for _, values := range c.When { + words = append(words, values...) + } + return words +} + +// listedUnder is the lines the list of damages prints under one damage's row. +func listedUnder(listing, id string) []string { + var block []string + inside := false + for _, line := range strings.Split(listing, "\n") { + switch { + case strings.HasPrefix(line, id+" "): + inside = true + case !strings.HasPrefix(line, " "): + inside = false + case inside: + block = append(block, line) + } + } + return block } // Every command that gets its own arguments is watched for --help. diff --git a/internal/guard/damagelength_test.go b/internal/guard/damagelength_test.go new file mode 100644 index 00000000..b8fe7f77 --- /dev/null +++ b/internal/guard/damagelength_test.go @@ -0,0 +1,369 @@ +package guard + +import ( + "bytes" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "os" + "path/filepath" + "testing" + + "github.com/donislawdev/TestingFilesGenerator/internal/core" + "github.com/donislawdev/TestingFilesGenerator/internal/damage" + "github.com/donislawdev/TestingFilesGenerator/internal/engine" + "github.com/donislawdev/TestingFilesGenerator/internal/format" + "github.com/donislawdev/TestingFilesGenerator/internal/manifest" +) + +// lengthSeed is the run seed these guards write with. Any number does, as long +// as the run being compared against uses the same one. +const lengthSeed = 11 + +// lengthRun plans and writes targets in one run and answers each target's +// manifest entry and path by its id. +func lengthRun(t *testing.T, out string, targets []engine.Target) (map[string]manifest.File, map[string]string) { + t.Helper() + opt := engine.Options{OutDir: out, Seed: lengthSeed, ManifestName: engine.DefaultManifestName} + planned, err := engine.Plan(targets, opt) + if err != nil { + t.Fatalf("planning in %s: %v", out, err) + } + var sum int64 + for _, tg := range targets { + for _, s := range tg.Sizes { + sum += s + } + } + if got := engine.TotalBytes(planned); got != sum { + t.Errorf("the plan in %s totals %d B and the sizes asked for add up to %d B - the free space check, the progress bar and --dry-run would all say the wrong number", + out, got, sum) + } + res, err := engine.Run(t.Context(), planned, opt) + if err != nil { + t.Fatalf("running in %s: %v", out, err) + } + entries, paths := map[string]manifest.File{}, map[string]string{} + for _, m := range res.Manifest.Files { + if m.Failed { + t.Fatalf("target %s failed: %s", m.TargetID, m.Error) + } + entries[m.TargetID] = m + paths[m.TargetID] = filepath.Join(out, filepath.FromSlash(m.Path)) + } + return entries, paths +} + +// lengthChangers is every damage that changes the length of what it is given, +// each at its defaults - which a guard has to ASSERT is not empty, or it +// would pass with nothing to judge. +func lengthChangers(t *testing.T) []damage.Chain { + t.Helper() + var out []damage.Chain + for _, d := range damage.All() { + if d.Length != nil { + out = append(out, damage.Chain{{ID: d.ID}}) + } + } + if len(out) < 2 { + t.Fatalf("%d damage(s) change the length, and truncate and append-junk both do - this guard would judge less than it claims", len(out)) + } + return out +} + +// TestADamagedFileIsExactlyTheSizeAskedFor holds untouchable rule 1 for every +// damage that changes the length of a file. +// +// Such a damage makes the generator write a different length from the one on +// disk, and until 2026-11-10 six places read the plan's count as the size of +// the file - the manifest, the length check, progress, the run's total, the +// free space check and --dry-run. One of them reading the wrong number gives a +// manifest that says 40 kB about a 20 kB file, or a file refused for being the +// size it was asked to be. So every format meets every such damage, at the +// smallest size the pair takes and at 20 kB, and the file on disk, the +// manifest and the plan's total all have to say the size that was asked for. +// +// A drawn size is asked too, in the formats with sizes they cannot reach - a +// band above a PNG's picture, odd counts in UTF-16 - because the draw has to +// ask the format about the size the damage starts from, not the one on disk. +func TestADamagedFileIsExactlyTheSizeAskedFor(t *testing.T) { + dir := t.TempDir() + judged := 0 + for _, f := range format.All() { + var targets []engine.Target + for n, chain := range lengthChangers(t) { + smallest := engine.SmallestDamaged(f, chain, format.Request{Label: true}) + for _, size := range witnessSizes(smallest) { + targets = append(targets, engine.Target{ID: fmt.Sprintf("d%d-%d", n, size), Format: f.ID, + Sizes: engine.Uniform(1, size), Damage: chain, Label: true}) + } + } + entries, paths := lengthRun(t, filepath.Join(dir, f.ID), targets) + for _, tg := range targets { + judged += checkLength(t, tg, entries[tg.ID], paths[tg.ID], tg.Sizes[0]) + } + } + + for _, c := range []struct { + format string + properties map[string]string + min, max int64 + }{ + {"png", nil, 150, 400}, + {"docx", nil, 1300, 4000}, + {"xml", map[string]string{"encoding": "utf-16le", "bom": "true"}, 600, 900}, + } { + for n, chain := range lengthChangers(t) { + tg := engine.Target{ID: fmt.Sprintf("range%d", n), Format: c.format, Properties: c.properties, + Range: engine.SizeRange{Used: true, Min: c.min, Max: c.max}, Sizes: engine.Uniform(40, 0), + Damage: chain, Label: true} + out := filepath.Join(dir, fmt.Sprintf("%s-range-%d", c.format, n)) + opt := engine.Options{OutDir: out, Seed: lengthSeed, ManifestName: engine.DefaultManifestName} + planned, err := engine.Plan([]engine.Target{tg}, opt) + if err != nil { + t.Errorf("%s %s: a range of %d to %d B was refused: %v", c.format, chain, c.min, c.max, err) + continue + } + res, err := engine.Run(t.Context(), planned, opt) + if err != nil { + t.Fatalf("%s %s: %v", c.format, chain, err) + } + for i, m := range res.Manifest.Files { + judged += checkLength(t, tg, m, filepath.Join(out, filepath.FromSlash(m.Path)), planned[i].Plan.Bytes) + if m.Bytes < c.min || m.Bytes > c.max { + t.Errorf("%s %s: a file of %d B was drawn from the range %d to %d B", c.format, chain, m.Bytes, c.min, c.max) + } + } + } + } + if judged == 0 { + t.Fatal("no damaged file was measured, so this guard proved nothing") + } +} + +// TestACutFromASizeTheFormatCannotWriteStartsFromTheNextOneItCan holds rule 1 +// where the file a cut starts from is not a size the format writes. +// +// The guard above never gets there. Half missing starts from twice the size, +// which is always even, and the sizes it asks are ones every format writes - +// the mutation run of 2026-11-10 took the move away and that guard stayed +// green. So a third missing is asked of a UTF-16 file, where the whole file +// would be an odd count the format refuses. +// +// It asserts that the whole file IS refused on its own. Without that, a +// format learning to write odd counts would leave this measuring a cut that +// needed no move, and passing. +func TestACutFromASizeTheFormatCannotWriteStartsFromTheNextOneItCan(t *testing.T) { + chain := damage.Chain{{ID: damage.Truncate, Values: damage.Values{damage.SettingMissing: "33"}}} + properties := map[string]string{"encoding": "utf-16le", "bom": "true"} + const size = 701 + whole, err := chain.Before(size) + if err != nil { + t.Fatal(err) + } + healthy := engine.Target{ID: "cut", Format: "xml", Properties: properties, Sizes: engine.Uniform(1, whole), Label: true} + if _, err := engine.Plan([]engine.Target{healthy}, engine.Options{OutDir: t.TempDir(), Seed: lengthSeed}); err == nil { + t.Fatalf("xml in UTF-16 writes %d B exactly, so a cut to %d B needs no move and this guard measures nothing", whole, size) + } + + cut := engine.Target{ID: "cut", Format: "xml", Properties: properties, Sizes: engine.Uniform(1, size), Damage: chain, Label: true} + entries, paths := lengthRun(t, t.TempDir(), []engine.Target{cut}) + if checkLength(t, cut, entries["cut"], paths["cut"], size) == 0 { + t.Fatal("the cut file was not measured, so this guard proved nothing") + } +} + +// checkLength is one damaged file against the size it was asked to be, and +// answers 1 for a file it measured. +func checkLength(t *testing.T, tg engine.Target, m manifest.File, path string, want int64) int { + t.Helper() + where := fmt.Sprintf("%s with %s", tg.Format, tg.Damage) + info, err := os.Stat(path) + if err != nil { + t.Errorf("%s: %v", where, err) + return 0 + } + if info.Size() != want || m.Bytes != want { + t.Errorf("%s: asked for %d B, the file on disk is %d B and the manifest says %d B", where, want, info.Size(), m.Bytes) + } + return 1 +} + +// TestATruncatedFileIsTheStartOfTheWholeOne holds D11 for truncate, and the +// early stop with it. +// +// A truncated file is the first bytes of the whole file the same recipe would +// write at the size the cut starts from - byte for byte, in every format. That +// is what makes the damage reproducible from its recipe, and it is also what +// proves stopping the generator early changes nothing: a generator writes from +// the front, so the bytes before the stop are the bytes it would have written +// anyway. A generator that wrote differently once stopped would turn this red. +// +// The whole file comes from a second run with the same seed and target id, so +// both files are drawn from the same seed. A size the cut would have to move +// up to - a band the format cannot write - is skipped, and the guard asserts +// that most formats were compared. +func TestATruncatedFileIsTheStartOfTheWholeOne(t *testing.T) { + dir := t.TempDir() + chain := damage.Chain{{ID: damage.Truncate}} + const size = witnessSize + whole, err := chain.Before(size) + if err != nil { + t.Fatal(err) + } + compared := 0 + for _, f := range format.All() { + cut := engine.Target{ID: "same", Format: f.ID, Sizes: engine.Uniform(1, size), Damage: chain, Label: true} + full := engine.Target{ID: "same", Format: f.ID, Sizes: engine.Uniform(1, whole), Label: true} + if _, err := engine.Plan([]engine.Target{full}, engine.Options{OutDir: t.TempDir(), Seed: lengthSeed}); err != nil { + // A size this format cannot write exactly: the cut started from + // the next one it can, which is a different whole file. + continue + } + _, cutPath := lengthRun(t, filepath.Join(dir, f.ID, "cut"), []engine.Target{cut}) + _, fullPath := lengthRun(t, filepath.Join(dir, f.ID, "whole"), []engine.Target{full}) + got, err := os.ReadFile(cutPath["same"]) + if err != nil { + t.Fatal(err) + } + all, err := os.ReadFile(fullPath["same"]) + if err != nil { + t.Fatal(err) + } + if int64(len(all)) != whole || !bytes.Equal(got, all[:size]) { + t.Errorf("%s: the truncated file is not the first %d B of the %d B file the same recipe writes", f.ID, size, whole) + } + compared++ + } + if compared < len(format.All())*3/4 { + t.Errorf("only %d of %d formats were compared, so this guard says little", compared, len(format.All())) + } +} + +// TestAppendedJunkIsTheWholeFileAndThenTheSameBytesEveryTime holds D11 and +// RC4 for append-junk. +// +// A file with junk appended is the whole file the same recipe writes at the +// smaller size, followed by the junk - nothing before the end moves. And the +// junk is frozen: drawn from the damage's own seed, derived from the file's, +// in a way the golden hash below pins. Changing how the bytes are drawn would +// change every file this damage ever wrote, which is a major version and a +// Breaking entry - this is the guard that says so. +func TestAppendedJunkIsTheWholeFileAndThenTheSameBytesEveryTime(t *testing.T) { + dir := t.TempDir() + chain := damage.Chain{{ID: damage.AppendJunk}} + const size = witnessSize + before, err := chain.Before(size) + if err != nil { + t.Fatal(err) + } + junk := size - before + + broken := engine.Target{ID: "same", Format: "txt", Sizes: engine.Uniform(1, size), Damage: chain, Label: true} + whole := engine.Target{ID: "same", Format: "txt", Sizes: engine.Uniform(1, before), Label: true} + _, brokenPath := lengthRun(t, filepath.Join(dir, "broken"), []engine.Target{broken}) + _, wholePath := lengthRun(t, filepath.Join(dir, "whole"), []engine.Target{whole}) + got, err := os.ReadFile(brokenPath["same"]) + if err != nil { + t.Fatal(err) + } + all, err := os.ReadFile(wholePath["same"]) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(got[:before], all) { + t.Errorf("the first %d B of the damaged file are not the %d B file the same recipe writes", before, before) + } + + // The junk is the damage's own draw for this file, and nothing else. + fileSeed := core.FileSeed(core.TargetSeed(lengthSeed, "same"), 0) + var tail bytes.Buffer + w, streams, err := chain.Open(&tail, junk, fileSeed) + if err != nil { + t.Fatal(err) + } + if _, err := w.Write(nil); err != nil { + t.Fatal(err) + } + if err := chain.Finish(streams); err != nil { + t.Fatal(err) + } + if !bytes.Equal(got[before:], tail.Bytes()) { + t.Error("the bytes after the whole file are not the damage's own draw for this file's seed") + } + + const golden = "62f8bf77f16ef1828d49e40899c817b3f7d811a242b03b6671a9b23827935f40" + sum := sha256.Sum256(got[before:]) + if hex.EncodeToString(sum[:]) != golden { + t.Errorf("the junk appended to the same recipe hashes to %s, and it was %s.\n"+ + "Reason: the bytes are frozen from the first release (D11). A change here changes every\n"+ + "file append-junk ever wrote - a major version and a Breaking entry, not a quiet edit.", + hex.EncodeToString(sum[:]), golden) + } +} + +// TestTheSmallestDamagedFileIsTakenAndOneByteLessIsNot asks every format and +// every damage that changes the length about the floor from both sides. +// +// The format's own minimum applies to the file the damage STARTS from, so a +// truncated PNG can be smaller than any healthy one and a PNG with junk +// appended has to be larger. The size a person sees is the one on disk, and +// the refusal has to name the smallest one that works in that size - not the +// size the generator was asked for, which nobody wrote. Planning only, so +// every pair is asked. +func TestTheSmallestDamagedFileIsTakenAndOneByteLessIsNot(t *testing.T) { + asked := 0 + for _, f := range format.All() { + for _, chain := range lengthChangers(t) { + smallest := engine.SmallestDamaged(f, chain, format.Request{Label: true}) + plan := func(size int64) error { + _, err := engine.Plan([]engine.Target{{ID: "t", Format: f.ID, Sizes: engine.Uniform(1, size), + Damage: chain, Label: true}}, engine.Options{OutDir: t.TempDir(), Seed: lengthSeed}) + return err + } + if err := plan(smallest); err != nil { + t.Errorf("%s with %s: %d B is announced as the smallest and was refused: %v", f.ID, chain, smallest, err) + continue + } + asked++ + if smallest == 0 { + continue + } + // One byte under, and far under: a refusal has to name the size + // that works from anywhere below it, not a floor of the damage + // that the format then refuses in turn. + for _, under := range []int64{smallest - 1, smallest / 3} { + if under == smallest-1 || under > 0 { + checkNamesTheSmallest(t, f.ID, chain, under, smallest, plan(under)) + } + } + } + } + if asked == 0 { + t.Fatal("no pair was asked, so this guard proved nothing") + } +} + +// checkNamesTheSmallest is one size under the smallest, refused in words that +// name the smallest. +func checkNamesTheSmallest(t *testing.T, formatID string, chain damage.Chain, under, smallest int64, err error) { + t.Helper() + var below *format.BelowMinimumError + var small *damage.TooSmallError + switch { + case err == nil: + t.Errorf("%s with %s: %d B was taken, under the %d B announced as the smallest", formatID, chain, under, smallest) + case errors.As(err, &below): + if below.Minimum != smallest || below.Requested != under { + t.Errorf("%s with %s: the refusal of %d B names %d B as the smallest and %d B as asked for: %v", + formatID, chain, under, below.Minimum, below.Requested, err) + } + case errors.As(err, &small): + if small.Floor != smallest { + t.Errorf("%s with %s: the refusal of %d B names %d B as the smallest: %v", formatID, chain, under, small.Floor, err) + } + default: + t.Errorf("%s with %s: %d B was refused with %T, which names no size: %v", formatID, chain, under, err, err) + } +} diff --git a/internal/guard/damagepolicy_test.go b/internal/guard/damagepolicy_test.go new file mode 100644 index 00000000..ac3006a5 --- /dev/null +++ b/internal/guard/damagepolicy_test.go @@ -0,0 +1,236 @@ +package guard + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "io" + "os" + "slices" + "strings" + "testing" + + "github.com/donislawdev/TestingFilesGenerator/internal/cli" + "github.com/donislawdev/TestingFilesGenerator/internal/damage" + "github.com/donislawdev/TestingFilesGenerator/internal/engine" + "github.com/donislawdev/TestingFilesGenerator/internal/format" + "github.com/donislawdev/TestingFilesGenerator/internal/manifest" + "github.com/donislawdev/TestingFilesGenerator/internal/recipe" +) + +// TestTheManifestSaysTheOutcomeDependsWhereReadersDiffer holds what the +// manifest says about a damaged file on both sides of the line. +// +// A file cut short is still text, so a txt cut short says unspecified, policy +// dependent, both answers offered and why - the shape MF5 gives a question +// with no single right answer. A PNG cut short has lost its end, which its +// rules forbid, and says reject with certainty. The settings count as well as +// the format: random bytes after UTF-8 text are not UTF-8, after UTF-16 text +// they mostly are. Owner's call on 2026-11-10. +// +// Read from the entry as it is written, because what a consumer finds there +// is the whole point of it. +func TestTheManifestSaysTheOutcomeDependsWhereReadersDiffer(t *testing.T) { + for _, c := range []struct { + format string + properties map[string]string + damage string + certain bool + }{ + {"txt", nil, damage.Truncate, false}, + {"png", nil, damage.Truncate, true}, + {"png", nil, damage.AppendJunk, false}, + {"txt", nil, damage.AppendJunk, true}, + {"txt", map[string]string{"encoding": "utf-16le"}, damage.AppendJunk, false}, + {"png", nil, damage.ZeroHead, true}, + } { + dir := t.TempDir() + target := engine.Target{ID: "d", Format: c.format, Properties: c.properties, + Sizes: engine.Uniform(1, 20<<10), Damage: damage.Chain{{ID: c.damage}}} + entries, _ := lengthRun(t, dir, []engine.Target{target}) + e := entries["d"].Expected + written, err := json.Marshal(entries["d"]) + if err != nil { + t.Fatal(err) + } + where := c.format + " with " + c.damage + if len(c.properties) > 0 { + where += " in utf-16le" + } + if c.certain { + if e.Outcome != manifest.OutcomeReject || e.Confidence != "certain" || bytes.Contains(written, []byte("policy_options")) { + t.Errorf("%s breaks the rules of the format and the manifest says %s", where, written) + } + continue + } + d, _ := damage.Get(c.damage) + pc, _ := d.PolicyFor(damage.Kind{Format: c.format, Settings: c.properties}) + if e.Outcome != manifest.OutcomeUnspecified || e.Confidence != "policy_dependent" || + !slices.Equal(e.PolicyOptions, []string{manifest.OutcomeAccept, manifest.OutcomeReject}) { + t.Errorf("%s leaves the outcome to the reader and the manifest says %+v", where, e) + } + if !bytes.Contains(written, []byte(`"policy_options":["accept","reject"]`)) { + t.Errorf("%s: the entry as it is written carries no policy_options: %s", where, written) + } + if e.Detail == "" || e.Detail != pc.Why { + t.Errorf("%s: the manifest explains it with %q, and the declaration says %q", where, e.Detail, pc.Why) + } + } +} + +// TestAcceptIsAllowedOnlyWhereTheOutcomeDependsOnTheReader asks both +// surfaces about the one expectation damage can rule out. +// +// Accept is refused beside a damage that breaks the rules of the format - a +// cut PNG - because nothing that follows them could accept it. Beside a +// damage that leaves the outcome to the reader - a cut txt - accept is one of +// the two answers the manifest offers, and a recipe stating it is writing +// down its own policy (owner's call, 2026-11-10, by the rule of 2026-09-09: +// refuse only what cannot be true). Both halves are asked, because a build +// refusing accept everywhere and a build refusing it nowhere would each pass +// one of them. +func TestAcceptIsAllowedOnlyWhereTheOutcomeDependsOnTheReader(t *testing.T) { + body := func(f string) []byte { + return []byte("version: 1\ntargets:\n - id: t\n format: " + f + + "\n size: 20kb\n damage: [truncate]\n expected: accept\n") + } + if _, err := recipe.Parse(body("png"), "r.yaml"); err == nil { + t.Error("a recipe cutting a png short and expecting it to be accepted was read without a word") + } + if _, err := recipe.Parse(body("txt"), "r.yaml"); err != nil { + t.Errorf("a recipe cutting a txt short and accepting it was refused, and accept is one of the answers its manifest offers: %v", err) + } + + run := func(f string) (int, int) { + dir := t.TempDir() + code := cli.Run(context.Background(), []string{"generate", "--format", f, "--size", "1kb", + "--damage", damage.Truncate, "--expected", "accept", "--out", dir}, io.Discard, io.Discard) + left, err := os.ReadDir(dir) + if err != nil { + t.Fatal(err) + } + return code, len(left) + } + if code, files := run("png"); code != cli.ExitUsage || files != 0 { + t.Errorf("png cut short with --expected accept ended with %d and left %d file(s), expected %d and none", + code, files, cli.ExitUsage) + } + if code, _ := run("txt"); code != cli.ExitOK { + t.Errorf("txt cut short with --expected accept ended with %d rather than %d", code, cli.ExitOK) + } +} + +// TestAChainThatCutsAwayWhatItAddedIsRefused asks both surfaces about a +// truncate placed after an append-junk. +// +// Each damage moves bytes, so the check that a damage changed something +// passes both, and the manifest would list junk the file no longer holds. +// Refused in the plan, with nothing written. The same two the other way round +// are a legitimate file - cut, then junk after the cut - and the guard asks +// that too, so a build refusing every chain of the two would be red. +func TestAChainThatCutsAwayWhatItAddedIsRefused(t *testing.T) { + eaten := damage.Chain{{ID: damage.AppendJunk}, {ID: damage.Truncate}} + fine := damage.Chain{{ID: damage.Truncate}, {ID: damage.AppendJunk}} + plan := func(chain damage.Chain) error { + _, err := engine.Plan([]engine.Target{{ID: "t", Format: "png", Sizes: engine.Uniform(1, 20<<10), Damage: chain}}, + engine.Options{OutDir: t.TempDir(), Seed: 5}) + return err + } + var refusal *damage.EatenError + if err := plan(eaten); !errors.As(err, &refusal) { + t.Errorf("truncate after append-junk was planned (%v), and the cut takes away what the junk added", err) + } + if err := plan(fine); err != nil { + t.Errorf("truncate before append-junk was refused, and it is a file with junk after a cut: %v", err) + } + + if _, err := recipe.Parse([]byte("version: 1\ntargets:\n - id: t\n format: png\n size: 20kb\n"+ + " damage: [append-junk, truncate]\n"), "r.yaml"); err == nil || !strings.Contains(err.Error(), "cuts away") { + t.Errorf("the recipe reader read truncate after append-junk without the refusal: %v", err) + } + + dir := t.TempDir() + var errOut bytes.Buffer + code := cli.Run(context.Background(), []string{"generate", "--format", "png", "--size", "20kb", + "--damage", damage.AppendJunk, "--damage", damage.Truncate, "--out", dir}, io.Discard, &errOut) + left, err := os.ReadDir(dir) + if err != nil { + t.Fatal(err) + } + if code != cli.ExitUsage || len(left) != 0 { + t.Errorf("the command line ended with %d and left %d file(s) for truncate after append-junk: %s", + code, len(left), errOut.String()) + } +} + +// TestAnArchiveSizedByItsContentsRefusesADamageThatChangesTheLength holds the +// line between two definitions of one number. +// +// An archive whose size comes from what it holds has no size until it is +// written, so a damage that needs to know the size on disk - to cut to it, +// or to leave room for junk - has nothing to work from, and would have to +// make one up. zero-head keeps the length and is allowed there, which is the +// control: a build refusing every damage on such an archive would be a wall. +func TestAnArchiveSizedByItsContentsRefusesADamageThatChangesTheLength(t *testing.T) { + body := func(d string) []byte { + return []byte("version: 1\ntargets:\n - id: t\n format: zip\n contains:\n - format: txt\n" + + " size: 1kb\n count: 2\n damage: [" + d + "]\n") + } + for _, c := range []struct { + damage string + refused bool + }{{damage.Truncate, true}, {damage.AppendJunk, true}, {damage.ZeroHead, false}} { + _, err := recipe.Parse(body(c.damage), "r.yaml") + if c.refused != (err != nil) || (err != nil && !strings.Contains(err.Error(), "changes the length")) { + t.Errorf("an archive sized by its contents with %s: refused %t, and the reader said %v", c.damage, c.refused, err) + } + + // The engine answers the same, for a caller that never read a recipe. + target := engine.Target{ID: "t", Format: "zip", SizeFromContents: true, Sizes: engine.Uniform(1, 0), + Contains: []format.Content{{Format: "txt", Bytes: 1024, Count: 2}}, + Damage: damage.Chain{{ID: c.damage}}} + _, err = engine.Plan([]engine.Target{target}, engine.Options{OutDir: t.TempDir(), Seed: 5}) + var refusal *damage.NeedsSizeError + if c.refused != errors.As(err, &refusal) || (!c.refused && err != nil) { + t.Errorf("the engine planned an archive sized by its contents with %s: %v", c.damage, err) + } + } +} + +// TestEveryPolicyCaseNamesAFormatAndSettingsThatExist keeps the declarations +// of where a damage depends on the reader honest about what they name. +// +// They are written by hand, as format ids and setting values, so a format +// renamed or a value dropped would leave a case that matches nothing - and +// the file it was written for would be called certain without anybody +// deciding it (pitfall 15). +func TestEveryPolicyCaseNamesAFormatAndSettingsThatExist(t *testing.T) { + cases := 0 + for _, d := range damage.All() { + for _, c := range d.Policy { + cases++ + f, err := format.Get(c.Format) + if err != nil { + t.Errorf("%s declares a policy case for %q, which is not a format", d.ID, c.Format) + continue + } + for name, values := range c.When { + i := slices.IndexFunc(f.Properties, func(p format.Property) bool { return p.Name == name }) + if i < 0 { + t.Errorf("%s on %s is narrowed to the setting %q, which %s does not have", d.ID, c.Format, name, c.Format) + continue + } + for _, v := range values { + if !slices.Contains(f.Properties[i].Choices, v) { + t.Errorf("%s on %s is narrowed to %s=%s, which is not one of its choices %v", + d.ID, c.Format, name, v, f.Properties[i].Choices) + } + } + } + } + } + if cases == 0 { + t.Fatal("no damage declares a policy case, so this guard checked nothing") + } +} diff --git a/internal/guard/damagerefused_test.go b/internal/guard/damagerefused_test.go index 2079a48e..ef51fa7b 100644 --- a/internal/guard/damagerefused_test.go +++ b/internal/guard/damagerefused_test.go @@ -100,7 +100,7 @@ func TestADamageThatWouldChangeNothingStopsTheFile(t *testing.T) { chain := damage.Chain{{ID: damage.ZeroHead}} var sink discard - w, streams, err := chain.Open(&sink) + w, streams, err := chain.Open(&sink, int64(len(source)), 1) if err != nil { t.Fatalf("opening the chain: %v", err) } diff --git a/internal/guard/damagewitness_test.go b/internal/guard/damagewitness_test.go index bad5a618..874294b9 100644 --- a/internal/guard/damagewitness_test.go +++ b/internal/guard/damagewitness_test.go @@ -1,8 +1,12 @@ package guard import ( + "errors" "fmt" + "os" "path/filepath" + "regexp" + "slices" "strconv" "strings" "testing" @@ -10,6 +14,7 @@ import ( "github.com/donislawdev/TestingFilesGenerator/internal/damage" "github.com/donislawdev/TestingFilesGenerator/internal/engine" "github.com/donislawdev/TestingFilesGenerator/internal/format" + "github.com/donislawdev/TestingFilesGenerator/internal/format/eml" "github.com/donislawdev/TestingFilesGenerator/internal/oracle" ) @@ -26,67 +31,137 @@ type damagePoint struct { label string } -// witnessCase is one damaged file and the healthy file of the same format -// and size beside it. +// witnessCase is one damaged file and the healthy file of the same format, +// settings and size beside it. type witnessCase struct { format string point damagePoint size int64 - good string - broken string + // setting is the one format setting moved from its default, empty for a + // file at its defaults. + setting map[string]string + kind damage.Kind + good string + broken string +} + +// policyRefusals is, for a pair the manifest calls policy dependent, every +// reason the structural check gives when it refuses one of these files - read +// and judged by a person as the shape this tool writes, or as a rule the +// format states and readers skip, rather than a rule a reader has to enforce. +// +// Written out rather than allowed wholesale, because the list is the second +// direction of the witness rule. A refusal for a reason NOT here may be the +// format's own rule broken, and then the pair should be certain - so it turns +// the guard red until somebody reads it and decides. A reason here that no +// file gives any more turns it red too, because a list nothing hits is a list +// nobody can trust (pitfall 15). Read from the measurements of 2026-11-10 +// (docs/DAMAGE-ETAP-2-2026-11-09.md sections 12.1 to 12.3). +var policyRefusals = map[[2]string][]string{ + // The shape this tool writes: the last line ends with a line break. + {damage.Truncate, "csv"}: {"the file does not end with"}, + {damage.Truncate, "log"}: {"the file does not end with a newline"}, + {damage.Truncate, "toml"}: {"the document does not end with a newline"}, + {damage.Truncate, "yaml"}: {"the document does not end with a newline"}, + {damage.Truncate, "html"}: {"was ordered and the page is not", "the document does not end with a closing"}, + {damage.Truncate, "md"}: {"so one block is never closed"}, + {damage.Truncate, "eml"}: {"the message has no blank line after its header lines", "layout=newsletter and the plain text", + // A rule the cut breaks only when it falls between a CR and its LF. + "a carriage return or a line feed on its own"}, + {damage.Truncate, "txt"}: {}, + // Bytes past the format's own end, which its rules forbid and readers skip. + {damage.AppendJunk, "png"}: {"declares"}, + {damage.AppendJunk, "gif"}: {"the trailer is at"}, + {damage.AppendJunk, "jpg"}: {"the file does not end with the end of image marker"}, + {damage.AppendJunk, "pdf"}: {"the end of file marker is not near the end"}, + {damage.AppendJunk, "rtf"}: {"outside 7 bit text"}, + {damage.AppendJunk, "zip"}: {"the comment declares"}, + {damage.AppendJunk, "docx"}: {"the comment declares"}, + {damage.AppendJunk, "xlsx"}: {"the comment declares"}, + {damage.AppendJunk, "pptx"}: {"the comment declares"}, + {damage.AppendJunk, "targz"}: {"of trailer should follow the stream"}, + {damage.AppendJunk, "eml"}: {"a part is not shaped the way this tool writes one", "RFC 5322 asks for", + "after its closing boundary", "is not a line of base64", + // A rule the junk breaks only when its random bytes hold a CR or an + // LF on its own - about four seeds in ten, so not every file. + "a carriage return or a line feed on its own"}, + // UTF-16 in the next two: the junk breaks the encoding only when its + // random bytes form a lone surrogate, which is a matter of the seed. + {damage.AppendJunk, "txt"}: {"that are not printable ASCII", "does not decode as utf-16"}, + {damage.AppendJunk, "md"}: {"that are not printable ASCII", "does not decode as utf-16"}, + // Bytes the format allows: nothing points at them. + {damage.AppendJunk, "ico"}: {}, + {damage.AppendJunk, "tiff"}: {}, } // TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat is the witness // rule as a guard rather than a measurement taken once. // -// A damage is offered for a format only when a judge refuses the result, so a -// test written against the file can tell an upload validator that let it -// through. Until 2026-11-09 that was a sweep from September, and the offer -// covered every format, including one added the day after the sweep. Now -// every format in the registry meets every damage in the registry, with its -// default values and each parameter at both ends, at the smallest size the -// pair can be given and at 20 kB. Each file comes from the real engine with -// its label, as both surfaces write by default, and the structural checker -// judges it beside a healthy file of the same format and size. +// Every format meets every damage, and the manifest says one of two things +// about the result (owner's call, 2026-11-10). Where the damage breaks the +// format's own rules it says reject with certainty, and that is only honest +// if a judge refuses the file - so a damaged file the structural check +// accepts is red. Where the file's fate depends on the reader it says so, and +// then the check has to accept the file or refuse it for one of the reasons +// in policyRefusals - a new reason is red until a person decides which side +// of the line it stands on. // -// Red when a damaged file is accepted, and red when the checker falls over -// instead of answering: a crash is a fault in the checker, never a witness. A -// format the checker does not know is red too. Nobody has decided whether a -// damage can be offered there, and the offer must not reach it by default. +// Two sweeps, each file from the real engine with its label, as both surfaces +// write by default, and each judged beside a healthy file of the same format, +// settings and size. The first takes every damage at its defaults and with +// each parameter at both ends, at the smallest size the pair can be given and +// at 20 kB. The second takes every setting of every format one at a time - a +// choice, or the other side of a flag - with each damage at its defaults at +// 20 kB, because whether a damage is noticed moves with the settings as well +// as the size: a cut line of one log shape is still a line and of another is +// not, and random bytes are text in UTF-16 and not in UTF-8 (section 12.2). // -// Measured at the default settings of each format. A format whose settings -// change the answer (the shape of a log line, the separator of a table) is -// judged here only as it comes by default, and that limit is written down -// in docs/DAMAGE-ETAP-2-2026-11-09.md section 2.2. +// Red as well when the checker falls over instead of answering - a crash is a +// fault in the checker, never a witness - and for a format the checker does +// not know: nobody has decided what a damage does there. +// +// A setting the format refuses on its own (a pair that needs another one +// beside it) is skipped, and so are numbers, which are dimensions and counts +// rather than shapes. Chains of several damages are not swept: the manifest +// is certain about a chain when one member is, by the reasoning in +// damage.Chain.Certain, and that is the limit written down in section 12.9. func TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat(t *testing.T) { dir := t.TempDir() - var points []damagePoint + var points, defaults []damagePoint for _, d := range damage.All() { points = append(points, damagePointsOf(t, d)...) + defaults = append(defaults, damagePoint{id: d.ID, values: damage.Values{}, label: d.ID + " at its defaults"}) } if len(points) == 0 { t.Fatal("the damage registry is empty, so this guard would pass without judging anything") } + told := strictTakesSettings(t) var cases []witnessCase formats := format.All() + swept := 0 for _, f := range formats { if !oracle.StrictKnows(f.ID) { - t.Errorf("%s has no structural check, so no damage has a witness there.\n"+ - "Reason: the offer of a damage reaches every format by default, so a format\n"+ - "nobody can judge would be handed damaged files nothing refuses. Give it a\n"+ - "check in internal/oracle/strict.py before it is registered.", f.ID) + t.Errorf("%s has no structural check, so nobody can say what a damage does there.\n"+ + "Reason: every damage reaches every format, so a format nobody can judge would be\n"+ + "handed damaged files the manifest says something about that nothing measured.\n"+ + "Give it a check in internal/oracle/strict.py before it is registered.", f.ID) continue } - cases = append(cases, witnessCasesOf(t, dir, f, points)...) + atDefaults := witnessCasesOf(t, dir, f, points) + bySetting := settingCasesOf(t, dir, f, defaults) + swept += len(bySetting) + cases = append(cases, atDefaults...) + cases = append(cases, bySetting...) } reqs := make([]oracle.StrictRequest, 0, 2*len(cases)) for _, c := range cases { + settings := settingsTold(told, c.format, c.setting) reqs = append(reqs, - oracle.StrictRequest{Format: c.format, Path: c.good}, - oracle.StrictRequest{Format: c.format, Path: c.broken}) + oracle.StrictRequest{Format: c.format, Path: c.good, Settings: settings}, + oracle.StrictRequest{Format: c.format, Path: c.broken, Settings: settings}) } got, available, err := oracle.StrictBatch(t.Context(), reqs) if !available { @@ -96,40 +171,117 @@ func TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat(t *testing.T) { t.Fatal(err) } - witnessed := 0 + j := witnessJudge{t: t, hit: map[[2]string]map[string]bool{}} for i, c := range cases { - good, broken := got[2*i], got[2*i+1] - where := fmt.Sprintf("%s at %d B with %s", c.format, c.size, c.point.label) - if good.Verdict != oracle.StrictOK { - // The control. A checker that refused everything would witness - // every damage, so a pair whose healthy file is refused says - // nothing about the damage. - t.Errorf("%s: the healthy file was answered %s %q, so this pair says nothing about the damage", - where, good.Verdict, good.Reason) - continue - } - switch broken.Verdict { - case oracle.StrictFail: - witnessed++ - case oracle.StrictOK: + j.judge(c, got[2*i], got[2*i+1]) + } + j.staleReasons() + + if want := len(formats) * len(points); len(cases)-swept < want { + t.Errorf("%d pair(s) judged at the defaults, and %d formats by %d damage points need at least %d", + len(cases)-swept, len(formats), len(points), want) + } + // Both answers reached, so neither half of this guard is green by never + // being asked. + if j.certain == 0 || j.depends == 0 { + t.Errorf("%d pair(s) judged certain and %d policy dependent - a guard that never meets one of the two proves nothing about it", + j.certain, j.depends) + } + if swept == 0 { + t.Error("no format setting was swept, so the second half of this guard judged nothing") + } + t.Logf("%d certain pair(s) refused and %d policy dependent pair(s) judged, %d of them with a setting moved, across %d formats", + j.certain, j.depends, swept, len(formats)) +} + +// witnessJudge holds what the guard above counts while it reads the answers. +type witnessJudge struct { + t *testing.T + certain int + depends int + // hit is every policy refusal reason a file gave, by pair. + hit map[[2]string]map[string]bool +} + +func (j *witnessJudge) judge(c witnessCase, good, broken oracle.StrictJudgement) { + t := j.t + where := fmt.Sprintf("%s at %d B with %s", c.format, c.size, c.point.label) + for name, value := range c.setting { + where += fmt.Sprintf(" and %s=%s", name, value) + } + if good.Verdict != oracle.StrictOK { + // The control. A checker that refused everything would witness every + // damage, so a pair whose healthy file is refused says nothing. + t.Errorf("%s: the healthy file was answered %s %q, so this pair says nothing about the damage", + where, good.Verdict, good.Reason) + return + } + if broken.Verdict != oracle.StrictOK && broken.Verdict != oracle.StrictFail { + t.Errorf("%s: the checker answered %s %q about the damaged file.\n"+ + "Reason: that is a fault in the checker, not a refusal of the file. Counted as\n"+ + "a witness, a checker that falls over on every damaged file would witness\n"+ + "every damage.", where, broken.Verdict, broken.Reason) + return + } + certain, _, err := damage.Chain{{ID: c.point.id, Values: c.point.values}}.Certain(c.kind) + if err != nil { + t.Errorf("%s: %v", where, err) + return + } + if certain { + j.certain++ + if broken.Verdict == oracle.StrictOK { t.Errorf("%s: the damaged file was ACCEPTED (%s).\n"+ - "Reason: a damaged file nothing refuses is one no test can be written against,\n"+ - "and its manifest calls it expected: reject. Either the damage is not offered\n"+ - "for this format, or the structural check is missing a rule of the format.", - where, broken.Reason) - default: - t.Errorf("%s: the checker answered %s %q about the damaged file.\n"+ - "Reason: that is a fault in the checker, not a refusal of the file. Counted as\n"+ - "a witness, a checker that falls over on every damaged file would witness\n"+ - "every damage.", where, broken.Verdict, broken.Reason) + "Reason: its manifest says reject with certainty, and nothing refuses it. Either the\n"+ + "structural check is missing a rule of the format, or the damage leaves this file's\n"+ + "fate to the reader and belongs in the damage's policy cases.", where, broken.Reason) + } + return + } + j.depends++ + if broken.Verdict == oracle.StrictOK { + return + } + pair := [2]string{c.point.id, c.format} + for _, reason := range policyRefusals[pair] { + if strings.Contains(broken.Reason, reason) { + if j.hit[pair] == nil { + j.hit[pair] = map[string]bool{} + } + j.hit[pair][reason] = true + return } } - if want := len(formats) * len(points); len(cases) < want { - t.Errorf("%d pair(s) judged, and %d formats by %d damage points need at least %d", - len(cases), len(formats), len(points), want) + t.Errorf("%s: the manifest says the outcome depends on the reader, and the structural check\n"+ + "refused the file for a reason nobody has read: %q.\n"+ + "Reason: if the damage breaks a rule of the format here, the pair is certain and the\n"+ + "manifest should say reject - take it out of the damage's policy cases. If the reason is\n"+ + "the shape this tool writes, or bytes past the format's end, add it to policyRefusals.", + where, broken.Reason) +} + +// staleReasons turns a reason no file gave into a failure, and a pair listed +// here that the registry no longer calls policy dependent. +func (j *witnessJudge) staleReasons() { + t := j.t + for pair, reasons := range policyRefusals { + d, err := damage.Get(pair[0]) + if err != nil { + t.Errorf("policyRefusals lists %s, which is not a damage", pair[0]) + continue + } + if !slices.ContainsFunc(d.Policy, func(c damage.PolicyCase) bool { return c.Format == pair[1] }) { + t.Errorf("policyRefusals lists %s on %s, and %s has no policy case for %s", + pair[0], pair[1], pair[0], pair[1]) + } + for _, reason := range reasons { + if !j.hit[pair][reason] { + t.Errorf("policyRefusals expects %s on %s to be refused with %q, and no file was.\n"+ + "Reason: a reason nothing gives any more is a list going stale - take it out.", + pair[0], pair[1], reason) + } + } } - t.Logf("%d damaged file(s) refused, across %d formats and %d damage point(s)", - witnessed, len(formats), len(points)) } // damagePointsOf is one damage with its defaults, and with each parameter at @@ -160,58 +312,158 @@ func damagePointsOf(t *testing.T, d damage.Descriptor) []damagePoint { return points } -// witnessCasesOf writes, for one format, a healthy and a damaged file of each -// damage point at the two sizes, all in one run of the engine. +// witnessCasesOf writes, for one format at its defaults, a healthy and a +// damaged file of each damage point at the two sizes, all in one run of the +// engine. func witnessCasesOf(t *testing.T, dir string, f format.Descriptor, points []damagePoint) []witnessCase { t.Helper() - // With the label, because that is how both surfaces write by default. - smallest := f.SmallestAccepted(format.Request{Label: true}) - var targets []engine.Target var cases []witnessCase sized := map[int64]bool{} for n, p := range points { chain := damage.Chain{{ID: p.id, Values: p.values}} - floor, _, err := chain.Floor() - if err != nil { - t.Fatalf("%s: %v", p.label, err) - } - for _, size := range witnessSizes(max(smallest, floor)) { + // With the label, because that is how both surfaces write by default. + smallest := engine.SmallestDamaged(f, chain, format.Request{Label: true}) + for _, size := range witnessSizes(smallest) { id := fmt.Sprintf("p%d-%d", n, size) targets = append(targets, engine.Target{ID: id, Format: f.ID, Sizes: engine.Uniform(1, size), Damage: chain, Label: true}) - cases = append(cases, witnessCase{format: f.ID, point: p, size: size, broken: id}) - if !sized[size] { - sized[size] = true - targets = append(targets, engine.Target{ID: fmt.Sprintf("good-%d", size), Format: f.ID, Sizes: engine.Uniform(1, size), Label: true}) + // A file cut short can be smaller than any healthy file of its + // format, so the control is the nearest size the format writes. + good := healthySizeFrom(t, f, nil, size) + cases = append(cases, witnessCase{format: f.ID, point: p, size: size, kind: damage.KindOf(f, nil), + good: fmt.Sprintf("good-%d", good), broken: id}) + if !sized[good] { + sized[good] = true + targets = append(targets, engine.Target{ID: fmt.Sprintf("good-%d", good), Format: f.ID, Sizes: engine.Uniform(1, good), Label: true}) } } } + paths := writeWitnessFiles(t, filepath.Join(dir, f.ID), f.ID, targets) + for i := range cases { + c := &cases[i] + c.good, c.broken = paths[c.good], paths[c.broken] + if c.good == "" || c.broken == "" { + t.Fatalf("%s: the run wrote no file for %s at %d B", f.ID, c.point.label, c.size) + } + } + return cases +} - out := filepath.Join(dir, f.ID) +// settingCasesOf writes, for one format, every setting moved from its default +// on its own - with a healthy file and one damaged by each damage at its +// defaults, at 20 kB or the smallest size the setting allows. +func settingCasesOf(t *testing.T, dir string, f format.Descriptor, defaults []damagePoint) []witnessCase { + t.Helper() + var targets []engine.Target + var cases []witnessCase + for n, setting := range settingVariants(f) { + healthy := engine.Target{ID: fmt.Sprintf("s%d-good", n), Format: f.ID, Properties: setting, Label: true} + size := max(witnessSize, f.SmallestAccepted(format.Request{Label: true, Properties: setting})) + healthy.Sizes = engine.Uniform(1, size) + // Planned alone first: a setting the format refuses without another + // beside it is skipped here rather than taking the whole run down. + if _, err := engine.Plan([]engine.Target{healthy}, engine.Options{OutDir: dir, Seed: 5}); err != nil { + continue + } + targets = append(targets, healthy) + for m, p := range defaults { + chain := damage.Chain{{ID: p.id, Values: p.values}} + at := max(size, engine.SmallestDamaged(f, chain, format.Request{Label: true, Properties: setting})) + id := fmt.Sprintf("s%d-d%d", n, m) + good := healthy.ID + if at != size { + good = id + "-good" + targets = append(targets, engine.Target{ID: good, Format: f.ID, Properties: setting, + Sizes: engine.Uniform(1, healthySizeFrom(t, f, setting, at)), Label: true}) + } + targets = append(targets, engine.Target{ID: id, Format: f.ID, Properties: setting, Sizes: engine.Uniform(1, at), Damage: chain, Label: true}) + cases = append(cases, witnessCase{format: f.ID, point: p, size: at, setting: setting, + kind: damage.KindOf(f, setting), good: good, broken: id}) + } + } + if len(targets) == 0 { + return nil + } + paths := writeWitnessFiles(t, filepath.Join(dir, f.ID+"-settings"), f.ID, targets) + for i := range cases { + c := &cases[i] + c.good, c.broken = paths[c.good], paths[c.broken] + if c.good == "" || c.broken == "" { + t.Fatalf("%s: the run wrote no file for %s with %v", f.ID, c.point.label, c.setting) + } + } + return cases +} + +// settingVariants is every setting of a format moved from its default on its +// own: each other choice of a closed set, and the other side of a flag. +func settingVariants(f format.Descriptor) []map[string]string { + var out []map[string]string + for _, p := range f.Properties { + var values []string + switch p.Kind { + case format.PropertyChoice: + values = p.Choices + case format.PropertyBool: + values = []string{"true", "false"} + case format.PropertyInt, format.PropertySize, format.PropertyDuration, format.PropertyText: + // A number, a size, a length of time and free text are how much + // of the file there is, not which shape it takes. Not swept. + } + for _, v := range values { + if v != p.Default { + out = append(out, map[string]string{p.Name: v}) + } + } + } + return out +} + +// writeWitnessFiles plans and writes the targets in one run, and answers the +// path of each target's file by its id. +func writeWitnessFiles(t *testing.T, out, formatID string, targets []engine.Target) map[string]string { + t.Helper() opt := engine.Options{OutDir: out, Seed: 5, ManifestName: engine.DefaultManifestName} planned, err := engine.Plan(targets, opt) if err != nil { - t.Fatalf("%s: planning the witness files: %v", f.ID, err) + t.Fatalf("%s: planning the witness files: %v", formatID, err) } res, err := engine.Run(t.Context(), planned, opt) if err != nil { - t.Fatalf("%s: writing the witness files: %v", f.ID, err) + t.Fatalf("%s: writing the witness files: %v", formatID, err) } paths := map[string]string{} for _, m := range res.Manifest.Files { if m.Failed { - t.Fatalf("%s: target %s failed: %s", f.ID, m.TargetID, m.Error) + t.Fatalf("%s: target %s failed: %s", formatID, m.TargetID, m.Error) + } + path := filepath.Join(out, filepath.FromSlash(m.Path)) + if !strings.HasPrefix(path, out) { + t.Fatalf("%s: target %s was written outside the run: %s", formatID, m.TargetID, path) } - paths[m.TargetID] = filepath.Join(out, filepath.FromSlash(m.Path)) + paths[m.TargetID] = path } - for i := range cases { - c := &cases[i] - c.good, c.broken = paths[fmt.Sprintf("good-%d", c.size)], paths[c.broken] - if c.good == "" || c.broken == "" || !strings.HasPrefix(c.broken, out) { - t.Fatalf("%s: the run wrote no file for %s at %d B", f.ID, c.point.label, c.size) + return paths +} + +// healthySizeFrom is the first size at or above from that the format writes +// as a healthy file, jumping along the sizes its refusals name. +func healthySizeFrom(t *testing.T, f format.Descriptor, setting map[string]string, from int64) int64 { + t.Helper() + at := from + for round := 0; round < 64; round++ { + _, err := f.Generator.Plan(format.Request{Bytes: at, Label: true, Properties: setting}) + if err == nil { + return at } + var below *format.BelowMinimumError + if !errors.As(err, &below) || below.Minimum <= at { + t.Fatalf("%s writes no healthy file at or above %d B: %v", f.ID, from, err) + } + at = below.Minimum } - return cases + t.Fatalf("%s kept naming larger sizes from %d B", f.ID, from) + return 0 } // witnessSizes is the smallest size a pair can be given, and the comfortable @@ -222,3 +474,43 @@ func witnessSizes(smallest int64) []int64 { } return []int64{smallest, witnessSize} } + +// strictTakesSettings is the formats whose structural check is told the +// settings of the file, read from the checker itself rather than copied, so a +// format that starts taking them is told them here the same day. +func strictTakesSettings(t *testing.T) map[string]bool { + t.Helper() + source, err := os.ReadFile(filepath.Join(repoRoot(t), "internal", "oracle", "strict.py")) + if err != nil { + t.Fatal(err) + } + line := regexp.MustCompile(`(?m)^TAKES_SETTINGS = \{([^}]*)\}`).FindSubmatch(source) + if line == nil { + t.Fatal("strict.py no longer declares TAKES_SETTINGS on one line, so this guard cannot tell which checks take settings") + } + out := map[string]bool{} + for _, name := range regexp.MustCompile(`"([a-z0-9]+)"`).FindAllSubmatch(line[1], -1) { + out[string(name[1])] = true + } + if !out["csv"] { + t.Fatalf("read %v as the checks that take settings, and csv - which always has - is not among them", out) + } + return out +} + +// settingsTold is the setting the checker is told about a file, for a check +// that takes them. The attachments of a message are not told, the way the +// mail guards leave them out (emlSettings). +func settingsTold(told map[string]bool, formatID string, setting map[string]string) map[string]string { + if !told[formatID] || len(setting) == 0 { + return nil + } + out := map[string]string{} + for name, value := range setting { + if formatID == "eml" && (name == eml.AttachmentFormat || name == eml.AttachmentSize) { + continue + } + out[name] = value + } + return out +} diff --git a/internal/guard/mutation/entries.py b/internal/guard/mutation/entries.py index e41d77be..e734040e 100644 --- a/internal/guard/mutation/entries.py +++ b/internal/guard/mutation/entries.py @@ -5850,8 +5850,8 @@ ( "a chain applies its damages in the order it was built rather than listed", "internal/damage/apply.go", - "\tfor i := len(c) - 1; i >= 0; i-- {", - "\tfor i := 0; i < len(c); i++ {", + "\tfor i := len(steps) - 1; i >= 0; i-- {\n\t\ts, err := steps[i].d.Open(", + "\tfor i := 0; i < len(steps); i++ {\n\t\ts, err := steps[i].d.Open(", "TestAChainAppliesItsDamagesInTheOrderItListsThem", ), ( @@ -5864,8 +5864,8 @@ ( "the writer stops putting damage in the path and writes a whole file", "internal/engine/parallel.go", - "\tif f.Damaged() {\n\t\tdamaged, opened, err := f.Target.Damage.Open(counter)", - "\tif f.Damaged() && false {\n\t\tdamaged, opened, err := f.Target.Damage.Open(counter)", + "\tif f.Damaged() {\n\t\tdamaged, opened, err := f.Target.Damage.Open(counter, f.Plan.Bytes, f.Seed)", + "\tif f.Damaged() && false {\n\t\tdamaged, opened, err := f.Target.Damage.Open(counter, f.Plan.Bytes, f.Seed)", "TestEveryDamageIsRefusedByTheStructuralCheckInEveryFormat", ), ( @@ -5878,8 +5878,8 @@ ( "a damaged file goes back to having no expectation declared for it", "internal/engine/engine.go", - "\t\tif f.Damaged() {\n\t\t\treturn manifest.Expected{\n\t\t\t\tOutcome: manifest.OutcomeReject,", - "\t\tif f.Damaged() && false {\n\t\t\treturn manifest.Expected{\n\t\t\t\tOutcome: manifest.OutcomeReject,", + "\t\tif f.Damaged() {\n\t\t\treturn damagedExpectation(f)", + "\t\tif f.Damaged() && false {\n\t\t\treturn damagedExpectation(f)", "TestTheManifestOfADamagedRunSaysSoAndExpectsRejection", ), ( @@ -5943,17 +5943,24 @@ # which is the state 0.3.0 shipped in. "the engine stops asking, so the command line writes a damaged file it calls acceptable", "internal/engine/damage.go", - "\treturn checkDamageExpectation(t)", + "\treturn checkDamageExpectation(t, desc)", "\treturn nil", "TestTheCommandLineRefusesDamageBesideAcceptToo", ), ( "the refusal comes back as a format problem, telling a script to try another format", "internal/cli/errors.go", - "\t\treturn ExitUsage, true\n\t}\n\tif code, ok := classifyFormat(err); ok {", - "\t\treturn ExitFormat, true\n\t}\n\tif code, ok := classifyFormat(err); ok {", + "\tif errors.As(err, &impossible) {\n\t\treturn ExitUsage, true", + "\tif errors.As(err, &impossible) {\n\t\treturn ExitFormat, true", "TestTheCommandLineRefusesDamageBesideAcceptToo", ), + ( + "a chain that cuts away what it added comes back as a format problem, telling a script to try another format", + "internal/cli/errors.go", + "\tif errors.As(err, &eaten) {\n\t\treturn ExitUsage, true", + "\tif errors.As(err, &eaten) {\n\t\treturn ExitFormat, true", + "TestAChainThatCutsAwayWhatItAddedIsRefused", + ), ( "the word damage rules out drifts from the one the manifest knows", "internal/damage/refusals.go", @@ -17155,4 +17162,221 @@ " ctx, cancel := context.WithTimeout(context.Background(), limit)", "TestAStoppedCallerStopsTheStructuralChecker", ), + ( + "a cut lets the generator write on after the bytes it keeps", + "internal/damage/truncate.go", + "\treturn n, ErrKeptEnough", + "\treturn n, nil", + "TestACutKeepsItsBytesAndStopsTheGenerator", + ), + ( + "a cut that dropped bytes says it touched nothing", + "internal/damage/truncate.go", + "\tt.cut = true\n", + "\n", + "TestACutKeepsItsBytesAndStopsTheGenerator", + ), + ( + "the floor of a cut lets through a file whose cut is three bytes", + "internal/damage/truncate.go", + "\treturn (smallestCut-1)*(100-m)/m + 1", + "\treturn (smallestCut-1)*(100-m)/m", + "TestTheFloorOfACutFollowsItsShareAndAChainAddsUp", + ), + ( + "the cut is rounded down, so less than the share stated goes missing", + "internal/damage/truncate.go", + "\treturn q*m + (r*m+rest-1)/rest", + "\treturn q*m + (r*m)/rest", + "TestTheFloorOfACutFollowsItsShareAndAChainAddsUp", + ), + ( + "the junk is drawn in another byte order, so every appended file changes", + "internal/damage/appendjunk.go", + "\t\t\tbinary.LittleEndian.PutUint64(buf[i:], r.Uint64())", + "\t\t\tbinary.BigEndian.PutUint64(buf[i:], r.Uint64())", + "TestAppendedJunkIsTheWholeFileAndThenTheSameBytesEveryTime", + ), + ( + "append-junk asks the generator for the whole size, so the file comes out larger", + "internal/damage/appendjunk.go", + "{ return out - int64(junkBytes(v)) }", + "{ return out }", + "TestADamagedFileIsExactlyTheSizeAskedFor", + ), + ( + "every damage of a chain draws from the seed of the place after its own", + "internal/damage/apply.go", + "File{Seed: core.DamageSeed(seed, i), Bytes: outs[i]}", + "File{Seed: core.DamageSeed(seed, i+1), Bytes: outs[i]}", + "TestAppendedJunkIsTheWholeFileAndThenTheSameBytesEveryTime", + ), + ( + "the seed of a damage is drawn from the same state as the size of a file", + "internal/core/seed.go", + "\th.Write([]byte{2})", + "\th.Write([]byte{1})", + "TestAppendedJunkIsTheWholeFileAndThenTheSameBytesEveryTime", + ), + ( + "a generator stopped by a cut is taken for a failed file", + "internal/engine/damage.go", + "\tif errors.Is(err, damage.ErrKeptEnough) {", + "\tif false && errors.Is(err, damage.ErrKeptEnough) {", + "TestATruncatedFileIsTheStartOfTheWholeOne", + ), + ( + "the damages are never finished, so no junk is ever appended", + "internal/engine/damage.go", + "\treturn f.Target.Damage.Finish(streams)", + "\treturn nil", + "TestAppendedJunkIsTheWholeFileAndThenTheSameBytesEveryTime", + ), + ( + "a damage that changes the length plans the generator at the size on disk", + "internal/engine/damage.go", + "\treturn damagedFormat(desc, t.Damage), nil", + "\treturn desc, nil", + "TestADamagedFileIsExactlyTheSizeAskedFor", + ), + ( + "the engine plans truncate after append-junk", + "internal/engine/damage.go", + "\tif eaten := t.Damage.Eaten(); eaten != nil {", + "\tif eaten := t.Damage.Eaten(); false && eaten != nil {", + "TestAChainThatCutsAwayWhatItAddedIsRefused", + ), + ( + "the engine plans a cut on an archive sized by what it holds", + "internal/engine/damage.go", + "\tif sized := t.Damage.SizeConflict(t.SizeFromContents); sized != nil {", + "\tif sized := t.Damage.SizeConflict(t.SizeFromContents); false && sized != nil {", + "TestAnArchiveSizedByItsContentsRefusesADamageThatChangesTheLength", + ), + ( + "the damage's own floor is named for a chain that changes the length, below the format's", + "internal/engine/damage.go", + "\tif len(t.Damage) == 0 || t.SizeFromContents || t.Damage.ChangesLength() {", + "\tif len(t.Damage) == 0 || t.SizeFromContents {", + "TestTheSmallestDamagedFileIsTakenAndOneByteLessIsNot", + ), + ( + "every damaged file is called certain, where readers differ too", + "internal/engine/damage.go", + "\tif err != nil || certain {", + "\tif err != nil || certain || true {", + "TestTheManifestSaysTheOutcomeDependsWhereReadersDiffer", + ), + ( + "a policy dependent file offers no answers to choose from", + "internal/engine/damage.go", + "\t\tPolicyOptions: []string{manifest.OutcomeAccept, manifest.OutcomeReject},\n", + "\n", + "TestTheManifestSaysTheOutcomeDependsWhereReadersDiffer", + ), + ( + "accept is refused beside every damage, where the outcome depends on the reader too", + "internal/damage/refusals.go", + "\tif certain, _, err := c.Certain(k); err != nil || !certain {", + "\tif certain, _, err := c.Certain(k); err != nil || false && !certain {", + "TestAcceptIsAllowedOnlyWhereTheOutcomeDependsOnTheReader", + ), + ( + "a damage with no policy case for the format is not called certain", + "internal/damage/policy.go", + "\t\tif !depends {", + "\t\tif false && !depends {", + "TestTheManifestSaysTheOutcomeDependsWhereReadersDiffer", + ), + ( + "a policy case narrowed to some settings covers every file of the format", + "internal/damage/policy.go", + "\t\tif !slices.Contains(values, k.Settings[name]) {", + "\t\tif false && !slices.Contains(values, k.Settings[name]) {", + "TestTheManifestSaysTheOutcomeDependsWhereReadersDiffer", + ), + ( + "a size a format cannot write exactly is refused rather than moved up for a cut", + "internal/engine/damagedformat.go", + "\tif err != nil && g.chain.Stretches() {", + "\tif false && err != nil && g.chain.Stretches() {", + "TestACutFromASizeTheFormatCannotWriteStartsFromTheNextOneItCan", + ), + ( + "the refusal of a damaged size names the format's first try rather than a size it writes", + "internal/engine/damagedformat.go", + "\t\t\tfirst = writable", + "\t\t\t_ = writable", + "TestTheSmallestDamagedFileIsTakenAndOneByteLessIsNot", + ), + ( + "a damaged plan says the size the generator writes rather than the size on disk", + "internal/engine/damagedformat.go", + "\t\tBytes: size,", + "\t\tBytes: p.Bytes,", + "TestADamagedFileIsExactlyTheSizeAskedFor", + ), + ( + "a refusal of a damaged size names the size the generator needed", + "internal/engine/damagedformat.go", + "\t\tleast := g.smallestReaching(first)", + "\t\tleast := first", + "TestTheSmallestDamagedFileIsTakenAndOneByteLessIsNot", + ), + ( + "the structural check takes a NUL inside a table", + "internal/oracle/strict.py", + " if control:", + " if False:", + "TestTheStructuralCheckerRefusesBrokenBytesInsteadOfStopping", + ), + ( + "the recipe reader reads truncate after append-junk", + "internal/recipe/damage.go", + "\tif eaten := t.Damage.Eaten(); eaten != nil {", + "\tif eaten := t.Damage.Eaten(); false && eaten != nil {", + "TestAChainThatCutsAwayWhatItAddedIsRefused", + ), + ( + "tfg damage --json leaves out where the outcome depends on the reader", + "internal/cli/damagecmd.go", + "PolicyDependent: policy,", + "PolicyDependent: nil,", + "TestTheMachineReadableDamageListCarriesTheWholeDeclaration", + ), + ( + "a policy case is narrowed by a setting the format does not have", + "internal/damage/appendjunk.go", + "{Format: \"txt\", When: map[string][]string{\"encoding\":", + "{Format: \"txt\", When: map[string][]string{\"encodings\":", + "TestEveryPolicyCaseNamesAFormatAndSettingsThatExist", + ), + ( + "the junk is drawn from one seed for every file", + "internal/damage/appendjunk.go", + "\tr := core.NewRand(j.seed)", + "\tr := core.NewRand(0)", + "TestJunkComesAfterTheFileAndOnlyThere", + ), + ( + "a cut larger than an int64 holds wraps to a smaller number instead of stopping at the largest", + "internal/damage/truncate.go", + "\tif q > (math.MaxInt64-m)/m {", + "\tif false && q > (math.MaxInt64-m)/m {", + "TestTheFloorOfACutFollowsItsShareAndAChainAddsUp", + ), + ( + "the description of one damage names the formats where readers differ and drops the reason", + "internal/cli/damagecmd.go", + "c.Format+settingsNarrowing(c.When), c.Why)", + "c.Format+settingsNarrowing(c.When), \"\")", + "TestTheDamageCommandSaysInWordsWhereTheOutcomeDependsOnTheReader", + ), + ( + "the list of damages stops saying where the outcome depends on the reader", + "internal/cli/damagecmd.go", + "\t\tif len(d.Policy) > 0 {", + "\t\tif len(d.Policy) > 0 && false {", + "TestTheDamageCommandSaysInWordsWhereTheOutcomeDependsOnTheReader", + ), ] diff --git a/internal/guard/strictverdict_test.go b/internal/guard/strictverdict_test.go index ab837522..05097c63 100644 --- a/internal/guard/strictverdict_test.go +++ b/internal/guard/strictverdict_test.go @@ -50,6 +50,7 @@ func TestTheStructuralCheckerRefusesBrokenBytesInsteadOfStopping(t *testing.T) { pptx := writeEncoded(t, "pptx", 8192, nil) pdf := writeEncoded(t, "pdf", 4096, nil) html := writeEncoded(t, "html", 2048, nil) + csv := writeEncoded(t, "csv", 2048, nil) cases := []strictCase{ {"healthy png", "png", png, oracle.StrictOK, ""}, @@ -59,6 +60,7 @@ func TestTheStructuralCheckerRefusesBrokenBytesInsteadOfStopping(t *testing.T) { {"healthy pptx", "pptx", pptx, oracle.StrictOK, ""}, {"healthy pdf", "pdf", pdf, oracle.StrictOK, ""}, {"healthy html", "html", html, oracle.StrictOK, ""}, + {"healthy csv", "csv", csv, oracle.StrictOK, ""}, {"png with nothing after the signature", "png", png[:8], oracle.StrictFail, "no chunk at all"}, {"png one byte short", "png", png[:len(png)-1], oracle.StrictFail, "before its 4 B checksum"}, @@ -72,6 +74,10 @@ func TestTheStructuralCheckerRefusesBrokenBytesInsteadOfStopping(t *testing.T) { {"docx with a compression method nobody knows", "docx", unknownMethod(t, docx), oracle.StrictFail, "compression method"}, {"pdf whose cross reference names an offset that is not a number", "pdf", garbledOffset(t, pdf), oracle.StrictFail, "which is not a number"}, {"html whose label heading is never closed", "html", unclosedLabel(t, html), oracle.StrictFail, "never closed on its line"}, + // RFC 4180 lets no control character into a field. Before 2026-11-10 a + // one row table with its head zeroed came back OK, because the only + // thing asked of it was whether its rows agreed with each other. + {"csv with a NUL inside a field", "csv", nulInField(t, csv), oracle.StrictFail, "control character"}, } dir := t.TempDir() @@ -397,3 +403,19 @@ func flipped(t *testing.T, b []byte) []byte { out[data] ^= 0xff return out } + +// nulInField is a healthy table with one letter of its second row replaced by +// a NUL - a field that is still a field, holding a byte RFC 4180 forbids. +func nulInField(t *testing.T, csv []byte) []byte { + t.Helper() + out := append([]byte(nil), csv...) + start := bytes.IndexByte(out, '\n') + 1 + for i := start; i < len(out); i++ { + if c := out[i]; (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') { + out[i] = 0 + return out + } + } + t.Fatal("the healthy table has no letter after its first line to replace") + return nil +} diff --git a/internal/gui/text/locale/registry/en.json b/internal/gui/text/locale/registry/en.json index d135a3d9..0c0d5531 100644 --- a/internal/gui/text/locale/registry/en.json +++ b/internal/gui/text/locale/registry/en.json @@ -1159,6 +1159,16 @@ "hash": "sha256-38755035559d", "other": "rfc7946" }, + "Detail.damage/append-junk.bytes": { + "description": "The sentence under the bytes setting of the append-junk damage, after what it takes.", + "hash": "sha256-4403339e8eb1", + "other": "How many random bytes come after the end. Below 64, some formats come out with damage no reader complains about." + }, + "Detail.damage/truncate.missing": { + "description": "The sentence under the missing setting of the truncate damage, after what it takes.", + "hash": "sha256-0b344bd34150", + "other": "How much of the whole file is gone, as a percentage. At least this much is cut - a few bytes more where the format cannot end exactly there." + }, "Detail.damage/zero-head.bytes": { "description": "The sentence under the bytes setting of the zero-head damage, after what it takes.", "hash": "sha256-8700126f6712", @@ -2554,6 +2564,11 @@ "hash": "sha256-8696622f3441", "other": "Methods" }, + "Label.missing": { + "description": "The name beside the box of a setting a recipe writes as missing.", + "hash": "sha256-6be36ca49ee8", + "other": "Missing" + }, "Label.modified": { "description": "The name beside the box of a setting a recipe writes as modified.", "hash": "sha256-e8ce5dcaf408", diff --git a/internal/gui/text/locale/registry/pl.json b/internal/gui/text/locale/registry/pl.json index e4119284..e528bdc6 100644 --- a/internal/gui/text/locale/registry/pl.json +++ b/internal/gui/text/locale/registry/pl.json @@ -231,6 +231,8 @@ "Choice.unlocated.some": { "hash": "sha256-a6b46dd0d1ae", "other": "niektóre" }, "Choice.winding.reversed": { "hash": "sha256-bc64f64fb479", "other": "odwrócony" }, "Choice.winding.rfc7946": { "hash": "sha256-38755035559d", "other": "rfc7946" }, + "Detail.damage/append-junk.bytes": { "hash": "sha256-4403339e8eb1", "other": "Ile losowych bajtów trafia za koniec pliku. Poniżej 64 niektóre formaty wychodzą z uszkodzeniem, na które żaden czytnik nie narzeka." }, + "Detail.damage/truncate.missing": { "hash": "sha256-0b344bd34150", "other": "Jaka część całego pliku znika, w procentach. Ucięte jest co najmniej tyle - o kilka bajtów więcej tam, gdzie format nie może się skończyć dokładnie w tym miejscu." }, "Detail.damage/zero-head.bytes": { "hash": "sha256-8700126f6712", "other": "Ile bajtów na początku zostaje wyzerowanych. Poniżej czterech niektóre formaty wychodzą z uszkodzeniem, na które żaden czytnik nie narzeka." }, "Detail.format/avif.height": { "hash": "sha256-4b336c279584", "other": "Jak wysoki jest obraz. Gdy pole zostaje puste, wybierany jest rozmiar, który mieści się w zamówionych bajtach." }, "Detail.format/avif.quality": { "hash": "sha256-f6e1c3e1af83", "other": "Ile szczegółów zachowuje obraz. Wyższa wartość wygląda lepiej i zajmuje więcej pliku, zostawiając mniej miejsca na dopełnienie." }, @@ -510,6 +512,7 @@ "Label.links": { "hash": "sha256-9024c197d849", "other": "Odnośniki" }, "Label.message_depth": { "hash": "sha256-8a18b09b1f37", "other": "Głębokość wiadomości" }, "Label.methods": { "hash": "sha256-8696622f3441", "other": "Metody" }, + "Label.missing": { "hash": "sha256-6be36ca49ee8", "other": "Brakuje" }, "Label.modified": { "hash": "sha256-e8ce5dcaf408", "other": "Zmodyfikowano" }, "Label.nesting": { "hash": "sha256-f342629fc0e1", "other": "Zagnieżdżenie" }, "Label.orientation": { "hash": "sha256-e3d11242200f", "other": "Orientacja" }, diff --git a/internal/gui/text/locale/said/en.json b/internal/gui/text/locale/said/en.json index f576a94c..73ef5403 100644 --- a/internal/gui/text/locale/said/en.json +++ b/internal/gui/text/locale/said/en.json @@ -797,6 +797,46 @@ "hash": "sha256-b90221bf868f", "other": "a damaged file is one a judge was measured to refuse, so expecting it to be accepted is an expectation nothing could meet" }, + "damage.Eaten": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.What}}, {{.Instead}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-6749ebcea05d", + "other": "{{.What}}. {{.Instead}}" + }, + "damage.EatenInstead": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Cut}}, {{.Added}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-a2a1e05e95ec", + "other": "Put {{.Cut}} before {{.Added}}." + }, + "damage.EatenWhat": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Cut}}, {{.Added}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-feb8c803f4e8", + "other": "{{.Cut}} comes after {{.Added}} and cuts away what it added" + }, + "damage.EatenWhy": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run.", + "hash": "sha256-bd0ef3c0b19a", + "other": "the manifest would list a damage whose bytes the file no longer holds" + }, + "damage.NeedsSize": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.What}}, {{.Instead}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-6749ebcea05d", + "other": "{{.What}}. {{.Instead}}" + }, + "damage.NeedsSizeInstead": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run.", + "hash": "sha256-bb7ddab6d307", + "other": "Give the target a size, or use a damage that keeps the length, such as zero-head." + }, + "damage.NeedsSizeWhat": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Damage}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-68ac855601a4", + "other": "{{.Damage}} changes the length of the file, and this target takes its size from what it holds" + }, + "damage.NeedsSizeWhy": { + "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run.", + "hash": "sha256-0e57e6efc6c1", + "other": "that size is only known once the file is written, so there is no length to work from" + }, "damage.NoChange": { "description": "Said by internal/damage - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.What}}, {{.Why}}, written the way the program writes them - keep each spelled exactly that way.", "hash": "sha256-14c3ed651919", @@ -978,6 +1018,31 @@ "hash": "sha256-1e90995301d1", "other": "writing {{.File}}" }, + "engine.DamagedAboveWhy": { + "description": "Said by internal/engine - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Damage}}, {{.Format}}, {{.Maximum}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-59fb1ae064c2", + "other": "{{.Damage}} has to start from a {{.Format}} file of at most {{.Maximum}}" + }, + "engine.DamagedBelowWhy": { + "description": "Said by internal/engine - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Damage}}, {{.Format}}, {{.Minimum}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-5fe7c6a1ed13", + "other": "{{.Damage}} has to start from a {{.Format}} file of at least {{.Minimum}}" + }, + "engine.DamagedFloorWhy": { + "description": "Said by internal/engine - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Damage}}, {{.Floor}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-7829143089a2", + "other": "{{.Damage}} needs at least {{.Floor}}" + }, + "engine.DamagedSizeAtLeast": { + "description": "Said by internal/engine - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Size}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-72fcb7ac9c01", + "other": "Ask for {{.Size}} or more." + }, + "engine.DamagedSizeAtMost": { + "description": "Said by internal/engine - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Size}}, written the way the program writes them - keep each spelled exactly that way.", + "hash": "sha256-2174959715d1", + "other": "Ask for {{.Size}} or less." + }, "engine.FilesSoFar": { "description": "Said by internal/engine - a refusal under the box it is about or at the foot of the form, or a note after a run. Carries {{.Count}}, written the way the program writes them - keep each spelled exactly that way.", "hash": "sha256-83589d643302", diff --git a/internal/gui/text/locale/said/pl.json b/internal/gui/text/locale/said/pl.json index 18245add..7b12604a 100644 --- a/internal/gui/text/locale/said/pl.json +++ b/internal/gui/text/locale/said/pl.json @@ -158,6 +158,14 @@ "damage.ConflictInstead": { "hash": "sha256-e4ac08c1d195", "other": "pomiń oczekiwany wynik, a będzie nim odrzucenie, albo wybierz oczyszczenie, jeśli system pod testem ma naprawić plik, albo nieokreślony, jeśli właśnie o to pytasz" }, "damage.ConflictWhat": { "hash": "sha256-a9099dc25467", "other": "ta partia uszkadza plik, a jako oczekiwany wynik ma {{.Outcome}}" }, "damage.ConflictWhy": { "hash": "sha256-b90221bf868f", "other": "uszkodzony plik to taki, który sędzia odrzucał w pomiarze, więc oczekiwanie, że zostanie przyjęty, jest oczekiwaniem, którego nic nie spełni" }, + "damage.Eaten": { "hash": "sha256-6749ebcea05d", "other": "{{.What}}. {{.Instead}}" }, + "damage.EatenInstead": { "hash": "sha256-a2a1e05e95ec", "other": "Postaw {{.Cut}} przed {{.Added}}." }, + "damage.EatenWhat": { "hash": "sha256-feb8c803f4e8", "other": "{{.Cut}} stoi po {{.Added}} i odcina to, co tamto dopisało" }, + "damage.EatenWhy": { "hash": "sha256-bd0ef3c0b19a", "other": "manifest wymieniałby uszkodzenie, którego bajtów plik już nie ma" }, + "damage.NeedsSize": { "hash": "sha256-6749ebcea05d", "other": "{{.What}}. {{.Instead}}" }, + "damage.NeedsSizeInstead": { "hash": "sha256-bb7ddab6d307", "other": "Podaj rozmiar tej partii albo użyj uszkodzenia, które nie zmienia długości, na przykład zero-head." }, + "damage.NeedsSizeWhat": { "hash": "sha256-68ac855601a4", "other": "{{.Damage}} zmienia długość pliku, a ta partia bierze rozmiar z tego, co zawiera" }, + "damage.NeedsSizeWhy": { "hash": "sha256-0e57e6efc6c1", "other": "ten rozmiar jest znany dopiero po zapisaniu pliku, więc nie ma długości, od której można wyjść" }, "damage.NoChange": { "hash": "sha256-14c3ed651919", "other": "{{.What}}. {{.Why}}" }, "damage.NoChangeInstead": { "hash": "sha256-7b4599e7c06c", "other": "daj uszkodzeniu większy plik albo inne ustawienia, albo zdejmij je z tej partii" }, "damage.NoChangeWhat": { "hash": "sha256-4f2698e2b793", "other": "{{.Damage}} nie zmieniło niczego w {{.File}}" }, @@ -194,6 +202,11 @@ "engine.CrashPlanning": { "hash": "sha256-90f7d5da615f", "other": "planowania pliku" }, "engine.CrashWhile": { "hash": "sha256-071bf3d7823e", "other": "{{.While}}" }, "engine.CrashWriting": { "hash": "sha256-1e90995301d1", "other": "zapisu {{.File}}" }, + "engine.DamagedAboveWhy": { "hash": "sha256-59fb1ae064c2", "other": "{{.Damage}} musi wyjść od pliku {{.Format}} o rozmiarze najwyżej {{.Maximum}}" }, + "engine.DamagedBelowWhy": { "hash": "sha256-5fe7c6a1ed13", "other": "{{.Damage}} musi wyjść od pliku {{.Format}} o rozmiarze co najmniej {{.Minimum}}" }, + "engine.DamagedFloorWhy": { "hash": "sha256-7829143089a2", "other": "{{.Damage}} potrzebuje co najmniej {{.Floor}}" }, + "engine.DamagedSizeAtLeast": { "hash": "sha256-72fcb7ac9c01", "other": "Poproś o {{.Size}} albo więcej." }, + "engine.DamagedSizeAtMost": { "hash": "sha256-2174959715d1", "other": "Poproś o {{.Size}} albo mniej." }, "engine.FilesSoFar": { "hash": "sha256-83589d643302", "one": "{{.Count}} pliku", "few": "{{.Count}} plikach", "many": "{{.Count}} plikach", "other": "{{.Count}} plikach" }, "engine.GeneratorCrash": { "hash": "sha256-82716d977373", "other": "generator {{.Format}} zatrzymał się na błędzie wewnętrznym w trakcie {{.While}}: {{.Value}}. To wada tego narzędzia, a nie coś, o co przepis może poprosić. Zgłoś ją razem z ustawieniami, które do niej doprowadziły" }, "engine.IDTwice": { "hash": "sha256-724959681477", "other": "nazwa partii {{.ID}} występuje dwa razy" }, diff --git a/internal/manifest/manifest.go b/internal/manifest/manifest.go index db3059df..99cdf7cb 100644 --- a/internal/manifest/manifest.go +++ b/internal/manifest/manifest.go @@ -280,6 +280,13 @@ type Expected struct { Reason string `json:"reason,omitempty"` Detail string `json:"detail,omitempty"` Confidence string `json:"confidence,omitempty"` + // PolicyOptions are the answers a policy dependent outcome leaves open, + // for a consumer that resolves it by its own policy (MF5). + // + // Absent from every file that has none, so each manifest written before + // it existed is byte for byte what it was, and the schema stays 1.0 by + // the rule MANIFEST.md section 10 states: it grows by adding. + PolicyOptions []string `json:"policy_options,omitempty"` } // Outcomes a manifest entry can declare. diff --git a/internal/oracle/strict.py b/internal/oracle/strict.py index 21ee0eb7..1c15bd25 100644 --- a/internal/oracle/strict.py +++ b/internal/oracle/strict.py @@ -395,6 +395,19 @@ def check_csv(data, settings=None): except UnicodeDecodeError as exc: fail(f"not valid UTF-8: {exc}") + # RFC 4180 lets a field hold text, separators, quotes and line breaks, and + # no control character below a space - a NUL least of all, which is what a + # table whose bytes were overwritten holds, and where every reader handing + # a field to C code stops. Asked before the shape, because a table of one + # row has no other row to disagree with: measured 2026-11-10, zero-head on + # a csv with header=false at its smallest size came back OK. A tab is let + # through, because a field holding one is common and a tab is a separator + # this format offers. + control = re.search(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]", text) + if control: + fail(f"the table holds the control character {control.group()!r} at {control.start()}, " + "which RFC 4180 does not allow in a field") + if not text.endswith(eol): fail(f"the file does not end with {settings.get('line_ending', 'lf')}, " "so the last row is unterminated") diff --git a/internal/recipe/damage.go b/internal/recipe/damage.go index 100601e5..89b6fecc 100644 --- a/internal/recipe/damage.go +++ b/internal/recipe/damage.go @@ -171,8 +171,18 @@ func reportDamageSetting(p *problems, at spot, bad error) { // address of the target and beside every other problem of the same recipe - // while the engine asking it is what covers the command line, which never // reads a recipe at all. One rule, two callers. See O199. +// +// Only where the damage breaks the rules of this format with these settings. +// Where the manifest says the outcome depends on the reader, accept is one of +// its two answers and a recipe stating it is writing down its own policy +// (owner's call, 2026-11-10). A format this build does not know is refused on +// its own line, so it is not asked about here. func refuseImpossibleExpectation(p *problems, where spot, t Target) { - refusal := t.Damage.ExpectationConflict(t.Expected) + d, err := format.Get(t.Format) + if err != nil { + return + } + refusal := t.Damage.ExpectationConflict(t.Expected, damage.KindOf(d, t.Properties)) if refusal == nil { return } @@ -180,6 +190,24 @@ func refuseImpossibleExpectation(p *problems, where spot, t Target) { p.add(where.of(refusal.AboutSetting()), whereWhat(where, what), why, instead) } +// refuseImpossibleDamage stops a chain that cuts away what it added, and a +// damage that changes the length on an archive sized by what it holds. +// +// Both are damage's rules rather than this file's, asked here as well as in +// the engine for the reason refuseImpossibleExpectation gives: here they are +// reported with the address of the target and beside every other problem of +// the recipe, and the engine asking too is what covers the command line. +func refuseImpossibleDamage(p *problems, where spot, t Target) { + if eaten := t.Damage.Eaten(); eaten != nil { + what, why, instead := eaten.Parts() + p.add(where.of(KeyDamage), whereWhat(where, what), why, instead) + } + if sized := t.Damage.SizeConflict(t.SizeFromContents); sized != nil { + what, why, instead := sized.Parts() + p.add(where.of(sized.AboutSetting()), whereWhat(where, what), why, instead) + } +} + // sortedKeysOf puts the settings of one entry in a stable order, so the same // recipe always reports the same one first. func sortedKeysOf(x map[string]any) []string { diff --git a/internal/recipe/target.go b/internal/recipe/target.go index 888e843d..a137db00 100644 --- a/internal/recipe/target.go +++ b/internal/recipe/target.go @@ -117,9 +117,12 @@ func (rt rawTarget) validate(p *problems, at func(id string) spot, def Defaults, t.Damage = damages(p, where, rt.Damage) t.Expected, t.ExpectedReason = expectation(p, where, rt.Expected) - refuseImpossibleExpectation(p, where, t) rt.describe(p, where, &t) t.Properties = properties(p, where, t.Format, rt.Properties) + // After the properties, because whether a damaged file can be expected to + // be accepted depends on its settings as well as its format. + refuseImpossibleExpectation(p, where, t) + refuseImpossibleDamage(p, where, t) // The count, whether or not its list was built and whether or not its size // was refused, so a run past the ceiling is said beside the target's own diff --git a/web/content/ar/damage.html b/web/content/ar/damage.html index a66cdf66..659502ae 100644 --- a/web/content/ar/damage.html +++ b/web/content/ar/damage.html @@ -49,7 +49,9 @@

الملف التالف يبقى بالحجم الذي طلبته

البايتات نفسها مرة أخرى. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

تُكتب الإعدادات بعد النقطتين. يمكن تكرار الخيار، وتُطبَّق أنواع التلف بالترتيب الذي تكتبه. ويعمل مع كل واحدة من {{ .Facts.FormatCount }} صيغة. @@ -65,18 +67,27 @@

ما أنواع التلف المتاحة؟

{{ template "damagesTable" . }}

- يكتب zero-head أصفارًا فوق بداية الملف. معظم القارئات تنظر إلى هناك أولًا، إلى التوقيع + يكتب zero-head يكتب أصفارًا فوق بداية الملف. معظم القارئات تنظر إلى هناك أولًا، إلى التوقيع والترويسة اللذين يقولان ما هو الملف، فتلاحظ ذلك أي قارئة تقريبًا. والنص العادي والسجلات لا توقيع لها وتُرفض أيضًا، لأن سلسلة من البايتات الصفرية ليست نصًّا. وتحت أربعة بايتات تخرج بعض الصيغ بتلف لا تشتكي منه أي قارئة، ولهذا يبدأ الإعداد من أربعة.

+

+ truncate يقطع نهاية الملف، كما يتركه رفع أو نسخ انقطع في منتصفه، ويبقى الملف بالحجم + الذي طلبته: يُولَّد أكبر ولا يصل إلى القرص إلا أوله. ويقول missing أي جزء من الملف + كله يضيع، النصف ما لم تحدد غير ذلك. +

+

+ append-junk يكتب بايتات عشوائية بعد نهاية الملف، 64 ما لم تحدد غير ذلك، كما يصل ملف + التصق شيء بنهايته. والأمر نفسه يضيف البايتات نفسها. +

ما يقوله البيان

بيان يقول ما ينبغي أن يحدث

- كل ملف تالف يحصل على مدخل يقول إن نظامك ينبغي أن يرفضه، ويُسجَّل التلف بجانبه: + حيث يخرق التلف قواعد الصيغة، يقول المدخل إن نظامك ينبغي أن يرفضه، ويُسجَّل التلف بجانبه: 

"expected": {
   "outcome": "reject",
@@ -92,13 +103,39 @@ 
بيان يقول ما ينبغي أن يحدث

   }
 ]

- يُرفض طلبان قبل أن يُكتب أي شيء، لأن كلًّا منهما سيترك على القرص ملفًا يصفه البيان وصفًا خاطئًا: + وحيث تختلف القارئات، يقول المدخل ذلك بدل التخمين. فالملف النصي المقطوع يبقى نصًّا، ومعظم عارضات + الصور تتجاهل البايتات بعد نهاية ملف PNG، لذلك يترك المدخل الجواب لنظامك ويعرض الجوابين ويقول + السبب: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ يسرد tfg damage <id> الصيغ التي يترك فيها التلف الجواب لنظامك، مع السبب. +

+

+ تُرفض أربعة طلبات قبل أن يُكتب أي شيء، لأن كلًّا منها سيترك على القرص ملفًا يصفه البيان وصفًا + خاطئًا:

  • ملف أصغر مما يحتاجه التلف، وكان سيخرج دون تغيير
  • - expected: accept بجانب تلف، لأن لا شيء يمكنه تحقيق ذلك. اكتب sanitize إن - كان نظامك مقصودًا به إصلاح الملف، أو unspecified إن كان هذا هو السؤال الذي تطرحه + expected: accept بجانب تلف يخرق قواعد الصيغة، لأن لا شيء يتبع تلك القواعد يقبل + الملف. وحيث يتوقف الجواب على نظامك، يُسمح بـ accept. اكتب sanitize + إن كان نظامك مقصودًا به إصلاح الملف، أو unspecified إن كان هذا هو السؤال الذي + تطرحه +
    • +
  • + truncate بعد append-junk، لأنه سيقطع البايتات التي أُضيفت. ضع + truncate أولًا +
    • +
  • + truncate أو append-junk على أرشيف يأخذ حجمه مما يحويه، لأنه لا يوجد + حجم يُنطلق منه
@@ -135,11 +172,18 @@

تحويله إلى اختبار

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ يُستبعد هنا الملف الذي نتيجته unspecified، لأن ما ينبغي لنظامك أن يفعله به هو + سياستك أنت. ومدخله يقول السبب، وpolicy_options يسرد الأجوبة الممكنة. +

الرفض الجيد هو الرفض النظيف. رسالة تقول ما الخطأ هي الجواب الذي تريده. أما خطأ الخادم أو التعليق أو ملف حُفظ نصفه فهو العيب الذي وُجد هذا الاختبار لاكتشافه. diff --git a/web/content/ar/site.json b/web/content/ar/site.json index ab7a46ea..38b3e70e 100644 --- a/web/content/ar/site.json +++ b/web/content/ar/site.json @@ -242,6 +242,8 @@ "unspecified": "يعتمد على قواعد نظامك. أنت من يقرر، ثم تتحقق من أن ما يحدث هو ما قصدته." }, "damages": { + "append-junk": "يكتب بايتات عشوائية بعد نهاية الملف ويحافظ على الحجم الذي طلبته بالضبط، كما يصل ملف التصق شيء بنهايته.", + "truncate": "يقطع نهاية الملف ويحافظ على الحجم الذي طلبته بالضبط، كما يتركه رفع أو نسخ انقطع في منتصفه.", "zero-head": "يكتب أصفارًا فوق أول بايتات الملف دون المساس بطوله. معظم القارئات تنظر إلى هناك أولًا، فيلاحظ هذا التلف كل شيء تقريبًا." }, "terms": { diff --git a/web/content/cs/damage.html b/web/content/cs/damage.html index a5f339cc..b401e308 100644 --- a/web/content/cs/damage.html +++ b/web/content/cs/damage.html @@ -51,7 +51,9 @@

Poškozený soubor má stále velikost, kterou jste chtěli

chtěli, a stejný příkaz zapíše znovu stejné bajty. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Nastavení se píše za dvojtečku. Přepínač lze opakovat a poškození se použijí v pořadí, v jakém je zapíšete. Funguje s každým z {{ .Facts.FormatCount }} formátů. @@ -73,13 +75,24 @@

Jaká poškození existují?

bajty některé formáty vyjdou s poškozením, na které si žádná čtečka nestěžuje, proto nastavení začíná na čtyřech.

+

+ truncate odřízne konec souboru, tak jak ho nechá přerušené nahrávání nebo + kopírování, a soubor má přesto velikost, o kterou jste požádali: vytvoří se větší a na disk + dorazí jen jeho začátek. missing říká, jaká část celého souboru chybí, polovina, + pokud neuvedete jinak. +

+

+ append-junk zapíše náhodné bajty za konec souboru, 64, pokud neuvedete jinak, tak + jak přijde soubor, na jehož konci něco visí. Stejný příkaz připojí stejné bajty. +

Co říká manifest

Manifest, který říká, co se má stát

- Každý poškozený soubor dostane záznam, že ho má váš systém odmítnout, s poškozením zapsaným vedle: + Tam, kde poškození porušuje pravidla formátu, záznam říká, že ho má váš systém odmítnout, s + poškozením zapsaným vedle: 

"expected": {
   "outcome": "reject",
@@ -95,15 +108,40 @@ 
Manifest, který říká, co se má stát

   }
 ]

- Dva požadavky se odmítnou dřív, než se cokoli zapíše, protože každý by na disku nechal soubor, který - manifest popisuje špatně: + Tam, kde se čtečky liší, to záznam řekne, místo aby hádal. Zkrácený textový soubor je pořád text + a většina prohlížečů přeskočí bajty za koncem PNG, takže záznam nechá odpověď na vašem systému, + nabídne obě a řekne proč: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> vypíše formáty, ve kterých poškození nechává odpověď na vašem + systému, a důvod. +

+

+ Čtyři požadavky se odmítnou dřív, než se cokoli zapíše, protože každý by na disku nechal soubor, + který manifest popisuje špatně:

  • soubor menší, než poškození potřebuje, který by vyšel nezměněný
  • - expected: accept vedle poškození, protože by to nic nemohlo splnit. Napište - sanitize, pokud má váš systém soubor opravit, nebo unspecified, - pokud je právě to vaše otázka + expected: accept vedle poškození, které porušuje pravidla formátu, protože nic, + co je dodržuje, takový soubor nepřijme. Tam, kde odpověď závisí na vašem systému, je + accept povoleno. Napište sanitize, pokud má váš systém soubor + opravit, nebo unspecified, pokud je právě to vaše otázka +
    • +
  • + truncate po append-junk, protože by odřízl připojené bajty. Dejte + truncate na začátek +
    • +
  • + truncate nebo append-junk na archiv, který bere svou velikost z + toho, co obsahuje, protože není velikost, ze které by se dalo vyjít
@@ -141,11 +179,19 @@

Z toho udělat test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Soubor s výsledkem unspecified se tu vynechá, protože co s ním má váš systém + udělat, určuje vaše vlastní pravidlo. Jeho záznam říká proč a policy_options + vypisuje možné odpovědi. +

Dobré odmítnutí je čisté. Zpráva, která říká, co bylo špatně, je odpověď, kterou chcete. Chyba serveru, zaseknutí nebo napůl uložený soubor je vada, kterou má tento test najít. diff --git a/web/content/cs/site.json b/web/content/cs/site.json index d2f4d048..45089f8e 100644 --- a/web/content/cs/site.json +++ b/web/content/cs/site.json @@ -241,6 +241,8 @@ "unspecified": "Záleží na pravidlech vašeho systému. Rozhodnete vy, a pak ověříte, že to, co se stane, je to, co jste chtěli." }, "damages": { + "append-junk": "Zapíše náhodné bajty za konec souboru a zachová přesně velikost, o kterou jste požádali, tak jak přijde soubor, na jehož konci něco visí.", + "truncate": "Odřízne konec souboru a zachová přesně velikost, o kterou jste požádali, tak jak ho nechá přerušené nahrávání nebo kopírování.", "zero-head": "Přepíše prvních několik bajtů souboru nulami a jeho délku nechá být. Většina čteček se dívá nejdřív tam, takže si tohoto poškození všimne téměř cokoli." }, "terms": { diff --git a/web/content/de/damage.html b/web/content/de/damage.html index 5cf84394..be18632e 100644 --- a/web/content/de/damage.html +++ b/web/content/de/damage.html @@ -53,7 +53,9 @@

Eine beschädigte Datei hat trotzdem die verlangte Größe

verlangte Größe, und derselbe Befehl schreibt wieder dieselben Bytes. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Einstellungen stehen nach einem Doppelpunkt. Die Option lässt sich wiederholen, und die Beschädigungen werden in der Reihenfolge angewendet, in der Sie sie schreiben. Es funktioniert @@ -77,14 +79,25 @@

Welche Beschädigungen gibt es?

eine Folge von Nullbytes kein Text ist. Unter vier Bytes entstehen bei manchen Formaten Schäden, über die sich kein Leser beschwert, deshalb beginnt die Einstellung bei vier.

+

+ truncate schneidet das Ende der Datei ab, so wie ein abgebrochener Upload oder + Kopiervorgang sie hinterlässt, und die Datei hat trotzdem die angegebene Größe: Sie wird größer + erzeugt, und nur ihr Anfang landet auf der Festplatte. missing gibt an, welcher + Anteil der ganzen Datei fehlt, die Hälfte, wenn Sie nichts anderes angeben. +

+

+ append-junk schreibt zufällige Bytes hinter das Ende der Datei, 64, wenn Sie nichts + anderes angeben, so wie eine Datei ankommt, an deren Ende etwas hängt. Derselbe Befehl hängt + dieselben Bytes an. +

Was das Manifest sagt

Ein Manifest, das sagt, was geschehen soll

- Jede beschädigte Datei bekommt einen Eintrag, der besagt, dass Ihr System sie ablehnen soll, mit der - Beschädigung daneben: + Wo die Beschädigung die Regeln des Formats bricht, besagt der Eintrag, dass Ihr System die Datei + ablehnen soll, mit der Beschädigung daneben: 

"expected": {
   "outcome": "reject",
@@ -100,15 +113,40 @@ 
Ein Manifest, das sagt, was geschehen soll

   }
 ]

- Zwei Anfragen werden abgelehnt, bevor etwas geschrieben wird, denn jede würde eine Datei + Wo sich Leser unterscheiden, sagt der Eintrag das, statt zu raten. Eine gekürzte Textdatei ist + immer noch Text, und die meisten Bildbetrachter überspringen Bytes hinter dem Ende einer PNG, + also überlässt der Eintrag die Antwort Ihrem System, nennt beide und sagt, warum: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> listet die Formate auf, in denen eine Beschädigung die + Antwort Ihrem System überlässt, und warum. +

+

+ Vier Anfragen werden abgelehnt, bevor etwas geschrieben wird, denn jede würde eine Datei hinterlassen, die das Manifest falsch beschreibt:

  • eine Datei, die kleiner ist, als die Beschädigung braucht, und unverändert herauskäme
  • - expected: accept neben einer Beschädigung, weil nichts es erfüllen könnte. Schreiben - Sie sanitize, wenn Ihr System die Datei reparieren soll, oder - unspecified, wenn genau das Ihre Frage ist + expected: accept neben einer Beschädigung, die die Regeln des Formats bricht, + weil nichts, was sich an sie hält, die Datei annimmt. Wo die Antwort von Ihrem System abhängt, + ist accept erlaubt. Schreiben Sie sanitize, wenn Ihr System die + Datei reparieren soll, oder unspecified, wenn genau das Ihre Frage ist +
    • +
  • + truncate nach append-junk, weil es die angehängten Bytes wieder + abschneiden würde. Stellen Sie truncate an den Anfang +
    • +
  • + truncate oder append-junk an einem Archiv, das seine Größe aus + seinem Inhalt bezieht, weil es keine Größe gibt, von der man ausgehen kann
@@ -146,11 +184,19 @@

Daraus einen Test machen

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Eine Datei mit dem Ergebnis unspecified wird hier ausgelassen, weil Ihre eigene + Richtlinie bestimmt, was Ihr System mit ihr tun soll. Ihr Eintrag sagt, warum, und + policy_options nennt die möglichen Antworten. +

Eine gute Ablehnung ist eine saubere. Eine Meldung, die sagt, was falsch war, ist die gewünschte Antwort. Ein Serverfehler, ein Hänger oder eine halb gespeicherte Datei ist der Fehler, den diff --git a/web/content/de/site.json b/web/content/de/site.json index e9e001a9..9d04098c 100644 --- a/web/content/de/site.json +++ b/web/content/de/site.json @@ -241,6 +241,8 @@ "unspecified": "Das hängt von den Regeln deines Systems ab. Du entscheidest, dann prüfst du, ob das Ergebnis dem entspricht, was du gemeint hast." }, "damages": { + "append-junk": "Schreibt zufällige Bytes hinter das Ende der Datei und behält genau die angegebene Größe, so wie eine Datei ankommt, an deren Ende etwas hängt.", + "truncate": "Schneidet das Ende der Datei ab und behält genau die angegebene Größe, so wie ein abgebrochener Upload oder Kopiervorgang die Datei hinterlässt.", "zero-head": "Überschreibt die ersten Bytes der Datei mit Nullen und lässt ihre Länge unverändert. Die meisten Leser schauen zuerst dorthin, deshalb bemerkt fast alles diese Beschädigung." }, "terms": { diff --git a/web/content/en/damage.html b/web/content/en/damage.html index 6dd3650c..781bcc04 100644 --- a/web/content/en/damage.html +++ b/web/content/en/damage.html @@ -52,7 +52,9 @@

A damaged file is still the size you asked for

you asked for, and the same command writes the same bytes again. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Settings go after a colon. The flag can be repeated, and the damages are applied in the order you write them. It works with every one of the {{ .Facts.FormatCount }} formats. @@ -74,14 +76,25 @@

Which damages are there?

Below four bytes some formats come out with damage that no reader complains about, which is why the setting starts at four.

+

+ truncate cuts the end off the file, the way an interrupted upload or copy leaves + it, and the file is still the size you asked for: it is generated larger, and only its start + reaches the disk. missing says how much of the whole file is gone, half unless you + say otherwise. +

+

+ append-junk writes random bytes after the end of the file, 64 unless you say + otherwise, the way a file arrives with something stuck to its end. The same command appends the + same bytes. +

What the manifest says

A manifest that says what should happen

- Every damaged file gets an entry saying your system should reject it, with the damage recorded - beside it: + Where the damage breaks the rules of the format, the entry says your system should reject it, + with the damage recorded beside it: 

"expected": {
   "outcome": "reject",
@@ -97,15 +110,40 @@ 
A manifest that says what should happen

   }
 ]

- Two requests are refused before anything is written, because each would leave a file on disk that - the manifest describes wrongly: + Where readers differ, the entry says so instead of guessing. A text file cut short is still + text, and most viewers skip bytes after the end of a PNG, so the entry leaves the answer to your + system, offers both, and says why: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> lists the formats where a damage leaves the answer to your + system, and why. +

+

+ Four requests are refused before anything is written, because each would leave a file on disk + that the manifest describes wrongly:

  • a file smaller than the damage needs, which would come out unchanged
  • - expected: accept beside a damage, because nothing could meet it. Write - sanitize if your system is meant to repair the file, or unspecified if - that is the question you are asking + expected: accept beside a damage that breaks the rules of the format, because + nothing that follows them accepts the file. Where the answer depends on your system, + accept is allowed. Write sanitize if your system is meant to repair + the file, or unspecified if that is the question you are asking +
    • +
  • + truncate after append-junk, which would cut away the bytes that were + added. Put truncate first +
    • +
  • + truncate or append-junk on an archive that takes its size from what + it holds, because there is no size to work from
@@ -143,11 +181,19 @@

Turning it into a test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ A file whose outcome is unspecified is left out here, because what your system + should do with it is your own policy. Its entry says why, and policy_options lists + the answers there are. +

A good refusal is a clean one. A message that says what was wrong is the answer you want. A server error, a hang or a half stored file is the defect this test exists to find. diff --git a/web/content/en/site.json b/web/content/en/site.json index 7a18f866..08b081ee 100644 --- a/web/content/en/site.json +++ b/web/content/en/site.json @@ -326,6 +326,8 @@ } ], "damages": { + "append-junk": "Writes random bytes after the end of the file and keeps exactly the size you asked for, the way a file with something stuck to its end arrives.", + "truncate": "Cuts the end off the file and keeps exactly the size you asked for, the way an interrupted upload or copy leaves it.", "zero-head": "Overwrites the first bytes of the file with zeros, leaving its length alone. Most readers look there first, so this is the damage almost anything notices." } } diff --git a/web/content/es/damage.html b/web/content/es/damage.html index 10d53d85..79996ee6 100644 --- a/web/content/es/damage.html +++ b/web/content/es/damage.html @@ -52,7 +52,9 @@

Un archivo dañado conserva el tamaño que pediste

pediste, y el mismo comando vuelve a escribir los mismos bytes. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Los ajustes van después de dos puntos. La opción se puede repetir, y los daños se aplican en el orden en que los escribes. Funciona con cada uno de los {{ .Facts.FormatCount }} formatos. @@ -74,14 +76,25 @@

¿Qué daños hay?

de bytes nulos no es texto. Por debajo de cuatro bytes algunos formatos salen con un daño del que ningún lector se queja, por eso el ajuste empieza en cuatro.

+

+ truncate corta el final del archivo, como lo deja una subida o una copia + interrumpida, y el archivo sigue teniendo el tamaño que pediste: se genera más grande y solo su + comienzo llega al disco. missing dice qué parte del archivo completo falta, la + mitad si no indicas otra cosa. +

+

+ append-junk escribe bytes aleatorios después del final del archivo, 64 si no + indicas otra cosa, como llega un archivo con algo pegado al final. El mismo comando añade los + mismos bytes. +

Lo que dice el manifiesto

Un manifiesto que dice lo que debe ocurrir

- Cada archivo dañado recibe una entrada que dice que tu sistema debe rechazarlo, con el daño anotado - al lado: + Donde el daño rompe las reglas del formato, la entrada dice que tu sistema debe rechazarlo, con + el daño anotado al lado: 

"expected": {
   "outcome": "reject",
@@ -97,15 +110,40 @@ 
Un manifiesto que dice lo que debe ocurrir

   }
 ]

- Se rechazan dos peticiones antes de escribir nada, porque cada una dejaría en el disco un archivo - que el manifiesto describe mal: + Donde los lectores no coinciden, la entrada lo dice en lugar de adivinar. Un archivo de texto + cortado sigue siendo texto, y la mayoría de los visores ignoran los bytes después del final de + un PNG, así que la entrada deja la respuesta a tu sistema, ofrece las dos y dice por qué: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> enumera los formatos en los que un daño deja la respuesta a + tu sistema, y por qué. +

+

+ Se rechazan cuatro peticiones antes de escribir nada, porque cada una dejaría en el disco un + archivo que el manifiesto describe mal:

  • un archivo más pequeño de lo que el daño necesita, que saldría intacto
  • - expected: accept junto a un daño, porque nada podría cumplirlo. Escribe - sanitize si tu sistema debe reparar el archivo, o unspecified si esa - es justo la pregunta que haces + expected: accept junto a un daño que rompe las reglas del formato, porque nada + que las siga acepta el archivo. Donde la respuesta depende de tu sistema, accept + está permitido. Escribe sanitize si tu sistema debe reparar el archivo, o + unspecified si esa es justo la pregunta que haces +
    • +
  • + truncate después de append-junk, porque cortaría los bytes añadidos. + Pon truncate primero +
    • +
  • + truncate o append-junk en un archivo comprimido que toma su tamaño + de lo que contiene, porque no hay un tamaño del que partir
@@ -143,11 +181,19 @@

Convertirlo en una prueba

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un archivo cuyo resultado es unspecified se omite aquí, porque lo que tu sistema + debe hacer con él lo decide tu propia política. Su entrada dice por qué, y + policy_options enumera las respuestas posibles. +

Un buen rechazo es un rechazo limpio. Un mensaje que dice qué estaba mal es la respuesta que quieres. Un error de servidor, un bloqueo o un archivo guardado a medias es el defecto que esta diff --git a/web/content/es/site.json b/web/content/es/site.json index 5a563885..9363f5d7 100644 --- a/web/content/es/site.json +++ b/web/content/es/site.json @@ -241,6 +241,8 @@ "unspecified": "Depende de las reglas de tu sistema. Tú decides y después compruebas que lo que ocurre es lo que querías." }, "damages": { + "append-junk": "Escribe bytes aleatorios después del final del archivo y mantiene exactamente el tamaño que pediste, como llega un archivo con algo pegado al final.", + "truncate": "Corta el final del archivo y mantiene exactamente el tamaño que pediste, como lo deja una subida o una copia interrumpida.", "zero-head": "Sobrescribe los primeros bytes del archivo con ceros sin tocar su longitud. La mayoría de los lectores miran ahí primero, así que casi todo nota este daño." }, "terms": { diff --git a/web/content/fr/damage.html b/web/content/fr/damage.html index b3ef8e08..eb64f017 100644 --- a/web/content/fr/damage.html +++ b/web/content/fr/damage.html @@ -53,7 +53,9 @@

Un fichier abîmé garde la taille demandée

et la même commande écrit de nouveau les mêmes octets. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Les réglages se placent après deux-points. L'option peut être répétée, et les dommages s'appliquent dans l'ordre où vous les écrivez. Ça marche avec chacun des {{ .Facts.FormatCount }} formats. @@ -77,14 +79,25 @@

Quels dommages existe-t-il ?

sortent avec un dommage dont aucun lecteur ne se plaint, c'est pourquoi le réglage commence à quatre.

+

+ truncate coupe la fin du fichier, comme le laisse un envoi ou une copie + interrompus, et le fichier garde la taille demandée : il est généré plus grand, et seul son + début atteint le disque. missing indique quelle part du fichier entier disparaît, + la moitié sauf indication contraire. +

+

+ append-junk écrit des octets aléatoires après la fin du fichier, 64 sauf indication + contraire, comme arrive un fichier avec quelque chose collé à sa fin. La même commande ajoute + les mêmes octets. +

Ce que dit le manifeste

Un manifeste qui dit ce qui doit se passer

- Chaque fichier abîmé reçoit une entrée disant que votre système doit le rejeter, avec le dommage - noté à côté : + Là où le dommage enfreint les règles du format, l'entrée dit que votre système doit le rejeter, + avec le dommage noté à côté : 

"expected": {
   "outcome": "reject",
@@ -100,15 +113,40 @@ 
Un manifeste qui dit ce qui doit se passer

   }
 ]

- Deux demandes sont refusées avant que rien ne soit écrit, car chacune laisserait sur le disque un - fichier que le manifeste décrit mal : + Là où les lecteurs divergent, l'entrée le dit au lieu de deviner. Un fichier texte coupé reste + du texte, et la plupart des visionneuses ignorent les octets après la fin d'un PNG, donc + l'entrée laisse la réponse à votre système, propose les deux et dit pourquoi : +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> liste les formats où un dommage laisse la réponse à votre + système, et pourquoi. +

+

+ Quatre demandes sont refusées avant que rien ne soit écrit, car chacune laisserait sur le disque + un fichier que le manifeste décrit mal :

  • un fichier plus petit que ce dont le dommage a besoin, qui sortirait intact
  • - expected: accept à côté d'un dommage, car rien ne pourrait le satisfaire. Écrivez - sanitize si votre système doit réparer le fichier, ou unspecified si - c'est justement la question que vous posez + expected: accept à côté d'un dommage qui enfreint les règles du format, car rien + de ce qui les suit n'accepte le fichier. Là où la réponse dépend de votre système, + accept est permis. Écrivez sanitize si votre système doit réparer le + fichier, ou unspecified si c'est justement la question que vous posez +
    • +
  • + truncate après append-junk, qui couperait les octets ajoutés. Placez + truncate en premier +
    • +
  • + truncate ou append-junk sur une archive qui tire sa taille de son + contenu, car il n'y a pas de taille d'où partir
@@ -146,11 +184,19 @@

En faire un test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un fichier dont le résultat est unspecified est laissé de côté ici, car ce que + votre système doit en faire relève de votre propre politique. Son entrée dit pourquoi, et + policy_options liste les réponses possibles. +

Un bon refus est un refus net. Un message qui dit ce qui n'allait pas est la réponse voulue. Une erreur serveur, un blocage ou un fichier à moitié enregistré est le défaut que ce test existe diff --git a/web/content/fr/site.json b/web/content/fr/site.json index 8d626b0a..298162ec 100644 --- a/web/content/fr/site.json +++ b/web/content/fr/site.json @@ -241,6 +241,8 @@ "unspecified": "Cela dépend des règles de votre système. Vous décidez, puis vous vérifiez que ce qui se passe est ce que vous aviez voulu." }, "damages": { + "append-junk": "Écrit des octets aléatoires après la fin du fichier et garde exactement la taille demandée, comme arrive un fichier avec quelque chose collé à sa fin.", + "truncate": "Coupe la fin du fichier et garde exactement la taille demandée, comme le laisse un envoi ou une copie interrompus.", "zero-head": "Écrase les premiers octets du fichier avec des zéros, sans toucher à sa longueur. La plupart des lecteurs regardent d'abord là, donc presque tout remarque ce dommage." }, "terms": { diff --git a/web/content/hi/damage.html b/web/content/hi/damage.html index 9d91a08b..ca7fdfe3 100644 --- a/web/content/hi/damage.html +++ b/web/content/hi/damage.html @@ -51,7 +51,9 @@

खराब फ़ाइल का आकार वही रहता ह रखती है, और वही कमांड फिर वही बाइट लिखता है। 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

सेटिंग कोलन के बाद लिखी जाती है। विकल्प दोहराया जा सकता है, और बिगाड़ आपके लिखे क्रम में लागू होते हैं। यह सभी {{ .Facts.FormatCount }} फ़ॉर्मैट के साथ चलता है। @@ -73,14 +75,24 @@

कौन-कौन से बिगाड़ हैं?

पाठ नहीं है। चार बाइट से नीचे कुछ फ़ॉर्मैट ऐसे बिगाड़ के साथ निकलते हैं जिसकी कोई रीडर शिकायत नहीं करता, इसीलिए सेटिंग चार से शुरू होती है।

+

+ truncate फ़ाइल का अंत काट देता है, जैसे बीच में टूटा अपलोड या कॉपी उसे छोड़ता है, + और फ़ाइल फिर भी उसी आकार की रहती है जो आपने माँगा: वह बड़ी बनाई जाती है और डिस्क तक सिर्फ़ उसकी + शुरुआत पहुँचती है। missing बताता है कि पूरी फ़ाइल का कितना हिस्सा गायब है, आधा, जब + तक आप कुछ और न कहें। +

+

+ append-junk फ़ाइल के अंत के बाद यादृच्छिक बाइट लिखता है, 64, जब तक आप कुछ और न + कहें, जैसे कोई फ़ाइल अपने अंत में कुछ चिपका हुआ लेकर आती है। एक ही कमांड वही बाइट जोड़ती है। +

मैनिफ़ेस्ट क्या कहता है

एक मैनिफ़ेस्ट जो बताता है कि क्या होना चाहिए

- हर खराब फ़ाइल को एक प्रविष्टि मिलती है जो कहती है कि आपके सिस्टम को उसे अस्वीकार करना चाहिए, और - बिगाड़ उसके बगल में दर्ज रहता है: + जहाँ बिगाड़ फ़ॉर्मैट के नियम तोड़ता है, वहाँ प्रविष्टि कहती है कि आपके सिस्टम को उसे अस्वीकार + करना चाहिए, और बिगाड़ उसके बगल में दर्ज रहता है: 

"expected": {
   "outcome": "reject",
@@ -96,15 +108,40 @@ 
एक मैनिफ़ेस्ट जो बताता है कि 
   }
 ]

- दो अनुरोध कुछ भी लिखे जाने से पहले ठुकरा दिए जाते हैं, क्योंकि हर एक डिस्क पर ऐसी फ़ाइल छोड़ देता - जिसका मैनिफ़ेस्ट गलत वर्णन करता: + जहाँ रीडर अलग-अलग बर्ताव करते हैं, वहाँ प्रविष्टि अंदाज़ा लगाने के बजाय यही कहती है। काटी गई पाठ + फ़ाइल फिर भी पाठ है, और ज़्यादातर इमेज व्यूअर PNG के अंत के बाद के बाइट छोड़ देते हैं, इसलिए + प्रविष्टि जवाब आपके सिस्टम पर छोड़ती है, दोनों विकल्प देती है और कारण बताती है: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> वे फ़ॉर्मैट और कारण बताता है जिनमें कोई बिगाड़ जवाब आपके + सिस्टम पर छोड़ता है। +

+

+ चार अनुरोध कुछ भी लिखे जाने से पहले ठुकरा दिए जाते हैं, क्योंकि हर एक डिस्क पर ऐसी फ़ाइल छोड़ + देता जिसका मैनिफ़ेस्ट गलत वर्णन करता:

  • बिगाड़ को जितना चाहिए उससे छोटी फ़ाइल, जो ज्यों की त्यों निकलती
  • - बिगाड़ के साथ expected: accept, क्योंकि कोई भी उसे पूरा नहीं कर सकता। अगर आपके सिस्टम - को फ़ाइल सुधारनी है तो sanitize लिखें, या अगर आप यही सवाल पूछ रहे हैं तो - unspecified + फ़ॉर्मैट के नियम तोड़ने वाले बिगाड़ के साथ expected: accept, क्योंकि उन नियमों को + मानने वाली कोई चीज़ फ़ाइल स्वीकार नहीं करती। जहाँ जवाब आपके सिस्टम पर निर्भर है, वहाँ + accept की अनुमति है। अगर आपके सिस्टम को फ़ाइल सुधारनी है तो sanitize + लिखें, और अगर आप ठीक यही पूछ रहे हैं तो unspecified +
    • +
  • + append-junk के बाद truncate, क्योंकि वह जोड़े गए बाइट काट देता। + truncate को पहले रखें +
    • +
  • + ऐसे आर्काइव पर truncate या append-junk जो अपना आकार अपनी सामग्री से + लेता है, क्योंकि शुरू करने के लिए कोई आकार नहीं है
@@ -142,11 +179,19 @@

इसे टेस्ट बनाना

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ जिस फ़ाइल का नतीजा unspecified है, उसे यहाँ छोड़ दिया जाता है, क्योंकि आपके सिस्टम + को उसके साथ क्या करना है, यह आपकी अपनी नीति तय करती है। उसकी प्रविष्टि कारण बताती है, और + policy_options संभव जवाब गिनाता है। +

अच्छा इनकार साफ़ इनकार होता है। जो संदेश बताए कि क्या गलत था, वही वह जवाब है जो आप चाहते हैं। सर्वर त्रुटि, अटक जाना या आधी सहेजी फ़ाइल वह खराबी है जिसे ढूँढ़ने के लिए यह टेस्ट है। diff --git a/web/content/hi/site.json b/web/content/hi/site.json index 06ad3ce6..f575a66d 100644 --- a/web/content/hi/site.json +++ b/web/content/hi/site.json @@ -241,6 +241,8 @@ "unspecified": "यह आपके सिस्टम के नियमों पर निर्भर है। आप तय करें, फिर जाँचें कि जो होता है वही है जो आप चाहते थे।" }, "damages": { + "append-junk": "फ़ाइल के अंत के बाद यादृच्छिक बाइट लिखता है और ठीक वही आकार रखता है जो आपने माँगा, जैसे कोई फ़ाइल अपने अंत में कुछ चिपका हुआ लेकर आती है।", + "truncate": "फ़ाइल का अंत काट देता है और ठीक वही आकार रखता है जो आपने माँगा, जैसे बीच में टूटा अपलोड या कॉपी उसे छोड़ता है।", "zero-head": "फ़ाइल के शुरुआती बाइट को शून्य से ढक देता है और लंबाई नहीं छेड़ता। ज़्यादातर रीडर सबसे पहले वहीं देखते हैं, इसलिए यह बिगाड़ लगभग हर चीज़ पकड़ लेती है।" }, "terms": { diff --git a/web/content/id/damage.html b/web/content/id/damage.html index 7922552b..cdbd3a79 100644 --- a/web/content/id/damage.html +++ b/web/content/id/damage.html @@ -52,7 +52,9 @@

File yang rusak tetap berukuran seperti yang Anda minta

minta, dan perintah yang sama menulis byte yang sama lagi. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Pengaturan ditulis setelah titik dua. Opsi ini bisa diulang, dan kerusakan diterapkan menurut urutan yang Anda tulis. Berlaku untuk semua {{ .Facts.FormatCount }} format. @@ -75,14 +77,25 @@

Kerusakan apa saja yang ada?

byte nol bukan teks. Di bawah empat byte, sebagian format keluar dengan kerusakan yang tidak dikeluhkan pembaca mana pun, itulah sebabnya pengaturan dimulai dari empat.

+

+ truncate memotong ujung file, seperti yang ditinggalkan unggahan atau salinan yang + terputus, dan file tetap berukuran sesuai permintaan Anda: file dibuat lebih besar dan hanya + awalnya yang sampai ke disk. missing menyatakan berapa bagian dari file utuh yang + hilang, separuh jika Anda tidak menyebut lain. +

+

+ append-junk menulis byte acak setelah akhir file, 64 jika Anda tidak menyebut lain, + seperti file yang tiba dengan sesuatu menempel di ujungnya. Perintah yang sama menambahkan byte + yang sama. +

Yang dikatakan manifes

Manifes yang menyatakan apa yang harus terjadi

- Setiap file yang rusak mendapat catatan yang menyatakan sistem Anda harus menolaknya, dengan - kerusakannya dicatat di sampingnya: + Jika kerusakan melanggar aturan format, catatannya menyatakan sistem Anda harus menolaknya, + dengan kerusakannya dicatat di sampingnya: 

"expected": {
   "outcome": "reject",
@@ -98,15 +111,42 @@ 
Manifes yang menyatakan apa yang harus terjadi

   }
 ]

- Dua permintaan ditolak sebelum apa pun ditulis, karena masing-masing akan meninggalkan file di disk - yang dijelaskan keliru oleh manifes: + Jika pembaca berbeda pendapat, catatannya menyatakan hal itu alih-alih menebak. File teks yang + terpotong tetap teks, dan sebagian besar penampil gambar melewati byte setelah akhir PNG, jadi + catatannya menyerahkan jawaban kepada sistem Anda, menawarkan keduanya, dan menjelaskan + alasannya: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> mencantumkan format tempat suatu kerusakan menyerahkan + jawaban kepada sistem Anda, beserta alasannya. +

+

+ Empat permintaan ditolak sebelum apa pun ditulis, karena masing-masing akan meninggalkan file di + disk yang dijelaskan keliru oleh manifes:

  • file yang lebih kecil dari yang dibutuhkan kerusakan, yang akan keluar tanpa perubahan
  • - expected: accept di samping kerusakan, karena tidak ada yang bisa memenuhinya. Tulis - sanitize jika sistem Anda memang harus memperbaiki file itu, atau - unspecified jika justru itu pertanyaan Anda + expected: accept di samping kerusakan yang melanggar aturan format, karena tidak + ada yang mengikuti aturan itu yang menerima file tersebut. Jika jawabannya bergantung pada + sistem Anda, accept diperbolehkan. Tulis sanitize jika sistem Anda + memang harus memperbaiki file itu, atau unspecified jika justru itu pertanyaan + Anda +
    • +
  • + truncate setelah append-junk, karena akan memotong byte yang + ditambahkan. Letakkan truncate lebih dulu +
    • +
  • + truncate atau append-junk pada arsip yang mengambil ukurannya dari + isinya, karena tidak ada ukuran untuk dijadikan titik awal
@@ -144,11 +184,19 @@

Menjadikannya tes

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ File dengan hasil unspecified dilewati di sini, karena apa yang harus dilakukan + sistem Anda terhadapnya adalah kebijakan Anda sendiri. Catatannya menjelaskan alasannya, dan + policy_options mencantumkan jawaban yang mungkin. +

Penolakan yang baik adalah penolakan yang bersih. Pesan yang menyebut apa yang salah adalah jawaban yang Anda inginkan. Kesalahan server, macet, atau file yang tersimpan setengah adalah cacat yang diff --git a/web/content/id/site.json b/web/content/id/site.json index 304beaa6..56d6e1a0 100644 --- a/web/content/id/site.json +++ b/web/content/id/site.json @@ -241,6 +241,8 @@ "unspecified": "Tergantung pada aturan sistem Anda. Anda yang memutuskan, lalu memeriksa apakah yang terjadi sesuai maksud Anda." }, "damages": { + "append-junk": "Menulis byte acak setelah akhir file dan mempertahankan ukuran persis yang Anda minta, seperti file yang tiba dengan sesuatu menempel di ujungnya.", + "truncate": "Memotong ujung file dan mempertahankan ukuran persis yang Anda minta, seperti yang ditinggalkan unggahan atau salinan yang terputus.", "zero-head": "Menimpa byte pertama file dengan nol tanpa mengubah panjangnya. Sebagian besar pembaca melihat ke sana lebih dulu, jadi hampir semua hal menyadari kerusakan ini." }, "terms": { diff --git a/web/content/it/damage.html b/web/content/it/damage.html index c6dd49fa..384bbb20 100644 --- a/web/content/it/damage.html +++ b/web/content/it/damage.html @@ -52,7 +52,9 @@

Un file danneggiato ha ancora la dimensione richiesta

richiesta, e lo stesso comando riscrive gli stessi byte. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Le impostazioni vanno dopo i due punti. L'opzione si può ripetere, e i danni vengono applicati nell'ordine in cui li scrivi. Funziona con ognuno dei {{ .Facts.FormatCount }} formati. @@ -75,14 +77,24 @@

Quali danni ci sono?

byte nulli non è testo. Sotto i quattro byte alcuni formati escono con un danno di cui nessun lettore si lamenta, ed è per questo che l'impostazione parte da quattro.

+

+ truncate taglia la fine del file, come la lascia un caricamento o una copia + interrotti, e il file ha comunque la dimensione richiesta: viene generato più grande e solo il + suo inizio arriva sul disco. missing dice quale parte del file intero manca, la + metà se non indichi altro. +

+

+ append-junk scrive byte casuali dopo la fine del file, 64 se non indichi altro, + come arriva un file con qualcosa attaccato in fondo. Lo stesso comando aggiunge gli stessi byte. +

Cosa dice il manifest

Un manifest che dice cosa deve succedere

- Ogni file danneggiato riceve una voce che dice che il tuo sistema deve rifiutarlo, con il danno - registrato accanto: + Dove il danno viola le regole del formato, la voce dice che il tuo sistema deve rifiutarlo, con + il danno registrato accanto: 

"expected": {
   "outcome": "reject",
@@ -98,15 +110,40 @@ 
Un manifest che dice cosa deve succedere

   }
 ]

- Due richieste vengono rifiutate prima che venga scritto qualcosa, perché ciascuna lascerebbe su - disco un file che il manifest descrive male: + Dove i lettori non concordano, la voce lo dice invece di tirare a indovinare. Un file di testo + tagliato è ancora testo, e la maggior parte dei visualizzatori ignora i byte dopo la fine di un + PNG, quindi la voce lascia la risposta al tuo sistema, offre entrambe e dice perché: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> elenca i formati in cui un danno lascia la risposta al tuo + sistema, e perché. +

+

+ Quattro richieste vengono rifiutate prima che venga scritto qualcosa, perché ciascuna lascerebbe + su disco un file che il manifest descrive male:

  • un file più piccolo di quanto serve al danno, che uscirebbe intatto
  • - expected: accept accanto a un danno, perché nulla potrebbe soddisfarlo. Scrivi - sanitize se il tuo sistema deve riparare il file, oppure unspecified - se è proprio questa la domanda che poni + expected: accept accanto a un danno che viola le regole del formato, perché nulla + che le segua accetta il file. Dove la risposta dipende dal tuo sistema, accept è + consentito. Scrivi sanitize se il tuo sistema deve riparare il file, oppure + unspecified se è proprio questa la domanda che poni +
    • +
  • + truncate dopo append-junk, perché taglierebbe via i byte aggiunti. + Metti truncate per primo +
    • +
  • + truncate o append-junk su un archivio che prende la dimensione da + ciò che contiene, perché non c'è una dimensione da cui partire
@@ -144,11 +181,19 @@

Trasformarlo in un test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un file il cui esito è unspecified viene escluso qui, perché cosa il tuo sistema + debba farne lo decide la tua politica. La sua voce dice perché, e policy_options + elenca le risposte possibili. +

Un buon rifiuto è un rifiuto pulito. Un messaggio che dice cosa non andava è la risposta che vuoi. Un errore del server, un blocco o un file salvato a metà è il difetto che questo test esiste per diff --git a/web/content/it/site.json b/web/content/it/site.json index 052f7688..3de28c2a 100644 --- a/web/content/it/site.json +++ b/web/content/it/site.json @@ -241,6 +241,8 @@ "unspecified": "Dipende dalle regole del tuo sistema. Decidi tu, poi controlli che ciò che accade sia ciò che intendevi." }, "damages": { + "append-junk": "Scrive byte casuali dopo la fine del file e mantiene esattamente la dimensione richiesta, come arriva un file con qualcosa attaccato in fondo.", + "truncate": "Taglia la fine del file e mantiene esattamente la dimensione richiesta, come la lascia un caricamento o una copia interrotti.", "zero-head": "Sovrascrive con zeri i primi byte del file, lasciandone intatta la lunghezza. La maggior parte dei lettori guarda prima lì, quindi quasi tutto si accorge di questo danno." }, "terms": { diff --git a/web/content/ja/damage.html b/web/content/ja/damage.html index 445d6497..7c70ad3c 100644 --- a/web/content/ja/damage.html +++ b/web/content/ja/damage.html @@ -41,7 +41,9 @@

破損したファイルも、求めたサイズのままです

ファイルは通常どおり生成され、ディスクへ向かう途中で壊されます。求めたサイズは保たれ、同じコマンドは同じバイトをもう一度書き出します。 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

設定はコロンの後ろに書きます。オプションは繰り返せて、破損は書いた順に適用されます。{{ .Facts.FormatCount }}種類すべての形式で使えます。

@@ -58,13 +60,19 @@

どんな破損がありますか。

zero-headはファイルの先頭をゼロで上書きします。ほとんどのリーダーはまずそこを見ます。ファイルが何であるかを示すシグネチャとヘッダーです。そのため、ほぼどのリーダーでも気づきます。プレーンテキストやログにはシグネチャがありませんが、これらも拒否されます。ゼロのバイトの連なりはテキストではないからです。4バイト未満では、どのリーダーも文句を言わない破損になる形式があります。設定が4から始まるのはそのためです。

+

+ truncateはファイルの末尾を切り落とします。途中で止まったアップロードやコピーが残すのと同じ形ですが、ファイルは指定したサイズのままです。大きめに生成され、その先頭だけがディスクに届くからです。missingはファイル全体のうちどれだけが失われるかを示し、指定しなければ半分です。 +

+

+ append-junkはファイルの末尾の後ろにランダムなバイトを書き込みます。指定しなければ64バイトで、末尾に何かが付いたまま届いたファイルと同じ形です。同じコマンドは同じバイトを付け加えます。 +

マニフェストの内容

何が起こるべきかを示すマニフェスト

- 破損したファイルにはそれぞれ、システムがそれを拒否すべきことを示す項目が付き、破損の内容がその横に記録されます。 + 破損が形式の規則に反する場合、項目はシステムがそれを拒否すべきことを示し、破損の内容がその横に記録されます。 

"expected": {
   "outcome": "reject",
@@ -80,13 +88,31 @@ 
何が起こるべきかを示すマニフェスト

   }
 ]

- 2種類のリクエストは、何かが書き込まれる前に拒否されます。どちらもマニフェストの記述が誤ったファイルをディスクに残してしまうからです。 + リーダーによって扱いが分かれる場合、項目は推測せずにそう伝えます。末尾を切られたテキストファイルもテキストのままですし、ほとんどの画像ビューアーはPNGの終わりの後ろのバイトを読み飛ばします。そのため項目は答えをシステムに委ね、両方の答えを示し、理由を書きます。 +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id>は、破損が答えをシステムに委ねる形式とその理由を一覧にします。 +

+

+ 4種類のリクエストは、何かが書き込まれる前に拒否されます。どれもマニフェストの記述が誤ったファイルをディスクに残してしまうからです。

  • 破損に必要なサイズより小さいファイル。そのままの状態で出てきてしまいます
  • - 破損と並べたexpected: - accept。何もそれを満たせないためです。システムがファイルを修復するはずならsanitizeを、まさにそれを確かめたいならunspecifiedを書いてください + 形式の規則に反する破損と並べたexpected: accept。規則に従うものはそのファイルを受け入れないためです。答えがシステム次第の場合はacceptを書けます。システムがファイルを修復するはずならsanitizeを、まさにそれを確かめたいならunspecifiedを書いてください +
    • +
  • + append-junkの後のtruncate。付け加えたバイトを切り落としてしまうためです。truncateを先に置いてください +
    • +
  • + 中身からサイズが決まるアーカイブに対するtruncateやappend-junk。基準になるサイズがないためです
@@ -122,11 +148,17 @@

テストにする

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 結果がunspecifiedのファイルはここでは除外します。システムがそれをどう扱うべきかは、あなた自身の方針で決まるからです。項目にはその理由が書かれ、policy_optionsには取りうる答えが並びます。 +

よい拒否は、きれいな拒否です。何が悪かったかを伝えるメッセージが、求めている答えです。サーバーエラー、ハング、書きかけで保存されたファイルは、このテストが見つけるために存在する欠陥です。

diff --git a/web/content/ja/site.json b/web/content/ja/site.json index 234cbed2..b77cb2cc 100644 --- a/web/content/ja/site.json +++ b/web/content/ja/site.json @@ -241,6 +241,8 @@ "unspecified": "システムのルール次第です。あなたが決め、実際に起きることが意図どおりかを確認してください。" }, "damages": { + "append-junk": "ファイルの末尾の後ろにランダムなバイトを書き込み、指定したサイズをそのまま保ちます。末尾に何かが付いたまま届いたファイルと同じ形です。", + "truncate": "ファイルの末尾を切り落とし、指定したサイズをそのまま保ちます。途中で止まったアップロードやコピーが残すのと同じ形です。", "zero-head": "ファイルの先頭のバイトをゼロで上書きし、長さは変えません。ほとんどのリーダーはまずそこを見るので、この破損にはほぼどれも気づきます。" }, "terms": { diff --git a/web/content/ko/damage.html b/web/content/ko/damage.html index e7252779..b9533e52 100644 --- a/web/content/ko/damage.html +++ b/web/content/ko/damage.html @@ -46,7 +46,9 @@

손상된 파일도 요청한 크기를 유지합니다

파일은 평소대로 생성된 뒤 디스크로 가는 길에 망가집니다. 요청한 크기는 그대로이고, 같은 명령은 같은 바이트를 다시 씁니다. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

설정은 콜론 뒤에 씁니다. 옵션은 반복할 수 있고, 손상은 쓴 순서대로 적용됩니다. {{ .Facts.FormatCount }}개 형식 모두에서 동작합니다.

@@ -65,13 +67,19 @@

어떤 손상이 있나요?

그래서 거의 모든 리더가 알아챕니다. 일반 텍스트와 로그에는 시그니처가 없지만 역시 거부됩니다. 0 바이트의 연속은 텍스트가 아니기 때문입니다. 4바이트 미만에서는 어떤 리더도 불평하지 않는 손상이 나오는 형식이 있으며, 설정이 4부터 시작하는 이유가 그것입니다.

+

+ truncate는 중간에 끊긴 업로드나 복사가 남기는 것처럼 파일의 끝을 잘라 내지만, 파일은 여전히 요청한 크기입니다. 더 크게 생성되고 그 앞부분만 디스크에 닿기 때문입니다. missing은 전체 파일 중 얼마가 사라지는지를 나타내며, 따로 지정하지 않으면 절반입니다. +

+

+ append-junk는 끝에 무언가가 붙은 채 도착한 파일처럼 파일의 끝 뒤에 무작위 바이트를 씁니다. 따로 지정하지 않으면 64바이트입니다. 같은 명령은 같은 바이트를 덧붙입니다. +

매니페스트가 말하는 것

무엇이 일어나야 하는지 말해 주는 매니페스트

- 손상된 파일마다 시스템이 그것을 거부해야 한다는 항목이 붙고, 손상 내용이 그 옆에 기록됩니다. + 손상이 형식의 규칙을 어기는 경우, 항목은 시스템이 그것을 거부해야 한다고 말하고 손상 내용이 그 옆에 기록됩니다. 

"expected": {
   "outcome": "reject",
@@ -87,13 +95,31 @@ 
무엇이 일어나야 하는지 말해 주는 매니페스트

   }
 ]

- 두 가지 요청은 무엇이든 쓰기 전에 거부됩니다. 둘 다 매니페스트가 잘못 설명하는 파일을 디스크에 남기게 되기 때문입니다. + 리더마다 다르게 다루는 경우, 항목은 추측하는 대신 그렇다고 말합니다. 끝이 잘린 텍스트 파일도 여전히 텍스트이고, 대부분의 이미지 뷰어는 PNG의 끝 뒤에 있는 바이트를 건너뜁니다. 그래서 항목은 답을 시스템에 맡기고, 두 가지 답을 모두 제시하며, 이유를 적습니다. +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id>는 손상이 답을 시스템에 맡기는 형식과 그 이유를 나열합니다. +

+

+ 네 가지 요청은 무엇이든 쓰기 전에 거부됩니다. 모두 매니페스트가 잘못 설명하는 파일을 디스크에 남기게 되기 때문입니다.

  • 손상에 필요한 크기보다 작은 파일. 그대로 나오게 됩니다
  • - 손상 옆의 expected: accept. 어떤 것도 그것을 충족할 수 없기 때문입니다. 시스템이 파일을 복구해야 한다면 - sanitize를, 바로 그것을 확인하려는 것이라면 unspecified를 쓰세요 + 형식의 규칙을 어기는 손상 옆의 expected: accept. 그 규칙을 따르는 것은 어떤 것도 그 파일을 받아들이지 않기 때문입니다. 답이 시스템에 달린 경우에는 accept를 쓸 수 있습니다. 시스템이 파일을 복구해야 한다면 sanitize를, 바로 그것을 확인하려는 것이라면 unspecified를 쓰세요 +
    • +
  • + append-junk 뒤의 truncate. 덧붙인 바이트를 잘라 내기 때문입니다. truncate를 먼저 두세요 +
    • +
  • + 담긴 내용에서 크기를 정하는 아카이브에 대한 truncate나 append-junk. 기준이 될 크기가 없기 때문입니다
@@ -129,11 +155,17 @@

테스트로 만들기

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 결과가 unspecified인 파일은 여기서 건너뜁니다. 시스템이 그 파일을 어떻게 다뤄야 하는지는 여러분의 정책이 정하기 때문입니다. 항목에는 그 이유가 적혀 있고, policy_options에는 가능한 답이 나열됩니다. +

좋은 거부는 깔끔한 거부입니다. 무엇이 잘못됐는지 알려 주는 메시지가 원하는 답입니다. 서버 오류, 멈춤, 반쯤 저장된 파일은 이 테스트가 찾아내려고 존재하는 결함입니다.

diff --git a/web/content/ko/site.json b/web/content/ko/site.json index 14f04a7a..edf73141 100644 --- a/web/content/ko/site.json +++ b/web/content/ko/site.json @@ -241,6 +241,8 @@ "unspecified": "시스템의 규칙에 따라 다릅니다. 직접 결정한 뒤 실제 일어나는 일이 의도한 것인지 확인하세요." }, "damages": { + "append-junk": "파일의 끝 뒤에 무작위 바이트를 쓰고 요청한 크기를 그대로 유지합니다. 끝에 무언가가 붙은 채 도착한 파일과 같은 모양입니다.", + "truncate": "파일의 끝을 잘라 내고 요청한 크기를 그대로 유지합니다. 중간에 끊긴 업로드나 복사가 남기는 모양과 같습니다.", "zero-head": "파일의 첫 바이트들을 길이는 그대로 둔 채 0으로 덮어씁니다. 대부분의 리더가 가장 먼저 그곳을 보므로 거의 모든 것이 이 손상을 알아챕니다." }, "terms": { diff --git a/web/content/nl/damage.html b/web/content/nl/damage.html index c5d39fc4..93ef2af3 100644 --- a/web/content/nl/damage.html +++ b/web/content/nl/damage.html @@ -53,7 +53,9 @@

Een beschadigd bestand heeft nog steeds de grootte die je vroeg

grootte die je vroeg, en hetzelfde commando schrijft opnieuw dezelfde bytes. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Instellingen komen achter een dubbele punt. De optie mag herhaald worden, en de beschadigingen worden toegepast in de volgorde waarin je ze schrijft. Het werkt met elk van de @@ -77,14 +79,25 @@

Welke beschadigingen zijn er?

reeks nulbytes geen tekst is. Onder vier bytes komen sommige formaten uit met schade waar geen lezer over klaagt, en daarom begint de instelling bij vier.

+

+ truncate knipt het einde van het bestand af, zoals een afgebroken upload of kopie + het achterlaat, en het bestand heeft toch de gevraagde grootte: het wordt groter gemaakt en + alleen het begin komt op de schijf. missing zegt welk deel van het hele bestand weg + is, de helft als je niets anders opgeeft. +

+

+ append-junk schrijft willekeurige bytes achter het einde van het bestand, 64 als je + niets anders opgeeft, zoals een bestand binnenkomt waar iets aan het einde aan vastzit. Dezelfde + opdracht voegt dezelfde bytes toe. +

Wat het manifest zegt

Een manifest dat zegt wat er moet gebeuren

- Elk beschadigd bestand krijgt een regel die zegt dat je systeem het moet weigeren, met de - beschadiging ernaast vastgelegd: + Waar de beschadiging de regels van het formaat breekt, zegt de vermelding dat je systeem het + bestand moet weigeren, met de beschadiging ernaast vastgelegd: 

"expected": {
   "outcome": "reject",
@@ -100,15 +113,40 @@ 
Een manifest dat zegt wat er moet gebeuren

   }
 ]

- Twee verzoeken worden geweigerd voordat er iets wordt geschreven, omdat elk een bestand op schijf - zou laten staan dat het manifest verkeerd beschrijft: + Waar lezers verschillen, zegt de vermelding dat in plaats van te gokken. Een afgeknipt + tekstbestand is nog steeds tekst, en de meeste viewers slaan bytes achter het einde van een PNG + over, dus laat de vermelding het antwoord aan je systeem, biedt beide aan en zegt waarom: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> noemt de formaten waarin een beschadiging het antwoord aan je + systeem laat, en waarom. +

+

+ Vier verzoeken worden geweigerd voordat er iets wordt geschreven, omdat elk een bestand op + schijf zou laten staan dat het manifest verkeerd beschrijft:

  • een bestand dat kleiner is dan de beschadiging nodig heeft en ongewijzigd zou uitkomen
  • - expected: accept naast een beschadiging, omdat niets daaraan kan voldoen. Schrijf - sanitize als je systeem het bestand moet repareren, of unspecified - als dat juist de vraag is die je stelt + expected: accept naast een beschadiging die de regels van het formaat breekt, + omdat niets dat ze volgt het bestand accepteert. Waar het antwoord van je systeem afhangt, mag + accept wel. Schrijf sanitize als je systeem het bestand moet + repareren, of unspecified als dat juist de vraag is die je stelt +
    • +
  • + truncate na append-junk, omdat het de toegevoegde bytes weer zou + afknippen. Zet truncate vooraan +
    • +
  • + truncate of append-junk op een archief dat zijn grootte haalt uit + wat het bevat, omdat er geen grootte is om van uit te gaan
@@ -146,11 +184,19 @@

Er een test van maken

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Een bestand met de uitkomst unspecified wordt hier overgeslagen, omdat je eigen + beleid bepaalt wat je systeem ermee moet doen. De vermelding zegt waarom, en + policy_options noemt de mogelijke antwoorden. +

Een goede weigering is een schone weigering. Een melding die zegt wat er mis was, is het antwoord dat je wilt. Een serverfout, een vastloper of een half opgeslagen bestand is het gebrek dat deze diff --git a/web/content/nl/site.json b/web/content/nl/site.json index 10139a11..a72a7f02 100644 --- a/web/content/nl/site.json +++ b/web/content/nl/site.json @@ -241,6 +241,8 @@ "unspecified": "Het hangt af van de regels van je systeem. Jij beslist, en controleert daarna of wat er gebeurt is wat je bedoelde." }, "damages": { + "append-junk": "Schrijft willekeurige bytes achter het einde van het bestand en houdt precies de gevraagde grootte, zoals een bestand binnenkomt waar iets aan het einde aan vastzit.", + "truncate": "Knipt het einde van het bestand af en houdt precies de gevraagde grootte, zoals een afgebroken upload of kopie het achterlaat.", "zero-head": "Overschrijft de eerste bytes van het bestand met nullen en laat de lengte ongemoeid. De meeste lezers kijken daar eerst, dus bijna alles merkt deze beschadiging." }, "terms": { diff --git a/web/content/pl/damage.html b/web/content/pl/damage.html index cd5ff132..c827f7c0 100644 --- a/web/content/pl/damage.html +++ b/web/content/pl/damage.html @@ -53,7 +53,9 @@

Uszkodzony plik ma nadal rozmiar, o który prosiłeś

a to samo polecenie zapisuje znowu te same bajty. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Ustawienia podaje się po dwukropku. Flagę można powtarzać, a uszkodzenia są stosowane w kolejności, w jakiej je zapiszesz. Działa z każdym z {{ .Facts.FormatCount }} formatów. @@ -76,14 +78,24 @@

Jakie są uszkodzenia?

czterech bajtów niektóre formaty wychodzą z uszkodzeniem, na które nie skarży się żaden czytnik, dlatego ustawienie zaczyna się od czterech.

+

+ truncate odcina koniec pliku, tak jak zostawia go przerwane wysyłanie albo + kopiowanie, a plik ma nadal rozmiar, o który prosiłeś: powstaje większy i na dysk trafia tylko + jego początek. missing mówi, jaka część całego pliku znika, połowa, jeśli nie + podasz inaczej. +

+

+ append-junk dopisuje losowe bajty za końcem pliku, 64, jeśli nie podasz inaczej, + tak jak przychodzi plik z czymś przyklejonym na końcu. To samo polecenie dopisuje te same bajty. +

Co mówi manifest

Manifest, który mówi, co ma się stać

- Każdy uszkodzony plik dostaje wpis mówiący, że system powinien go odrzucić, a obok zapisane jest - uszkodzenie: + Tam, gdzie uszkodzenie łamie reguły formatu, wpis mówi, że system powinien plik odrzucić, a obok + zapisane jest uszkodzenie: 

"expected": {
   "outcome": "reject",
@@ -99,15 +111,40 @@ 
Manifest, który mówi, co ma się stać

   }
 ]

- Dwie prośby są odrzucane, zanim cokolwiek powstanie, bo każda zostawiłaby na dysku plik, który + Tam, gdzie czytniki się różnią, wpis mówi to wprost, zamiast zgadywać. Ucięty plik tekstowy jest + nadal tekstem, a większość przeglądarek obrazów pomija bajty za końcem PNG, więc wpis zostawia + odpowiedź twojemu systemowi, podaje obie i mówi dlaczego: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> wypisuje formaty, w których uszkodzenie zostawia odpowiedź + twojemu systemowi, i powód. +

+

+ Cztery prośby są odrzucane, zanim cokolwiek powstanie, bo każda zostawiłaby na dysku plik, który manifest opisuje błędnie:

  • plik mniejszy, niż potrzebuje uszkodzenie, bo wyszedłby nietknięty
  • - expected: accept obok uszkodzenia, bo nic nie mogłoby tego spełnić. Napisz - sanitize, jeśli system ma plik naprawić, albo unspecified, jeśli - właśnie o to pytasz + expected: accept obok uszkodzenia, które łamie reguły formatu, bo nic, co ich + przestrzega, nie przyjmie takiego pliku. Tam, gdzie odpowiedź zależy od twojego systemu, + accept jest dozwolone. Napisz sanitize, jeśli system ma plik + naprawić, albo unspecified, jeśli właśnie o to pytasz +
    • +
  • + truncate po append-junk, bo odcięłoby dopisane bajty. Postaw + truncate pierwsze +
    • +
  • + truncate albo append-junk na archiwum, które bierze rozmiar z tego, + co zawiera, bo nie ma rozmiaru, od którego można wyjść
@@ -145,11 +182,19 @@

Zamiana tego w test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Plik o wyniku unspecified jest tu pomijany, bo to, co twój system ma z nim zrobić, + jest twoją własną polityką. Jego wpis mówi dlaczego, a policy_options wymienia + możliwe odpowiedzi. +

Dobre odrzucenie jest czyste. Komunikat mówiący, co było nie tak, to odpowiedź, o którą chodzi. Błąd serwera, zawieszenie albo plik zapisany do połowy to wada, którą ten test ma znaleźć. diff --git a/web/content/pl/site.json b/web/content/pl/site.json index 54f43a9a..6bec8999 100644 --- a/web/content/pl/site.json +++ b/web/content/pl/site.json @@ -326,6 +326,8 @@ } ], "damages": { + "append-junk": "Dopisuje losowe bajty za końcem pliku i zachowuje dokładnie rozmiar, o który prosiłeś, tak jak przychodzi plik z czymś przyklejonym na końcu.", + "truncate": "Odcina koniec pliku i zachowuje dokładnie rozmiar, o który prosiłeś, tak jak zostawia go przerwane wysyłanie albo kopiowanie.", "zero-head": "Nadpisuje pierwsze bajty pliku zerami, nie zmieniając jego długości. Większość czytników zagląda tam najpierw, więc to uszkodzenie zauważy prawie wszystko." } } diff --git a/web/content/pt-BR/damage.html b/web/content/pt-BR/damage.html index cfa96876..06c1a884 100644 --- a/web/content/pt-BR/damage.html +++ b/web/content/pt-BR/damage.html @@ -52,7 +52,9 @@

Um arquivo danificado continua com o tamanho que você pediu

o mesmo comando grava os mesmos bytes outra vez. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

As configurações vão depois de dois-pontos. A opção pode ser repetida, e os danos são aplicados na ordem em que você os escreve. Funciona com cada um dos {{ .Facts.FormatCount }} formatos. @@ -74,14 +76,25 @@

Quais danos existem?

bytes zero não é texto. Abaixo de quatro bytes, alguns formatos saem com um dano de que nenhum leitor reclama, e por isso a configuração começa em quatro.

+

+ truncate corta o fim do arquivo, como um upload ou uma cópia interrompidos o + deixam, e o arquivo continua com o tamanho que você pediu: ele é gerado maior e só o seu início + chega ao disco. missing diz que parte do arquivo inteiro some, metade se você não + disser outra coisa. +

+

+ append-junk grava bytes aleatórios depois do fim do arquivo, 64 se você não disser + outra coisa, como chega um arquivo com algo grudado no fim. O mesmo comando acrescenta os mesmos + bytes. +

O que o manifesto diz

Um manifesto que diz o que deve acontecer

- Cada arquivo danificado recebe uma entrada dizendo que o seu sistema deve rejeitá-lo, com o dano - registrado ao lado: + Onde o dano quebra as regras do formato, a entrada diz que o seu sistema deve rejeitá-lo, com o + dano registrado ao lado: 

"expected": {
   "outcome": "reject",
@@ -97,15 +110,40 @@ 
Um manifesto que diz o que deve acontecer

   }
 ]

- Dois pedidos são recusados antes de qualquer coisa ser gravada, porque cada um deixaria no disco um - arquivo que o manifesto descreve errado: + Onde os leitores divergem, a entrada diz isso em vez de adivinhar. Um arquivo de texto cortado + continua sendo texto, e a maioria dos visualizadores ignora bytes depois do fim de um PNG, então + a entrada deixa a resposta para o seu sistema, oferece as duas e diz por quê: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> lista os formatos em que um dano deixa a resposta para o seu + sistema, e por quê. +

+

+ Quatro pedidos são recusados antes de qualquer coisa ser gravada, porque cada um deixaria no + disco um arquivo que o manifesto descreve errado:

  • um arquivo menor do que o dano precisa, que sairia intacto
  • - expected: accept ao lado de um dano, porque nada poderia cumprir isso. Escreva - sanitize se o seu sistema deve consertar o arquivo, ou unspecified - se essa é justamente a pergunta que você faz + expected: accept ao lado de um dano que quebra as regras do formato, porque nada + que as siga aceita o arquivo. Onde a resposta depende do seu sistema, accept é + permitido. Escreva sanitize se o seu sistema deve consertar o arquivo, ou + unspecified se essa é justamente a pergunta que você faz +
    • +
  • + truncate depois de append-junk, porque cortaria os bytes + acrescentados. Coloque truncate primeiro +
    • +
  • + truncate ou append-junk em um arquivo compactado que tira o tamanho + do que contém, porque não há um tamanho de onde partir
@@ -143,11 +181,19 @@

Transformando em um teste

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Um arquivo cujo resultado é unspecified fica de fora aqui, porque o que o seu + sistema deve fazer com ele é a sua própria política. A entrada dele diz por quê, e + policy_options lista as respostas possíveis. +

Uma boa recusa é uma recusa limpa. Uma mensagem que diz o que estava errado é a resposta que você quer. Um erro de servidor, uma trava ou um arquivo salvo pela metade é o defeito que este teste diff --git a/web/content/pt-BR/site.json b/web/content/pt-BR/site.json index 6ade62a6..cdc2c1dc 100644 --- a/web/content/pt-BR/site.json +++ b/web/content/pt-BR/site.json @@ -241,6 +241,8 @@ "unspecified": "Depende das regras do seu sistema. Você decide e depois confere se o que acontece é o que você queria." }, "damages": { + "append-junk": "Grava bytes aleatórios depois do fim do arquivo e mantém exatamente o tamanho que você pediu, como chega um arquivo com algo grudado no fim.", + "truncate": "Corta o fim do arquivo e mantém exatamente o tamanho que você pediu, como um upload ou uma cópia interrompidos o deixam.", "zero-head": "Sobrescreve os primeiros bytes do arquivo com zeros, sem mexer no comprimento. A maioria dos leitores olha ali primeiro, então quase tudo percebe este dano." }, "terms": { diff --git a/web/content/ro/damage.html b/web/content/ro/damage.html index 381ce3ea..a1abad86 100644 --- a/web/content/ro/damage.html +++ b/web/content/ro/damage.html @@ -53,7 +53,9 @@

Un fișier deteriorat are în continuare mărimea cerută

iar aceeași comandă scrie din nou aceiași octeți. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Setările se scriu după două puncte. Opțiunea poate fi repetată, iar deteriorările se aplică în ordinea în care le scrii. Funcționează cu fiecare dintre cele {{ .Facts.FormatCount }} formate. @@ -76,14 +78,25 @@

Ce deteriorări există?

octeți nuli nu este text. Sub patru octeți, unele formate ies cu o deteriorare de care nu se plânge niciun cititor, de aceea setarea începe de la patru.

+

+ truncate taie sfârșitul fișierului, așa cum îl lasă o încărcare sau o copiere + întreruptă, iar fișierul are totuși dimensiunea cerută: este generat mai mare și doar începutul + lui ajunge pe disc. missing spune ce parte din fișierul întreg lipsește, jumătate + dacă nu spui altceva. +

+

+ append-junk scrie octeți aleatori după sfârșitul fișierului, 64 dacă nu spui + altceva, așa cum sosește un fișier cu ceva lipit la capăt. Aceeași comandă adaugă aceiași + octeți. +

Ce spune manifestul

Un manifest care spune ce trebuie să se întâmple

- Fiecare fișier deteriorat primește o intrare care spune că sistemul tău trebuie să îl respingă, cu - deteriorarea notată alături: + Acolo unde deteriorarea încalcă regulile formatului, intrarea spune că sistemul tău trebuie să + îl respingă, cu deteriorarea notată alături: 

"expected": {
   "outcome": "reject",
@@ -99,15 +112,40 @@ 
Un manifest care spune ce trebuie să se întâmple

   }
 ]

- Două cereri sunt refuzate înainte să se scrie ceva, pentru că fiecare ar lăsa pe disc un fișier pe - care manifestul îl descrie greșit: + Acolo unde cititorii diferă, intrarea spune asta în loc să ghicească. Un fișier text tăiat + rămâne text, iar majoritatea vizualizatoarelor sar peste octeții de după sfârșitul unui PNG, așa + că intrarea lasă răspunsul sistemului tău, le oferă pe amândouă și spune de ce: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> enumeră formatele în care o deteriorare lasă răspunsul + sistemului tău, și de ce. +

+

+ Patru cereri sunt refuzate înainte să se scrie ceva, pentru că fiecare ar lăsa pe disc un fișier + pe care manifestul îl descrie greșit:

  • un fișier mai mic decât are nevoie deteriorarea, care ar ieși neschimbat
  • - expected: accept lângă o deteriorare, pentru că nimic nu ar putea îndeplini asta. Scrie - sanitize dacă sistemul tău trebuie să repare fișierul sau - unspecified dacă tocmai asta este întrebarea pe care o pui + expected: accept lângă o deteriorare care încalcă regulile formatului, pentru că + nimic din ce le respectă nu acceptă fișierul. Acolo unde răspunsul depinde de sistemul tău, + accept este permis. Scrie sanitize dacă sistemul tău trebuie să + repare fișierul sau unspecified dacă tocmai asta este întrebarea pe care o pui +
    • +
  • + truncate după append-junk, pentru că ar tăia octeții adăugați. Pune + truncate primul +
    • +
  • + truncate sau append-junk pe o arhivă care își ia dimensiunea din ce + conține, pentru că nu există o dimensiune de la care să se plece
@@ -145,11 +183,19 @@

Transformarea într-un test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un fișier al cărui rezultat este unspecified este lăsat deoparte aici, pentru că ce + trebuie să facă sistemul tău cu el ține de propria ta politică. Intrarea lui spune de ce, iar + policy_options enumeră răspunsurile posibile. +

O respingere bună este una curată. Un mesaj care spune ce a fost greșit este răspunsul pe care îl vrei. O eroare de server, o blocare sau un fișier salvat pe jumătate este defectul pe care acest diff --git a/web/content/ro/site.json b/web/content/ro/site.json index 512ec33b..8b6de87e 100644 --- a/web/content/ro/site.json +++ b/web/content/ro/site.json @@ -241,6 +241,8 @@ "unspecified": "Depinde de regulile sistemului tău. Tu decizi, apoi verifici că ce se întâmplă este ce ai vrut." }, "damages": { + "append-junk": "Scrie octeți aleatori după sfârșitul fișierului și păstrează exact dimensiunea cerută, așa cum sosește un fișier cu ceva lipit la capăt.", + "truncate": "Taie sfârșitul fișierului și păstrează exact dimensiunea cerută, așa cum îl lasă o încărcare sau o copiere întreruptă.", "zero-head": "Suprascrie cu zerouri primii octeți ai fișierului, fără să îi schimbe lungimea. Majoritatea cititorilor se uită întâi acolo, deci aproape orice observă această deteriorare." }, "terms": { diff --git a/web/content/ru/damage.html b/web/content/ru/damage.html index 8fc7fbbe..cc2d8257 100644 --- a/web/content/ru/damage.html +++ b/web/content/ru/damage.html @@ -53,7 +53,9 @@

Повреждённый файл остаётся нужного разме команда снова записывает те же байты. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Настройки пишутся после двоеточия. Параметр можно повторять, а повреждения применяются в том порядке, в каком вы их записали. Это работает с каждым из {{ .Facts.FormatCount }} форматов. @@ -77,14 +79,25 @@

Какие бывают повреждения?

форматов получается повреждение, на которое не жалуется ни одна программа чтения, поэтому настройка начинается с четырёх.

+

+ truncate отрезает конец файла, как оставляет его прерванная загрузка или + копирование, и файл всё равно имеет указанный вами размер: он создаётся больше, и на диск + попадает только его начало. missing говорит, какая часть всего файла пропадает, + половина, если не указать иное. +

+

+ append-junk дописывает случайные байты после конца файла, 64, если не указать иное, + как приходит файл, к концу которого что-то прилипло. Одна и та же команда дописывает одни и те + же байты. +

Что говорит манифест

Манифест, который говорит, что должно произойти

- Каждый повреждённый файл получает запись о том, что ваша система должна его отклонить, а рядом - записано повреждение: + Там, где повреждение нарушает правила формата, запись говорит, что ваша система должна его + отклонить, а рядом записано повреждение: 

"expected": {
   "outcome": "reject",
@@ -100,15 +113,40 @@ 
Манифест, который говорит, что должно про
   }
 ]

- Два запроса отклоняются до того, как что-либо записано, потому что каждый оставил бы на диске файл, - который манифест описывает неверно: + Там, где программы чтения расходятся, запись говорит об этом, а не гадает. Обрезанный текстовый + файл остаётся текстом, а большинство просмотрщиков пропускают байты после конца PNG, поэтому + запись оставляет ответ вашей системе, предлагает оба и объясняет почему: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> перечисляет форматы, в которых повреждение оставляет ответ + вашей системе, и причину. +

+

+ Четыре запроса отклоняются до того, как что-либо записано, потому что каждый оставил бы на диске + файл, который манифест описывает неверно:

  • файл меньше, чем нужно повреждению, который вышел бы нетронутым
  • - expected: accept рядом с повреждением, потому что ничто не могло бы этого выполнить. - Напишите sanitize, если ваша система должна починить файл, или - unspecified, если именно это вы и проверяете + expected: accept рядом с повреждением, которое нарушает правила формата, потому + что ничто, что им следует, такой файл не примет. Там, где ответ зависит от вашей системы, + accept разрешён. Напишите sanitize, если ваша система должна + починить файл, или unspecified, если именно это вы и проверяете +
    • +
  • + truncate после append-junk, потому что он отрезал бы дописанные + байты. Поставьте truncate первым +
    • +
  • + truncate или append-junk для архива, размер которого берётся из + содержимого, потому что нет размера, от которого можно отталкиваться
@@ -146,11 +184,19 @@

Превращаем это в тест

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Файл с результатом unspecified здесь пропускается, потому что то, что ваша система + должна с ним сделать, определяет ваша собственная политика. Его запись объясняет почему, а + policy_options перечисляет возможные ответы. +

Хороший отказ - это чистый отказ. Сообщение, говорящее, что было не так, - тот ответ, который вам нужен. Ошибка сервера, зависание или наполовину сохранённый файл - тот дефект, ради которого diff --git a/web/content/ru/site.json b/web/content/ru/site.json index 31250c92..38fa40a5 100644 --- a/web/content/ru/site.json +++ b/web/content/ru/site.json @@ -241,6 +241,8 @@ "unspecified": "Зависит от правил вашей системы. Вы решаете, а затем проверяете, что происходит именно то, что вы имели в виду." }, "damages": { + "append-junk": "Дописывает случайные байты после конца файла и сохраняет ровно тот размер, который вы указали, как приходит файл, к концу которого что-то прилипло.", + "truncate": "Отрезает конец файла и сохраняет ровно тот размер, который вы указали, как оставляет файл прерванная загрузка или копирование.", "zero-head": "Перезаписывает первые байты файла нулями, не меняя его длину. Большинство программ чтения смотрят сначала туда, поэтому это повреждение замечает почти всё." }, "terms": { diff --git a/web/content/th/damage.html b/web/content/th/damage.html index 7c2d680f..d15df661 100644 --- a/web/content/th/damage.html +++ b/web/content/th/damage.html @@ -50,7 +50,9 @@

ไฟล์ที่เสียหายยังมีขนาดตา และคำสั่งเดิมเขียนไบต์เดิมซ้ำอีกครั้ง 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

การตั้งค่าเขียนหลังเครื่องหมายทวิภาค ตัวเลือกนี้ใส่ซ้ำได้ และความเสียหายจะถูกนำมาใช้ตามลำดับที่คุณเขียน ใช้ได้กับทั้ง {{ .Facts.FormatCount }} รูปแบบ @@ -72,13 +74,19 @@

มีความเสียหายแบบไหนบ้าง

ต่ำกว่าสี่ไบต์ บางรูปแบบออกมาพร้อมความเสียหายที่ไม่มีตัวอ่านใดบ่น นั่นคือเหตุผลที่การตั้งค่าเริ่มที่สี่

+

+ truncate ตัดส่วนท้ายของไฟล์ทิ้ง เหมือนที่การอัปโหลดหรือการคัดลอกที่ถูกขัดจังหวะทิ้งไว้ แต่ไฟล์ยังมีขนาดตามที่คุณขอ เพราะไฟล์ถูกสร้างให้ใหญ่กว่า และมีเพียงส่วนต้นที่ไปถึงดิสก์ missing บอกว่าไฟล์ทั้งไฟล์หายไปเท่าใด คือครึ่งหนึ่งหากคุณไม่ได้ระบุเป็นอย่างอื่น +

+

+ append-junk เขียนไบต์สุ่มต่อจากจุดจบของไฟล์ 64 ไบต์หากคุณไม่ได้ระบุเป็นอย่างอื่น เหมือนไฟล์ที่มาถึงพร้อมของบางอย่างติดอยู่ที่ท้าย คำสั่งเดียวกันจะต่อท้ายด้วยไบต์ชุดเดียวกัน +

แมนิเฟสต์บอกอะไร

แมนิเฟสต์ที่บอกว่าควรเกิดอะไรขึ้น

- ไฟล์ที่เสียหายทุกไฟล์ได้รายการที่บอกว่าระบบของคุณควรปฏิเสธ โดยบันทึกความเสียหายไว้ข้างๆ: + ในกรณีที่ความเสียหายละเมิดกฎของรูปแบบ รายการจะบอกว่าระบบของคุณควรปฏิเสธ โดยบันทึกความเสียหายไว้ข้างๆ: 

"expected": {
   "outcome": "reject",
@@ -94,14 +102,31 @@ 
แมนิเฟสต์ที่บอกว่าควรเกิด
   }
 ]

- คำขอสองแบบถูกปฏิเสธก่อนที่จะมีอะไรถูกเขียน เพราะแต่ละแบบจะทิ้งไฟล์ไว้บนดิสก์ที่แมนิเฟสต์อธิบายผิด: + ในกรณีที่ตัวอ่านแต่ละตัวต่างกัน รายการจะบอกเช่นนั้นแทนการเดา ไฟล์ข้อความที่ถูกตัดก็ยังเป็นข้อความ และโปรแกรมดูภาพส่วนใหญ่ข้ามไบต์ที่อยู่หลังจุดจบของ PNG รายการจึงปล่อยคำตอบให้ระบบของคุณ เสนอทั้งสองคำตอบ และบอกเหตุผล: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> แสดงรายการรูปแบบที่ความเสียหายปล่อยคำตอบให้ระบบของคุณ พร้อมเหตุผล +

+

+ คำขอสี่แบบถูกปฏิเสธก่อนที่จะมีอะไรถูกเขียน เพราะแต่ละแบบจะทิ้งไฟล์ไว้บนดิสก์ที่แมนิเฟสต์อธิบายผิด:

  • ไฟล์ที่เล็กกว่าที่ความเสียหายต้องการ ซึ่งจะออกมาโดยไม่เปลี่ยนแปลง
  • - expected: accept คู่กับความเสียหาย เพราะไม่มีอะไรทำให้เป็นจริงได้ เขียน - sanitize ถ้าระบบของคุณควรซ่อมไฟล์ หรือ unspecified - ถ้านั่นคือคำถามที่คุณกำลังถาม + expected: accept คู่กับความเสียหายที่ละเมิดกฎของรูปแบบ เพราะไม่มีสิ่งใดที่ทำตามกฎเหล่านั้นจะยอมรับไฟล์ ในกรณีที่คำตอบขึ้นกับระบบของคุณ accept ใช้ได้ เขียน sanitize ถ้าระบบของคุณควรซ่อมไฟล์ หรือ unspecified ถ้านั่นคือคำถามที่คุณถามอยู่พอดี +
    • +
  • + truncate หลัง append-junk เพราะจะตัดไบต์ที่เพิ่มเข้าไปทิ้ง ให้วาง truncate ไว้ก่อน +
    • +
  • + truncate หรือ append-junk กับไฟล์เก็บถาวรที่เอาขนาดมาจากสิ่งที่อยู่ข้างใน เพราะไม่มีขนาดให้ตั้งต้น
@@ -138,11 +163,17 @@

เปลี่ยนให้เป็นการทดสอบ

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ ไฟล์ที่ผลลัพธ์เป็น unspecified จะถูกข้ามไปตรงนี้ เพราะสิ่งที่ระบบของคุณควรทำกับไฟล์นั้นเป็นนโยบายของคุณเอง รายการของไฟล์บอกเหตุผล และ policy_options แสดงคำตอบที่เป็นไปได้ +

การปฏิเสธที่ดีคือการปฏิเสธที่สะอาด ข้อความที่บอกว่าผิดตรงไหนคือคำตอบที่คุณต้องการ ส่วนข้อผิดพลาดของเซิร์ฟเวอร์ การค้าง diff --git a/web/content/th/site.json b/web/content/th/site.json index 44727f79..82fd796f 100644 --- a/web/content/th/site.json +++ b/web/content/th/site.json @@ -241,6 +241,8 @@ "unspecified": "ขึ้นอยู่กับกฎของระบบของคุณ คุณเป็นผู้ตัดสิน แล้วตรวจสอบว่าสิ่งที่เกิดขึ้นตรงกับที่ตั้งใจ" }, "damages": { + "append-junk": "เขียนไบต์สุ่มต่อจากจุดจบของไฟล์และคงขนาดที่คุณขอไว้พอดี เหมือนไฟล์ที่มาถึงพร้อมของบางอย่างติดอยู่ที่ท้าย", + "truncate": "ตัดส่วนท้ายของไฟล์ทิ้งและคงขนาดที่คุณขอไว้พอดี เหมือนที่การอัปโหลดหรือการคัดลอกที่ถูกขัดจังหวะทิ้งไว้", "zero-head": "เขียนทับไบต์แรกๆ ของไฟล์ด้วยศูนย์ โดยไม่แตะความยาว ตัวอ่านส่วนใหญ่มองตรงนั้นก่อน เกือบทุกอย่างจึงสังเกตเห็นความเสียหายนี้" }, "terms": { diff --git a/web/content/tr/damage.html b/web/content/tr/damage.html index df1d5aad..9d9ff9bf 100644 --- a/web/content/tr/damage.html +++ b/web/content/tr/damage.html @@ -52,7 +52,9 @@

Hasarlı bir dosya yine istediğiniz boyuttadır

aynı baytları yazar. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Ayarlar iki noktadan sonra yazılır. Seçenek tekrarlanabilir ve hasarlar yazdığınız sırayla uygulanır. {{ .Facts.FormatCount }} biçimin her biriyle çalışır. @@ -75,14 +77,25 @@

Hangi hasarlar var?

Dört baytın altında bazı biçimler hiçbir okuyucunun şikâyet etmediği bir hasarla çıkar, ayarın dörtten başlamasının nedeni budur.

+

+ truncate dosyanın sonunu keser, tıpkı yarıda kalan bir yüklemenin ya da + kopyalamanın bıraktığı gibi, ve dosya yine de istediğiniz boyuttadır: daha büyük üretilir ve + diske yalnızca başı ulaşır. missing, dosyanın tamamının ne kadarının kaybolduğunu + söyler, başka bir şey belirtmezseniz yarısı. +

+

+ append-junk dosyanın sonundan sonra rastgele baytlar yazar, başka bir şey + belirtmezseniz 64, tıpkı sonuna bir şey yapışmış bir dosyanın gelişi gibi. Aynı komut aynı + baytları ekler. +

Bildirim ne söyler

Ne olması gerektiğini söyleyen bir bildirim

- Hasarlı her dosya, sisteminizin onu reddetmesi gerektiğini söyleyen bir kayıt alır ve hasar yanına - yazılır: + Hasarın biçimin kurallarını çiğnediği yerde kayıt, sisteminizin dosyayı reddetmesi gerektiğini + söyler ve hasar yanına yazılır: 

"expected": {
   "outcome": "reject",
@@ -98,15 +111,40 @@ 
Ne olması gerektiğini söyleyen bir bildirim

   }
 ]

- İki istek, bir şey yazılmadan önce reddedilir, çünkü her biri diskte bildirimin yanlış tarif ettiği - bir dosya bırakırdı: + Okuyucuların ayrıştığı yerde kayıt, tahmin yürütmek yerine bunu söyler. Kesilmiş bir metin + dosyası yine metindir ve çoğu görüntüleyici bir PNG'nin sonundan sonraki baytları atlar, bu + yüzden kayıt yanıtı sisteminize bırakır, ikisini de sunar ve nedenini söyler: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id>, bir hasarın yanıtı sisteminize bıraktığı biçimleri nedeniyle + birlikte listeler. +

+

+ Dört istek, bir şey yazılmadan önce reddedilir, çünkü her biri diskte bildirimin yanlış tarif + ettiği bir dosya bırakırdı:

  • hasarın gerektirdiğinden küçük bir dosya, ki değişmeden çıkardı
  • - Bir hasarın yanında expected: accept, çünkü hiçbir şey bunu karşılayamaz. Sisteminiz - dosyayı onarmalıysa sanitize, tam da bunu soruyorsanız unspecified - yazın + Biçimin kurallarını çiğneyen bir hasarın yanında expected: accept, çünkü bu + kurallara uyan hiçbir şey dosyayı kabul etmez. Yanıtın sisteminize bağlı olduğu yerde + accept serbesttir. Sisteminiz dosyayı onarmalıysa sanitize, tam da + bunu soruyorsanız unspecified yazın +
    • +
  • + append-junk sonrasında truncate, çünkü eklenen baytları keserdi. + truncate öğesini başa koyun +
    • +
  • + İçeriğinden boyut alan bir arşivde truncate ya da append-junk, çünkü + yola çıkılacak bir boyut yoktur
@@ -144,11 +182,19 @@

Bunu bir teste dönüştürmek

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Sonucu unspecified olan bir dosya burada atlanır, çünkü sisteminizin onunla ne + yapması gerektiği kendi politikanıza bağlıdır. Kaydı nedenini söyler, + policy_options ise olası yanıtları listeler. +

İyi bir ret temiz bir rettir. Neyin yanlış olduğunu söyleyen bir ileti istediğiniz yanıttır. Sunucu hatası, takılma ya da yarım kaydedilmiş dosya, bu testin bulmak için var olduğu kusurdur. diff --git a/web/content/tr/site.json b/web/content/tr/site.json index 35532514..5bae3269 100644 --- a/web/content/tr/site.json +++ b/web/content/tr/site.json @@ -241,6 +241,8 @@ "unspecified": "Sisteminizin kurallarına bağlı. Siz karar verirsiniz, sonra olanın amaçladığınız şey olduğunu doğrularsınız." }, "damages": { + "append-junk": "Dosyanın sonundan sonra rastgele baytlar yazar ve istediğiniz boyutu tam olarak korur, tıpkı sonuna bir şey yapışmış bir dosyanın gelişi gibi.", + "truncate": "Dosyanın sonunu keser ve istediğiniz boyutu tam olarak korur, tıpkı yarıda kalan bir yüklemenin ya da kopyalamanın bıraktığı gibi.", "zero-head": "Dosyanın ilk baytlarını uzunluğuna dokunmadan sıfırlarla ezer. Okuyucuların çoğu önce oraya bakar, bu yüzden hemen her şey bu hasarı fark eder." }, "terms": { diff --git a/web/content/uk/damage.html b/web/content/uk/damage.html index 907a4b0c..800fdf04 100644 --- a/web/content/uk/damage.html +++ b/web/content/uk/damage.html @@ -53,7 +53,9 @@

Пошкоджений файл лишається потрібного ро сама команда знову записує ті самі байти. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Налаштування пишуться після двокрапки. Параметр можна повторювати, а пошкодження застосовуються в тому порядку, в якому ви їх записали. Це працює з кожним із {{ .Facts.FormatCount }} форматів. @@ -76,14 +78,24 @@

Які бувають пошкодження?

байтів не є текстом. Менше ніж чотири байти - і в деяких форматів виходить пошкодження, на яке не скаржиться жодна програма читання, тому налаштування починається з чотирьох.

+

+ truncate відрізає кінець файлу, як залишає його перерване завантаження чи + копіювання, і файл однаково має вказаний вами розмір: він створюється більшим, і на диск + потрапляє лише його початок. missing каже, яка частина всього файлу зникає, + половина, якщо не вказати інше. +

+

+ append-junk дописує випадкові байти після кінця файлу, 64, якщо не вказати інше, як + приходить файл, до кінця якого щось прилипло. Та сама команда дописує ті самі байти. +

Що каже маніфест

Маніфест, який каже, що має статися

- Кожен пошкоджений файл отримує запис про те, що ваша система має його відхилити, а поруч записано - пошкодження: + Там, де пошкодження порушує правила формату, запис каже, що ваша система має його відхилити, а + поруч записано пошкодження: 

"expected": {
   "outcome": "reject",
@@ -99,15 +111,40 @@ 
Маніфест, який каже, що має статися

   }
 ]

- Два запити відхиляються, перш ніж щось буде записано, бо кожен залишив би на диску файл, який + Там, де програми читання розходяться, запис каже про це, а не вгадує. Обрізаний текстовий файл + лишається текстом, а більшість переглядачів пропускають байти після кінця PNG, тому запис + залишає відповідь вашій системі, пропонує обидві й пояснює чому: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> перелічує формати, у яких пошкодження залишає відповідь вашій + системі, і причину. +

+

+ Чотири запити відхиляються, перш ніж щось буде записано, бо кожен залишив би на диску файл, який маніфест описує хибно:

  • файл менший, ніж потрібно пошкодженню, який вийшов би недоторканим
  • - expected: accept поруч із пошкодженням, бо ніщо не могло б цього виконати. Напишіть - sanitize, якщо ваша система має полагодити файл, або unspecified, - якщо саме це ви й перевіряєте + expected: accept поруч із пошкодженням, яке порушує правила формату, бо ніщо, що + їх дотримується, такого файлу не прийме. Там, де відповідь залежить від вашої системи, + accept дозволено. Напишіть sanitize, якщо ваша система має + полагодити файл, або unspecified, якщо саме це ви й перевіряєте +
    • +
  • + truncate після append-junk, бо він відрізав би дописані байти. + Поставте truncate першим +
    • +
  • + truncate або append-junk для архіву, розмір якого береться зі + вмісту, бо немає розміру, від якого можна відштовхнутися
@@ -145,11 +182,19 @@

Перетворюємо це на тест

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Файл із результатом unspecified тут пропускається, бо те, що ваша система має з ним + зробити, визначає ваша власна політика. Його запис пояснює чому, а policy_options + перелічує можливі відповіді. +

Гарна відмова - це чиста відмова. Повідомлення, що каже, що було не так, - та відповідь, яка вам потрібна. Помилка сервера, зависання або наполовину збережений файл - той дефект, заради якого diff --git a/web/content/uk/site.json b/web/content/uk/site.json index 2badcbad..1545de34 100644 --- a/web/content/uk/site.json +++ b/web/content/uk/site.json @@ -241,6 +241,8 @@ "unspecified": "Залежить від правил вашої системи. Ви вирішуєте, а потім перевіряєте, що відбувається саме те, що ви мали на увазі." }, "damages": { + "append-junk": "Дописує випадкові байти після кінця файлу й зберігає саме той розмір, який ви вказали, як приходить файл, до кінця якого щось прилипло.", + "truncate": "Відрізає кінець файлу й зберігає саме той розмір, який ви вказали, як залишає файл перерване завантаження чи копіювання.", "zero-head": "Перезаписує перші байти файлу нулями, не змінюючи його довжину. Більшість програм читання дивляться спочатку туди, тому це пошкодження помічає майже все." }, "terms": { diff --git a/web/content/vi/damage.html b/web/content/vi/damage.html index 562d1961..13298678 100644 --- a/web/content/vi/damage.html +++ b/web/content/vi/damage.html @@ -50,7 +50,9 @@

Tệp bị hỏng vẫn có kích thước bạn đã yêu cầu

cầu, và cùng một lệnh ghi lại đúng các byte đó. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Cài đặt viết sau dấu hai chấm. Tùy chọn có thể lặp lại, và các hỏng hóc được áp dụng theo thứ tự bạn viết. Nó dùng được với từng định dạng trong {{ .Facts.FormatCount }} định dạng. @@ -73,14 +75,24 @@

Có những kiểu hỏng nào?

bốn byte, một số định dạng ra với hỏng hóc mà không trình đọc nào phàn nàn, đó là lý do cài đặt bắt đầu từ bốn.

+

+ truncate cắt bỏ phần cuối tệp, giống như một lần tải lên hoặc sao chép bị ngắt giữa + chừng để lại, và tệp vẫn có kích thước bạn yêu cầu: tệp được tạo lớn hơn và chỉ phần đầu của nó + tới đĩa. missing cho biết bao nhiêu phần của toàn bộ tệp bị mất, một nửa nếu bạn + không nói khác. +

+

+ append-junk ghi các byte ngẫu nhiên sau phần cuối tệp, 64 nếu bạn không nói khác, + giống như một tệp đến nơi với thứ gì đó dính ở cuối. Cùng một lệnh sẽ thêm cùng những byte đó. +

Manifest nói gì

Một manifest nói điều phải xảy ra

- Mỗi tệp bị hỏng nhận một mục nói rằng hệ thống của bạn phải từ chối nó, với hỏng hóc được ghi bên - cạnh: + Ở chỗ hỏng hóc vi phạm quy tắc của định dạng, mục nói rằng hệ thống của bạn phải từ chối tệp, + với hỏng hóc được ghi bên cạnh: 

"expected": {
   "outcome": "reject",
@@ -96,15 +108,40 @@ 
Một manifest nói điều phải xảy ra

   }
 ]

- Hai yêu cầu bị từ chối trước khi có gì được ghi, vì mỗi yêu cầu sẽ để lại trên đĩa một tệp mà + Ở chỗ các trình đọc khác nhau, mục nói rõ điều đó thay vì đoán. Một tệp văn bản bị cắt vẫn là + văn bản, và hầu hết trình xem ảnh bỏ qua các byte sau phần cuối của PNG, nên mục để câu trả lời + cho hệ thống của bạn, đưa ra cả hai và nói lý do: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> liệt kê các định dạng mà ở đó một hỏng hóc để câu trả lời cho + hệ thống của bạn, kèm lý do. +

+

+ Bốn yêu cầu bị từ chối trước khi có gì được ghi, vì mỗi yêu cầu sẽ để lại trên đĩa một tệp mà manifest mô tả sai:

  • một tệp nhỏ hơn mức hỏng hóc cần, sẽ ra mà không bị đổi
  • - expected: accept cạnh một hỏng hóc, vì không gì có thể đáp ứng được. Viết - sanitize nếu hệ thống của bạn phải sửa tệp, hoặc unspecified nếu đó - chính là câu hỏi bạn đang đặt ra + expected: accept cạnh một hỏng hóc vi phạm quy tắc của định dạng, vì không thứ gì + tuân theo các quy tắc đó chấp nhận tệp. Ở chỗ câu trả lời tùy vào hệ thống của bạn, + accept được phép. Viết sanitize nếu hệ thống của bạn phải sửa tệp, + hoặc unspecified nếu đó chính là câu hỏi bạn đang đặt ra +
    • +
  • + truncate sau append-junk, vì nó sẽ cắt mất các byte đã thêm. Hãy đặt + truncate trước +
    • +
  • + truncate hoặc append-junk trên một tệp nén lấy kích thước từ những + gì nó chứa, vì không có kích thước nào để bắt đầu
@@ -142,11 +179,19 @@

Biến nó thành một bài kiểm thử

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Một tệp có kết quả unspecified được bỏ qua ở đây, vì việc hệ thống của bạn phải làm + gì với nó là chính sách của riêng bạn. Mục của nó nói lý do, và policy_options liệt + kê các câu trả lời có thể. +

Một lần từ chối tốt là một lần từ chối gọn. Một thông báo nói điều gì sai là câu trả lời bạn muốn. Lỗi máy chủ, treo máy hoặc một tệp lưu dở là khiếm khuyết mà bài kiểm thử này sinh ra để tìm. diff --git a/web/content/vi/site.json b/web/content/vi/site.json index 02c6f74e..8f29596a 100644 --- a/web/content/vi/site.json +++ b/web/content/vi/site.json @@ -241,6 +241,8 @@ "unspecified": "Tùy vào quy tắc của hệ thống bạn. Bạn quyết định, rồi kiểm tra điều xảy ra có đúng ý bạn không." }, "damages": { + "append-junk": "Ghi các byte ngẫu nhiên sau phần cuối tệp và giữ đúng kích thước bạn yêu cầu, giống như một tệp đến nơi với thứ gì đó dính ở cuối.", + "truncate": "Cắt bỏ phần cuối tệp và giữ đúng kích thước bạn yêu cầu, giống như một lần tải lên hoặc sao chép bị ngắt giữa chừng để lại.", "zero-head": "Ghi đè các byte đầu của tệp bằng số không, giữ nguyên độ dài. Hầu hết trình đọc nhìn vào đó trước, nên gần như mọi thứ đều nhận ra hỏng hóc này." }, "terms": { diff --git a/web/content/zh-Hans/damage.html b/web/content/zh-Hans/damage.html index 5d31022c..ab9285b7 100644 --- a/web/content/zh-Hans/damage.html +++ b/web/content/zh-Hans/damage.html @@ -41,7 +41,9 @@

损坏的文件仍然是你要求的大小

文件先正常生成，再在写入磁盘的途中被弄坏。它保持你要求的大小，同一条命令再次写出的字节也完全相同。 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

设置写在冒号之后。该选项可以重复，损坏按你写下的顺序依次应用。它适用于全部 {{ .Facts.FormatCount }} 种格式。

@@ -59,13 +61,19 @@

有哪些损坏方式？

zero-head 把文件开头写成零。大多数读取器最先看的就是那里，也就是说明文件是什么的签名和文件头，所以几乎任何读取器都会发现。纯文本和日志没有签名，同样会被拒绝，因为一串零字节不是文本。低于四个字节时，有些格式产生的损坏没有任何读取器会抱怨，这就是该设置从四开始的原因。

+

+ truncate 截掉文件的末尾，就像中断的上传或复制留下的那样，而文件仍然是你要求的大小：它先按更大的尺寸生成，只有开头部分写入磁盘。missing 表示整个文件丢失了多少，不另行指定时为一半。 +

+

+ append-junk 在文件末尾之后写入随机字节，不另行指定时为 64 个，就像末尾粘着东西送来的文件那样。同一条命令追加的字节也相同。 +

清单怎么说

一份说明应发生什么的清单

- 每个损坏的文件都会得到一条记录，说明你的系统应该拒绝它，损坏方式记在旁边： + 当损坏违反该格式的规则时，记录会说明你的系统应该拒绝它，损坏方式记在旁边： 

"expected": {
   "outcome": "reject",
@@ -81,13 +89,31 @@ 
一份说明应发生什么的清单

   }
 ]

- 有两种请求会在写入任何内容之前被拒绝，因为各自都会在磁盘上留下一个清单描述有误的文件： + 当各读取器的处理不一致时，记录会如实说明，而不是去猜测。被截短的文本文件仍然是文本，而大多数看图软件会跳过 PNG 结尾之后的字节，所以记录把答案留给你的系统，给出两种答案，并说明原因： +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> 会列出某种损坏把答案留给你的系统的格式及其原因。 +

+

+ 有四种请求会在写入任何内容之前被拒绝，因为各自都会在磁盘上留下一个清单描述有误的文件：

  • 比损坏所需更小的文件，它会原样输出
  • - 在损坏旁边写 expected: accept，因为没有什么能满足它。如果你的系统本应修复文件，请写 - sanitize，如果你要问的恰恰就是这一点，请写 unspecified + 在违反格式规则的损坏旁边写 expected: accept，因为遵守这些规则的读取器都不会接受这个文件。当答案取决于你的系统时，可以写 accept。如果你的系统本应修复文件，请写 sanitize，如果你要问的恰恰就是这一点，请写 unspecified +
    • +
  • + 在 append-junk 之后使用 truncate，因为它会把追加的字节截掉。请把 truncate 放在前面 +
    • +
  • + 对按所含内容决定大小的压缩包使用 truncate 或 append-junk，因为没有可以作为出发点的大小
@@ -123,11 +149,17 @@

把它变成测试

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 结果为 unspecified 的文件在这里被跳过，因为你的系统该如何处理它由你自己的策略决定。它的记录说明了原因，policy_options 列出了可能的答案。 +

好的拒绝是干净的拒绝。一条说明错在哪里的消息，就是你想要的答案。服务器错误、卡死或只保存了一半的文件，正是这个测试要找出来的缺陷。

diff --git a/web/content/zh-Hans/site.json b/web/content/zh-Hans/site.json index cf5fc6b6..4c173277 100644 --- a/web/content/zh-Hans/site.json +++ b/web/content/zh-Hans/site.json @@ -241,6 +241,8 @@ "unspecified": "取决于你的系统规则。由你决定，然后检查实际发生的是否符合你的本意。" }, "damages": { + "append-junk": "在文件末尾之后写入随机字节，并完全保持你要求的大小，就像末尾粘着东西送来的文件那样。", + "truncate": "截掉文件的末尾，并完全保持你要求的大小，就像中断的上传或复制留下的那样。", "zero-head": "用零覆盖文件开头的若干字节，长度保持不变。大多数读取器最先看的就是那里，所以几乎任何东西都会发现这种损坏。" }, "terms": { diff --git a/web/content/zh-Hant/damage.html b/web/content/zh-Hant/damage.html index 5a71331f..aa4a1f11 100644 --- a/web/content/zh-Hant/damage.html +++ b/web/content/zh-Hant/damage.html @@ -41,7 +41,9 @@

損毀的檔案仍然是你要求的大小

檔案先正常產生，再在寫入磁碟的途中被弄壞。它保持你要求的大小，同一個指令再次寫出的位元組也完全相同。 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

設定寫在冒號之後。該選項可以重複，損毀依你寫下的順序逐一套用。它適用於全部 {{ .Facts.FormatCount }} 種格式。

@@ -59,13 +61,19 @@

有哪些損毀方式？

zero-head 把檔案開頭寫成零。大多數讀取器最先看的就是那裡，也就是說明檔案是什麼的簽章和檔頭，所以幾乎任何讀取器都會發現。純文字和日誌沒有簽章，同樣會被拒絕，因為一串零位元組不是文字。低於四個位元組時，有些格式產生的損毀沒有任何讀取器會抱怨，這就是該設定從四開始的原因。

+

+ truncate 截掉檔案的結尾，就像中斷的上傳或複製留下的那樣，而檔案仍然是你要求的大小：它先按更大的尺寸產生，只有開頭部分寫入磁碟。missing 表示整個檔案遺失了多少，不另行指定時為一半。 +

+

+ append-junk 在檔案結尾之後寫入隨機位元組，不另行指定時為 64 個，就像結尾黏著東西送來的檔案那樣。同一道指令附加的位元組也相同。 +

清單怎麼說

一份說明應發生什麼的清單

- 每個損毀的檔案都會得到一筆記錄，說明你的系統應該拒絕它，損毀方式記在旁邊： + 當損毀違反該格式的規則時，記錄會說明你的系統應該拒絕它，損毀方式記在旁邊： 

"expected": {
   "outcome": "reject",
@@ -81,13 +89,31 @@ 
一份說明應發生什麼的清單

   }
 ]

- 有兩種請求會在寫入任何內容之前被拒絕，因為各自都會在磁碟上留下一個清單描述有誤的檔案： + 當各讀取器的處理不一致時，記錄會如實說明，而不是去猜測。被截短的文字檔仍然是文字，而大多數看圖軟體會跳過 PNG 結尾之後的位元組，所以記錄把答案留給你的系統，給出兩種答案，並說明原因： +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> 會列出某種損毀把答案留給你的系統的格式及其原因。 +

+

+ 有四種請求會在寫入任何內容之前被拒絕，因為各自都會在磁碟上留下一個清單描述有誤的檔案：

  • 比損毀所需更小的檔案，它會原樣輸出
  • - 在損毀旁邊寫 expected: accept，因為沒有什麼能滿足它。如果你的系統本應修復檔案，請寫 - sanitize，如果你要問的恰恰就是這一點，請寫 unspecified + 在違反格式規則的損毀旁邊寫 expected: accept，因為遵守這些規則的讀取器都不會接受這個檔案。當答案取決於你的系統時，可以寫 accept。如果你的系統本應修復檔案，請寫 sanitize，如果你要問的恰恰就是這一點，請寫 unspecified +
    • +
  • + 在 append-junk 之後使用 truncate，因為它會把附加的位元組截掉。請把 truncate 放在前面 +
    • +
  • + 對依所含內容決定大小的壓縮檔使用 truncate 或 append-junk，因為沒有可以作為出發點的大小
@@ -123,11 +149,17 @@

把它變成測試

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 結果為 unspecified 的檔案在這裡被略過，因為你的系統該如何處理它由你自己的策略決定。它的記錄說明了原因，policy_options 列出了可能的答案。 +

好的拒絕是乾淨的拒絕。一則說明錯在哪裡的訊息，就是你想要的答案。伺服器錯誤、卡死或只儲存了一半的檔案，正是這個測試要找出來的缺陷。

diff --git a/web/content/zh-Hant/site.json b/web/content/zh-Hant/site.json index a43653e0..d091991c 100644 --- a/web/content/zh-Hant/site.json +++ b/web/content/zh-Hant/site.json @@ -241,6 +241,8 @@ "unspecified": "取決於你的系統規則。由你決定，然後檢查實際發生的是否符合你的本意。" }, "damages": { + "append-junk": "在檔案結尾之後寫入隨機位元組，並完全保持你要求的大小，就像結尾黏著東西送來的檔案那樣。", + "truncate": "截掉檔案的結尾，並完全保持你要求的大小，就像中斷的上傳或複製留下的那樣。", "zero-head": "用零覆蓋檔案開頭的若干位元組，長度保持不變。大多數讀取器最先看的就是那裡，所以幾乎任何東西都會發現這種損毀。" }, "terms": { diff --git a/web/public/ar/corrupt-test-files/index.html b/web/public/ar/corrupt-test-files/index.html index eee82cb0..80168e88 100644 --- a/web/public/ar/corrupt-test-files/index.html +++ b/web/public/ar/corrupt-test-files/index.html @@ -195,7 +195,9 @@

الملف التالف يبقى بالحجم الذي طلبته

البايتات نفسها مرة أخرى. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

تُكتب الإعدادات بعد النقطتين. يمكن تكرار الخيار، وتُطبَّق أنواع التلف بالترتيب الذي تكتبه. ويعمل مع كل واحدة من 31 صيغة. @@ -220,6 +222,18 @@

ما أنواع التلف المتاحة؟

+ + append-junk + يكتب بايتات عشوائية بعد نهاية الملف ويحافظ على الحجم الذي طلبته بالضبط، كما يصل ملف التصق شيء بنهايته. + 64 + bytes + + + truncate + يقطع نهاية الملف ويحافظ على الحجم الذي طلبته بالضبط، كما يتركه رفع أو نسخ انقطع في منتصفه. + 4 + missing + zero-head يكتب أصفارًا فوق أول بايتات الملف دون المساس بطوله. معظم القارئات تنظر إلى هناك أولًا، فيلاحظ هذا التلف كل شيء تقريبًا. @@ -230,18 +244,27 @@

ما أنواع التلف المتاحة؟

- يكتب zero-head أصفارًا فوق بداية الملف. معظم القارئات تنظر إلى هناك أولًا، إلى التوقيع + يكتب zero-head يكتب أصفارًا فوق بداية الملف. معظم القارئات تنظر إلى هناك أولًا، إلى التوقيع والترويسة اللذين يقولان ما هو الملف، فتلاحظ ذلك أي قارئة تقريبًا. والنص العادي والسجلات لا توقيع لها وتُرفض أيضًا، لأن سلسلة من البايتات الصفرية ليست نصًّا. وتحت أربعة بايتات تخرج بعض الصيغ بتلف لا تشتكي منه أي قارئة، ولهذا يبدأ الإعداد من أربعة.

+

+ truncate يقطع نهاية الملف، كما يتركه رفع أو نسخ انقطع في منتصفه، ويبقى الملف بالحجم + الذي طلبته: يُولَّد أكبر ولا يصل إلى القرص إلا أوله. ويقول missing أي جزء من الملف + كله يضيع، النصف ما لم تحدد غير ذلك. +

+

+ append-junk يكتب بايتات عشوائية بعد نهاية الملف، 64 ما لم تحدد غير ذلك، كما يصل ملف + التصق شيء بنهايته. والأمر نفسه يضيف البايتات نفسها. +

ما يقوله البيان

بيان يقول ما ينبغي أن يحدث

- كل ملف تالف يحصل على مدخل يقول إن نظامك ينبغي أن يرفضه، ويُسجَّل التلف بجانبه: + حيث يخرق التلف قواعد الصيغة، يقول المدخل إن نظامك ينبغي أن يرفضه، ويُسجَّل التلف بجانبه: 

"expected": {
   "outcome": "reject",
@@ -257,13 +280,39 @@ 
بيان يقول ما ينبغي أن يحدث

   }
 ]

- يُرفض طلبان قبل أن يُكتب أي شيء، لأن كلًّا منهما سيترك على القرص ملفًا يصفه البيان وصفًا خاطئًا: + وحيث تختلف القارئات، يقول المدخل ذلك بدل التخمين. فالملف النصي المقطوع يبقى نصًّا، ومعظم عارضات + الصور تتجاهل البايتات بعد نهاية ملف PNG، لذلك يترك المدخل الجواب لنظامك ويعرض الجوابين ويقول + السبب: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ يسرد tfg damage <id> الصيغ التي يترك فيها التلف الجواب لنظامك، مع السبب. +

+

+ تُرفض أربعة طلبات قبل أن يُكتب أي شيء، لأن كلًّا منها سيترك على القرص ملفًا يصفه البيان وصفًا + خاطئًا:

  • ملف أصغر مما يحتاجه التلف، وكان سيخرج دون تغيير
  • - expected: accept بجانب تلف، لأن لا شيء يمكنه تحقيق ذلك. اكتب sanitize إن - كان نظامك مقصودًا به إصلاح الملف، أو unspecified إن كان هذا هو السؤال الذي تطرحه + expected: accept بجانب تلف يخرق قواعد الصيغة، لأن لا شيء يتبع تلك القواعد يقبل + الملف. وحيث يتوقف الجواب على نظامك، يُسمح بـ accept. اكتب sanitize + إن كان نظامك مقصودًا به إصلاح الملف، أو unspecified إن كان هذا هو السؤال الذي + تطرحه +
    • +
  • + truncate بعد append-junk، لأنه سيقطع البايتات التي أُضيفت. ضع + truncate أولًا +
    • +
  • + truncate أو append-junk على أرشيف يأخذ حجمه مما يحويه، لأنه لا يوجد + حجم يُنطلق منه
@@ -300,11 +349,18 @@

تحويله إلى اختبار

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ يُستبعد هنا الملف الذي نتيجته unspecified، لأن ما ينبغي لنظامك أن يفعله به هو + سياستك أنت. ومدخله يقول السبب، وpolicy_options يسرد الأجوبة الممكنة. +

الرفض الجيد هو الرفض النظيف. رسالة تقول ما الخطأ هي الجواب الذي تريده. أما خطأ الخادم أو التعليق أو ملف حُفظ نصفه فهو العيب الذي وُجد هذا الاختبار لاكتشافه. diff --git a/web/public/corrupt-test-files/index.html b/web/public/corrupt-test-files/index.html index 685c618c..3cf40aec 100644 --- a/web/public/corrupt-test-files/index.html +++ b/web/public/corrupt-test-files/index.html @@ -198,7 +198,9 @@

A damaged file is still the size you asked for

you asked for, and the same command writes the same bytes again. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Settings go after a colon. The flag can be repeated, and the damages are applied in the order you write them. It works with every one of the 31 formats. @@ -223,6 +225,18 @@

Which damages are there?

+ + append-junk + Writes random bytes after the end of the file and keeps exactly the size you asked for, the way a file with something stuck to its end arrives. + 64 + bytes + + + truncate + Cuts the end off the file and keeps exactly the size you asked for, the way an interrupted upload or copy leaves it. + 4 + missing + zero-head Overwrites the first bytes of the file with zeros, leaving its length alone. Most readers look there first, so this is the damage almost anything notices. @@ -239,14 +253,25 @@

Which damages are there?

Below four bytes some formats come out with damage that no reader complains about, which is why the setting starts at four.

+

+ truncate cuts the end off the file, the way an interrupted upload or copy leaves + it, and the file is still the size you asked for: it is generated larger, and only its start + reaches the disk. missing says how much of the whole file is gone, half unless you + say otherwise. +

+

+ append-junk writes random bytes after the end of the file, 64 unless you say + otherwise, the way a file arrives with something stuck to its end. The same command appends the + same bytes. +

What the manifest says

A manifest that says what should happen

- Every damaged file gets an entry saying your system should reject it, with the damage recorded - beside it: + Where the damage breaks the rules of the format, the entry says your system should reject it, + with the damage recorded beside it: 

"expected": {
   "outcome": "reject",
@@ -262,15 +287,40 @@ 
A manifest that says what should happen

   }
 ]

- Two requests are refused before anything is written, because each would leave a file on disk that - the manifest describes wrongly: + Where readers differ, the entry says so instead of guessing. A text file cut short is still + text, and most viewers skip bytes after the end of a PNG, so the entry leaves the answer to your + system, offers both, and says why: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> lists the formats where a damage leaves the answer to your + system, and why. +

+

+ Four requests are refused before anything is written, because each would leave a file on disk + that the manifest describes wrongly:

  • a file smaller than the damage needs, which would come out unchanged
  • - expected: accept beside a damage, because nothing could meet it. Write - sanitize if your system is meant to repair the file, or unspecified if - that is the question you are asking + expected: accept beside a damage that breaks the rules of the format, because + nothing that follows them accepts the file. Where the answer depends on your system, + accept is allowed. Write sanitize if your system is meant to repair + the file, or unspecified if that is the question you are asking +
    • +
  • + truncate after append-junk, which would cut away the bytes that were + added. Put truncate first +
    • +
  • + truncate or append-junk on an archive that takes its size from what + it holds, because there is no size to work from
@@ -308,11 +358,19 @@

Turning it into a test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ A file whose outcome is unspecified is left out here, because what your system + should do with it is your own policy. Its entry says why, and policy_options lists + the answers there are. +

A good refusal is a clean one. A message that says what was wrong is the answer you want. A server error, a hang or a half stored file is the defect this test exists to find. diff --git a/web/public/cs/poskozene-testovaci-soubory/index.html b/web/public/cs/poskozene-testovaci-soubory/index.html index c6622c33..edeb3d6c 100644 --- a/web/public/cs/poskozene-testovaci-soubory/index.html +++ b/web/public/cs/poskozene-testovaci-soubory/index.html @@ -197,7 +197,9 @@

Poškozený soubor má stále velikost, kterou jste chtěli

chtěli, a stejný příkaz zapíše znovu stejné bajty. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Nastavení se píše za dvojtečku. Přepínač lze opakovat a poškození se použijí v pořadí, v jakém je zapíšete. Funguje s každým z 31 formátů. @@ -222,6 +224,18 @@

Jaká poškození existují?

+ + append-junk + Zapíše náhodné bajty za konec souboru a zachová přesně velikost, o kterou jste požádali, tak jak přijde soubor, na jehož konci něco visí. + 64 + bytes + + + truncate + Odřízne konec souboru a zachová přesně velikost, o kterou jste požádali, tak jak ho nechá přerušené nahrávání nebo kopírování. + 4 + missing + zero-head Přepíše prvních několik bajtů souboru nulami a jeho délku nechá být. Většina čteček se dívá nejdřív tam, takže si tohoto poškození všimne téměř cokoli. @@ -238,13 +252,24 @@

Jaká poškození existují?

bajty některé formáty vyjdou s poškozením, na které si žádná čtečka nestěžuje, proto nastavení začíná na čtyřech.

+

+ truncate odřízne konec souboru, tak jak ho nechá přerušené nahrávání nebo + kopírování, a soubor má přesto velikost, o kterou jste požádali: vytvoří se větší a na disk + dorazí jen jeho začátek. missing říká, jaká část celého souboru chybí, polovina, + pokud neuvedete jinak. +

+

+ append-junk zapíše náhodné bajty za konec souboru, 64, pokud neuvedete jinak, tak + jak přijde soubor, na jehož konci něco visí. Stejný příkaz připojí stejné bajty. +

Co říká manifest

Manifest, který říká, co se má stát

- Každý poškozený soubor dostane záznam, že ho má váš systém odmítnout, s poškozením zapsaným vedle: + Tam, kde poškození porušuje pravidla formátu, záznam říká, že ho má váš systém odmítnout, s + poškozením zapsaným vedle: 

"expected": {
   "outcome": "reject",
@@ -260,15 +285,40 @@ 
Manifest, který říká, co se má stát

   }
 ]

- Dva požadavky se odmítnou dřív, než se cokoli zapíše, protože každý by na disku nechal soubor, který - manifest popisuje špatně: + Tam, kde se čtečky liší, to záznam řekne, místo aby hádal. Zkrácený textový soubor je pořád text + a většina prohlížečů přeskočí bajty za koncem PNG, takže záznam nechá odpověď na vašem systému, + nabídne obě a řekne proč: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> vypíše formáty, ve kterých poškození nechává odpověď na vašem + systému, a důvod. +

+

+ Čtyři požadavky se odmítnou dřív, než se cokoli zapíše, protože každý by na disku nechal soubor, + který manifest popisuje špatně:

  • soubor menší, než poškození potřebuje, který by vyšel nezměněný
  • - expected: accept vedle poškození, protože by to nic nemohlo splnit. Napište - sanitize, pokud má váš systém soubor opravit, nebo unspecified, - pokud je právě to vaše otázka + expected: accept vedle poškození, které porušuje pravidla formátu, protože nic, + co je dodržuje, takový soubor nepřijme. Tam, kde odpověď závisí na vašem systému, je + accept povoleno. Napište sanitize, pokud má váš systém soubor + opravit, nebo unspecified, pokud je právě to vaše otázka +
    • +
  • + truncate po append-junk, protože by odřízl připojené bajty. Dejte + truncate na začátek +
    • +
  • + truncate nebo append-junk na archiv, který bere svou velikost z + toho, co obsahuje, protože není velikost, ze které by se dalo vyjít
@@ -306,11 +356,19 @@

Z toho udělat test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Soubor s výsledkem unspecified se tu vynechá, protože co s ním má váš systém + udělat, určuje vaše vlastní pravidlo. Jeho záznam říká proč a policy_options + vypisuje možné odpovědi. +

Dobré odmítnutí je čisté. Zpráva, která říká, co bylo špatně, je odpověď, kterou chcete. Chyba serveru, zaseknutí nebo napůl uložený soubor je vada, kterou má tento test najít. diff --git a/web/public/de/beschaedigte-testdateien/index.html b/web/public/de/beschaedigte-testdateien/index.html index 9effcfad..9aa093c5 100644 --- a/web/public/de/beschaedigte-testdateien/index.html +++ b/web/public/de/beschaedigte-testdateien/index.html @@ -199,7 +199,9 @@

Eine beschädigte Datei hat trotzdem die verlangte Größe

verlangte Größe, und derselbe Befehl schreibt wieder dieselben Bytes. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Einstellungen stehen nach einem Doppelpunkt. Die Option lässt sich wiederholen, und die Beschädigungen werden in der Reihenfolge angewendet, in der Sie sie schreiben. Es funktioniert @@ -226,6 +228,18 @@

Welche Beschädigungen gibt es?

+ + append-junk + Schreibt zufällige Bytes hinter das Ende der Datei und behält genau die angegebene Größe, so wie eine Datei ankommt, an deren Ende etwas hängt. + 64 + bytes + + + truncate + Schneidet das Ende der Datei ab und behält genau die angegebene Größe, so wie ein abgebrochener Upload oder Kopiervorgang die Datei hinterlässt. + 4 + missing + zero-head Überschreibt die ersten Bytes der Datei mit Nullen und lässt ihre Länge unverändert. Die meisten Leser schauen zuerst dorthin, deshalb bemerkt fast alles diese Beschädigung. @@ -242,14 +256,25 @@

Welche Beschädigungen gibt es?

eine Folge von Nullbytes kein Text ist. Unter vier Bytes entstehen bei manchen Formaten Schäden, über die sich kein Leser beschwert, deshalb beginnt die Einstellung bei vier.

+

+ truncate schneidet das Ende der Datei ab, so wie ein abgebrochener Upload oder + Kopiervorgang sie hinterlässt, und die Datei hat trotzdem die angegebene Größe: Sie wird größer + erzeugt, und nur ihr Anfang landet auf der Festplatte. missing gibt an, welcher + Anteil der ganzen Datei fehlt, die Hälfte, wenn Sie nichts anderes angeben. +

+

+ append-junk schreibt zufällige Bytes hinter das Ende der Datei, 64, wenn Sie nichts + anderes angeben, so wie eine Datei ankommt, an deren Ende etwas hängt. Derselbe Befehl hängt + dieselben Bytes an. +

Was das Manifest sagt

Ein Manifest, das sagt, was geschehen soll

- Jede beschädigte Datei bekommt einen Eintrag, der besagt, dass Ihr System sie ablehnen soll, mit der - Beschädigung daneben: + Wo die Beschädigung die Regeln des Formats bricht, besagt der Eintrag, dass Ihr System die Datei + ablehnen soll, mit der Beschädigung daneben: 

"expected": {
   "outcome": "reject",
@@ -265,15 +290,40 @@ 
Ein Manifest, das sagt, was geschehen soll

   }
 ]

- Zwei Anfragen werden abgelehnt, bevor etwas geschrieben wird, denn jede würde eine Datei + Wo sich Leser unterscheiden, sagt der Eintrag das, statt zu raten. Eine gekürzte Textdatei ist + immer noch Text, und die meisten Bildbetrachter überspringen Bytes hinter dem Ende einer PNG, + also überlässt der Eintrag die Antwort Ihrem System, nennt beide und sagt, warum: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> listet die Formate auf, in denen eine Beschädigung die + Antwort Ihrem System überlässt, und warum. +

+

+ Vier Anfragen werden abgelehnt, bevor etwas geschrieben wird, denn jede würde eine Datei hinterlassen, die das Manifest falsch beschreibt:

  • eine Datei, die kleiner ist, als die Beschädigung braucht, und unverändert herauskäme
  • - expected: accept neben einer Beschädigung, weil nichts es erfüllen könnte. Schreiben - Sie sanitize, wenn Ihr System die Datei reparieren soll, oder - unspecified, wenn genau das Ihre Frage ist + expected: accept neben einer Beschädigung, die die Regeln des Formats bricht, + weil nichts, was sich an sie hält, die Datei annimmt. Wo die Antwort von Ihrem System abhängt, + ist accept erlaubt. Schreiben Sie sanitize, wenn Ihr System die + Datei reparieren soll, oder unspecified, wenn genau das Ihre Frage ist +
    • +
  • + truncate nach append-junk, weil es die angehängten Bytes wieder + abschneiden würde. Stellen Sie truncate an den Anfang +
    • +
  • + truncate oder append-junk an einem Archiv, das seine Größe aus + seinem Inhalt bezieht, weil es keine Größe gibt, von der man ausgehen kann
@@ -311,11 +361,19 @@

Daraus einen Test machen

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Eine Datei mit dem Ergebnis unspecified wird hier ausgelassen, weil Ihre eigene + Richtlinie bestimmt, was Ihr System mit ihr tun soll. Ihr Eintrag sagt, warum, und + policy_options nennt die möglichen Antworten. +

Eine gute Ablehnung ist eine saubere. Eine Meldung, die sagt, was falsch war, ist die gewünschte Antwort. Ein Serverfehler, ein Hänger oder eine halb gespeicherte Datei ist der Fehler, den diff --git a/web/public/es/archivos-de-prueba-corruptos/index.html b/web/public/es/archivos-de-prueba-corruptos/index.html index 1785cc04..9aa91668 100644 --- a/web/public/es/archivos-de-prueba-corruptos/index.html +++ b/web/public/es/archivos-de-prueba-corruptos/index.html @@ -198,7 +198,9 @@

Un archivo dañado conserva el tamaño que pediste

pediste, y el mismo comando vuelve a escribir los mismos bytes. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Los ajustes van después de dos puntos. La opción se puede repetir, y los daños se aplican en el orden en que los escribes. Funciona con cada uno de los 31 formatos. @@ -223,6 +225,18 @@

¿Qué daños hay?

+ + append-junk + Escribe bytes aleatorios después del final del archivo y mantiene exactamente el tamaño que pediste, como llega un archivo con algo pegado al final. + 64 + bytes + + + truncate + Corta el final del archivo y mantiene exactamente el tamaño que pediste, como lo deja una subida o una copia interrumpida. + 4 + missing + zero-head Sobrescribe los primeros bytes del archivo con ceros sin tocar su longitud. La mayoría de los lectores miran ahí primero, así que casi todo nota este daño. @@ -239,14 +253,25 @@

¿Qué daños hay?

de bytes nulos no es texto. Por debajo de cuatro bytes algunos formatos salen con un daño del que ningún lector se queja, por eso el ajuste empieza en cuatro.

+

+ truncate corta el final del archivo, como lo deja una subida o una copia + interrumpida, y el archivo sigue teniendo el tamaño que pediste: se genera más grande y solo su + comienzo llega al disco. missing dice qué parte del archivo completo falta, la + mitad si no indicas otra cosa. +

+

+ append-junk escribe bytes aleatorios después del final del archivo, 64 si no + indicas otra cosa, como llega un archivo con algo pegado al final. El mismo comando añade los + mismos bytes. +

Lo que dice el manifiesto

Un manifiesto que dice lo que debe ocurrir

- Cada archivo dañado recibe una entrada que dice que tu sistema debe rechazarlo, con el daño anotado - al lado: + Donde el daño rompe las reglas del formato, la entrada dice que tu sistema debe rechazarlo, con + el daño anotado al lado: 

"expected": {
   "outcome": "reject",
@@ -262,15 +287,40 @@ 
Un manifiesto que dice lo que debe ocurrir

   }
 ]

- Se rechazan dos peticiones antes de escribir nada, porque cada una dejaría en el disco un archivo - que el manifiesto describe mal: + Donde los lectores no coinciden, la entrada lo dice en lugar de adivinar. Un archivo de texto + cortado sigue siendo texto, y la mayoría de los visores ignoran los bytes después del final de + un PNG, así que la entrada deja la respuesta a tu sistema, ofrece las dos y dice por qué: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> enumera los formatos en los que un daño deja la respuesta a + tu sistema, y por qué. +

+

+ Se rechazan cuatro peticiones antes de escribir nada, porque cada una dejaría en el disco un + archivo que el manifiesto describe mal:

  • un archivo más pequeño de lo que el daño necesita, que saldría intacto
  • - expected: accept junto a un daño, porque nada podría cumplirlo. Escribe - sanitize si tu sistema debe reparar el archivo, o unspecified si esa - es justo la pregunta que haces + expected: accept junto a un daño que rompe las reglas del formato, porque nada + que las siga acepta el archivo. Donde la respuesta depende de tu sistema, accept + está permitido. Escribe sanitize si tu sistema debe reparar el archivo, o + unspecified si esa es justo la pregunta que haces +
    • +
  • + truncate después de append-junk, porque cortaría los bytes añadidos. + Pon truncate primero +
    • +
  • + truncate o append-junk en un archivo comprimido que toma su tamaño + de lo que contiene, porque no hay un tamaño del que partir
@@ -308,11 +358,19 @@

Convertirlo en una prueba

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un archivo cuyo resultado es unspecified se omite aquí, porque lo que tu sistema + debe hacer con él lo decide tu propia política. Su entrada dice por qué, y + policy_options enumera las respuestas posibles. +

Un buen rechazo es un rechazo limpio. Un mensaje que dice qué estaba mal es la respuesta que quieres. Un error de servidor, un bloqueo o un archivo guardado a medias es el defecto que esta diff --git a/web/public/fr/fichiers-de-test-corrompus/index.html b/web/public/fr/fichiers-de-test-corrompus/index.html index ef54d815..b65a855c 100644 --- a/web/public/fr/fichiers-de-test-corrompus/index.html +++ b/web/public/fr/fichiers-de-test-corrompus/index.html @@ -199,7 +199,9 @@

Un fichier abîmé garde la taille demandée

et la même commande écrit de nouveau les mêmes octets. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Les réglages se placent après deux-points. L'option peut être répétée, et les dommages s'appliquent dans l'ordre où vous les écrivez. Ça marche avec chacun des 31 formats. @@ -225,6 +227,18 @@

Quels dommages existe-t-il ?

+ + append-junk + Écrit des octets aléatoires après la fin du fichier et garde exactement la taille demandée, comme arrive un fichier avec quelque chose collé à sa fin. + 64 + bytes + + + truncate + Coupe la fin du fichier et garde exactement la taille demandée, comme le laisse un envoi ou une copie interrompus. + 4 + missing + zero-head Écrase les premiers octets du fichier avec des zéros, sans toucher à sa longueur. La plupart des lecteurs regardent d'abord là, donc presque tout remarque ce dommage. @@ -242,14 +256,25 @@

Quels dommages existe-t-il ?

sortent avec un dommage dont aucun lecteur ne se plaint, c'est pourquoi le réglage commence à quatre.

+

+ truncate coupe la fin du fichier, comme le laisse un envoi ou une copie + interrompus, et le fichier garde la taille demandée : il est généré plus grand, et seul son + début atteint le disque. missing indique quelle part du fichier entier disparaît, + la moitié sauf indication contraire. +

+

+ append-junk écrit des octets aléatoires après la fin du fichier, 64 sauf indication + contraire, comme arrive un fichier avec quelque chose collé à sa fin. La même commande ajoute + les mêmes octets. +

Ce que dit le manifeste

Un manifeste qui dit ce qui doit se passer

- Chaque fichier abîmé reçoit une entrée disant que votre système doit le rejeter, avec le dommage - noté à côté : + Là où le dommage enfreint les règles du format, l'entrée dit que votre système doit le rejeter, + avec le dommage noté à côté : 

"expected": {
   "outcome": "reject",
@@ -265,15 +290,40 @@ 
Un manifeste qui dit ce qui doit se passer

   }
 ]

- Deux demandes sont refusées avant que rien ne soit écrit, car chacune laisserait sur le disque un - fichier que le manifeste décrit mal : + Là où les lecteurs divergent, l'entrée le dit au lieu de deviner. Un fichier texte coupé reste + du texte, et la plupart des visionneuses ignorent les octets après la fin d'un PNG, donc + l'entrée laisse la réponse à votre système, propose les deux et dit pourquoi : +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> liste les formats où un dommage laisse la réponse à votre + système, et pourquoi. +

+

+ Quatre demandes sont refusées avant que rien ne soit écrit, car chacune laisserait sur le disque + un fichier que le manifeste décrit mal :

  • un fichier plus petit que ce dont le dommage a besoin, qui sortirait intact
  • - expected: accept à côté d'un dommage, car rien ne pourrait le satisfaire. Écrivez - sanitize si votre système doit réparer le fichier, ou unspecified si - c'est justement la question que vous posez + expected: accept à côté d'un dommage qui enfreint les règles du format, car rien + de ce qui les suit n'accepte le fichier. Là où la réponse dépend de votre système, + accept est permis. Écrivez sanitize si votre système doit réparer le + fichier, ou unspecified si c'est justement la question que vous posez +
    • +
  • + truncate après append-junk, qui couperait les octets ajoutés. Placez + truncate en premier +
    • +
  • + truncate ou append-junk sur une archive qui tire sa taille de son + contenu, car il n'y a pas de taille d'où partir
@@ -311,11 +361,19 @@

En faire un test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un fichier dont le résultat est unspecified est laissé de côté ici, car ce que + votre système doit en faire relève de votre propre politique. Son entrée dit pourquoi, et + policy_options liste les réponses possibles. +

Un bon refus est un refus net. Un message qui dit ce qui n'allait pas est la réponse voulue. Une erreur serveur, un blocage ou un fichier à moitié enregistré est le défaut que ce test existe diff --git a/web/public/hi/corrupt-test-files/index.html b/web/public/hi/corrupt-test-files/index.html index 694dee68..e5b71f5d 100644 --- a/web/public/hi/corrupt-test-files/index.html +++ b/web/public/hi/corrupt-test-files/index.html @@ -197,7 +197,9 @@

खराब फ़ाइल का आकार वही रहता ह रखती है, और वही कमांड फिर वही बाइट लिखता है। 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

सेटिंग कोलन के बाद लिखी जाती है। विकल्प दोहराया जा सकता है, और बिगाड़ आपके लिखे क्रम में लागू होते हैं। यह सभी 31 फ़ॉर्मैट के साथ चलता है। @@ -222,6 +224,18 @@

कौन-कौन से बिगाड़ हैं?

+ + append-junk + फ़ाइल के अंत के बाद यादृच्छिक बाइट लिखता है और ठीक वही आकार रखता है जो आपने माँगा, जैसे कोई फ़ाइल अपने अंत में कुछ चिपका हुआ लेकर आती है। + 64 + bytes + + + truncate + फ़ाइल का अंत काट देता है और ठीक वही आकार रखता है जो आपने माँगा, जैसे बीच में टूटा अपलोड या कॉपी उसे छोड़ता है। + 4 + missing + zero-head फ़ाइल के शुरुआती बाइट को शून्य से ढक देता है और लंबाई नहीं छेड़ता। ज़्यादातर रीडर सबसे पहले वहीं देखते हैं, इसलिए यह बिगाड़ लगभग हर चीज़ पकड़ लेती है। @@ -238,14 +252,24 @@

कौन-कौन से बिगाड़ हैं?

पाठ नहीं है। चार बाइट से नीचे कुछ फ़ॉर्मैट ऐसे बिगाड़ के साथ निकलते हैं जिसकी कोई रीडर शिकायत नहीं करता, इसीलिए सेटिंग चार से शुरू होती है।

+

+ truncate फ़ाइल का अंत काट देता है, जैसे बीच में टूटा अपलोड या कॉपी उसे छोड़ता है, + और फ़ाइल फिर भी उसी आकार की रहती है जो आपने माँगा: वह बड़ी बनाई जाती है और डिस्क तक सिर्फ़ उसकी + शुरुआत पहुँचती है। missing बताता है कि पूरी फ़ाइल का कितना हिस्सा गायब है, आधा, जब + तक आप कुछ और न कहें। +

+

+ append-junk फ़ाइल के अंत के बाद यादृच्छिक बाइट लिखता है, 64, जब तक आप कुछ और न + कहें, जैसे कोई फ़ाइल अपने अंत में कुछ चिपका हुआ लेकर आती है। एक ही कमांड वही बाइट जोड़ती है। +

मैनिफ़ेस्ट क्या कहता है

एक मैनिफ़ेस्ट जो बताता है कि क्या होना चाहिए

- हर खराब फ़ाइल को एक प्रविष्टि मिलती है जो कहती है कि आपके सिस्टम को उसे अस्वीकार करना चाहिए, और - बिगाड़ उसके बगल में दर्ज रहता है: + जहाँ बिगाड़ फ़ॉर्मैट के नियम तोड़ता है, वहाँ प्रविष्टि कहती है कि आपके सिस्टम को उसे अस्वीकार + करना चाहिए, और बिगाड़ उसके बगल में दर्ज रहता है: 

"expected": {
   "outcome": "reject",
@@ -261,15 +285,40 @@ 
एक मैनिफ़ेस्ट जो बताता है कि 
   }
 ]

- दो अनुरोध कुछ भी लिखे जाने से पहले ठुकरा दिए जाते हैं, क्योंकि हर एक डिस्क पर ऐसी फ़ाइल छोड़ देता - जिसका मैनिफ़ेस्ट गलत वर्णन करता: + जहाँ रीडर अलग-अलग बर्ताव करते हैं, वहाँ प्रविष्टि अंदाज़ा लगाने के बजाय यही कहती है। काटी गई पाठ + फ़ाइल फिर भी पाठ है, और ज़्यादातर इमेज व्यूअर PNG के अंत के बाद के बाइट छोड़ देते हैं, इसलिए + प्रविष्टि जवाब आपके सिस्टम पर छोड़ती है, दोनों विकल्प देती है और कारण बताती है: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> वे फ़ॉर्मैट और कारण बताता है जिनमें कोई बिगाड़ जवाब आपके + सिस्टम पर छोड़ता है। +

+

+ चार अनुरोध कुछ भी लिखे जाने से पहले ठुकरा दिए जाते हैं, क्योंकि हर एक डिस्क पर ऐसी फ़ाइल छोड़ + देता जिसका मैनिफ़ेस्ट गलत वर्णन करता:

  • बिगाड़ को जितना चाहिए उससे छोटी फ़ाइल, जो ज्यों की त्यों निकलती
  • - बिगाड़ के साथ expected: accept, क्योंकि कोई भी उसे पूरा नहीं कर सकता। अगर आपके सिस्टम - को फ़ाइल सुधारनी है तो sanitize लिखें, या अगर आप यही सवाल पूछ रहे हैं तो - unspecified + फ़ॉर्मैट के नियम तोड़ने वाले बिगाड़ के साथ expected: accept, क्योंकि उन नियमों को + मानने वाली कोई चीज़ फ़ाइल स्वीकार नहीं करती। जहाँ जवाब आपके सिस्टम पर निर्भर है, वहाँ + accept की अनुमति है। अगर आपके सिस्टम को फ़ाइल सुधारनी है तो sanitize + लिखें, और अगर आप ठीक यही पूछ रहे हैं तो unspecified +
    • +
  • + append-junk के बाद truncate, क्योंकि वह जोड़े गए बाइट काट देता। + truncate को पहले रखें +
    • +
  • + ऐसे आर्काइव पर truncate या append-junk जो अपना आकार अपनी सामग्री से + लेता है, क्योंकि शुरू करने के लिए कोई आकार नहीं है
@@ -307,11 +356,19 @@

इसे टेस्ट बनाना

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ जिस फ़ाइल का नतीजा unspecified है, उसे यहाँ छोड़ दिया जाता है, क्योंकि आपके सिस्टम + को उसके साथ क्या करना है, यह आपकी अपनी नीति तय करती है। उसकी प्रविष्टि कारण बताती है, और + policy_options संभव जवाब गिनाता है। +

अच्छा इनकार साफ़ इनकार होता है। जो संदेश बताए कि क्या गलत था, वही वह जवाब है जो आप चाहते हैं। सर्वर त्रुटि, अटक जाना या आधी सहेजी फ़ाइल वह खराबी है जिसे ढूँढ़ने के लिए यह टेस्ट है। diff --git a/web/public/id/file-uji-rusak/index.html b/web/public/id/file-uji-rusak/index.html index 18ca49ae..3d8be848 100644 --- a/web/public/id/file-uji-rusak/index.html +++ b/web/public/id/file-uji-rusak/index.html @@ -198,7 +198,9 @@

File yang rusak tetap berukuran seperti yang Anda minta

minta, dan perintah yang sama menulis byte yang sama lagi. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Pengaturan ditulis setelah titik dua. Opsi ini bisa diulang, dan kerusakan diterapkan menurut urutan yang Anda tulis. Berlaku untuk semua 31 format. @@ -224,6 +226,18 @@

Kerusakan apa saja yang ada?

+ + append-junk + Menulis byte acak setelah akhir file dan mempertahankan ukuran persis yang Anda minta, seperti file yang tiba dengan sesuatu menempel di ujungnya. + 64 + bytes + + + truncate + Memotong ujung file dan mempertahankan ukuran persis yang Anda minta, seperti yang ditinggalkan unggahan atau salinan yang terputus. + 4 + missing + zero-head Menimpa byte pertama file dengan nol tanpa mengubah panjangnya. Sebagian besar pembaca melihat ke sana lebih dulu, jadi hampir semua hal menyadari kerusakan ini. @@ -240,14 +254,25 @@

Kerusakan apa saja yang ada?

byte nol bukan teks. Di bawah empat byte, sebagian format keluar dengan kerusakan yang tidak dikeluhkan pembaca mana pun, itulah sebabnya pengaturan dimulai dari empat.

+

+ truncate memotong ujung file, seperti yang ditinggalkan unggahan atau salinan yang + terputus, dan file tetap berukuran sesuai permintaan Anda: file dibuat lebih besar dan hanya + awalnya yang sampai ke disk. missing menyatakan berapa bagian dari file utuh yang + hilang, separuh jika Anda tidak menyebut lain. +

+

+ append-junk menulis byte acak setelah akhir file, 64 jika Anda tidak menyebut lain, + seperti file yang tiba dengan sesuatu menempel di ujungnya. Perintah yang sama menambahkan byte + yang sama. +

Yang dikatakan manifes

Manifes yang menyatakan apa yang harus terjadi

- Setiap file yang rusak mendapat catatan yang menyatakan sistem Anda harus menolaknya, dengan - kerusakannya dicatat di sampingnya: + Jika kerusakan melanggar aturan format, catatannya menyatakan sistem Anda harus menolaknya, + dengan kerusakannya dicatat di sampingnya: 

"expected": {
   "outcome": "reject",
@@ -263,15 +288,42 @@ 
Manifes yang menyatakan apa yang harus terjadi

   }
 ]

- Dua permintaan ditolak sebelum apa pun ditulis, karena masing-masing akan meninggalkan file di disk - yang dijelaskan keliru oleh manifes: + Jika pembaca berbeda pendapat, catatannya menyatakan hal itu alih-alih menebak. File teks yang + terpotong tetap teks, dan sebagian besar penampil gambar melewati byte setelah akhir PNG, jadi + catatannya menyerahkan jawaban kepada sistem Anda, menawarkan keduanya, dan menjelaskan + alasannya: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> mencantumkan format tempat suatu kerusakan menyerahkan + jawaban kepada sistem Anda, beserta alasannya. +

+

+ Empat permintaan ditolak sebelum apa pun ditulis, karena masing-masing akan meninggalkan file di + disk yang dijelaskan keliru oleh manifes:

  • file yang lebih kecil dari yang dibutuhkan kerusakan, yang akan keluar tanpa perubahan
  • - expected: accept di samping kerusakan, karena tidak ada yang bisa memenuhinya. Tulis - sanitize jika sistem Anda memang harus memperbaiki file itu, atau - unspecified jika justru itu pertanyaan Anda + expected: accept di samping kerusakan yang melanggar aturan format, karena tidak + ada yang mengikuti aturan itu yang menerima file tersebut. Jika jawabannya bergantung pada + sistem Anda, accept diperbolehkan. Tulis sanitize jika sistem Anda + memang harus memperbaiki file itu, atau unspecified jika justru itu pertanyaan + Anda +
    • +
  • + truncate setelah append-junk, karena akan memotong byte yang + ditambahkan. Letakkan truncate lebih dulu +
    • +
  • + truncate atau append-junk pada arsip yang mengambil ukurannya dari + isinya, karena tidak ada ukuran untuk dijadikan titik awal
@@ -309,11 +361,19 @@

Menjadikannya tes

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ File dengan hasil unspecified dilewati di sini, karena apa yang harus dilakukan + sistem Anda terhadapnya adalah kebijakan Anda sendiri. Catatannya menjelaskan alasannya, dan + policy_options mencantumkan jawaban yang mungkin. +

Penolakan yang baik adalah penolakan yang bersih. Pesan yang menyebut apa yang salah adalah jawaban yang Anda inginkan. Kesalahan server, macet, atau file yang tersimpan setengah adalah cacat yang diff --git a/web/public/it/file-di-test-corrotti/index.html b/web/public/it/file-di-test-corrotti/index.html index 97760b95..f385b52a 100644 --- a/web/public/it/file-di-test-corrotti/index.html +++ b/web/public/it/file-di-test-corrotti/index.html @@ -198,7 +198,9 @@

Un file danneggiato ha ancora la dimensione richiesta

richiesta, e lo stesso comando riscrive gli stessi byte. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Le impostazioni vanno dopo i due punti. L'opzione si può ripetere, e i danni vengono applicati nell'ordine in cui li scrivi. Funziona con ognuno dei 31 formati. @@ -224,6 +226,18 @@

Quali danni ci sono?

+ + append-junk + Scrive byte casuali dopo la fine del file e mantiene esattamente la dimensione richiesta, come arriva un file con qualcosa attaccato in fondo. + 64 + bytes + + + truncate + Taglia la fine del file e mantiene esattamente la dimensione richiesta, come la lascia un caricamento o una copia interrotti. + 4 + missing + zero-head Sovrascrive con zeri i primi byte del file, lasciandone intatta la lunghezza. La maggior parte dei lettori guarda prima lì, quindi quasi tutto si accorge di questo danno. @@ -240,14 +254,24 @@

Quali danni ci sono?

byte nulli non è testo. Sotto i quattro byte alcuni formati escono con un danno di cui nessun lettore si lamenta, ed è per questo che l'impostazione parte da quattro.

+

+ truncate taglia la fine del file, come la lascia un caricamento o una copia + interrotti, e il file ha comunque la dimensione richiesta: viene generato più grande e solo il + suo inizio arriva sul disco. missing dice quale parte del file intero manca, la + metà se non indichi altro. +

+

+ append-junk scrive byte casuali dopo la fine del file, 64 se non indichi altro, + come arriva un file con qualcosa attaccato in fondo. Lo stesso comando aggiunge gli stessi byte. +

Cosa dice il manifest

Un manifest che dice cosa deve succedere

- Ogni file danneggiato riceve una voce che dice che il tuo sistema deve rifiutarlo, con il danno - registrato accanto: + Dove il danno viola le regole del formato, la voce dice che il tuo sistema deve rifiutarlo, con + il danno registrato accanto: 

"expected": {
   "outcome": "reject",
@@ -263,15 +287,40 @@ 
Un manifest che dice cosa deve succedere

   }
 ]

- Due richieste vengono rifiutate prima che venga scritto qualcosa, perché ciascuna lascerebbe su - disco un file che il manifest descrive male: + Dove i lettori non concordano, la voce lo dice invece di tirare a indovinare. Un file di testo + tagliato è ancora testo, e la maggior parte dei visualizzatori ignora i byte dopo la fine di un + PNG, quindi la voce lascia la risposta al tuo sistema, offre entrambe e dice perché: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> elenca i formati in cui un danno lascia la risposta al tuo + sistema, e perché. +

+

+ Quattro richieste vengono rifiutate prima che venga scritto qualcosa, perché ciascuna lascerebbe + su disco un file che il manifest descrive male:

  • un file più piccolo di quanto serve al danno, che uscirebbe intatto
  • - expected: accept accanto a un danno, perché nulla potrebbe soddisfarlo. Scrivi - sanitize se il tuo sistema deve riparare il file, oppure unspecified - se è proprio questa la domanda che poni + expected: accept accanto a un danno che viola le regole del formato, perché nulla + che le segua accetta il file. Dove la risposta dipende dal tuo sistema, accept è + consentito. Scrivi sanitize se il tuo sistema deve riparare il file, oppure + unspecified se è proprio questa la domanda che poni +
    • +
  • + truncate dopo append-junk, perché taglierebbe via i byte aggiunti. + Metti truncate per primo +
    • +
  • + truncate o append-junk su un archivio che prende la dimensione da + ciò che contiene, perché non c'è una dimensione da cui partire
@@ -309,11 +358,19 @@

Trasformarlo in un test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un file il cui esito è unspecified viene escluso qui, perché cosa il tuo sistema + debba farne lo decide la tua politica. La sua voce dice perché, e policy_options + elenca le risposte possibili. +

Un buon rifiuto è un rifiuto pulito. Un messaggio che dice cosa non andava è la risposta che vuoi. Un errore del server, un blocco o un file salvato a metà è il difetto che questo test esiste per diff --git a/web/public/ja/corrupt-test-files/index.html b/web/public/ja/corrupt-test-files/index.html index 2c9e2ee9..068c4dc1 100644 --- a/web/public/ja/corrupt-test-files/index.html +++ b/web/public/ja/corrupt-test-files/index.html @@ -187,7 +187,9 @@

破損したファイルも、求めたサイズのままです

ファイルは通常どおり生成され、ディスクへ向かう途中で壊されます。求めたサイズは保たれ、同じコマンドは同じバイトをもう一度書き出します。 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

設定はコロンの後ろに書きます。オプションは繰り返せて、破損は書いた順に適用されます。31種類すべての形式で使えます。

@@ -211,6 +213,18 @@

どんな破損がありますか。

+ + append-junk + ファイルの末尾の後ろにランダムなバイトを書き込み、指定したサイズをそのまま保ちます。末尾に何かが付いたまま届いたファイルと同じ形です。 + 64 + bytes + + + truncate + ファイルの末尾を切り落とし、指定したサイズをそのまま保ちます。途中で止まったアップロードやコピーが残すのと同じ形です。 + 4 + missing + zero-head ファイルの先頭のバイトをゼロで上書きし、長さは変えません。ほとんどのリーダーはまずそこを見るので、この破損にはほぼどれも気づきます。 @@ -223,13 +237,19 @@

どんな破損がありますか。

zero-headはファイルの先頭をゼロで上書きします。ほとんどのリーダーはまずそこを見ます。ファイルが何であるかを示すシグネチャとヘッダーです。そのため、ほぼどのリーダーでも気づきます。プレーンテキストやログにはシグネチャがありませんが、これらも拒否されます。ゼロのバイトの連なりはテキストではないからです。4バイト未満では、どのリーダーも文句を言わない破損になる形式があります。設定が4から始まるのはそのためです。

+

+ truncateはファイルの末尾を切り落とします。途中で止まったアップロードやコピーが残すのと同じ形ですが、ファイルは指定したサイズのままです。大きめに生成され、その先頭だけがディスクに届くからです。missingはファイル全体のうちどれだけが失われるかを示し、指定しなければ半分です。 +

+

+ append-junkはファイルの末尾の後ろにランダムなバイトを書き込みます。指定しなければ64バイトで、末尾に何かが付いたまま届いたファイルと同じ形です。同じコマンドは同じバイトを付け加えます。 +

マニフェストの内容

何が起こるべきかを示すマニフェスト

- 破損したファイルにはそれぞれ、システムがそれを拒否すべきことを示す項目が付き、破損の内容がその横に記録されます。 + 破損が形式の規則に反する場合、項目はシステムがそれを拒否すべきことを示し、破損の内容がその横に記録されます。 

"expected": {
   "outcome": "reject",
@@ -245,13 +265,31 @@ 
何が起こるべきかを示すマニフェスト

   }
 ]

- 2種類のリクエストは、何かが書き込まれる前に拒否されます。どちらもマニフェストの記述が誤ったファイルをディスクに残してしまうからです。 + リーダーによって扱いが分かれる場合、項目は推測せずにそう伝えます。末尾を切られたテキストファイルもテキストのままですし、ほとんどの画像ビューアーはPNGの終わりの後ろのバイトを読み飛ばします。そのため項目は答えをシステムに委ね、両方の答えを示し、理由を書きます。 +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id>は、破損が答えをシステムに委ねる形式とその理由を一覧にします。 +

+

+ 4種類のリクエストは、何かが書き込まれる前に拒否されます。どれもマニフェストの記述が誤ったファイルをディスクに残してしまうからです。

  • 破損に必要なサイズより小さいファイル。そのままの状態で出てきてしまいます
  • - 破損と並べたexpected: - accept。何もそれを満たせないためです。システムがファイルを修復するはずならsanitizeを、まさにそれを確かめたいならunspecifiedを書いてください + 形式の規則に反する破損と並べたexpected: accept。規則に従うものはそのファイルを受け入れないためです。答えがシステム次第の場合はacceptを書けます。システムがファイルを修復するはずならsanitizeを、まさにそれを確かめたいならunspecifiedを書いてください +
    • +
  • + append-junkの後のtruncate。付け加えたバイトを切り落としてしまうためです。truncateを先に置いてください +
    • +
  • + 中身からサイズが決まるアーカイブに対するtruncateやappend-junk。基準になるサイズがないためです
@@ -287,11 +325,17 @@

テストにする

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 結果がunspecifiedのファイルはここでは除外します。システムがそれをどう扱うべきかは、あなた自身の方針で決まるからです。項目にはその理由が書かれ、policy_optionsには取りうる答えが並びます。 +

よい拒否は、きれいな拒否です。何が悪かったかを伝えるメッセージが、求めている答えです。サーバーエラー、ハング、書きかけで保存されたファイルは、このテストが見つけるために存在する欠陥です。

diff --git a/web/public/ko/corrupt-test-files/index.html b/web/public/ko/corrupt-test-files/index.html index 2213c9c0..f6acc4c2 100644 --- a/web/public/ko/corrupt-test-files/index.html +++ b/web/public/ko/corrupt-test-files/index.html @@ -192,7 +192,9 @@

손상된 파일도 요청한 크기를 유지합니다

파일은 평소대로 생성된 뒤 디스크로 가는 길에 망가집니다. 요청한 크기는 그대로이고, 같은 명령은 같은 바이트를 다시 씁니다. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

설정은 콜론 뒤에 씁니다. 옵션은 반복할 수 있고, 손상은 쓴 순서대로 적용됩니다. 31개 형식 모두에서 동작합니다.

@@ -216,6 +218,18 @@

어떤 손상이 있나요?

+ + append-junk + 파일의 끝 뒤에 무작위 바이트를 쓰고 요청한 크기를 그대로 유지합니다. 끝에 무언가가 붙은 채 도착한 파일과 같은 모양입니다. + 64 + bytes + + + truncate + 파일의 끝을 잘라 내고 요청한 크기를 그대로 유지합니다. 중간에 끊긴 업로드나 복사가 남기는 모양과 같습니다. + 4 + missing + zero-head 파일의 첫 바이트들을 길이는 그대로 둔 채 0으로 덮어씁니다. 대부분의 리더가 가장 먼저 그곳을 보므로 거의 모든 것이 이 손상을 알아챕니다. @@ -230,13 +244,19 @@

어떤 손상이 있나요?

그래서 거의 모든 리더가 알아챕니다. 일반 텍스트와 로그에는 시그니처가 없지만 역시 거부됩니다. 0 바이트의 연속은 텍스트가 아니기 때문입니다. 4바이트 미만에서는 어떤 리더도 불평하지 않는 손상이 나오는 형식이 있으며, 설정이 4부터 시작하는 이유가 그것입니다.

+

+ truncate는 중간에 끊긴 업로드나 복사가 남기는 것처럼 파일의 끝을 잘라 내지만, 파일은 여전히 요청한 크기입니다. 더 크게 생성되고 그 앞부분만 디스크에 닿기 때문입니다. missing은 전체 파일 중 얼마가 사라지는지를 나타내며, 따로 지정하지 않으면 절반입니다. +

+

+ append-junk는 끝에 무언가가 붙은 채 도착한 파일처럼 파일의 끝 뒤에 무작위 바이트를 씁니다. 따로 지정하지 않으면 64바이트입니다. 같은 명령은 같은 바이트를 덧붙입니다. +

매니페스트가 말하는 것

무엇이 일어나야 하는지 말해 주는 매니페스트

- 손상된 파일마다 시스템이 그것을 거부해야 한다는 항목이 붙고, 손상 내용이 그 옆에 기록됩니다. + 손상이 형식의 규칙을 어기는 경우, 항목은 시스템이 그것을 거부해야 한다고 말하고 손상 내용이 그 옆에 기록됩니다. 

"expected": {
   "outcome": "reject",
@@ -252,13 +272,31 @@ 
무엇이 일어나야 하는지 말해 주는 매니페스트

   }
 ]

- 두 가지 요청은 무엇이든 쓰기 전에 거부됩니다. 둘 다 매니페스트가 잘못 설명하는 파일을 디스크에 남기게 되기 때문입니다. + 리더마다 다르게 다루는 경우, 항목은 추측하는 대신 그렇다고 말합니다. 끝이 잘린 텍스트 파일도 여전히 텍스트이고, 대부분의 이미지 뷰어는 PNG의 끝 뒤에 있는 바이트를 건너뜁니다. 그래서 항목은 답을 시스템에 맡기고, 두 가지 답을 모두 제시하며, 이유를 적습니다. +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id>는 손상이 답을 시스템에 맡기는 형식과 그 이유를 나열합니다. +

+

+ 네 가지 요청은 무엇이든 쓰기 전에 거부됩니다. 모두 매니페스트가 잘못 설명하는 파일을 디스크에 남기게 되기 때문입니다.

  • 손상에 필요한 크기보다 작은 파일. 그대로 나오게 됩니다
  • - 손상 옆의 expected: accept. 어떤 것도 그것을 충족할 수 없기 때문입니다. 시스템이 파일을 복구해야 한다면 - sanitize를, 바로 그것을 확인하려는 것이라면 unspecified를 쓰세요 + 형식의 규칙을 어기는 손상 옆의 expected: accept. 그 규칙을 따르는 것은 어떤 것도 그 파일을 받아들이지 않기 때문입니다. 답이 시스템에 달린 경우에는 accept를 쓸 수 있습니다. 시스템이 파일을 복구해야 한다면 sanitize를, 바로 그것을 확인하려는 것이라면 unspecified를 쓰세요 +
    • +
  • + append-junk 뒤의 truncate. 덧붙인 바이트를 잘라 내기 때문입니다. truncate를 먼저 두세요 +
    • +
  • + 담긴 내용에서 크기를 정하는 아카이브에 대한 truncate나 append-junk. 기준이 될 크기가 없기 때문입니다
@@ -294,11 +332,17 @@

테스트로 만들기

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 결과가 unspecified인 파일은 여기서 건너뜁니다. 시스템이 그 파일을 어떻게 다뤄야 하는지는 여러분의 정책이 정하기 때문입니다. 항목에는 그 이유가 적혀 있고, policy_options에는 가능한 답이 나열됩니다. +

좋은 거부는 깔끔한 거부입니다. 무엇이 잘못됐는지 알려 주는 메시지가 원하는 답입니다. 서버 오류, 멈춤, 반쯤 저장된 파일은 이 테스트가 찾아내려고 존재하는 결함입니다.

diff --git a/web/public/nl/beschadigde-testbestanden/index.html b/web/public/nl/beschadigde-testbestanden/index.html index 34c76d93..7e3b3de0 100644 --- a/web/public/nl/beschadigde-testbestanden/index.html +++ b/web/public/nl/beschadigde-testbestanden/index.html @@ -199,7 +199,9 @@

Een beschadigd bestand heeft nog steeds de grootte die je vroeg

grootte die je vroeg, en hetzelfde commando schrijft opnieuw dezelfde bytes. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Instellingen komen achter een dubbele punt. De optie mag herhaald worden, en de beschadigingen worden toegepast in de volgorde waarin je ze schrijft. Het werkt met elk van de @@ -226,6 +228,18 @@

Welke beschadigingen zijn er?

+ + append-junk + Schrijft willekeurige bytes achter het einde van het bestand en houdt precies de gevraagde grootte, zoals een bestand binnenkomt waar iets aan het einde aan vastzit. + 64 + bytes + + + truncate + Knipt het einde van het bestand af en houdt precies de gevraagde grootte, zoals een afgebroken upload of kopie het achterlaat. + 4 + missing + zero-head Overschrijft de eerste bytes van het bestand met nullen en laat de lengte ongemoeid. De meeste lezers kijken daar eerst, dus bijna alles merkt deze beschadiging. @@ -242,14 +256,25 @@

Welke beschadigingen zijn er?

reeks nulbytes geen tekst is. Onder vier bytes komen sommige formaten uit met schade waar geen lezer over klaagt, en daarom begint de instelling bij vier.

+

+ truncate knipt het einde van het bestand af, zoals een afgebroken upload of kopie + het achterlaat, en het bestand heeft toch de gevraagde grootte: het wordt groter gemaakt en + alleen het begin komt op de schijf. missing zegt welk deel van het hele bestand weg + is, de helft als je niets anders opgeeft. +

+

+ append-junk schrijft willekeurige bytes achter het einde van het bestand, 64 als je + niets anders opgeeft, zoals een bestand binnenkomt waar iets aan het einde aan vastzit. Dezelfde + opdracht voegt dezelfde bytes toe. +

Wat het manifest zegt

Een manifest dat zegt wat er moet gebeuren

- Elk beschadigd bestand krijgt een regel die zegt dat je systeem het moet weigeren, met de - beschadiging ernaast vastgelegd: + Waar de beschadiging de regels van het formaat breekt, zegt de vermelding dat je systeem het + bestand moet weigeren, met de beschadiging ernaast vastgelegd: 

"expected": {
   "outcome": "reject",
@@ -265,15 +290,40 @@ 
Een manifest dat zegt wat er moet gebeuren

   }
 ]

- Twee verzoeken worden geweigerd voordat er iets wordt geschreven, omdat elk een bestand op schijf - zou laten staan dat het manifest verkeerd beschrijft: + Waar lezers verschillen, zegt de vermelding dat in plaats van te gokken. Een afgeknipt + tekstbestand is nog steeds tekst, en de meeste viewers slaan bytes achter het einde van een PNG + over, dus laat de vermelding het antwoord aan je systeem, biedt beide aan en zegt waarom: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> noemt de formaten waarin een beschadiging het antwoord aan je + systeem laat, en waarom. +

+

+ Vier verzoeken worden geweigerd voordat er iets wordt geschreven, omdat elk een bestand op + schijf zou laten staan dat het manifest verkeerd beschrijft:

  • een bestand dat kleiner is dan de beschadiging nodig heeft en ongewijzigd zou uitkomen
  • - expected: accept naast een beschadiging, omdat niets daaraan kan voldoen. Schrijf - sanitize als je systeem het bestand moet repareren, of unspecified - als dat juist de vraag is die je stelt + expected: accept naast een beschadiging die de regels van het formaat breekt, + omdat niets dat ze volgt het bestand accepteert. Waar het antwoord van je systeem afhangt, mag + accept wel. Schrijf sanitize als je systeem het bestand moet + repareren, of unspecified als dat juist de vraag is die je stelt +
    • +
  • + truncate na append-junk, omdat het de toegevoegde bytes weer zou + afknippen. Zet truncate vooraan +
    • +
  • + truncate of append-junk op een archief dat zijn grootte haalt uit + wat het bevat, omdat er geen grootte is om van uit te gaan
@@ -311,11 +361,19 @@

Er een test van maken

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Een bestand met de uitkomst unspecified wordt hier overgeslagen, omdat je eigen + beleid bepaalt wat je systeem ermee moet doen. De vermelding zegt waarom, en + policy_options noemt de mogelijke antwoorden. +

Een goede weigering is een schone weigering. Een melding die zegt wat er mis was, is het antwoord dat je wilt. Een serverfout, een vastloper of een half opgeslagen bestand is het gebrek dat deze diff --git a/web/public/pl/uszkodzone-pliki-testowe/index.html b/web/public/pl/uszkodzone-pliki-testowe/index.html index 59df7c42..af5bbd7a 100644 --- a/web/public/pl/uszkodzone-pliki-testowe/index.html +++ b/web/public/pl/uszkodzone-pliki-testowe/index.html @@ -199,7 +199,9 @@

Uszkodzony plik ma nadal rozmiar, o który prosiłeś

a to samo polecenie zapisuje znowu te same bajty. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Ustawienia podaje się po dwukropku. Flagę można powtarzać, a uszkodzenia są stosowane w kolejności, w jakiej je zapiszesz. Działa z każdym z 31 formatów. @@ -225,6 +227,18 @@

Jakie są uszkodzenia?

+ + append-junk + Dopisuje losowe bajty za końcem pliku i zachowuje dokładnie rozmiar, o który prosiłeś, tak jak przychodzi plik z czymś przyklejonym na końcu. + 64 + bytes + + + truncate + Odcina koniec pliku i zachowuje dokładnie rozmiar, o który prosiłeś, tak jak zostawia go przerwane wysyłanie albo kopiowanie. + 4 + missing + zero-head Nadpisuje pierwsze bajty pliku zerami, nie zmieniając jego długości. Większość czytników zagląda tam najpierw, więc to uszkodzenie zauważy prawie wszystko. @@ -241,14 +255,24 @@

Jakie są uszkodzenia?

czterech bajtów niektóre formaty wychodzą z uszkodzeniem, na które nie skarży się żaden czytnik, dlatego ustawienie zaczyna się od czterech.

+

+ truncate odcina koniec pliku, tak jak zostawia go przerwane wysyłanie albo + kopiowanie, a plik ma nadal rozmiar, o który prosiłeś: powstaje większy i na dysk trafia tylko + jego początek. missing mówi, jaka część całego pliku znika, połowa, jeśli nie + podasz inaczej. +

+

+ append-junk dopisuje losowe bajty za końcem pliku, 64, jeśli nie podasz inaczej, + tak jak przychodzi plik z czymś przyklejonym na końcu. To samo polecenie dopisuje te same bajty. +

Co mówi manifest

Manifest, który mówi, co ma się stać

- Każdy uszkodzony plik dostaje wpis mówiący, że system powinien go odrzucić, a obok zapisane jest - uszkodzenie: + Tam, gdzie uszkodzenie łamie reguły formatu, wpis mówi, że system powinien plik odrzucić, a obok + zapisane jest uszkodzenie: 

"expected": {
   "outcome": "reject",
@@ -264,15 +288,40 @@ 
Manifest, który mówi, co ma się stać

   }
 ]

- Dwie prośby są odrzucane, zanim cokolwiek powstanie, bo każda zostawiłaby na dysku plik, który + Tam, gdzie czytniki się różnią, wpis mówi to wprost, zamiast zgadywać. Ucięty plik tekstowy jest + nadal tekstem, a większość przeglądarek obrazów pomija bajty za końcem PNG, więc wpis zostawia + odpowiedź twojemu systemowi, podaje obie i mówi dlaczego: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> wypisuje formaty, w których uszkodzenie zostawia odpowiedź + twojemu systemowi, i powód. +

+

+ Cztery prośby są odrzucane, zanim cokolwiek powstanie, bo każda zostawiłaby na dysku plik, który manifest opisuje błędnie:

  • plik mniejszy, niż potrzebuje uszkodzenie, bo wyszedłby nietknięty
  • - expected: accept obok uszkodzenia, bo nic nie mogłoby tego spełnić. Napisz - sanitize, jeśli system ma plik naprawić, albo unspecified, jeśli - właśnie o to pytasz + expected: accept obok uszkodzenia, które łamie reguły formatu, bo nic, co ich + przestrzega, nie przyjmie takiego pliku. Tam, gdzie odpowiedź zależy od twojego systemu, + accept jest dozwolone. Napisz sanitize, jeśli system ma plik + naprawić, albo unspecified, jeśli właśnie o to pytasz +
    • +
  • + truncate po append-junk, bo odcięłoby dopisane bajty. Postaw + truncate pierwsze +
    • +
  • + truncate albo append-junk na archiwum, które bierze rozmiar z tego, + co zawiera, bo nie ma rozmiaru, od którego można wyjść
@@ -310,11 +359,19 @@

Zamiana tego w test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Plik o wyniku unspecified jest tu pomijany, bo to, co twój system ma z nim zrobić, + jest twoją własną polityką. Jego wpis mówi dlaczego, a policy_options wymienia + możliwe odpowiedzi. +

Dobre odrzucenie jest czyste. Komunikat mówiący, co było nie tak, to odpowiedź, o którą chodzi. Błąd serwera, zawieszenie albo plik zapisany do połowy to wada, którą ten test ma znaleźć. diff --git a/web/public/pt-br/arquivos-de-teste-corrompidos/index.html b/web/public/pt-br/arquivos-de-teste-corrompidos/index.html index a499309c..acadc891 100644 --- a/web/public/pt-br/arquivos-de-teste-corrompidos/index.html +++ b/web/public/pt-br/arquivos-de-teste-corrompidos/index.html @@ -198,7 +198,9 @@

Um arquivo danificado continua com o tamanho que você pediu

o mesmo comando grava os mesmos bytes outra vez. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

As configurações vão depois de dois-pontos. A opção pode ser repetida, e os danos são aplicados na ordem em que você os escreve. Funciona com cada um dos 31 formatos. @@ -223,6 +225,18 @@

Quais danos existem?

+ + append-junk + Grava bytes aleatórios depois do fim do arquivo e mantém exatamente o tamanho que você pediu, como chega um arquivo com algo grudado no fim. + 64 + bytes + + + truncate + Corta o fim do arquivo e mantém exatamente o tamanho que você pediu, como um upload ou uma cópia interrompidos o deixam. + 4 + missing + zero-head Sobrescreve os primeiros bytes do arquivo com zeros, sem mexer no comprimento. A maioria dos leitores olha ali primeiro, então quase tudo percebe este dano. @@ -239,14 +253,25 @@

Quais danos existem?

bytes zero não é texto. Abaixo de quatro bytes, alguns formatos saem com um dano de que nenhum leitor reclama, e por isso a configuração começa em quatro.

+

+ truncate corta o fim do arquivo, como um upload ou uma cópia interrompidos o + deixam, e o arquivo continua com o tamanho que você pediu: ele é gerado maior e só o seu início + chega ao disco. missing diz que parte do arquivo inteiro some, metade se você não + disser outra coisa. +

+

+ append-junk grava bytes aleatórios depois do fim do arquivo, 64 se você não disser + outra coisa, como chega um arquivo com algo grudado no fim. O mesmo comando acrescenta os mesmos + bytes. +

O que o manifesto diz

Um manifesto que diz o que deve acontecer

- Cada arquivo danificado recebe uma entrada dizendo que o seu sistema deve rejeitá-lo, com o dano - registrado ao lado: + Onde o dano quebra as regras do formato, a entrada diz que o seu sistema deve rejeitá-lo, com o + dano registrado ao lado: 

"expected": {
   "outcome": "reject",
@@ -262,15 +287,40 @@ 
Um manifesto que diz o que deve acontecer

   }
 ]

- Dois pedidos são recusados antes de qualquer coisa ser gravada, porque cada um deixaria no disco um - arquivo que o manifesto descreve errado: + Onde os leitores divergem, a entrada diz isso em vez de adivinhar. Um arquivo de texto cortado + continua sendo texto, e a maioria dos visualizadores ignora bytes depois do fim de um PNG, então + a entrada deixa a resposta para o seu sistema, oferece as duas e diz por quê: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> lista os formatos em que um dano deixa a resposta para o seu + sistema, e por quê. +

+

+ Quatro pedidos são recusados antes de qualquer coisa ser gravada, porque cada um deixaria no + disco um arquivo que o manifesto descreve errado:

  • um arquivo menor do que o dano precisa, que sairia intacto
  • - expected: accept ao lado de um dano, porque nada poderia cumprir isso. Escreva - sanitize se o seu sistema deve consertar o arquivo, ou unspecified - se essa é justamente a pergunta que você faz + expected: accept ao lado de um dano que quebra as regras do formato, porque nada + que as siga aceita o arquivo. Onde a resposta depende do seu sistema, accept é + permitido. Escreva sanitize se o seu sistema deve consertar o arquivo, ou + unspecified se essa é justamente a pergunta que você faz +
    • +
  • + truncate depois de append-junk, porque cortaria os bytes + acrescentados. Coloque truncate primeiro +
    • +
  • + truncate ou append-junk em um arquivo compactado que tira o tamanho + do que contém, porque não há um tamanho de onde partir
@@ -308,11 +358,19 @@

Transformando em um teste

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Um arquivo cujo resultado é unspecified fica de fora aqui, porque o que o seu + sistema deve fazer com ele é a sua própria política. A entrada dele diz por quê, e + policy_options lista as respostas possíveis. +

Uma boa recusa é uma recusa limpa. Uma mensagem que diz o que estava errado é a resposta que você quer. Um erro de servidor, uma trava ou um arquivo salvo pela metade é o defeito que este teste diff --git a/web/public/ro/fisiere-de-test-corupte/index.html b/web/public/ro/fisiere-de-test-corupte/index.html index 060643e8..ba6c171b 100644 --- a/web/public/ro/fisiere-de-test-corupte/index.html +++ b/web/public/ro/fisiere-de-test-corupte/index.html @@ -199,7 +199,9 @@

Un fișier deteriorat are în continuare mărimea cerută

iar aceeași comandă scrie din nou aceiași octeți. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Setările se scriu după două puncte. Opțiunea poate fi repetată, iar deteriorările se aplică în ordinea în care le scrii. Funcționează cu fiecare dintre cele 31 formate. @@ -225,6 +227,18 @@

Ce deteriorări există?

+ + append-junk + Scrie octeți aleatori după sfârșitul fișierului și păstrează exact dimensiunea cerută, așa cum sosește un fișier cu ceva lipit la capăt. + 64 + bytes + + + truncate + Taie sfârșitul fișierului și păstrează exact dimensiunea cerută, așa cum îl lasă o încărcare sau o copiere întreruptă. + 4 + missing + zero-head Suprascrie cu zerouri primii octeți ai fișierului, fără să îi schimbe lungimea. Majoritatea cititorilor se uită întâi acolo, deci aproape orice observă această deteriorare. @@ -241,14 +255,25 @@

Ce deteriorări există?

octeți nuli nu este text. Sub patru octeți, unele formate ies cu o deteriorare de care nu se plânge niciun cititor, de aceea setarea începe de la patru.

+

+ truncate taie sfârșitul fișierului, așa cum îl lasă o încărcare sau o copiere + întreruptă, iar fișierul are totuși dimensiunea cerută: este generat mai mare și doar începutul + lui ajunge pe disc. missing spune ce parte din fișierul întreg lipsește, jumătate + dacă nu spui altceva. +

+

+ append-junk scrie octeți aleatori după sfârșitul fișierului, 64 dacă nu spui + altceva, așa cum sosește un fișier cu ceva lipit la capăt. Aceeași comandă adaugă aceiași + octeți. +

Ce spune manifestul

Un manifest care spune ce trebuie să se întâmple

- Fiecare fișier deteriorat primește o intrare care spune că sistemul tău trebuie să îl respingă, cu - deteriorarea notată alături: + Acolo unde deteriorarea încalcă regulile formatului, intrarea spune că sistemul tău trebuie să + îl respingă, cu deteriorarea notată alături: 

"expected": {
   "outcome": "reject",
@@ -264,15 +289,40 @@ 
Un manifest care spune ce trebuie să se întâmple

   }
 ]

- Două cereri sunt refuzate înainte să se scrie ceva, pentru că fiecare ar lăsa pe disc un fișier pe - care manifestul îl descrie greșit: + Acolo unde cititorii diferă, intrarea spune asta în loc să ghicească. Un fișier text tăiat + rămâne text, iar majoritatea vizualizatoarelor sar peste octeții de după sfârșitul unui PNG, așa + că intrarea lasă răspunsul sistemului tău, le oferă pe amândouă și spune de ce: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> enumeră formatele în care o deteriorare lasă răspunsul + sistemului tău, și de ce. +

+

+ Patru cereri sunt refuzate înainte să se scrie ceva, pentru că fiecare ar lăsa pe disc un fișier + pe care manifestul îl descrie greșit:

  • un fișier mai mic decât are nevoie deteriorarea, care ar ieși neschimbat
  • - expected: accept lângă o deteriorare, pentru că nimic nu ar putea îndeplini asta. Scrie - sanitize dacă sistemul tău trebuie să repare fișierul sau - unspecified dacă tocmai asta este întrebarea pe care o pui + expected: accept lângă o deteriorare care încalcă regulile formatului, pentru că + nimic din ce le respectă nu acceptă fișierul. Acolo unde răspunsul depinde de sistemul tău, + accept este permis. Scrie sanitize dacă sistemul tău trebuie să + repare fișierul sau unspecified dacă tocmai asta este întrebarea pe care o pui +
    • +
  • + truncate după append-junk, pentru că ar tăia octeții adăugați. Pune + truncate primul +
    • +
  • + truncate sau append-junk pe o arhivă care își ia dimensiunea din ce + conține, pentru că nu există o dimensiune de la care să se plece
@@ -310,11 +360,19 @@

Transformarea într-un test

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Un fișier al cărui rezultat este unspecified este lăsat deoparte aici, pentru că ce + trebuie să facă sistemul tău cu el ține de propria ta politică. Intrarea lui spune de ce, iar + policy_options enumeră răspunsurile posibile. +

O respingere bună este una curată. Un mesaj care spune ce a fost greșit este răspunsul pe care îl vrei. O eroare de server, o blocare sau un fișier salvat pe jumătate este defectul pe care acest diff --git a/web/public/ru/corrupt-test-files/index.html b/web/public/ru/corrupt-test-files/index.html index 638d3330..460ca1aa 100644 --- a/web/public/ru/corrupt-test-files/index.html +++ b/web/public/ru/corrupt-test-files/index.html @@ -199,7 +199,9 @@

Повреждённый файл остаётся нужного разме команда снова записывает те же байты. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Настройки пишутся после двоеточия. Параметр можно повторять, а повреждения применяются в том порядке, в каком вы их записали. Это работает с каждым из 31 форматов. @@ -225,6 +227,18 @@

Какие бывают повреждения?

+ + append-junk + Дописывает случайные байты после конца файла и сохраняет ровно тот размер, который вы указали, как приходит файл, к концу которого что-то прилипло. + 64 + bytes + + + truncate + Отрезает конец файла и сохраняет ровно тот размер, который вы указали, как оставляет файл прерванная загрузка или копирование. + 4 + missing + zero-head Перезаписывает первые байты файла нулями, не меняя его длину. Большинство программ чтения смотрят сначала туда, поэтому это повреждение замечает почти всё. @@ -242,14 +256,25 @@

Какие бывают повреждения?

форматов получается повреждение, на которое не жалуется ни одна программа чтения, поэтому настройка начинается с четырёх.

+

+ truncate отрезает конец файла, как оставляет его прерванная загрузка или + копирование, и файл всё равно имеет указанный вами размер: он создаётся больше, и на диск + попадает только его начало. missing говорит, какая часть всего файла пропадает, + половина, если не указать иное. +

+

+ append-junk дописывает случайные байты после конца файла, 64, если не указать иное, + как приходит файл, к концу которого что-то прилипло. Одна и та же команда дописывает одни и те + же байты. +

Что говорит манифест

Манифест, который говорит, что должно произойти

- Каждый повреждённый файл получает запись о том, что ваша система должна его отклонить, а рядом - записано повреждение: + Там, где повреждение нарушает правила формата, запись говорит, что ваша система должна его + отклонить, а рядом записано повреждение: 

"expected": {
   "outcome": "reject",
@@ -265,15 +290,40 @@ 
Манифест, который говорит, что должно про
   }
 ]

- Два запроса отклоняются до того, как что-либо записано, потому что каждый оставил бы на диске файл, - который манифест описывает неверно: + Там, где программы чтения расходятся, запись говорит об этом, а не гадает. Обрезанный текстовый + файл остаётся текстом, а большинство просмотрщиков пропускают байты после конца PNG, поэтому + запись оставляет ответ вашей системе, предлагает оба и объясняет почему: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> перечисляет форматы, в которых повреждение оставляет ответ + вашей системе, и причину. +

+

+ Четыре запроса отклоняются до того, как что-либо записано, потому что каждый оставил бы на диске + файл, который манифест описывает неверно:

  • файл меньше, чем нужно повреждению, который вышел бы нетронутым
  • - expected: accept рядом с повреждением, потому что ничто не могло бы этого выполнить. - Напишите sanitize, если ваша система должна починить файл, или - unspecified, если именно это вы и проверяете + expected: accept рядом с повреждением, которое нарушает правила формата, потому + что ничто, что им следует, такой файл не примет. Там, где ответ зависит от вашей системы, + accept разрешён. Напишите sanitize, если ваша система должна + починить файл, или unspecified, если именно это вы и проверяете +
    • +
  • + truncate после append-junk, потому что он отрезал бы дописанные + байты. Поставьте truncate первым +
    • +
  • + truncate или append-junk для архива, размер которого берётся из + содержимого, потому что нет размера, от которого можно отталкиваться
@@ -311,11 +361,19 @@

Превращаем это в тест

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Файл с результатом unspecified здесь пропускается, потому что то, что ваша система + должна с ним сделать, определяет ваша собственная политика. Его запись объясняет почему, а + policy_options перечисляет возможные ответы. +

Хороший отказ - это чистый отказ. Сообщение, говорящее, что было не так, - тот ответ, который вам нужен. Ошибка сервера, зависание или наполовину сохранённый файл - тот дефект, ради которого diff --git a/web/public/th/corrupt-test-files/index.html b/web/public/th/corrupt-test-files/index.html index 35cb69f4..f9ebb833 100644 --- a/web/public/th/corrupt-test-files/index.html +++ b/web/public/th/corrupt-test-files/index.html @@ -196,7 +196,9 @@

ไฟล์ที่เสียหายยังมีขนาดตา และคำสั่งเดิมเขียนไบต์เดิมซ้ำอีกครั้ง 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

การตั้งค่าเขียนหลังเครื่องหมายทวิภาค ตัวเลือกนี้ใส่ซ้ำได้ และความเสียหายจะถูกนำมาใช้ตามลำดับที่คุณเขียน ใช้ได้กับทั้ง 31 รูปแบบ @@ -221,6 +223,18 @@

มีความเสียหายแบบไหนบ้าง

+ + append-junk + เขียนไบต์สุ่มต่อจากจุดจบของไฟล์และคงขนาดที่คุณขอไว้พอดี เหมือนไฟล์ที่มาถึงพร้อมของบางอย่างติดอยู่ที่ท้าย + 64 + bytes + + + truncate + ตัดส่วนท้ายของไฟล์ทิ้งและคงขนาดที่คุณขอไว้พอดี เหมือนที่การอัปโหลดหรือการคัดลอกที่ถูกขัดจังหวะทิ้งไว้ + 4 + missing + zero-head เขียนทับไบต์แรกๆ ของไฟล์ด้วยศูนย์ โดยไม่แตะความยาว ตัวอ่านส่วนใหญ่มองตรงนั้นก่อน เกือบทุกอย่างจึงสังเกตเห็นความเสียหายนี้ @@ -237,13 +251,19 @@

มีความเสียหายแบบไหนบ้าง

ต่ำกว่าสี่ไบต์ บางรูปแบบออกมาพร้อมความเสียหายที่ไม่มีตัวอ่านใดบ่น นั่นคือเหตุผลที่การตั้งค่าเริ่มที่สี่

+

+ truncate ตัดส่วนท้ายของไฟล์ทิ้ง เหมือนที่การอัปโหลดหรือการคัดลอกที่ถูกขัดจังหวะทิ้งไว้ แต่ไฟล์ยังมีขนาดตามที่คุณขอ เพราะไฟล์ถูกสร้างให้ใหญ่กว่า และมีเพียงส่วนต้นที่ไปถึงดิสก์ missing บอกว่าไฟล์ทั้งไฟล์หายไปเท่าใด คือครึ่งหนึ่งหากคุณไม่ได้ระบุเป็นอย่างอื่น +

+

+ append-junk เขียนไบต์สุ่มต่อจากจุดจบของไฟล์ 64 ไบต์หากคุณไม่ได้ระบุเป็นอย่างอื่น เหมือนไฟล์ที่มาถึงพร้อมของบางอย่างติดอยู่ที่ท้าย คำสั่งเดียวกันจะต่อท้ายด้วยไบต์ชุดเดียวกัน +

แมนิเฟสต์บอกอะไร

แมนิเฟสต์ที่บอกว่าควรเกิดอะไรขึ้น

- ไฟล์ที่เสียหายทุกไฟล์ได้รายการที่บอกว่าระบบของคุณควรปฏิเสธ โดยบันทึกความเสียหายไว้ข้างๆ: + ในกรณีที่ความเสียหายละเมิดกฎของรูปแบบ รายการจะบอกว่าระบบของคุณควรปฏิเสธ โดยบันทึกความเสียหายไว้ข้างๆ: 

"expected": {
   "outcome": "reject",
@@ -259,14 +279,31 @@ 
แมนิเฟสต์ที่บอกว่าควรเกิด
   }
 ]

- คำขอสองแบบถูกปฏิเสธก่อนที่จะมีอะไรถูกเขียน เพราะแต่ละแบบจะทิ้งไฟล์ไว้บนดิสก์ที่แมนิเฟสต์อธิบายผิด: + ในกรณีที่ตัวอ่านแต่ละตัวต่างกัน รายการจะบอกเช่นนั้นแทนการเดา ไฟล์ข้อความที่ถูกตัดก็ยังเป็นข้อความ และโปรแกรมดูภาพส่วนใหญ่ข้ามไบต์ที่อยู่หลังจุดจบของ PNG รายการจึงปล่อยคำตอบให้ระบบของคุณ เสนอทั้งสองคำตอบ และบอกเหตุผล: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> แสดงรายการรูปแบบที่ความเสียหายปล่อยคำตอบให้ระบบของคุณ พร้อมเหตุผล +

+

+ คำขอสี่แบบถูกปฏิเสธก่อนที่จะมีอะไรถูกเขียน เพราะแต่ละแบบจะทิ้งไฟล์ไว้บนดิสก์ที่แมนิเฟสต์อธิบายผิด:

  • ไฟล์ที่เล็กกว่าที่ความเสียหายต้องการ ซึ่งจะออกมาโดยไม่เปลี่ยนแปลง
  • - expected: accept คู่กับความเสียหาย เพราะไม่มีอะไรทำให้เป็นจริงได้ เขียน - sanitize ถ้าระบบของคุณควรซ่อมไฟล์ หรือ unspecified - ถ้านั่นคือคำถามที่คุณกำลังถาม + expected: accept คู่กับความเสียหายที่ละเมิดกฎของรูปแบบ เพราะไม่มีสิ่งใดที่ทำตามกฎเหล่านั้นจะยอมรับไฟล์ ในกรณีที่คำตอบขึ้นกับระบบของคุณ accept ใช้ได้ เขียน sanitize ถ้าระบบของคุณควรซ่อมไฟล์ หรือ unspecified ถ้านั่นคือคำถามที่คุณถามอยู่พอดี +
    • +
  • + truncate หลัง append-junk เพราะจะตัดไบต์ที่เพิ่มเข้าไปทิ้ง ให้วาง truncate ไว้ก่อน +
    • +
  • + truncate หรือ append-junk กับไฟล์เก็บถาวรที่เอาขนาดมาจากสิ่งที่อยู่ข้างใน เพราะไม่มีขนาดให้ตั้งต้น
@@ -303,11 +340,17 @@

เปลี่ยนให้เป็นการทดสอบ

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ ไฟล์ที่ผลลัพธ์เป็น unspecified จะถูกข้ามไปตรงนี้ เพราะสิ่งที่ระบบของคุณควรทำกับไฟล์นั้นเป็นนโยบายของคุณเอง รายการของไฟล์บอกเหตุผล และ policy_options แสดงคำตอบที่เป็นไปได้ +

การปฏิเสธที่ดีคือการปฏิเสธที่สะอาด ข้อความที่บอกว่าผิดตรงไหนคือคำตอบที่คุณต้องการ ส่วนข้อผิดพลาดของเซิร์ฟเวอร์ การค้าง diff --git a/web/public/tr/bozuk-test-dosyalari/index.html b/web/public/tr/bozuk-test-dosyalari/index.html index ac7df427..207cee25 100644 --- a/web/public/tr/bozuk-test-dosyalari/index.html +++ b/web/public/tr/bozuk-test-dosyalari/index.html @@ -198,7 +198,9 @@

Hasarlı bir dosya yine istediğiniz boyuttadır

aynı baytları yazar. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Ayarlar iki noktadan sonra yazılır. Seçenek tekrarlanabilir ve hasarlar yazdığınız sırayla uygulanır. 31 biçimin her biriyle çalışır. @@ -224,6 +226,18 @@

Hangi hasarlar var?

+ + append-junk + Dosyanın sonundan sonra rastgele baytlar yazar ve istediğiniz boyutu tam olarak korur, tıpkı sonuna bir şey yapışmış bir dosyanın gelişi gibi. + 64 + bytes + + + truncate + Dosyanın sonunu keser ve istediğiniz boyutu tam olarak korur, tıpkı yarıda kalan bir yüklemenin ya da kopyalamanın bıraktığı gibi. + 4 + missing + zero-head Dosyanın ilk baytlarını uzunluğuna dokunmadan sıfırlarla ezer. Okuyucuların çoğu önce oraya bakar, bu yüzden hemen her şey bu hasarı fark eder. @@ -240,14 +254,25 @@

Hangi hasarlar var?

Dört baytın altında bazı biçimler hiçbir okuyucunun şikâyet etmediği bir hasarla çıkar, ayarın dörtten başlamasının nedeni budur.

+

+ truncate dosyanın sonunu keser, tıpkı yarıda kalan bir yüklemenin ya da + kopyalamanın bıraktığı gibi, ve dosya yine de istediğiniz boyuttadır: daha büyük üretilir ve + diske yalnızca başı ulaşır. missing, dosyanın tamamının ne kadarının kaybolduğunu + söyler, başka bir şey belirtmezseniz yarısı. +

+

+ append-junk dosyanın sonundan sonra rastgele baytlar yazar, başka bir şey + belirtmezseniz 64, tıpkı sonuna bir şey yapışmış bir dosyanın gelişi gibi. Aynı komut aynı + baytları ekler. +

Bildirim ne söyler

Ne olması gerektiğini söyleyen bir bildirim

- Hasarlı her dosya, sisteminizin onu reddetmesi gerektiğini söyleyen bir kayıt alır ve hasar yanına - yazılır: + Hasarın biçimin kurallarını çiğnediği yerde kayıt, sisteminizin dosyayı reddetmesi gerektiğini + söyler ve hasar yanına yazılır: 

"expected": {
   "outcome": "reject",
@@ -263,15 +288,40 @@ 
Ne olması gerektiğini söyleyen bir bildirim

   }
 ]

- İki istek, bir şey yazılmadan önce reddedilir, çünkü her biri diskte bildirimin yanlış tarif ettiği - bir dosya bırakırdı: + Okuyucuların ayrıştığı yerde kayıt, tahmin yürütmek yerine bunu söyler. Kesilmiş bir metin + dosyası yine metindir ve çoğu görüntüleyici bir PNG'nin sonundan sonraki baytları atlar, bu + yüzden kayıt yanıtı sisteminize bırakır, ikisini de sunar ve nedenini söyler: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id>, bir hasarın yanıtı sisteminize bıraktığı biçimleri nedeniyle + birlikte listeler. +

+

+ Dört istek, bir şey yazılmadan önce reddedilir, çünkü her biri diskte bildirimin yanlış tarif + ettiği bir dosya bırakırdı:

  • hasarın gerektirdiğinden küçük bir dosya, ki değişmeden çıkardı
  • - Bir hasarın yanında expected: accept, çünkü hiçbir şey bunu karşılayamaz. Sisteminiz - dosyayı onarmalıysa sanitize, tam da bunu soruyorsanız unspecified - yazın + Biçimin kurallarını çiğneyen bir hasarın yanında expected: accept, çünkü bu + kurallara uyan hiçbir şey dosyayı kabul etmez. Yanıtın sisteminize bağlı olduğu yerde + accept serbesttir. Sisteminiz dosyayı onarmalıysa sanitize, tam da + bunu soruyorsanız unspecified yazın +
    • +
  • + append-junk sonrasında truncate, çünkü eklenen baytları keserdi. + truncate öğesini başa koyun +
    • +
  • + İçeriğinden boyut alan bir arşivde truncate ya da append-junk, çünkü + yola çıkılacak bir boyut yoktur
@@ -309,11 +359,19 @@

Bunu bir teste dönüştürmek

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Sonucu unspecified olan bir dosya burada atlanır, çünkü sisteminizin onunla ne + yapması gerektiği kendi politikanıza bağlıdır. Kaydı nedenini söyler, + policy_options ise olası yanıtları listeler. +

İyi bir ret temiz bir rettir. Neyin yanlış olduğunu söyleyen bir ileti istediğiniz yanıttır. Sunucu hatası, takılma ya da yarım kaydedilmiş dosya, bu testin bulmak için var olduğu kusurdur. diff --git a/web/public/uk/corrupt-test-files/index.html b/web/public/uk/corrupt-test-files/index.html index 20206f96..71a15cdc 100644 --- a/web/public/uk/corrupt-test-files/index.html +++ b/web/public/uk/corrupt-test-files/index.html @@ -199,7 +199,9 @@

Пошкоджений файл лишається потрібного ро сама команда знову записує ті самі байти. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Налаштування пишуться після двокрапки. Параметр можна повторювати, а пошкодження застосовуються в тому порядку, в якому ви їх записали. Це працює з кожним із 31 форматів. @@ -225,6 +227,18 @@

Які бувають пошкодження?

+ + append-junk + Дописує випадкові байти після кінця файлу й зберігає саме той розмір, який ви вказали, як приходить файл, до кінця якого щось прилипло. + 64 + bytes + + + truncate + Відрізає кінець файлу й зберігає саме той розмір, який ви вказали, як залишає файл перерване завантаження чи копіювання. + 4 + missing + zero-head Перезаписує перші байти файлу нулями, не змінюючи його довжину. Більшість програм читання дивляться спочатку туди, тому це пошкодження помічає майже все. @@ -241,14 +255,24 @@

Які бувають пошкодження?

байтів не є текстом. Менше ніж чотири байти - і в деяких форматів виходить пошкодження, на яке не скаржиться жодна програма читання, тому налаштування починається з чотирьох.

+

+ truncate відрізає кінець файлу, як залишає його перерване завантаження чи + копіювання, і файл однаково має вказаний вами розмір: він створюється більшим, і на диск + потрапляє лише його початок. missing каже, яка частина всього файлу зникає, + половина, якщо не вказати інше. +

+

+ append-junk дописує випадкові байти після кінця файлу, 64, якщо не вказати інше, як + приходить файл, до кінця якого щось прилипло. Та сама команда дописує ті самі байти. +

Що каже маніфест

Маніфест, який каже, що має статися

- Кожен пошкоджений файл отримує запис про те, що ваша система має його відхилити, а поруч записано - пошкодження: + Там, де пошкодження порушує правила формату, запис каже, що ваша система має його відхилити, а + поруч записано пошкодження: 

"expected": {
   "outcome": "reject",
@@ -264,15 +288,40 @@ 
Маніфест, який каже, що має статися

   }
 ]

- Два запити відхиляються, перш ніж щось буде записано, бо кожен залишив би на диску файл, який + Там, де програми читання розходяться, запис каже про це, а не вгадує. Обрізаний текстовий файл + лишається текстом, а більшість переглядачів пропускають байти після кінця PNG, тому запис + залишає відповідь вашій системі, пропонує обидві й пояснює чому: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> перелічує формати, у яких пошкодження залишає відповідь вашій + системі, і причину. +

+

+ Чотири запити відхиляються, перш ніж щось буде записано, бо кожен залишив би на диску файл, який маніфест описує хибно:

  • файл менший, ніж потрібно пошкодженню, який вийшов би недоторканим
  • - expected: accept поруч із пошкодженням, бо ніщо не могло б цього виконати. Напишіть - sanitize, якщо ваша система має полагодити файл, або unspecified, - якщо саме це ви й перевіряєте + expected: accept поруч із пошкодженням, яке порушує правила формату, бо ніщо, що + їх дотримується, такого файлу не прийме. Там, де відповідь залежить від вашої системи, + accept дозволено. Напишіть sanitize, якщо ваша система має + полагодити файл, або unspecified, якщо саме це ви й перевіряєте +
    • +
  • + truncate після append-junk, бо він відрізав би дописані байти. + Поставте truncate першим +
    • +
  • + truncate або append-junk для архіву, розмір якого береться зі + вмісту, бо немає розміру, від якого можна відштовхнутися
@@ -310,11 +359,19 @@

Перетворюємо це на тест

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Файл із результатом unspecified тут пропускається, бо те, що ваша система має з ним + зробити, визначає ваша власна політика. Його запис пояснює чому, а policy_options + перелічує можливі відповіді. +

Гарна відмова - це чиста відмова. Повідомлення, що каже, що було не так, - та відповідь, яка вам потрібна. Помилка сервера, зависання або наполовину збережений файл - той дефект, заради якого diff --git a/web/public/vi/tep-kiem-thu-bi-hong/index.html b/web/public/vi/tep-kiem-thu-bi-hong/index.html index 16fc5787..f9fb5d03 100644 --- a/web/public/vi/tep-kiem-thu-bi-hong/index.html +++ b/web/public/vi/tep-kiem-thu-bi-hong/index.html @@ -196,7 +196,9 @@

Tệp bị hỏng vẫn có kích thước bạn đã yêu cầu

cầu, và cùng một lệnh ghi lại đúng các byte đó. 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

Cài đặt viết sau dấu hai chấm. Tùy chọn có thể lặp lại, và các hỏng hóc được áp dụng theo thứ tự bạn viết. Nó dùng được với từng định dạng trong 31 định dạng. @@ -222,6 +224,18 @@

Có những kiểu hỏng nào?

+ + append-junk + Ghi các byte ngẫu nhiên sau phần cuối tệp và giữ đúng kích thước bạn yêu cầu, giống như một tệp đến nơi với thứ gì đó dính ở cuối. + 64 + bytes + + + truncate + Cắt bỏ phần cuối tệp và giữ đúng kích thước bạn yêu cầu, giống như một lần tải lên hoặc sao chép bị ngắt giữa chừng để lại. + 4 + missing + zero-head Ghi đè các byte đầu của tệp bằng số không, giữ nguyên độ dài. Hầu hết trình đọc nhìn vào đó trước, nên gần như mọi thứ đều nhận ra hỏng hóc này. @@ -238,14 +252,24 @@

Có những kiểu hỏng nào?

bốn byte, một số định dạng ra với hỏng hóc mà không trình đọc nào phàn nàn, đó là lý do cài đặt bắt đầu từ bốn.

+

+ truncate cắt bỏ phần cuối tệp, giống như một lần tải lên hoặc sao chép bị ngắt giữa + chừng để lại, và tệp vẫn có kích thước bạn yêu cầu: tệp được tạo lớn hơn và chỉ phần đầu của nó + tới đĩa. missing cho biết bao nhiêu phần của toàn bộ tệp bị mất, một nửa nếu bạn + không nói khác. +

+

+ append-junk ghi các byte ngẫu nhiên sau phần cuối tệp, 64 nếu bạn không nói khác, + giống như một tệp đến nơi với thứ gì đó dính ở cuối. Cùng một lệnh sẽ thêm cùng những byte đó. +

Manifest nói gì

Một manifest nói điều phải xảy ra

- Mỗi tệp bị hỏng nhận một mục nói rằng hệ thống của bạn phải từ chối nó, với hỏng hóc được ghi bên - cạnh: + Ở chỗ hỏng hóc vi phạm quy tắc của định dạng, mục nói rằng hệ thống của bạn phải từ chối tệp, + với hỏng hóc được ghi bên cạnh: 

"expected": {
   "outcome": "reject",
@@ -261,15 +285,40 @@ 
Một manifest nói điều phải xảy ra

   }
 ]

- Hai yêu cầu bị từ chối trước khi có gì được ghi, vì mỗi yêu cầu sẽ để lại trên đĩa một tệp mà + Ở chỗ các trình đọc khác nhau, mục nói rõ điều đó thay vì đoán. Một tệp văn bản bị cắt vẫn là + văn bản, và hầu hết trình xem ảnh bỏ qua các byte sau phần cuối của PNG, nên mục để câu trả lời + cho hệ thống của bạn, đưa ra cả hai và nói lý do: +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> liệt kê các định dạng mà ở đó một hỏng hóc để câu trả lời cho + hệ thống của bạn, kèm lý do. +

+

+ Bốn yêu cầu bị từ chối trước khi có gì được ghi, vì mỗi yêu cầu sẽ để lại trên đĩa một tệp mà manifest mô tả sai:

  • một tệp nhỏ hơn mức hỏng hóc cần, sẽ ra mà không bị đổi
  • - expected: accept cạnh một hỏng hóc, vì không gì có thể đáp ứng được. Viết - sanitize nếu hệ thống của bạn phải sửa tệp, hoặc unspecified nếu đó - chính là câu hỏi bạn đang đặt ra + expected: accept cạnh một hỏng hóc vi phạm quy tắc của định dạng, vì không thứ gì + tuân theo các quy tắc đó chấp nhận tệp. Ở chỗ câu trả lời tùy vào hệ thống của bạn, + accept được phép. Viết sanitize nếu hệ thống của bạn phải sửa tệp, + hoặc unspecified nếu đó chính là câu hỏi bạn đang đặt ra +
    • +
  • + truncate sau append-junk, vì nó sẽ cắt mất các byte đã thêm. Hãy đặt + truncate trước +
    • +
  • + truncate hoặc append-junk trên một tệp nén lấy kích thước từ những + gì nó chứa, vì không có kích thước nào để bắt đầu
@@ -307,11 +356,19 @@

Biến nó thành một bài kiểm thử

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ Một tệp có kết quả unspecified được bỏ qua ở đây, vì việc hệ thống của bạn phải làm + gì với nó là chính sách của riêng bạn. Mục của nó nói lý do, và policy_options liệt + kê các câu trả lời có thể. +

Một lần từ chối tốt là một lần từ chối gọn. Một thông báo nói điều gì sai là câu trả lời bạn muốn. Lỗi máy chủ, treo máy hoặc một tệp lưu dở là khiếm khuyết mà bài kiểm thử này sinh ra để tìm. diff --git a/web/public/zh-hans/corrupt-test-files/index.html b/web/public/zh-hans/corrupt-test-files/index.html index 39d6f9f6..9d1ec75a 100644 --- a/web/public/zh-hans/corrupt-test-files/index.html +++ b/web/public/zh-hans/corrupt-test-files/index.html @@ -187,7 +187,9 @@

损坏的文件仍然是你要求的大小

文件先正常生成，再在写入磁盘的途中被弄坏。它保持你要求的大小，同一条命令再次写出的字节也完全相同。 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

设置写在冒号之后。该选项可以重复，损坏按你写下的顺序依次应用。它适用于全部 31 种格式。

@@ -211,6 +213,18 @@

有哪些损坏方式？

+ + append-junk + 在文件末尾之后写入随机字节，并完全保持你要求的大小，就像末尾粘着东西送来的文件那样。 + 64 + bytes + + + truncate + 截掉文件的末尾，并完全保持你要求的大小，就像中断的上传或复制留下的那样。 + 4 + missing + zero-head 用零覆盖文件开头的若干字节，长度保持不变。大多数读取器最先看的就是那里，所以几乎任何东西都会发现这种损坏。 @@ -224,13 +238,19 @@

有哪些损坏方式？

zero-head 把文件开头写成零。大多数读取器最先看的就是那里，也就是说明文件是什么的签名和文件头，所以几乎任何读取器都会发现。纯文本和日志没有签名，同样会被拒绝，因为一串零字节不是文本。低于四个字节时，有些格式产生的损坏没有任何读取器会抱怨，这就是该设置从四开始的原因。

+

+ truncate 截掉文件的末尾，就像中断的上传或复制留下的那样，而文件仍然是你要求的大小：它先按更大的尺寸生成，只有开头部分写入磁盘。missing 表示整个文件丢失了多少，不另行指定时为一半。 +

+

+ append-junk 在文件末尾之后写入随机字节，不另行指定时为 64 个，就像末尾粘着东西送来的文件那样。同一条命令追加的字节也相同。 +

清单怎么说

一份说明应发生什么的清单

- 每个损坏的文件都会得到一条记录，说明你的系统应该拒绝它，损坏方式记在旁边： + 当损坏违反该格式的规则时，记录会说明你的系统应该拒绝它，损坏方式记在旁边： 

"expected": {
   "outcome": "reject",
@@ -246,13 +266,31 @@ 
一份说明应发生什么的清单

   }
 ]

- 有两种请求会在写入任何内容之前被拒绝，因为各自都会在磁盘上留下一个清单描述有误的文件： + 当各读取器的处理不一致时，记录会如实说明，而不是去猜测。被截短的文本文件仍然是文本，而大多数看图软件会跳过 PNG 结尾之后的字节，所以记录把答案留给你的系统，给出两种答案，并说明原因： +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> 会列出某种损坏把答案留给你的系统的格式及其原因。 +

+

+ 有四种请求会在写入任何内容之前被拒绝，因为各自都会在磁盘上留下一个清单描述有误的文件：

  • 比损坏所需更小的文件，它会原样输出
  • - 在损坏旁边写 expected: accept，因为没有什么能满足它。如果你的系统本应修复文件，请写 - sanitize，如果你要问的恰恰就是这一点，请写 unspecified + 在违反格式规则的损坏旁边写 expected: accept，因为遵守这些规则的读取器都不会接受这个文件。当答案取决于你的系统时，可以写 accept。如果你的系统本应修复文件，请写 sanitize，如果你要问的恰恰就是这一点，请写 unspecified +
    • +
  • + 在 append-junk 之后使用 truncate，因为它会把追加的字节截掉。请把 truncate 放在前面 +
    • +
  • + 对按所含内容决定大小的压缩包使用 truncate 或 append-junk，因为没有可以作为出发点的大小
@@ -288,11 +326,17 @@

把它变成测试

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 结果为 unspecified 的文件在这里被跳过，因为你的系统该如何处理它由你自己的策略决定。它的记录说明了原因，policy_options 列出了可能的答案。 +

好的拒绝是干净的拒绝。一条说明错在哪里的消息，就是你想要的答案。服务器错误、卡死或只保存了一半的文件，正是这个测试要找出来的缺陷。

diff --git a/web/public/zh-hant/corrupt-test-files/index.html b/web/public/zh-hant/corrupt-test-files/index.html index 4d8fa363..401e769c 100644 --- a/web/public/zh-hant/corrupt-test-files/index.html +++ b/web/public/zh-hant/corrupt-test-files/index.html @@ -187,7 +187,9 @@

損毀的檔案仍然是你要求的大小

檔案先正常產生，再在寫入磁碟的途中被弄壞。它保持你要求的大小，同一個指令再次寫出的位元組也完全相同。 

tfg generate --format png --size 2mb --damage zero-head --out ./out
-tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken
+tfg generate --format pdf --size 1mb --count 5 --damage zero-head:bytes=16 --out ./broken +tfg generate --format zip --size 5mb --damage truncate:missing=30 --out ./cut +tfg generate --format jpg --size 500kb --damage append-junk --out ./tail

設定寫在冒號之後。該選項可以重複，損毀依你寫下的順序逐一套用。它適用於全部 31 種格式。

@@ -211,6 +213,18 @@

有哪些損毀方式？

+ + append-junk + 在檔案結尾之後寫入隨機位元組，並完全保持你要求的大小，就像結尾黏著東西送來的檔案那樣。 + 64 + bytes + + + truncate + 截掉檔案的結尾，並完全保持你要求的大小，就像中斷的上傳或複製留下的那樣。 + 4 + missing + zero-head 用零覆蓋檔案開頭的若干位元組，長度保持不變。大多數讀取器最先看的就是那裡，所以幾乎任何東西都會發現這種損毀。 @@ -224,13 +238,19 @@

有哪些損毀方式？

zero-head 把檔案開頭寫成零。大多數讀取器最先看的就是那裡，也就是說明檔案是什麼的簽章和檔頭，所以幾乎任何讀取器都會發現。純文字和日誌沒有簽章，同樣會被拒絕，因為一串零位元組不是文字。低於四個位元組時，有些格式產生的損毀沒有任何讀取器會抱怨，這就是該設定從四開始的原因。

+

+ truncate 截掉檔案的結尾，就像中斷的上傳或複製留下的那樣，而檔案仍然是你要求的大小：它先按更大的尺寸產生，只有開頭部分寫入磁碟。missing 表示整個檔案遺失了多少，不另行指定時為一半。 +

+

+ append-junk 在檔案結尾之後寫入隨機位元組，不另行指定時為 64 個，就像結尾黏著東西送來的檔案那樣。同一道指令附加的位元組也相同。 +

清單怎麼說

一份說明應發生什麼的清單

- 每個損毀的檔案都會得到一筆記錄，說明你的系統應該拒絕它，損毀方式記在旁邊： + 當損毀違反該格式的規則時，記錄會說明你的系統應該拒絕它，損毀方式記在旁邊： 

"expected": {
   "outcome": "reject",
@@ -246,13 +266,31 @@ 
一份說明應發生什麼的清單

   }
 ]

- 有兩種請求會在寫入任何內容之前被拒絕，因為各自都會在磁碟上留下一個清單描述有誤的檔案： + 當各讀取器的處理不一致時，記錄會如實說明，而不是去猜測。被截短的文字檔仍然是文字，而大多數看圖軟體會跳過 PNG 結尾之後的位元組，所以記錄把答案留給你的系統，給出兩種答案，並說明原因： +

+ 
"expected": {
+  "outcome": "unspecified",
+  "reason": "content_malformed",
+  "detail": "A PNG ends at its IEND chunk and most viewers skip what follows it.",
+  "confidence": "policy_dependent",
+  "policy_options": ["accept", "reject"]
+}
+

+ tfg damage <id> 會列出某種損毀把答案留給你的系統的格式及其原因。 +

+

+ 有四種請求會在寫入任何內容之前被拒絕，因為各自都會在磁碟上留下一個清單描述有誤的檔案：

  • 比損毀所需更小的檔案，它會原樣輸出
  • - 在損毀旁邊寫 expected: accept，因為沒有什麼能滿足它。如果你的系統本應修復檔案，請寫 - sanitize，如果你要問的恰恰就是這一點，請寫 unspecified + 在違反格式規則的損毀旁邊寫 expected: accept，因為遵守這些規則的讀取器都不會接受這個檔案。當答案取決於你的系統時，可以寫 accept。如果你的系統本應修復檔案，請寫 sanitize，如果你要問的恰恰就是這一點，請寫 unspecified +
    • +
  • + 在 append-junk 之後使用 truncate，因為它會把附加的位元組截掉。請把 truncate 放在前面 +
    • +
  • + 對依所含內容決定大小的壓縮檔使用 truncate 或 append-junk，因為沒有可以作為出發點的大小
@@ -288,11 +326,17 @@

把它變成測試

manifest = json.load(open(os.path.join(directory, "manifest.json"))) for entry in manifest["files"]: + outcome = entry["expected"]["outcome"] + if outcome == "unspecified": + continue response = upload(os.path.join(directory, entry["path"])) - if entry["expected"]["outcome"] == "reject": + if outcome == "reject": assert not response.ok else: assert response.ok +

+ 結果為 unspecified 的檔案在這裡被略過，因為你的系統該如何處理它由你自己的策略決定。它的記錄說明了原因，policy_options 列出了可能的答案。 +

好的拒絕是乾淨的拒絕。一則說明錯在哪裡的訊息，就是你想要的答案。伺服器錯誤、卡死或只儲存了一半的檔案，正是這個測試要找出來的缺陷。