diff --git a/in-depth.html b/in-depth.html index 543eb2e2..00d51235 100644 --- a/in-depth.html +++ b/in-depth.html @@ -24,6 +24,7 @@ --> + @@ -32,769 +33,810 @@ - -

-
- JetStream 3 -
-

-
-
-

In-Depth Analysis

- -

- JetStream 3 combines together a variety of JavaScript and WebAssembly benchmarks, covering a variety of - advanced workloads and programming techniques, and reports a single score that - balances them using a geometric mean. -

- -

- Each benchmark measures a distinct workload, and no single optimization - technique is sufficient to speed up all benchmarks. Some benchmarks demonstrate tradeoffs, and - aggressive or specialized optimizations for one benchmark might make another benchmark slower. - JetStream 3 rewards browsers that start up quickly, execute code quickly, and continue running smoothly. -

- -

- Each benchmark in JetStream 3 computes its own individual score. - Scores in JetStream are dimensionless floats, where a higher score is better. - JetStream weighs each benchmark equally, taking the geometric mean over each individual benchmark's score to compute the overall JetStream 3 score. - The geometric mean ensures that a multiplicative improvement of any individual score has the same effect on the aggregated score, regardless of the absolute value of the individual score. - For example, an improvement by 5% of the sub score of benchmark A has the same effect on the total score as an improvement by 5% of the sub score of benchmark B. -

- -

- It's not enough to just measure the total running time of a workload. - Browsers may perform differently for the same JavaScript workload depending on how many times it - has run. For example, garbage collection runs periodically, making some iterations take longer than - others. Code that runs repeatedly gets optimized by the browser, so the first iteration - of any workload is usually more expensive than the rest. -

- -

- For most of the JavaScript and WebAssembly benchmarks in JetStream 3, individual scores - equally weigh startup performance, worst case performance, and average case - performance. These three metrics are crucial to running performant JavaScript - in the browser. Fast startup times lead browsers to loading pages more quickly. Good - worst case performance ensures web applications can run without hiccups. Fast average - case performance makes it so that the most advanced web applications can run at all. -

- -

- An important component of JetStream 1 were the asm.js subset of benchmarks. With the release - of WebAssembly, the importance of asm.js has lessened since many users of asm.js are - now using WebAssembly. JetStream 3 has converted many of the asm.js benchmarks from - JetStream 1 into WebAssembly. -

- -

- All but one of JetStream 3's JavaScript benchmarks run for N iterations, where - N is usually 120. JetStream 3 reports the startup score as the time it takes to run the first iteration. - The worst case score is the average of the worst M iterations, excluding the first iteration. - M is always less than N, and is usually 4. The average case score is the average - of all but the first iteration. These three scores are weighed equally using the geometric - mean. -

- -

- JetStream 3 also includes a JavaScript benchmark named WSL. WSL is an implementation of a - GPU shading language written in JavaScript. WSL does not use the above mechanism for scoring - because it has a long running time. Instead, the WSL benchmark computes its score as the - geometric mean over two metrics: the time it takes to compile the WSL standard library, and the time - it takes to run through the WSL specification test suite. -

- -

- JetStream 3 includes parts of these benchmark suites that came before it: SunSpider, - Octane 2, JetStream 1, - ARES-6, Web Tooling Benchmark, and benchmarks inspired by Kraken. - JetStream 3 also includes a new set of benchmarks that measure the performance of WebAssembly, Web Workers, - Promises, async iteration, unicode regular expressions, and JavaScript parsing. -

- -

- JetStream 3 includes several benchmarks from earlier JetStream versions, but updates the benchmark driver to - improve score stability. This is achieved by pre-fetching network resources prior to running - the benchmarks. This can reduce perturbations on the measurement of JavaScript execution - time due to second order effects of pause times induced by network latency. -

- -

- Note that scores from JetStream 3 are not comparable to scores to other versions - of any JetStream benchmark. -

- -

- JetStream 3 has 64 subtests: -

- -
- -
WSL
-
- WSL is an implementation of a GPU shading language written in JavaScript. - WSL measures the time it takes to compile the WSL standard library and the time - it takes to run through the WSL specification test suite. - Source code: WSL -
- -
UniPoker
-
- UniPoker is a 5 card stud poker simulation using the Unicode playing card code points, U+1F0A1..U+1F0DE, - as the card representation in code. Scoring of hands is done with three regular expressions, one to check - for a flush, one to check for straights, and one to check for pairs, three of a kind, and four of a kind. - Source code: poker.js -
- -
uglify-js-wtb
-
- UglifyJS is a JavaScript parser, minifier, compressor, and beautifier toolkit. It is commonly - used to minimize JavaScript bundles. - This benchmark runs UglifyJS on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: uglify.js -
- -
typescript
-
- Tests how quickly Microsoft's TypeScript compiler can - compile itself. More than anything else, this tests how quickly a JavaScript runtime can optimize - a large pile of code. - A similar version of this benchmark was previously published in Octane version 2. - Source code: typescript.js -
- -
tsf-wasm
-
- Runs Filip Pizlo's — of the WebKit team — implementation of a Typed Stream Format - in WebAssembly. The original code is compiled from C to WebAssembly using Emscripten. - Source code: TSF -
- -
tfjs-wasm
-
- Tests Tensorflow.js pre-trained machine learning models supported by WebAssembly backend. The current benchmark includes models: mobilenet, knn-classifier, coco-ssd, universal-sentence-encoder. Source code: TFJS. -
- -
argon2-wasm
-
- Tests Argon2, a password-hashing function (the winner of Password Hashing Competition), in WebAssembly. This is test is based on argon2-browser library. Source code: ARGON2. -
- -
tagcloud-SP
-
- Parses JSON and generates markup for a tag - cloud view of the data. Written by Maciej Stachowiak of the WebKit team. Exercises string - parsing and manipulation. A similar version of this benchmark was originally published in SunSpider. - Source code: tagcloud.js -
- -
string-unpack-code-SP
-
- This benchmark unpacks various minified JavaScript libraries. It stresses the speed of various string manipulation - operations. - A similar version of this benchmark was previously published in SunSpider. - Source code: string-unpack-code.js -
- -
stanford-crypto-sha256
-
- Measures the performance of the SHA256 hashing algorithm as implemented by the Stanford JavaScript Crypto Library. This benchmark stresses numeric analysis and array access. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: stanford-crypto-sha256.js -
- -
stanford-crypto-pbkdf2
-
- Measures the performance of the PBKDF2 hashing algorithm as implemented by the Stanford JavaScript Crypto Library. This benchmark stresses numeric analysis and array access. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: stanford-crypto-pbkdf2.js -
- -
stanford-crypto-aes
-
- Measures the performance of the AES hashing algorithm as implemented by the Stanford JavaScript Crypto Library. This benchmark stresses numeric analysis and array access. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: stanford-crypto-aes.js -
- -
splay
-
- Tests the manipulation of splay trees - represented using plain JavaScript objects. This benchmark stresses the performance of the garbage collector. - A similar version of this benchmark was previously published in Octane version 2. - Source code: splay.js -
- -
segmentation
-
- Uses Web Workers to parallelize the computation of a - time series segmentation algorithm over a sample data set. This code is adapted from an algorithm used in the - WebKit performance dashboard. - Source code: segmentation.js -
- -
richards
-
- Martin Richard's system language - benchmark ported to JavaScript. Tests object property access performance. - A similar version of this benchmark was previously published in Octane version 2. - Source code: richards.js -
- -
richards-wasm
-
- Martin Richard's system language - benchmark compiled to a hybrid of WebAssembly and JavaScript. It stresses how quickly - JavaScript can call into WebAssembly code. - Source code: richards.c, richards.js -
- -
8bitbench-wasm
-
- A simple 8-bit emulator targeting wasm, written in rust. It aims to represent what an emulator or small game might act like in the real world. -
- Attribution: See 8bitbench/ABOUT.md -
- Source code: 8bitbench/js3harness.js, based off https://github.com/justinmichaud/8bitbench -
- -
regexp
-
- Collection of regular expressions found by the V8 team in 2010, curated into a benchmark. - A similar version of this benchmark was previously published in Octane version 2. - Source code: regexp.js -
- -
regex-dna-SP
-
- Regular-expression-based solution to DNA manipulation from - The Great Computer Language Shootout, - contributed by Jesse Millikan. - A similar version of this benchmark was previously published in SunSpider. - Source code: regex-dna.js -
- -
raytrace
-
- Ray tracer written in JavaScript using ES6 classes. - Tests object construction performance and floating point math. - A similar version of this benchmark was previously published in Octane version 2. - Source code: raytrace.js -
- -
raytrace-private-class-fields
-
- Ray tracer written in JavaScript using ES6 classes and private fields. - Tests object construction performance, along with setting default values of private fields, their access speed, and floating point math. - A similar version of this benchmark was previously published in Octane version 2. - Source code: raytrace-private-class-fields.js -
- -
raytrace-public-class-fields
-
- Ray tracer written in JavaScript using ES6 classes and public fields. - Tests object construction performance, along with setting default values of public fields, and floating point math. - A similar version of this benchmark was previously published in Octane version 2. - Source code: raytrace-public-class-fields.js -
- -
quicksort-wasm
-
- Quicksort benchmark, compiled to WebAssembly with Emscripten. - The original C version of this benchmark was previously published in the LLVM test suite. - Source code: quicksort.c, quicksort.js -
- -
prepack-wtb
-
- Prepack is a tool that optimizes JavaScript source code by performing computations at compile - time instead of run time where possible. - This benchmark runs Prepack on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: prepack.js -
- -
pdfjs
-
- Mozilla's PDF reader written in JavaScript. - This benchmark emphasizes array manipulation and bit operations. - A similar version of this benchmark was previously published in Octane version 2. - Source code: pdfjs.js -
- -
OfflineAssembler
-
- Offline Assembler is the lexer, parser, and AST layer of the offline assembler for JavaScriptCore. - It has been ported to JavaScript from the original Ruby implementation. This test stresses regular expression - performance. - Source code: OfflineAssembler.js -
- -
octane-code-load
-
- Test of code load speed of the jQuery and Closure libraries. Because this test allows - caching, this is representative of revisiting the same website. - A similar version of this benchmark was previously published in Octane version 2. - Source code: code-load.js -
- - -
- Fluid simulation written by Oliver Hunt. Emphasizes floating point array performance. - A similar version of this benchmark was previously published in Octane version 2. - Source code: navier-stokes.js -
- -
n-body-SP
-
- Classic solar system simulation benchmark from - The Great Computer Language Shootout, - contributed by Isaac Guy. Tests math and object access performance. - A similar version of this benchmark was previously published in SunSpider. - Source code: n-body.js -
- -
multi-inspector-code-load
-
- Measures the repeated parsing of a modern JavaScript code base: WebKit's Web Inspector. - Because this test allows caching, this is representative of revisiting the same website. - Source code: code-multi-load.js -
- -
ML
-
- ML is an implementation of a feedforward neural network. - The benchmark trains several networks using different activation functions - and several sample data sets. ML makes heavy use of classes. It relies on the ml-matrix library and does non-trivial matrix math. - This benchmark was previously published in ARES-6. - Source code: ML -
- -
mandreel
-
- Tests the Bullet physics engine. - The physics engine is compiled to JavaScript with Mandreel. - A similar version of this benchmark was previously published in Octane version 2. - Source code: mandreel.js -
- -
lebab-wtb
-
- Lebab transpiles ES5 code into ES6/ES7. - This benchmark runs Lebab on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: lebab.js -
- -
json-stringify-inspector
-
- Measures JSON.stringify performance on a set of objects that WebKit's Web Inspector - stringifies when communicating between the UI and web pages. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: json-stringify-inspector.js -
- -
json-parse-inspector
-
- Measures JSON.parse performance on a set of objects that WebKit's Web Inspector - parses when communicating between the UI and web pages. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: json-parse-inspector.js -
-
jshint-wtb
-
- JSHint is a static analysis tool that warns about errors - and potential problems in JavaScript programs. - This benchmark runs JSHint on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: jshint.js -
- -
HashSet-wasm
-
- A WebAssembly benchmark replaying a set of hash table operations performed in WebKit when loading - a web page. This benchmark was compiled from C++ to WebAssembly using Emscripten. - Source code: HashSet.cpp, HashSet.js -
- -
hash-map
-
- Apache Harmony java.util.HashMap implementation ported to JavaScript and benchmarked by - doing hash table insertions, queries, and then iterating the associated entrySet. Tests - object-oriented JavaScript idioms and object construction. A similar JavaScript version - of this benchmark was originally published as part of the WebKit test suite. - Source code: hash-map.js -
- -
gcc-loops-wasm
-
- Example loops used to tune the GCC and LLVM vectorizers, compiled to WebAssembly with - Emscripten. The original C++ version of this benchmark was previously published in the LLVM test suite. - Source code: gcc-loops.cpp, gcc-loops.js -
- -
gbemu
-
- Gameboy emulator written in JavaScript. Tests typed array and - property access performance. - A similar version of this benchmark was previously published in Octane version 2. - Source code: gbemu-part1.js, gbemu-part2.js -
- -
gaussian-blur
-
- Tests the performance of a JavaScript implementation of gaussian - blur on a test image. Tests numeric analysis speed and uses typed arrays. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: gaussian-blur.js -
- -
FlightPlanner
-
- Flight Planner is a benchmark taken from a flight management web application. - Flight Planner parses aircraft flight plans and computes distance, courses, and elapsed times for legs of flight plans. - It uses FAA data for airports, navigation aids, and airways. The flight management app was originally written to help - compete in a flying proficiency event. It stresses regular expression performance. - Source code: flight_planner.js -
- -
first-inspector-code-load
-
- Measures the first-time parsing of a modern JavaScript code base: WebKit's Web Inspector. - This models the parsing time of visiting a web site for the first time. - Source code: code-first-load.js -
- -
espree-wtb
-
- Espree is a JavaScript parser written in JavaScript. - This benchmark runs Espree on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: espree.js -
- -
earley-boyer
-
- Tests two classic Scheme benchmarks translated to JavaScript using scheme2js. The first - benchmark is Earley, is a chart parser algorithm created by Jay Earley. The second is - Boyer, a logic programming benchmark by Bob Boyer. Measures variadic functions and object - construction. - A similar version of this benchmark was previously published in Octane version 2. - Source code: earley-boyer.js -
- -
delta-blue
-
- The classic DeltaBlue benchmark derived from a Smalltalk implementation by Maloney and - Wolczko. Tests devirtualization of JavaScript code that uses an idiomatic class hierarchy - construction. - A similar version of this benchmark was previously published in Octane version 2. - Source code: deltablue.js -
- -
date-format-xparb-SP
-
- Sophisticated date formatting and parsing library test, based on code by Barin Schwartz. - A similar version of this benchmark was previously published in SunSpider. - Source code: date-format-xparb.js -
- -
date-format-tofte-SP
-
- Date and time formatting test, based on code by Svend Tofte. Involves an interesting use - of eval and also covers string manipulation and JavaScript library functions. - A similar version of this benchmark was previously published in SunSpider. - Source code: date-format-tofte.js -
- -
crypto-sha1-SP
-
- SHA-1 implementation in JavaScript by - Paul Johnston and others. Tests interesting integer math idioms. - A similar version of this benchmark was previously published in SunSpider. - Source code: crypto-sha1.js -
- -
crypto-md5-SP
-
- MD5 implementation in JavaScript by - Paul Johnston and others. Tests interesting integer math idioms. - A similar version of this benchmark was previously published in SunSpider. - Source code: crypto-md5.js -
- -
crypto-aes-SP
-
- AES implementation - in JavaScript by Chris Veness. A newer version can be - found here. Tests integer math. - A similar version of this benchmark was previously published in SunSpider. - Source code: crypto-aes.js -
- -
crypto
-
- RSA cypher implemented in JavaScript by Tom Wu. Tests integer math and arrays. - A similar version of this benchmark was previously published in Octane version 2. - Source code: crypto.js -
- -
coffeescript-wtb
-
- CoffeeScript is a programming language that attempts to expose the - good parts of JavaScript in a simple way. - This benchmark tests the CoffeeScript compiler on test programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: coffeescript.js -
- -
chai-wtb
-
- Chai is a BDD / TDD assertion library for - node.js and the browser. It is commonly used to write unit and integration tests. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: chai.js -
- -
cdjs
-
- JavaScript implementation of the CDx collision detection - benchmark. Measures the performance of over 200 collision detection runs. - Source code: cdjs -
- -
box2d
-
- The Box2D physics engine ported to JavaScript. Tests floating - point math and data structures. - A similar version of this benchmark was previously published in Octane version 2. - Source code: box2d.js -
- -
bomb-workers
-
- Tests running various subtests of the SunSpider benchmark in parallel using Web Workers. - Stresses the browser's ability to run JavaScript code in parallel. - Source code: bomb.js -
- -
Basic
-
- Basic is an ES2015 implementation of the ECMA-55 BASIC standard. - Basic stresses performance of generator functions, classes, Map, and WeakMap. - The benchmark runs a handful of simple programs, the most complex of which finds prime numbers. - This benchmark was previously published in ARES-6. - Source code: Basic -
- -
base64-SP
-
- Base64 encoder/decoder written in JavaScript, originally from the Mozilla XML-RPC client component. - Tests string manipulation. - A similar version of this benchmark was previously published in SunSpider. - Source code: base64.js -
- -
babylon-wtb
-
- Babylon is the frontend for the Babel transpiler. - It is a JavaScript parser written in JavaScript. It computes the Abstract Syntax Tree of the input JavaScript program. - This benchmark runs Babylon on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: babylon.js -
- -
Babylon
-
- Babylon is an implementation of a parser for the JavaScript language. - Babylon is the parser used in the Babel JavaScript transpiler. The benchmark runs the - Babylon parser on four different JavaScript sources. Babylon makes heavy use of classes, does non trivial string processing, - and creates non-trivial object graphs. - This benchmark was previously published in ARES-6. - Source code: Babylon -
- -
async-fs
-
- This is an implementation of a mock file system that stresses the performance of DataView, Promises, and async - iteration. The benchmark simulates adding and removing files, and swapping the byte order of existing files. - Source code: async-file-system.js -
- -
sync-fs
-
- This is an implementation of a mock file system that stresses the performance of DataView, Promises, and synchronous - generators / iterators. The benchmark simulates adding and removing files, and swapping the byte order of existing files. - Source code: sync-file-system.js -
- -
lazy-collections
-
- This benchmark iterates over common integer sequences (fibonacci, prime numbers, etc) as lazy collections using eponymous - library by Robin Malfait that stresses the performance of generators. - Source code: lazy-collections.js -
- -
js-tokens
-
- This benchmarks runs js-tokens, a lenient and almost spec-compliant JavaScript - tokenizer by Simon Lydell, over its own source code and a JSX snippet. Stresses the performance of generators and regular - expressions, especially sticky and unicode RegExp patterns with property escapes \p{}. - Source code: js-tokens.js -
- -
doxbee-promise
-
- A typical CRUD method extracted from DoxBee that is called when uploading files. The benchmark emulates a situation where - 10k requests are being made concurrently to execute some mixed async / sync action with fast I/O response times. - Authored by Gorki Kosev, - packed by Benedikt Meurer. - Source code: doxbee-promise.js -
- -
doxbee-async
-
- A typical CRUD method extracted from DoxBee that is called when uploading files. The benchmark emulates a situation where - 10k requests are being made concurrently to execute some mixed async / sync action with fast I/O response times. - Uses async / await instead of plain Promise. - Authored by Gorki Kosev, - packed by Benedikt Meurer. - Source code: doxbee-async.js -
- -
Air
-
- Air is an ES2015 port of the WebKit B3 JIT's Air::allocateStack phase. - This code is a heavy user of Map, Set, classes, spread, and for-of. The benchmark runs allocateStack on hot function - bodies from other popular JavaScript benchmarks. This benchmark was previously published in ARES-6. - Source code: Air -
- -
ai-astar
-
- This benchmark runs a JavaScript implementation of the A* search algorithm - written by Brian Grinstead. - This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. - Source code: ai-astar.js -
- -
acorn-wtb
-
- Acorn is a JavaScript parser written in JavaScript. - This benchmark runs Acorn on test JavaScript programs. - This benchmark stresses string manipulation and regular expression performance. - A similar version of this benchmark was previously published in the Web Tooling Benchmark. - Source code: acorn.js -
- -
3d-raytrace-SP
-
- Simple raytracer written by Oliver Hunt. - Tests arrays and floating-point math in relatively short-running code. - A similar version of this benchmark was previously published in SunSpider. - Source code: 3d-raytrace.js -
- -
3d-cube-SP
-
- 3D cube rotation benchmark by Simon Speich. The original can be found - on Simon's - web page. Tests arrays and floating-point math in relatively - short-running code. - A similar version of this benchmark was previously published in SunSpider. - Source code: 3d-cube.js -
- -
bigint-noble-bls12-381
-
- BLS12-381, pairing-friendly - Barreto-Lynn-Scott elliptic curve construction, - implemented in JavaScript - by Paul Miller. Tests typed arrays and arithmetic operations on BigInt. - Source code: noble-bls12-381-bundle.js -
- -
bigint-noble-secp256k1
-
- secp256k1, an elliptic curve that could - be used for asymmetric encryption, ECDH key agreement protocol and signature schemes, - implemented in JavaScript - by Paul Miller. Tests typed arrays and arithmetic operations on BigInt. - Source code: noble-secp256k1-bundle.js -
- -
bigint-noble-ed25519
-
- ed25519, an elliptic curve that could - be used for EDDSA signature scheme and X25519 ECDH key agreement, - implemented in JavaScript - by Paul Miller. Tests typed arrays and arithmetic operations on BigInt. - Source code: noble-ed25519-bundle.js -
- -
bigint-paillier
-
- Paillier cryptosystem, - a probabilistic asymmetric algorithm for public key cryptography, - implemented in JavaScript - by Juan Hernández Serrano. Tests arithmetic operations on BigInt. - Source code: paillier-bundle.js -
- -
bigint-bigdenary
-
- BigDenary, an arbitrary-precision - decimal arithmetic, implemented in JavaScript by U-Zyn Chua. - Tests arithmetic operations on BigInt. - Source code: bigdenary-bundle.js -
- -
proxy-mobx
-
- A super minimal, store-only implementation of Reminders.app, written in MobX. - Heavily relies on computed getters and utilizes Map / Set collections. - Synchronously re-renders into text on every data manipulation. - Tests get / set Proxy traps, as well as various Array methods. - Source code: mobx-benchmark.js -
- -
proxy-vue
-
- A super minimal, store-only implementation of Reminders.app, written using reactivity - API of Vue.js 3.0. Heavily relies on computed getters and utilizes Map / Set collections. - Synchronously re-renders into text on every data manipulation. - Tests get / set Proxy traps, as well as various Array methods. - Source code: vue-benchmark.js -
- -
dotnet-interp-wasm
-
- Tests .NET on WebAssembly. This benchmark tests operations - on .NET implementation of String, JSON serialization, specifics of .NET exceptions and computation - of a 3D scene using Mono Interpreter. Source code: .NET. -
- -
dotnet-aot-wasm
-
- Tests .NET on WebAssembly. This benchmark tests operations - on .NET implementation of String, JSON serialization, specifics of .NET exceptions and computation - of a 3D scene using Mono AOT. Source code: .NET. -
- -
- -

← Return to Tests

-
- -
+ +

+
+ JetStream 3 +
+

+
+
+

In-Depth Analysis

+ +

+ JetStream 3 combines together a variety of JavaScript and WebAssembly benchmarks, covering a variety of + advanced workloads and programming techniques, and reports a single score that + balances them using a geometric mean. +

+ +

+ Each benchmark measures a distinct workload, and no single optimization + technique is sufficient to speed up all benchmarks. Some benchmarks demonstrate tradeoffs, and + aggressive or specialized optimizations for one benchmark might make another benchmark slower. + JetStream 3 rewards browsers that start up quickly, execute code quickly, and continue running smoothly. +

+ +

+ Each benchmark in JetStream 3 computes its own individual score. + Scores in JetStream are dimensionless floats, where a higher score is better. + JetStream weighs each benchmark equally, taking the geometric mean over each individual + benchmark's score to compute the overall JetStream 3 score. + The geometric mean ensures that a multiplicative improvement of any individual score has the same effect + on the aggregated score, regardless of the absolute value of the individual score. + For example, an improvement by 5% of the sub score of benchmark A has the same effect on the total score + as an improvement by 5% of the sub score of benchmark B. +

+ +

+ It's not enough to just measure the total running time of a workload. + Browsers may perform differently for the same JavaScript workload depending on how many times it + has run. For example, garbage collection runs periodically, making some iterations take longer than + others. Code that runs repeatedly gets optimized by the browser, so the first iteration + of any workload is usually more expensive than the rest. +

+ +

+ For most of the JavaScript and WebAssembly benchmarks in JetStream 3, individual scores + equally weigh startup performance, worst case performance, and average case + performance. These three metrics are crucial to running performant JavaScript + in the browser. Fast startup times lead browsers to loading pages more quickly. Good + worst case performance ensures web applications can run without hiccups. Fast average + case performance makes it so that the most advanced web applications can run at all. +

+ +

+ An important component of JetStream 1 were the asm.js subset of benchmarks. With the release + of WebAssembly, the importance of asm.js has lessened since many users of asm.js are + now using WebAssembly. JetStream 3 has converted many of the asm.js benchmarks from + JetStream 1 into WebAssembly. +

+ +

+ All but one of JetStream 3's JavaScript benchmarks run for N iterations, where + N is usually 120. JetStream 3 reports the startup score as the time it takes to run the first iteration. + The worst case score is the average of the worst M iterations, excluding the first iteration. + M is always less than N, and is usually 4. The average case score is the average + of all but the first iteration. These three scores are weighed equally using the geometric + mean. +

+ +

+ JetStream 3 also includes a JavaScript benchmark named WSL. WSL is an implementation of a + GPU shading language written in JavaScript. WSL does not use the above mechanism for scoring + because it has a long running time. Instead, the WSL benchmark computes its score as the + geometric mean over two metrics: the time it takes to compile the WSL standard library, and the time + it takes to run through the WSL specification test suite. +

+ +

+ JetStream 3 includes parts of these benchmark suites that came before it: SunSpider, + Octane 2, JetStream 1, + ARES-6, Web Tooling Benchmark, and benchmarks + inspired by Kraken. + JetStream 3 also includes a new set of benchmarks that measure the performance of WebAssembly, Web + Workers, + Promises, async iteration, unicode regular expressions, and JavaScript parsing. +

+ +

+ JetStream 3 includes several benchmarks from earlier JetStream versions, but updates the benchmark + driver to + improve score stability. This is achieved by pre-fetching network resources prior to running + the benchmarks. This can reduce perturbations on the measurement of JavaScript execution + time due to second order effects of pause times induced by network latency. +

+ +

+ Note that scores from JetStream 3 are not comparable to scores to other versions + of any JetStream benchmark. +

+ +

+ JetStream 3 has 64 subtests: +

+ +
+
8bitbench-wasm
+
+ A simple 8-bit emulator targeting wasm, written in rust. It aims to represent what an emulator or + small game might act like in the real world. +
+ Attribution: See 8bitbench/ABOUT.md +
+ Source code: 8bitbench/js3harness.js, based off https://github.com/justinmichaud/8bitbench +
+
acorn-wtb
+
+ Acorn is a JavaScript parser written in JavaScript. + This benchmark runs Acorn on test JavaScript programs. + This benchmark stresses string manipulation and regular expression performance. + A similar version of this benchmark was previously published in the Web Tooling Benchmark. + Source code: acorn.js +
+
ai-astar
+
+ This benchmark runs a JavaScript implementation of the A* search algorithm + written by Brian Grinstead. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: ai-astar.js +
+
Air
+
+ Air is an ES2015 port of the WebKit B3 JIT's Air::allocateStack phase. + This code is a heavy user of Map, Set, classes, spread, and for-of. The benchmark runs allocateStack + on hot function + bodies from other popular JavaScript benchmarks. This benchmark was previously published in ARES-6. + Source code: Air +
+
argon2-wasm
+
+ Tests Argon2, a password-hashing function + (the winner of Password Hashing Competition), in WebAssembly. This is test is based on argon2-browser library. Source code: ARGON2. +
+
async-fs
+
+ This is an implementation of a mock file system that stresses the performance of DataView, Promises, + and async + iteration. The benchmark simulates adding and removing files, and swapping the byte order of + existing files. + Source code: async-file-system.js +
+
babel-minify-wtb
+
babel-wtb
+
TODO
+
Babylon
+
+ Babylon is an implementation of a parser for the + JavaScript language. + Babylon is the parser used in the Babel JavaScript transpiler. The + benchmark runs the + Babylon parser on four different JavaScript sources. Babylon makes heavy use of classes, does non + trivial string processing, + and creates non-trivial object graphs. + This benchmark was previously published in ARES-6. + Source code: Babylon +
+
babylon-wtb
+
+ Babylon is the + frontend for the Babel transpiler. + It is a JavaScript parser written in JavaScript. It computes the Abstract Syntax Tree of the input + JavaScript program. + This benchmark runs Babylon on test JavaScript programs. + This benchmark stresses string manipulation and regular expression performance. + A similar version of this benchmark was previously published in the Web Tooling Benchmark. + Source code: babylon.js +
+
babylon-scene-es6
+
babylon-scene-es6
+
babylon-startup-es5
+
babylon-startup-es6
+
TODO
+
Basic
+
+ Basic is an ES2015 implementation of the ECMA-55 + BASIC standard. + Basic stresses performance of generator functions, classes, Map, and WeakMap. + The benchmark runs a handful of simple programs, the most complex of which finds prime numbers. + This benchmark was previously published in ARES-6. + Source code: Basic +
+
bigint-bigdenary
+
+ BigDenary, an arbitrary-precision + decimal arithmetic, implemented in JavaScript by U-Zyn Chua. + Tests arithmetic operations on BigInt. + Source code: bigdenary-bundle.js +
+
bigint-noble-bls12-381
+
+ BLS12-381, pairing-friendly + Barreto-Lynn-Scott elliptic curve construction, + implemented in JavaScript + by Paul Miller. Tests typed arrays and arithmetic operations on BigInt. + Source code: noble-bls12-381-bundle.js +
+
bigint-noble-ed25519
+
+ ed25519, an elliptic curve that could + be used for EDDSA signature scheme and X25519 ECDH key agreement, + implemented in JavaScript + by Paul Miller. Tests typed arrays and arithmetic operations on BigInt. + Source code: noble-ed25519-bundle.js +
+
bigint-noble-secp256k1
+
+ secp256k1, an elliptic curve that could + be used for asymmetric encryption, ECDH key agreement protocol and signature schemes, + implemented in JavaScript + by Paul Miller. Tests typed arrays and arithmetic operations on BigInt. + Source code: noble-secp256k1-bundle.js +
+
bigint-paillier
+
+ Paillier cryptosystem, + a probabilistic asymmetric algorithm for public key cryptography, + implemented in JavaScript + by Juan Hernández Serrano. Tests arithmetic operations on BigInt. + Source code: paillier-bundle.js +
+
bomb-workers
+
+ Tests running various subtests of the SunSpider benchmark in parallel using Web Workers. + Stresses the browser's ability to run JavaScript code in parallel. + Source code: bomb.js +
+
Box2D
+
+ The Box2D physics engine ported to JavaScript. + Tests floating + point math and data structures. + A similar version of this benchmark was previously published in Octane version 2. + Source code: box2d.js +
+
cdjs
+
+ JavaScript implementation of the CDx collision + detection + benchmark. Measures the performance of over 200 collision detection runs. + Source code: cdjs +
+
chai-wtb
+
+ Chai is a BDD / TDD assertion + library for + node.js and the browser. It is commonly used to write unit and integration tests. + A similar version of this benchmark was previously published in the Web Tooling Benchmark. + Source code: chai.js +
+
crypto
+
+ RSA cypher implemented in JavaScript by Tom Wu. Tests integer math and arrays. + A similar version of this benchmark was previously published in Octane version 2. + Source code: crypto.js +
+
Dart-flute-complex-wasm
+
TODO
+
Dart-flute-todomvc-wasm
+
TODO
+
delta-blue
+
+ The classic DeltaBlue benchmark derived from a Smalltalk implementation by Maloney and + Wolczko. Tests devirtualization of JavaScript code that uses an idiomatic class hierarchy + construction. + A similar version of this benchmark was previously published in Octane version 2. + Source code: deltablue.js +
+
dotnet-aot-wasm
+
+ Tests .NET on WebAssembly. This benchmark tests + operations + on .NET implementation of String, JSON serialization, specifics of .NET exceptions and computation + of a 3D scene using Mono AOT. Source code: .NET. +
+
dotnet-interp-wasm
+
+ Tests .NET on WebAssembly. This benchmark tests + operations + on .NET implementation of String, JSON serialization, specifics of .NET exceptions and computation + of a 3D scene using Mono Interpreter. Source code: .NET. +
+
doxbee-async
+
+ A typical CRUD method extracted from DoxBee that is called when uploading files. The benchmark + emulates a situation where + 10k requests are being made concurrently to execute some mixed async / sync action with fast I/O + response times. + Uses async / await instead of plain Promise. + Authored by Gorki + Kosev, + packed by Benedikt Meurer. + Source code: doxbee-async.js +
+
doxbee-promise
+
+ A typical CRUD method extracted from DoxBee that is called when uploading files. The benchmark + emulates a situation where + 10k requests are being made concurrently to execute some mixed async / sync action with fast I/O + response times. + Authored by Gorki + Kosev, + packed by Benedikt Meurer. + Source code: doxbee-promise.js +
+
earley-boyer
+
+ Tests two classic Scheme benchmarks translated to JavaScript using scheme2js. The first + benchmark is Earley, is a chart parser algorithm created by Jay Earley. The second is + Boyer, a logic programming benchmark by Bob Boyer. Measures variadic functions and object + construction. + A similar version of this benchmark was previously published in Octane version 2. + Source code: earley-boyer.js +
+
espree-wtb
+
+ Espree is a JavaScript parser written in JavaScript. + This benchmark runs Espree on test JavaScript programs. + This benchmark stresses string manipulation and regular expression performance. + A similar version of this benchmark was previously published in the Web Tooling Benchmark. + Source code: espree.js +
+
esprima-next-wtb
+
+
first-inspector-code-load
+
+ Measures the first-time parsing of a modern JavaScript code base: WebKit's Web Inspector. + This models the parsing time of visiting a web site for the first time. + Source code: code-first-load.js +
+
FlightPlanner
+
+ Flight Planner is a benchmark taken from a flight management web application. + Flight Planner parses aircraft flight plans and computes distance, courses, and elapsed times for + legs of flight plans. + It uses FAA data for airports, navigation aids, and airways. The flight management app was + originally written to help + compete in a flying proficiency event. It stresses regular expression performance. + Source code: flight_planner.js +
+
gaussian-blur
+
+ Tests the performance of a JavaScript implementation of gaussian + blur on a test image. Tests numeric analysis speed and uses typed arrays. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: gaussian-blur.js +
+
gbemu
+
+ Gameboy emulator written in JavaScript. Tests typed array and + property access performance. + A similar version of this benchmark was previously published in Octane version 2. + Source code: gbemu-part1.js, gbemu-part2.js +
+
gcc-loops-wasm
+
+ Example loops used to tune the GCC and LLVM vectorizers, compiled to WebAssembly with + Emscripten. The original C++ version of this benchmark was + previously published in the LLVM test suite. + Source code: gcc-loops.cpp, gcc-loops.js +
+
hash-map
+
+ Apache Harmony java.util.HashMap implementation ported to JavaScript and benchmarked by + doing hash table insertions, queries, and then iterating the associated entrySet. Tests + object-oriented JavaScript idioms and object construction. A similar JavaScript version + of this benchmark was originally published as part of the WebKit test suite. + Source code: hash-map.js +
+
HashSet-wasm
+
+ A WebAssembly benchmark replaying a set of hash table operations performed in WebKit when loading + a web page. This benchmark was compiled from C++ to WebAssembly using Emscripten. + Source code: HashSet.cpp, HashSet.js +
+
intl
+
TODO
+
j2cl-box2d-wasm
+
TODO
+
js-tokens
+
+ This benchmarks runs js-tokens, a lenient and + almost spec-compliant JavaScript + tokenizer by Simon Lydell, over its own source code and a JSX snippet. Stresses the performance of + generators and regular + expressions, especially sticky and unicode RegExp patterns with property escapes \p{}. + Source code: js-tokens.js +
+
jsdom-d3-startup
+
TODO
+
json-parse-inspector
+
+ Measures JSON.parse performance on a set of objects that WebKit's Web Inspector + parses when communicating between the UI and web pages. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: json-parse-inspector.js +
+
json-stringify-inspector
+
+ Measures JSON.stringify performance on a set of objects that WebKit's Web Inspector + stringifies when communicating between the UI and web pages. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: json-stringify-inspector.js +
+
Kotlin-compose-wasm
+
+
lazy-collections
+
+ This benchmark iterates over common integer sequences (fibonacci, prime numbers, etc) as lazy + collections using eponymous + library by Robin Malfait that stresses the performance of + generators. + Source code: lazy-collections.js +
+
+
+ Lebab transpiles ES5 code into ES6/ES7. + This benchmark runs Lebab on test JavaScript programs. + This benchmark stresses string manipulation and regular expression performance. + A similar version of this benchmark was previously published in the Web Tooling Benchmark. + Source code: lebab.js +
+
mandreel
+
+ Tests the Bullet physics engine. + The physics engine is compiled to JavaScript with Mandreel. + A similar version of this benchmark was previously published in Octane version 2. + Source code: mandreel.js +
+
ML
+
+ ML is an implementation of a + feedforward neural network. + The benchmark trains several networks using different activation functions + and several sample data sets. ML makes heavy use of classes. It relies on the ml-matrix library and + does non-trivial matrix math. + This benchmark was previously published in ARES-6. + Source code: ML +
+
mobx-startup
+
+
multi-inspector-code-load
+
+ Measures the repeated parsing of a modern JavaScript code base: WebKit's Web Inspector. + Because this test allows caching, this is representative of revisiting the same website. + Source code: code-multi-load.js +
+ +
+ Fluid simulation written by Oliver Hunt. Emphasizes floating point + array performance. + A similar version of this benchmark was previously published in Octane version 2. + Source code: navier-stokes.js +
+
octane-code-load
+
+ Test of code load speed of the jQuery and Closure libraries. Because this test allows + caching, this is representative of revisiting the same website. + A similar version of this benchmark was previously published in Octane version 2. + Source code: code-load.js +
+
OfflineAssembler
+
+ Offline Assembler is the lexer, parser, and AST layer of the offline assembler for JavaScriptCore. + It has been ported to JavaScript from the original Ruby implementation. This test stresses regular + expression + performance. + Source code: OfflineAssembler.js +
+
pdfjs
+
+ Mozilla's PDF reader written in JavaScript. + This benchmark emphasizes array manipulation and bit operations. + A similar version of this benchmark was previously published in Octane version 2. + Source code: pdfjs.js +
+
postcss-wtb
+
TODO
+
+
TODO
+
prismjs-startup-es5
+
prismjs-startup-es6
+
TODO
+
proxy-mobx
+
+ A super minimal, store-only implementation of Reminders.app, written in MobX. + Heavily relies on computed getters and utilizes Map / Set collections. + Synchronously re-renders into text on every data manipulation. + Tests get / set Proxy traps, as well as various Array methods. + Source code: mobx-benchmark.js +
+
proxy-vue
+
+ A super minimal, store-only implementation of Reminders.app, written using reactivity + API of Vue.js 3.0. Heavily relies on computed getters and utilizes Map / Set collections. + Synchronously re-renders into text on every data manipulation. + Tests get / set Proxy traps, as well as various Array methods. + Source code: vue-benchmark.js +
+
quicksort-wasm
+
+ Quicksort benchmark, compiled to WebAssembly with Emscripten. + The original C version of this benchmark was previously published in the LLVM test suite. + Source code: quicksort.c, quicksort.js +
+
raytrace
+
+ Ray tracer written in JavaScript + using ES6 classes. + Tests object construction performance and floating point math. + A similar version of this benchmark was previously published in Octane version 2. + Source code: raytrace.js +
+
raytrace-private-class-fields
+
+ Ray tracer written in JavaScript + using ES6 classes and private fields. + Tests object construction performance, along with setting default values of private fields, their + access speed, and floating point math. + A similar version of this benchmark was previously published in Octane version 2. + Source code: raytrace-private-class-fields.js +
+
raytrace-public-class-fields
+
+ Ray tracer written in JavaScript + using ES6 classes and public fields. + Tests object construction performance, along with setting default values of public fields, and + floating point math. + A similar version of this benchmark was previously published in Octane version 2. + Source code: raytrace-public-class-fields.js +
+
regexp-octane
+
+ Collection of regular expressions found by the V8 team in 2010, curated into a benchmark. + A similar version of this benchmark was previously published in Octane version 2. + Source code: regexp.js +
+
richards
+
+ Martin Richard's system language + benchmark ported to JavaScript. Tests object property access performance. + A similar version of this benchmark was previously published in Octane version 2. + Source code: richards.js +
+
richards-wasm
+
+ Martin Richard's system language + benchmark compiled to a hybrid of WebAssembly and JavaScript. It stresses how quickly + JavaScript can call into WebAssembly code. + Source code: richards.c, richards.js +
+
segmentation
+
+ Uses Web Workers to parallelize the computation of a + time series segmentation algorithm over a sample data set. This code is adapted from an + algorithm used in the + WebKit performance dashboard. + Source code: segmentation.js +
+
source-map-wtb
+
TODO
+
splay
+
+ Tests the manipulation of splay trees + represented using plain JavaScript objects. This benchmark stresses the performance of the garbage + collector. + A similar version of this benchmark was previously published in Octane version 2. + Source code: splay.js +
+
sqlite3-wasm
+
TODO
+
stanford-crypto-aes
+
+ Measures the performance of the AES hashing algorithm as + implemented by the Stanford JavaScript Crypto + Library. This benchmark stresses numeric analysis and array access. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: stanford-crypto-aes.js +
+
stanford-crypto-pbkdf2
+
+ Measures the performance of the PBKDF2 hashing + algorithm as implemented by the Stanford JavaScript + Crypto Library. This benchmark stresses numeric analysis and array access. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: stanford-crypto-pbkdf2.js +
+
stanford-crypto-sha256
+
+ Measures the performance of the SHA256 hashing + algorithm as implemented by the Stanford JavaScript + Crypto Library. This benchmark stresses numeric analysis and array access. + This benchmark was inspired by a similar benchmark in the Kraken benchmark suite. + Source code: stanford-crypto-sha256.js +
+
Sunspider
+
+
+
3d-cube-SP
+
+ 3D cube rotation benchmark by Simon Speich. The original can be found + on Simon's + web page. Tests arrays and floating-point math in relatively + short-running code. + A similar version of this benchmark was previously published in SunSpider. + Source code: 3d-cube.js +
+
3d-raytrace-SP
+
+ Simple raytracer written by Oliver Hunt. + Tests arrays and floating-point math in relatively short-running code. + A similar version of this benchmark was previously published in SunSpider. + Source code: 3d-raytrace.js +
+
base64-SP
+
+ Base64 encoder/decoder written in JavaScript, originally from the Mozilla XML-RPC client + component. + Tests string manipulation. + A similar version of this benchmark was previously published in SunSpider. + Source code: base64.js +
+
crypto-aes-SP
+
+ AES implementation + in JavaScript by Chris Veness. A newer version can be + found here. Tests integer math. + A similar version of this benchmark was previously published in SunSpider. + Source code: crypto-aes.js +
+
crypto-md5-SP
+
+ MD5 implementation in JavaScript by + Paul Johnston and others. Tests interesting integer math idioms. + A similar version of this benchmark was previously published in SunSpider. + Source code: crypto-md5.js +
+
crypto-sha1-SP
+
+ SHA-1 implementation in JavaScript by + Paul Johnston and others. Tests interesting integer math idioms. + A similar version of this benchmark was previously published in SunSpider. + Source code: crypto-sha1.js +
+
date-format-tofte-SP
+
+ Date and time formatting test, based on code by Svend Tofte. Involves an interesting use + of eval and also covers string manipulation and JavaScript library functions. + A similar version of this benchmark was previously published in SunSpider. + Source code: date-format-tofte.js +
+
date-format-xparb-SP
+
+ Sophisticated date formatting and parsing library test, based on code by Barin Schwartz. + A similar version of this benchmark was previously published in SunSpider. + Source code: date-format-xparb.js +
+
n-body-SP
+
+ Classic solar system simulation benchmark from + The Great + Computer + Language Shootout, + contributed by Isaac Guy. Tests math and object access performance. + A similar version of this benchmark was previously published in SunSpider. + Source code: n-body.js +
+
regex-dna-SP
+
+ Regular-expression-based solution to DNA manipulation from + The Great + Computer + Language Shootout, + contributed by Jesse Millikan. + A similar version of this benchmark was previously published in SunSpider. + Source code: regex-dna.js +
+
string-unpack-code-SP
+
+ This benchmark unpacks various minified JavaScript libraries. It stresses the speed of + various + string manipulation + operations. + A similar version of this benchmark was previously published in SunSpider. + Source code: string-unpack-code.js +
+
tagcloud-SP
+
+ Parses JSON and generates markup for a tag + cloud view of the data. Written by Maciej Stachowiak of the WebKit team. Exercises + string + parsing and manipulation. A similar version of this benchmark was originally published in + SunSpider. + Source code: tagcloud.js +
+
+
+
sync-fs
+
+ This is an implementation of a mock file system that stresses the performance of DataView, Promises, + and synchronous + generators / iterators. The benchmark simulates adding and removing files, and swapping the byte + order of existing files. + Source code: sync-file-system.js +
+
tfjs-wasm
+
+ Tests Tensorflow.js pre-trained machine learning + models supported by WebAssembly backend. + The current benchmark includes models: mobilenet, + knn-classifier, coco-ssd, universal-sentence-encoder. + Source code: TFJS. +
+
tfjs-wasm-simd
+
TODO
+
threejs
+
+
transformersjs-bert-wasm"
+
TODO
+
transformersjs-whisper-wasm
+
TODO
+
tsf-wasm
+
+ Runs Filip Pizlo's — of the WebKit team — implementation of a Typed Stream Format + in WebAssembly. The original code is compiled from C to WebAssembly using Emscripten. + Source code: TSF +
+
typescript-lib
+
+ Tests how quickly Microsoft's TypeScript compiler can + compile itself. More than anything else, this tests how quickly a JavaScript runtime can optimize + a large pile of code. + A similar version of this benchmark was previously published in Octane version 2. + Source code: typescript.js +
+
typescript-octane
+
TODO
+
UniPoker
+
+ UniPoker is a 5 card stud poker simulation using the Unicode playing card code points, + U+1F0A1..U+1F0DE, + as the card representation in code. Scoring of hands is done with three regular expressions, one to + check + for a flush, one to check for straights, and one to check for pairs, three of a kind, and four of a + kind. + Source code: poker.js +
+
validatorjs
+
TODO
+
web-ssr
+
TODO
+
WSL
+
+ WSL is an implementation of a GPU shading language written in JavaScript. + WSL measures the time it takes to compile the WSL standard library and the time + it takes to run through the WSL specification test suite. + Source code: WSL +
+
zlib-wasm
+
zlib-wasm
+
+ +

← Return to Tests

+
+ +
- + + \ No newline at end of file diff --git a/tests/run-browser.mjs b/tests/run-browser.mjs index d5917b2c..3657367f 100644 --- a/tests/run-browser.mjs +++ b/tests/run-browser.mjs @@ -70,6 +70,13 @@ const TESTS = [ run() { return runEnd2EndTest("Run Default Suite"); } + }, + { + name: "Verify In Depth Info", + tags: ["all", "in-depth"], + run() { + return runBrowserDriverTest("In Depth Page Check", inDepthPageTest); + } } ]; @@ -166,11 +173,12 @@ async function runTests() { process.exit(1); } -async function runEnd2EndTest(name, params) { - return runTest(name, () => testEnd2End(params)); + +async function runBrowserDriverTest(name, body) { + return runTest(name, () => runBrowserDriver(body)) } -async function testEnd2End(params) { +async function runBrowserDriver(body) { const builder = new Builder().withCapabilities(capabilities); if (browserOptions) { switch(BROWSER) { @@ -188,28 +196,9 @@ async function testEnd2End(params) { const sessionId = (await driver.getSession()).getId(); const driverCapabilities = await driver.getCapabilities(); logInfo(`Browser: ${driverCapabilities.getBrowserName()} ${driverCapabilities.getBrowserVersion()}`); - const urlParams = Object.assign({ - worstCaseCount: 2, - iterationCount: 3 - }, params); - let results; let success = true; try { - const url = new URL(`http://localhost:${PORT}/index.html`); - url.search = new URLSearchParams(urlParams).toString(); - logInfo(`JetStream PREPARE ${url}`); - await driver.get(url.toString()); - await driver.executeAsyncScript((callback) => { - // callback() is explicitly called without the default event - // as argument to avoid serialization issues with chromedriver. - globalThis.addEventListener("JetStreamReady", () => callback()); - // We might not get a chance to install the on-ready listener, thus - // we also check if the runner is ready synchronously. - if (globalThis?.JetStream?.isReady) - callback(); - }); - results = await benchmarkResults(driver); - // FIXME: validate results; + await body(driver); } catch(e) { success = false; throw e; @@ -223,6 +212,33 @@ async function testEnd2End(params) { } } +async function runEnd2EndTest(name, params) { + return runBrowserDriverTest(name, (driver) => testEnd2End(driver, params)); +} + +async function testEnd2End(driver, params) { + const urlParams = Object.assign({ + worstCaseCount: 2, + iterationCount: 3 + }, params); + let results; + const url = new URL(`http://localhost:${PORT}/index.html`); + url.search = new URLSearchParams(urlParams).toString(); + logInfo(`JetStream PREPARE ${url}`); + await driver.get(url.toString()); + await driver.executeAsyncScript((callback) => { + // callback() is explicitly called without the default event + // as argument to avoid serialization issues with chromedriver. + globalThis.addEventListener("JetStreamReady", () => callback()); + // We might not get a chance to install the on-ready listener, thus + // we also check if the runner is ready synchronously. + if (globalThis?.JetStream?.isReady) + callback(); + }); + results = await benchmarkResults(driver); + // FIXME: validate results; +} + async function benchmarkResults(driver) { logInfo("JetStream START"); await driver.manage().setTimeouts({ script: 2 * 60_000 }); @@ -238,6 +254,45 @@ async function benchmarkResults(driver) { return JSON.parse(resultsJSON); } +async function inDepthPageTest(driver) { + await driver.get( `http://localhost:${PORT}/in-depth.html`); + const ids = await driver.executeScript(() => { + return Array.from(document.querySelectorAll("#workload-details dt[id]")).map(each => each.id); + }); + const sortedIds = ids.slice().sort((a, b) => { + return a.toLowerCase().localeCompare(b.toLowerCase()); + }); + const sectionErrors = [] + sortedIds.forEach((id, index) => { + if (id !== ids[index]) { + sectionErrors.push( + `Expected index ${index} to be '${id}' but got '${ids[index]}' `); + } + }); + const idSet = new Set(ids); + await driver.get( `http://localhost:${PORT}/index.html?tags=all`); + const benchmarkNames = await driver.executeScript(() => { + return globalThis.JetStream.benchmarks.map(each => each.name); + }); + benchmarkNames.sort((a,b) => { + return a.toLowerCase().localeCompare(b.toLowerCase()); + }); + + const missingIds = benchmarkNames.filter(name => !idSet.has(name)); + if (missingIds.length > 0) { + sectionErrors.push(`Missing in-depth.html info section: ${JSON.stringify(missingIds, undefined, 2)}`); + } + + const benchmarkNamesSet = new Set(benchmarkNames); + const unusedIds = sortedIds.filter(id => !benchmarkNamesSet.has(id)); + if (unusedIds.length > 0) { + sectionErrors.push(`Unused in-depth.html info section: ${JSON.stringify(unusedIds, undefined, 2)}`); + } + if (sectionErrors.length > 0) { + throw new Error(`info section errors: ${sectionErrors.join("\n")}`); + } +} + class JetStreamTestError extends Error { constructor(errors) { super(`Tests failed: ${errors.map(e => e.stack).join(", ")}`);