diff --git a/CHANGELOG.md b/CHANGELOG.md index f8eb117cf..7fd0889f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,7 @@ * [#2940](https://github.com/ruby-grape/grape/pull/2940): Speed up matching fully static routes - [@ericproulx](https://github.com/ericproulx). * [#2942](https://github.com/ruby-grape/grape/pull/2942): Route a path carrying raw non-ASCII bytes on an API with a non-ASCII route instead of raising `Encoding::CompatibilityError` - [@ericproulx](https://github.com/ericproulx). * [#2920](https://github.com/ruby-grape/grape/pull/2920): Look for a 405 neighbour only on paths the request's method has no route on - [@ericproulx](https://github.com/ericproulx). +* [#2943](https://github.com/ruby-grape/grape/pull/2943): Speed up matching among many parameterized routes - [@ericproulx](https://github.com/ericproulx). * Your contribution here. ### 4.0.1 (2026-09-15) diff --git a/benchmark/version_throughput/RESULTS.md b/benchmark/version_throughput/RESULTS.md index bcf191491..d2b0ab4f2 100644 --- a/benchmark/version_throughput/RESULTS.md +++ b/benchmark/version_throughput/RESULTS.md @@ -1,6 +1,6 @@ # Grape route throughput by version -Generated: 2026-09-13 19:27:30 CEST +Generated: 2026-09-16 00:01:04 CEST Ruby: ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [arm64-darwin25] Host: Darwin 25.6.0 arm64 JIT modes benched: No JIT, YJIT, ZJIT @@ -13,14 +13,14 @@ Single-threaded `Benchmark.ips`, 2s warmup + 5s measure. Each route shape has an | Version | No JIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT speedup | ZJIT (i/s) | μs/req | vs prev | vs 3.0.1 | ZJIT speedup | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 3.0.1 | 34,383 | 29.08 | — | — | 56,257 | 17.78 | — | — | +63.6% | 41,398 | 24.16 | — | — | +20.4% | -| 3.1.1 | 43,533 | 22.97 | +26.6% | +26.6% | 84,684 | 11.81 | +50.5% | +50.5% | +94.5% | 53,632 | 18.65 | +29.6% | +29.6% | +23.2% | -| 3.2.1 | 44,507 | 22.47 | +2.2% | +29.4% | 85,762 | 11.66 | +1.3% | +52.4% | +92.7% | 54,489 | 18.35 | +1.6% | +31.6% | +22.4% | -| 3.3.5 | 63,684 | 15.70 | +43.1% | +85.2% | 134,370 | 7.44 | +56.7% | +138.8% | +111.0% | 85,579 | 11.69 | +57.1% | +106.7% | +34.4% | -| 4.0.0 | 118,375 | 8.45 | +85.9% | +244.3% | 225,178 | 4.44 | +67.6% | +300.3% | +90.2% | 146,964 | 6.80 | +71.7% | +255.0% | +24.2% | -| master | 157,683 | 6.34 | +33.2% | +358.6% | 313,952 | 3.19 | +39.4% | +458.1% | +99.1% | 208,519 | 4.80 | +41.9% | +403.7% | +32.2% | +| 3.0.1 | 33,175 | 30.14 | — | — | 55,370 | 18.06 | — | — | +66.9% | 41,927 | 23.85 | — | — | +26.4% | +| 3.1.1 | 45,369 | 22.04 | +36.8% | +36.8% | 84,068 | 11.90 | +51.8% | +51.8% | +85.3% | 55,237 | 18.10 | +31.7% | +31.7% | +21.8% | +| 3.2.1 | 46,594 | 21.46 | +2.7% | +40.4% | 88,161 | 11.34 | +4.9% | +59.2% | +89.2% | 56,755 | 17.62 | +2.7% | +35.4% | +21.8% | +| 3.3.5 | 67,489 | 14.82 | +44.8% | +103.4% | 138,268 | 7.23 | +56.8% | +149.7% | +104.9% | 87,007 | 11.49 | +53.3% | +107.5% | +28.9% | +| 4.0.1 | 119,363 | 8.38 | +76.9% | +259.8% | 234,084 | 4.27 | +69.3% | +322.8% | +96.1% | 151,349 | 6.61 | +74.0% | +261.0% | +26.8% | +| master | 158,444 | 6.31 | +32.7% | +377.6% | 326,376 | 3.06 | +39.4% | +489.4% | +106.0% | 205,603 | 4.86 | +35.8% | +390.4% | +29.8% | -Over time, 3.0.1 → master: **+358.6%** without a JIT, **+458.1%** with YJIT, **+403.7%** with ZJIT. +Over time, 3.0.1 → master: **+377.6%** without a JIT, **+489.4%** with YJIT, **+390.4%** with ZJIT. ## Parameterized route @@ -28,14 +28,14 @@ Over time, 3.0.1 → master: **+358.6%** without a JIT, **+458.1%** with YJIT, * | Version | No JIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT speedup | ZJIT (i/s) | μs/req | vs prev | vs 3.0.1 | ZJIT speedup | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 3.0.1 | 33,318 | 30.01 | — | — | 52,802 | 18.94 | — | — | +58.5% | 39,428 | 25.36 | — | — | +18.3% | -| 3.1.1 | 41,793 | 23.93 | +25.4% | +25.4% | 79,385 | 12.60 | +50.3% | +50.3% | +89.9% | 54,451 | 18.37 | +38.1% | +38.1% | +30.3% | -| 3.2.1 | 43,069 | 23.22 | +3.1% | +29.3% | 81,429 | 12.28 | +2.6% | +54.2% | +89.1% | 51,927 | 19.26 | -4.6% | +31.7% | +20.6% | -| 3.3.5 | 60,972 | 16.40 | +41.6% | +83.0% | 124,143 | 8.06 | +52.5% | +135.1% | +103.6% | 79,978 | 12.50 | +54.0% | +102.8% | +31.2% | -| 4.0.0 | 108,826 | 9.19 | +78.5% | +226.6% | 213,895 | 4.68 | +72.3% | +305.1% | +96.5% | 141,123 | 7.09 | +76.5% | +257.9% | +29.7% | -| master | 130,330 | 7.67 | +19.8% | +291.2% | 249,438 | 4.01 | +16.6% | +372.4% | +91.4% | 161,566 | 6.19 | +14.5% | +309.8% | +24.0% | +| 3.0.1 | 34,022 | 29.39 | — | — | 53,966 | 18.53 | — | — | +58.6% | 41,163 | 24.29 | — | — | +21.0% | +| 3.1.1 | 42,336 | 23.62 | +24.4% | +24.4% | 81,484 | 12.27 | +51.0% | +51.0% | +92.5% | 55,170 | 18.13 | +34.0% | +34.0% | +30.3% | +| 3.2.1 | 44,905 | 22.27 | +6.1% | +32.0% | 85,515 | 11.69 | +4.9% | +58.5% | +90.4% | 56,074 | 17.83 | +1.6% | +36.2% | +24.9% | +| 3.3.5 | 64,020 | 15.62 | +42.6% | +88.2% | 125,728 | 7.95 | +47.0% | +133.0% | +96.4% | 82,136 | 12.17 | +46.5% | +99.5% | +28.3% | +| 4.0.1 | 110,980 | 9.01 | +73.4% | +226.2% | 213,641 | 4.68 | +69.9% | +295.9% | +92.5% | 141,087 | 7.09 | +71.8% | +242.7% | +27.1% | +| master | 131,221 | 7.62 | +18.2% | +285.7% | 258,773 | 3.86 | +21.1% | +379.5% | +97.2% | 166,831 | 5.99 | +18.2% | +305.3% | +27.1% | -Over time, 3.0.1 → master: **+291.2%** without a JIT, **+372.4%** with YJIT, **+309.8%** with ZJIT. +Over time, 3.0.1 → master: **+285.7%** without a JIT, **+379.5%** with YJIT, **+305.3%** with ZJIT. ## Many static routes @@ -43,14 +43,14 @@ Over time, 3.0.1 → master: **+291.2%** without a JIT, **+372.4%** with YJIT, * | Version | No JIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT speedup | ZJIT (i/s) | μs/req | vs prev | vs 3.0.1 | ZJIT speedup | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 3.0.1 | 8,373 | 119.44 | — | — | 10,653 | 93.87 | — | — | +27.2% | 8,826 | 113.31 | — | — | +5.4% | -| 3.1.1 | 11,855 | 84.35 | +41.6% | +41.6% | 14,302 | 69.92 | +34.3% | +34.3% | +20.6% | 12,784 | 78.22 | +44.9% | +44.9% | +7.8% | -| 3.2.1 | 11,833 | 84.51 | -0.2% | +41.3% | 14,353 | 69.67 | +0.4% | +34.7% | +21.3% | 12,556 | 79.65 | -1.8% | +42.3% | +6.1% | -| 3.3.5 | 12,820 | 78.00 | +8.3% | +53.1% | 14,996 | 66.69 | +4.5% | +40.8% | +17.0% | 13,603 | 73.52 | +8.3% | +54.1% | +6.1% | -| 4.0.0 | 17,377 | 57.55 | +35.5% | +107.5% | 19,777 | 50.56 | +31.9% | +85.6% | +13.8% | 18,787 | 53.23 | +38.1% | +112.9% | +8.1% | -| master | 154,126 | 6.49 | +787.0% | +1740.8% | 307,715 | 3.25 | +1455.9% | +2788.5% | +99.7% | 201,657 | 4.96 | +973.4% | +2184.9% | +30.8% | +| 3.0.1 | 8,488 | 117.82 | — | — | 10,877 | 91.94 | — | — | +28.1% | 8,821 | 113.37 | — | — | +3.9% | +| 3.1.1 | 12,201 | 81.96 | +43.7% | +43.7% | 14,771 | 67.70 | +35.8% | +35.8% | +21.1% | 13,129 | 76.17 | +48.8% | +48.8% | +7.6% | +| 3.2.1 | 12,338 | 81.05 | +1.1% | +45.4% | 14,871 | 67.25 | +0.7% | +36.7% | +20.5% | 13,120 | 76.22 | -0.1% | +48.7% | +6.3% | +| 3.3.5 | 13,192 | 75.80 | +6.9% | +55.4% | 15,261 | 65.53 | +2.6% | +40.3% | +15.7% | 13,975 | 71.56 | +6.5% | +58.4% | +5.9% | +| 4.0.1 | 17,308 | 57.78 | +31.2% | +103.9% | 19,912 | 50.22 | +30.5% | +83.1% | +15.0% | 18,496 | 54.07 | +32.3% | +109.7% | +6.9% | +| master | 158,539 | 6.31 | +816.0% | +1767.9% | 318,447 | 3.14 | +1499.3% | +2827.8% | +100.9% | 211,627 | 4.73 | +1044.2% | +2299.2% | +33.5% | -Over time, 3.0.1 → master: **+1740.8%** without a JIT, **+2788.5%** with YJIT, **+2184.9%** with ZJIT. +Over time, 3.0.1 → master: **+1767.9%** without a JIT, **+2827.8%** with YJIT, **+2299.2%** with ZJIT. ## Many parameterized routes @@ -58,14 +58,14 @@ Over time, 3.0.1 → master: **+1740.8%** without a JIT, **+2788.5%** with YJIT, | Version | No JIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT (i/s) | μs/req | vs prev | vs 3.0.1 | YJIT speedup | ZJIT (i/s) | μs/req | vs prev | vs 3.0.1 | ZJIT speedup | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 3.0.1 | 7,561 | 132.26 | — | — | 9,565 | 104.55 | — | — | +26.5% | 8,080 | 123.76 | — | — | +6.9% | -| 3.1.1 | 10,412 | 96.04 | +37.7% | +37.7% | 12,333 | 81.09 | +28.9% | +28.9% | +18.4% | 11,122 | 89.91 | +37.6% | +37.6% | +6.8% | -| 3.2.1 | 10,382 | 96.32 | -0.3% | +37.3% | 12,305 | 81.26 | -0.2% | +28.6% | +18.5% | 11,091 | 90.16 | -0.3% | +37.3% | +6.8% | -| 3.3.5 | 11,364 | 88.00 | +9.5% | +50.3% | 12,868 | 77.71 | +4.6% | +34.5% | +13.2% | 11,750 | 85.11 | +5.9% | +45.4% | +3.4% | -| 4.0.0 | 14,535 | 68.80 | +27.9% | +92.2% | 16,321 | 61.27 | +26.8% | +70.6% | +12.3% | 15,377 | 65.03 | +30.9% | +90.3% | +5.8% | -| master | 14,456 | 69.18 | -0.5% | +91.2% | 16,189 | 61.77 | -0.8% | +69.3% | +12.0% | 15,352 | 65.14 | -0.2% | +90.0% | +6.2% | - -Over time, 3.0.1 → master: **+91.2%** without a JIT, **+69.3%** with YJIT, **+90.0%** with ZJIT. +| 3.0.1 | 7,596 | 131.65 | — | — | 9,838 | 101.65 | — | — | +29.5% | 8,274 | 120.85 | — | — | +8.9% | +| 3.1.1 | 10,610 | 94.25 | +39.7% | +39.7% | 12,669 | 78.93 | +28.8% | +28.8% | +19.4% | 11,453 | 87.31 | +38.4% | +38.4% | +7.9% | +| 3.2.1 | 10,810 | 92.51 | +1.9% | +42.3% | 12,760 | 78.37 | +0.7% | +29.7% | +18.0% | 11,394 | 87.76 | -0.5% | +37.7% | +5.4% | +| 3.3.5 | 11,523 | 86.79 | +6.6% | +51.7% | 13,170 | 75.93 | +3.2% | +33.9% | +14.3% | 12,106 | 82.60 | +6.2% | +46.3% | +5.1% | +| 4.0.1 | 14,497 | 68.98 | +25.8% | +90.8% | 16,322 | 61.27 | +23.9% | +65.9% | +12.6% | 15,433 | 64.80 | +27.5% | +86.5% | +6.5% | +| master | 100,913 | 9.91 | +596.1% | +1228.5% | 176,391 | 5.67 | +980.7% | +1693.0% | +74.8% | 124,391 | 8.04 | +706.0% | +1403.3% | +23.3% | + +Over time, 3.0.1 → master: **+1228.5%** without a JIT, **+1693.0%** with YJIT, **+1403.3%** with ZJIT. ## Notes - All versions exercise the same route-shape APIs, kept stable in `app.rb`. diff --git a/benchmark/version_throughput/run.rb b/benchmark/version_throughput/run.rb index 99dd891a6..d7a2247af 100644 --- a/benchmark/version_throughput/run.rb +++ b/benchmark/version_throughput/run.rb @@ -34,7 +34,7 @@ HERE = __dir__ TMP = File.join(ROOT, 'tmp', 'bench-versions') -DEFAULT_VERSIONS = %w[3.0.1 3.1.1 3.2.1 3.3.5 4.0.0 master].freeze +DEFAULT_VERSIONS = %w[3.0.1 3.1.1 3.2.1 3.3.5 4.0.1 master].freeze versions = (ENV['GRAPE_VERSIONS']&.split(',')&.map(&:strip) || DEFAULT_VERSIONS).freeze # One pass per JIT mode, in report column order. +flag+ is what `ruby` is diff --git a/lib/grape/router.rb b/lib/grape/router.rb index d1d27803d..f0808e8a4 100644 --- a/lib/grape/router.rb +++ b/lib/grape/router.rb @@ -20,6 +20,9 @@ def compile! return if @compiled @union = resolve_capture_groups(Regexp.union(@neutral_regexes), @neutral_map) + # Shared by the methods below: HEAD's routes mirror GET's, so their + # buckets compile to the same unions. + bucket_unions = {} # Compiled from the routes actually registered rather than from # Grape::HTTP_SUPPORTED_METHODS. A route declared with any other verb # (`route :purge, '/cache'`) is accepted at definition time and is @@ -31,8 +34,9 @@ def compile! optimized_map = routes.map.with_index { |route, index| route.to_regexp(index) } union = resolve_capture_groups(Regexp.union(optimized_map), routes) # Paired with the routes it was built from, so a match reads both - # through a single lookup. - @optimized_map[method] = [union, routes].freeze + # through a single lookup, and with the buckets that narrow those + # routes by a path segment (nil when no segment splits them well). + @optimized_map[method] = [union, routes, RouteBuckets.build(routes, optimized_map, bucket_unions)].freeze # Left out when no route spells out a path in full, so a request for # +method+ goes straight to the union rather than missing a lookup first. static_routes = static_routes_for(union, routes) @@ -126,12 +130,21 @@ def transaction(input, method, env) exact_route, captures = static response = process_static_route(exact_route, captures, env) else - # Matched here rather than through #match? so the MatchData survives: the - # route's path captures are groups of it (see Route#params_for). - union, routes = @optimized_map[method] - union&.match(input) do |m| - exact_route = routes.detect { |route| m[route.regexp_capture_group] } - response = process_route(exact_route, input, env, m) if exact_route + union, routes, buckets = @optimized_map[method] + if (bucket = buckets&.bucket_for(input)) + # A bucket numbers its groups its own way, so it hands over the + # captures numbered for its own union along with the match. + bucket.match(input) do |route, m, captures| + exact_route = route + response = process_route(route, input, env, m, captures) + end + else + # Matched here rather than through #match? so the MatchData survives: the + # route's path captures are groups of it (see Route#params_for). + union&.match(input) do |m| + exact_route = routes.detect { |route| m[route.regexp_capture_group] } + response = process_route(exact_route, input, env, m, exact_route.union_captures) if exact_route + end end end return response if halt?(response) @@ -216,10 +229,12 @@ def close_body(response) # Routing args are rebuilt for every attempt: when a route cascades # (X-Cascade pass), the next candidate must not observe the previous # attempt's +route_info+ or path captures. - def process_route(route, input, env, match = nil, include_allow_header: false) + def process_route(route, input, env, match = nil, captures = nil, include_allow_header: false) # The path captures are the hash: +route_info+ is written into them - # rather than merged in from a second one. - routing_args = route.params_for(input, match) || {} + # rather than merged in from a second one. +captures+ names the groups of + # +match+ that hold them: the router's union numbers them one way, a + # bucket's another. + routing_args = route.params_for(input, match, captures) || {} routing_args[:route_info] = route env[Grape::Env::GRAPE_ROUTING_ARGS] = routing_args env[Grape::Env::GRAPE_ALLOWED_METHODS] = route.allow_header if include_allow_header @@ -230,7 +245,7 @@ def process_route(route, input, env, match = nil, include_allow_header: false) # read when the table was built; each request gets its own copy of each, # the way it gets fresh strings out of a match. def process_static_route(route, captures, env) - routing_args = captures ? captures.transform_values(&:dup) : {} + routing_args = captures&.transform_values(&:dup) || {} routing_args[:route_info] = route env[Grape::Env::GRAPE_ROUTING_ARGS] = routing_args route.call(env) diff --git a/lib/grape/router/base_route.rb b/lib/grape/router/base_route.rb index 021a4d3bc..1a96fa612 100644 --- a/lib/grape/router/base_route.rb +++ b/lib/grape/router/base_route.rb @@ -72,6 +72,19 @@ def to_regexp(index) # never at request time -- instances are shared across threads. attr_reader :union_captures + # {#union_captures} renumbered for a union in which this route's own + # group is +group+ -- one of a bucket's, say. Every capture sits at a + # fixed offset from the route's own group, so the whole map shifts by + # the same number. Nil when the route has no reusable captures. + def union_captures_at(group) + return if @union_captures.nil? + + shift = group - @regexp_capture_group + return @union_captures if shift.zero? + + @union_captures.transform_values { |capture| capture + shift }.freeze + end + # @api private # @see #regexp_capture_group def resolve_capture_group!(union_named_captures) diff --git a/lib/grape/router/greedy_route.rb b/lib/grape/router/greedy_route.rb index ef45f3360..7d33cc546 100644 --- a/lib/grape/router/greedy_route.rb +++ b/lib/grape/router/greedy_route.rb @@ -22,7 +22,7 @@ def params nil end - def params_for(_input, _match = nil) + def params_for(_input, _match = nil, _captures = nil) nil end end diff --git a/lib/grape/router/route.rb b/lib/grape/router/route.rb index 38524ec83..875346c32 100644 --- a/lib/grape/router/route.rb +++ b/lib/grape/router/route.rb @@ -53,13 +53,15 @@ def params # # +match+, when given, is the union MatchData the router matched this route # with; its groups already hold the substrings a second match would produce - # (see {BaseRoute#union_captures}). Mustermann percent-decodes a captured - # value containing '%', and none can unless the path does -- so a '%'-free - # path reads the groups, anything else the full match below. - def params_for(input, match = nil) + # (see {BaseRoute#union_captures}). +captures+ names those groups, since a + # route sits at one number in the router's union and at another in each + # bucket that holds it (see {BaseRoute#union_captures_at}); nil there means + # the match cannot stand in for Mustermann. Mustermann percent-decodes a + # captured value containing '%', and none can unless the path does -- so a + # '%'-free path reads the groups, anything else the full match below. + def params_for(input, match = nil, captures = union_captures) return unless pattern.captures? - captures = union_captures return params_from_union(captures, match) if match && captures && !input.include?('%') parsed = pattern.params(input) diff --git a/lib/grape/router/route_buckets.rb b/lib/grape/router/route_buckets.rb new file mode 100644 index 000000000..6bd9d7a5f --- /dev/null +++ b/lib/grape/router/route_buckets.rb @@ -0,0 +1,223 @@ +# frozen_string_literal: true + +module Grape + class Router + # Narrows the routes a request is matched against to the ones that can + # match its path, by one literal path segment. + # + # A route that spells out a literal at segment +k+ of its path -- with + # every segment ahead of it a literal or a lone +:param+ that cannot span + # a '/' -- matches only paths whose segment +k+ is that literal. Grouping + # the routes by it, and keeping every route that cannot be grouped in each + # group, gives each group the routes that could match a path with that + # segment, still in registration order, so the first of them to match is + # the route the full union would have picked. + # + # Mustermann lets a literal character match its percent-encoding, and a + # route's pattern ends in \Z, which also matches before a trailing + # newline, so a path holding either is left to the full union. + # + # For example, a method with these routes, in registration order: + # + # /users /users/:id /users/:id/posts /posts /posts/:id + # /comments/:id /tags/:name /:slug + # + # is split by segment 1, which names every route but +/:slug+: + # + # "users" => /users, /users/:id, /users/:id/posts, /:slug + # "posts" => /posts, /posts/:id, /:slug + # "comments" => /comments/:id, /:slug + # "tags" => /tags/:name, /:slug + # fallback => /:slug + # + # A request for +/users/42+ is matched against the four routes under + # "users" instead of all eight, +/about+ names no bucket and is matched + # against the fallback, and +/users%2F42+ goes to the full union. + class RouteBuckets + # The routes one segment value can reach, with the number each one's + # group has in +union+: a route sits in several buckets, at a different + # position in each, so the numbers cannot be kept on the route. + class Bucket + def initialize(union, routes, groups, captures) + @union = union + @routes = routes + @groups = groups + @captures = captures + freeze + end + + # Yields the route +input+ matched, the match, and that route's + # captures numbered for this union -- nil when a match cannot stand in + # for what Mustermann returns. Yields nothing when no route here + # matches. + def match(input) + @union&.match(input) do |m| + index = @groups.index { |group| m[group] } + yield(@routes[index], m, @captures[index]) if index + end + end + end + + NONE = Bucket.new(nil, [].freeze, [].freeze, [].freeze) + FULL_UNION_INPUT = /[%\n]/ + # A segment ahead of the key: plain characters, or one +:param+. + LITERAL_SEGMENT = /\A[^:*?(){}|\\%]+\z/ + CAPTURE_SEGMENT = /\A:(\w+)\z/ + # The key itself is compared with a slice of the path cut at the first + # '.', so it holds no '.', and nothing a path could spell differently. + KEY_SEGMENT = /\A[^:*?(){}|\\%.+\s]+\z/ + # Below this many routes the full union is already cheap to walk, and + # the extra unions would only lengthen boot. + MINIMUM_ROUTES = 8 + private_constant :Bucket, :NONE, :FULL_UNION_INPUT, :LITERAL_SEGMENT, :CAPTURE_SEGMENT, :KEY_SEGMENT, :MINIMUM_ROUTES + + class << self + # Buckets for +routes+ (with +regexps+, their union alternatives), or + # nil when no segment splits them well enough to be worth the unions. + # +unions+ is shared by a router's methods, so a bucket whose + # alternatives another method already compiled -- the HEAD routes + # mirroring GET's -- reuses that union. + # + # Every segment position is tried, and the one leaving the fewest + # routes to match wins. For the routes in the class example: + # + # position 1: users x3, posts x2, comments, tags, nil cost 1 + 3 = 4 + # position 2: nil for all eight cost 8 + # position 3: posts for /users/:id/posts, nil for 7 cost 7 + 1 = 8 + # + # Position 1 wins, and a cost of 4 is within half of the 8 routes, so + # the buckets are built; at 5 they would not be. + def build(routes, regexps, unions) + return if routes.size < MINIMUM_ROUTES + + segments = routes.map { |route| route.origin.split('/') } + depth = segments.map(&:size).max + keys_by_position = Array.new(depth) { Array.new(routes.size) } + routes.each_with_index { |route, index| fill_keys(keys_by_position, index, route, segments[index]) } + candidates = (1...depth).map { |position| [position, keys_by_position[position]] } + position, keys = candidates.min_by { |_, route_keys| cost(route_keys) } + return unless position && cost(keys) <= routes.size / 2 + + new(position, routes, regexps, keys, unions) + end + + private + + # Writes into +keys_by_position+, at +index+, the literal +route+ + # requires at each segment position, leaving nil where that segment + # cannot tell it apart. One walk over +segments+: a segment that does + # not span exactly one path segment ends it, as no position past that + # segment can tell the route apart either. + def fill_keys(keys_by_position, index, route, segments) + last = segments.size - 1 + position = 1 + while position <= last + segment = segments[position] + keys_by_position[position][index] = segment if key?(route, segment, position == last) + return unless single_segment?(route, segment) + + position += 1 + end + end + + def key?(route, segment, last_segment) + return false unless segment.match?(KEY_SEGMENT) && segment.ascii_only? + + # An unanchored route's trailing '/?*path' can run on into this segment. + !last_segment || route.anchor + end + + def single_segment?(route, segment) + return true if segment.match?(LITERAL_SEGMENT) + + name = segment[CAPTURE_SEGMENT, 1] + return false unless name + return Array(route.version).none? { |version| version.to_s.include?('/') } if name == 'version' + + requirements = route.requirements + !(requirements && (requirements.key?(name.to_sym) || requirements.key?(name))) + end + + # The most routes one request can still be matched against. + def cost(keys) + keys.count(nil) + (keys.compact.tally.values.max || 0) + end + end + + def initialize(position, routes, regexps, keys, unions) + @position = position + ungrouped, grouped = split_indices(keys) + # Every bucket holds the routes no segment tells apart, so none can be + # built before the whole of +ungrouped+ is known. + grouped.transform_values! { |indices| bucket(routes, regexps, (indices + ungrouped).sort!, unions) } + @buckets = grouped.freeze + @fallback = ungrouped.empty? ? NONE : bucket(routes, regexps, ungrouped, unions) + freeze + end + + # The bucket for +input+, or nil when it has to go through the full union. + # With the class example's buckets: + # + # /users/42, /users.json, /users/42.json => "users" + # /about, / => the fallback + # /users%2F42, "/users\n" => nil + def bucket_for(input) + return if input.match?(FULL_UNION_INPUT) + + key = segment_key(input) + (key && @buckets[key]) || @fallback + end + + private + + # The indices of the routes no key names, and the indices each key does + # name, in one pass over +keys+ -- +keys[n]+ being the key of the route + # at +n+, or nil when no segment tells that route apart. + def split_indices(keys) + ungrouped = [] + grouped = {} + keys.each_with_index do |key, index| + next ungrouped << index if key.nil? + + (grouped[key] ||= []) << index + end + [ungrouped, grouped] + end + + def bucket(routes, regexps, indices, unions) + alternatives = regexps.values_at(*indices).freeze + members = routes.values_at(*indices).freeze + union, groups = unions[alternatives] ||= compile_union(alternatives, members) + # A member sits at a different group here than it does in the router's + # union, which is what its own captures are numbered for. + captures = members.map.with_index { |route, index| route.union_captures_at(groups[index]) }.freeze + Bucket.new(union, members, groups, captures) + end + + # The union of +alternatives+ and the group each of +members+ has in it. + # Each alternative names its group after its route's position, so equal + # alternatives spell out the same union with the same groups, whichever + # method's routes they came from. + def compile_union(alternatives, members) + union = Regexp.union(alternatives) + named_captures = union.named_captures + [union, members.map { |route| named_captures.fetch(route.regexp_capture_index).first }.freeze].freeze + end + + # Segment +@position+ of +input+, cut at its first '.'. + def segment_key(input) + start = 0 + @position.times do + start = input.index('/', start) + return unless start + + start += 1 + end + stop = input.index('/', start) || input.length + dot = input.index('.', start) + stop = dot if dot && dot < stop + input[start, stop - start] + end + end + end +end diff --git a/spec/grape/router/route_buckets_spec.rb b/spec/grape/router/route_buckets_spec.rb new file mode 100644 index 000000000..81ee427f2 --- /dev/null +++ b/spec/grape/router/route_buckets_spec.rb @@ -0,0 +1,121 @@ +# frozen_string_literal: true + +# A method with enough routes is matched only against the routes that can +# match a request's path, told apart by one of its literal segments. Each +# example routes a request that narrowing could send to the wrong route, or to +# none. +describe Grape::Router::RouteBuckets do + def route_for(path) + get path + expect(last_response.status).to eq(200) + JSON.parse(last_response.body) + end + + context 'with path versioning and a JSON format' do + let(:app) do + Class.new(Grape::API) do + prefix :api + format :json + version 'v1', using: :path + + get('/:anything/first') { { route: 'anything first' } } + 10.times do |i| + get("/resource#{i}") { { route: "index #{i}" } } + get("/resource#{i}/:id") { { route: "show #{i}", id: params[:id] } } + end + get('/v1.2/items/:id') { { route: 'dotted literal' } } + get('/proxy', anchor: false) { { route: 'unanchored' } } + get('/:slug') { { route: 'slug', slug: params[:slug] } } + end + end + + it 'routes a request to the route named by its segment' do + expect(route_for('/api/v1/resource7/42')).to eq('route' => 'show 7', 'id' => '42') + end + + it 'routes a request whose later segment spells the segment of another route' do + expect(route_for('/api/v1/resource1/resource0')).to eq('route' => 'show 1', 'id' => 'resource0') + end + + it 'prefers an earlier route whose segment is a param' do + expect(route_for('/api/v1/resource1/first')).to eq('route' => 'anything first') + end + + it 'falls back to the routes whose segment is a param when no route names it' do + expect(route_for('/api/v1/unknown')).to eq('route' => 'slug', 'slug' => 'unknown') + end + + it 'routes a segment spelled with a percent-encoding' do + expect(route_for('/api/v1/%72esource7/42')).to eq('route' => 'show 7', 'id' => '42') + end + + it 'routes a segment followed by the format extension' do + expect(route_for('/api/v1/resource7.json')).to eq('route' => 'index 7') + end + + it 'captures a param followed by the format extension' do + expect(route_for('/api/v1/resource7/42.json')).to eq('route' => 'show 7', 'id' => '42') + end + + it 'routes a literal segment holding a dot' do + expect(route_for('/api/v1/v1.2/items/42')).to eq('route' => 'dotted literal') + end + + it 'routes an unanchored route on a path running on from its last segment' do + expect(route_for('/api/v1/proxyextra')).to eq('route' => 'unanchored') + end + end + + context 'with a param whose requirement spans a slash ahead of the literal' do + let(:app) do + Class.new(Grape::API) do + format :json + 10.times do |i| + get("/:tenant/resource#{i}/:id", requirements: { tenant: %r{[^/]+/[^/]+} }) do + { route: "show #{i}", tenant: params[:tenant] } + end + end + end + end + + it 'routes a request whose param covers two segments' do + expect(route_for('/acme/eu/resource3/42')).to eq('route' => 'show 3', 'tenant' => 'acme/eu') + end + end + + context 'when the routes are told apart by their first segment' do + let(:app) do + Class.new(Grape::API) do + format :json + 10.times do |i| + get("/resource#{i}/:id") { { route: "show #{i}" } } + end + end + end + + it 'answers the root path as one no route matched' do + get '/' + + expect(last_response.status).to eq(404) + end + end + + context 'with methods whose routes line up position for position' do + let(:app) do + Class.new(Grape::API) do + format :json + 10.times do |i| + get("/resource#{i}/:id") { { route: "show #{i}" } } + post("/resource#{i}/:id/items") { { route: "add item #{i}" } } + end + end + end + + it 'matches each method against its own routes' do + post '/resource3/42/items' + + expect(last_response.status).to eq(201) + expect(JSON.parse(last_response.body)).to eq('route' => 'add item 3') + end + end +end