Skip to content

skip gforce() setup when j is only .N - #7920

Open
m-muecke wants to merge 1 commit into
masterfrom
gforce-dotN
Open

m-muecke wants to merge 1 commit into
masterfrom
gforce-dotN

Conversation

@m-muecke

Copy link
Copy Markdown
Contributor

On small tables data.table seems to be a bit slower for DT[, .N, by = g] than dplyr. AFAICT, this is due to overhead from gforce() that is not needed. An alternative approach would've been returning a GForce_N element in the output list of .optimize_gforce() and checking with that.

master

library(data.table, warn.conflicts = FALSE)

set.seed(1)
n = 1e7
bench::press(
  groups = c(10, 1e6),
  {
    DT = data.table(g = sample.int(groups, n, TRUE), v = runif(n))
    df = as.data.frame(DT)
    bench::mark(
      data.table = DT[, .N, by = g],
      dplyr = dplyr::count(df, g),
      check = FALSE,
      iterations = 20,
      filter_gc = FALSE
    )
  }
)
#> Running with:
#>    groups
#> 1      10
#> 2 1000000
#> # A tibble: 4 × 7
#>   expression  groups      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>   <dbl> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 data.table      10  37.33ms   38.1ms    22.8     268.8MB    22.8 
#> 2 dplyr           10  22.75ms   26.3ms    38.7      80.9MB     9.68
#> 3 data.table 1000000 107.77ms  112.4ms     8.01    297.6MB     8.01
#> 4 dplyr      1000000    3.67s     3.8s     0.260   219.7MB    13.9

Created on 2026-10-13 with reprex v2.1.1

PR

library(data.table, warn.conflicts = FALSE)

set.seed(1)
n = 1e7
bench::press(
  groups = c(10, 1e6),
  {
    DT = data.table(g = sample.int(groups, n, TRUE), v = runif(n))
    df = as.data.frame(DT)
    bench::mark(
      data.table = DT[, .N, by = g],
      dplyr = dplyr::count(df, g),
      check = FALSE,
      iterations = 20,
      filter_gc = FALSE
    )
  }
)
#> Running with:
#>    groups
#> 1      10
#> 2 1000000
#> # A tibble: 4 × 7
#>   expression  groups      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>   <dbl> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 data.table      10   17.8ms   18.5ms    52.5      39.9MB    18.4 
#> 2 dplyr           10   22.4ms   23.7ms    39.1      80.9MB    23.5 
#> 3 data.table 1000000   94.5ms   97.6ms    10.1      68.7MB     4.02
#> 4 dplyr      1000000     3.7s     3.9s     0.255   219.7MB    14.6

Created on 2026-10-13 with reprex v2.1.1

@codecov

codecov Bot commented Oct 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.02%. Comparing base (1d87418) to head (4b729cd).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7920   +/-   ##
=======================================
  Coverage   99.02%   99.02%           
=======================================
  Files          88       88           
  Lines       17381    17383    +2     
=======================================
+ Hits        17211    17213    +2     
  Misses        170      170           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jangorecki
jangorecki requested a review from ben-schwen October 13, 2026 21:19

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant