[pull] master from ruby:master - #1367
Merged
Merged
Conversation
We were missing a lot of calls in our top-called list.
Before:
```
Top-20 most called JIT functions (20.4% of total 12,040,141):
fetch@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/map.rb:183: 268,895 ( 2.2%)
fetch_or_store@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/map.rb:205: 267,411 ( 2.2%)
each@<internal:array>:222: 252,044 ( 2.1%)
klass@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/reflection.rb:423: 145,723 ( 1.2%)
connection_specification_name@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/connection_handling.rb:320: 126,207 ( 1.0%)
[]@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:187: 124,689 ( 1.0%)
fetch@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/result.rb:76: 123,449 ( 1.0%)
html_escape@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/core_ext/erb/util.rb:11: 115,758 ( 1.0%)
foreign_key@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/reflection.rb:559: 113,200 ( 0.9%)
association@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations.rb:54: 98,595 ( 0.8%)
map@<internal:array>:240: 97,415 ( 0.8%)
klass@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations/association.rb:166: 93,497 ( 0.8%)
===@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/core_ext/time/calculations.rb:19: 86,726 ( 0.7%)
for@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/tzinfo-2.0.6/lib/tzinfo/timestamp.rb:112: 86,521 ( 0.7%)
connected_to_stack@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/core.rb:216: 80,057 ( 0.7%)
[]@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/isolated_execution_state.rb:32: 78,839 ( 0.7%)
context@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/isolated_execution_state.rb:55: 77,134 ( 0.6%)
initialize@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/tzinfo-2.0.6/lib/tzinfo/timezone_transition.rb:35: 76,603 ( 0.6%)
block in redefine@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/class_attribute.rb:15: 73,963 ( 0.6%)
association_instance_get@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations.rb:84: 72,814 ( 0.6%)
```
After:
```
Top-20 most called JIT functions (22.6% of total 22,686,232):
block in _load_from_sql@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/querying.rb:77: 564,328 ( 2.5%)
each@<internal:array>:222: 467,559 ( 2.1%)
block in redefine@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/class_attribute.rb:15: 381,214 ( 1.7%)
fetch_value@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activemodel-8.1.1/lib/active_model/attribute_set/builder.rb:42: 365,814 ( 1.6%)
block in merge@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/merger.rb:60: 314,382 ( 1.4%)
block (2 levels) in clean_load_path@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/bundler-4.0.12/lib/bundler/shared_helpers.rb:370: 311,248 ( 1.4%)
block in column_types@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/result.rb:172: 282,164 ( 1.2%)
fetch@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/map.rb:183: 268,899 ( 1.2%)
fetch_or_store@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/concurrent-ruby-1.3.5/lib/concurrent-ruby/concurrent/map.rb:205: 267,411 ( 1.2%)
block in perform_query@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/connection_adapters/sqlite3/database_statements.rb:123: 267,148 ( 1.2%)
_read_attribute@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/attribute_methods/read.rb:39: 248,941 ( 1.1%)
klass@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/reflection.rb:423: 219,426 ( 1.0%)
block in period_for@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/tzinfo-2.0.6/lib/tzinfo/data_sources/transitions_data_timezone_info.rb:45: 189,701 ( 0.8%)
safe_concat@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/actionview-8.1.1/lib/action_view/buffers.rb:57: 153,995 ( 0.7%)
block in print_counters_with_prefix@<internal:zjit>:253: 143,882 ( 0.6%)
connection_specification_name@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/connection_handling.rb:320: 140,225 ( 0.6%)
association_instance_get@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations.rb:84: 138,333 ( 0.6%)
association@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations.rb:54: 138,322 ( 0.6%)
klass@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations/association.rb:166: 137,059 ( 0.6%)
[]@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:187: 124,689 ( 0.5%)
```
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
This is a small cleanup, but since `BitSet::insert` returns whether the value was newly inserted, we don't need a separate `necessary.get(insn_id)` lookup.
Box-local extension loading does not work in processes that fork. The process-private directory holding the extension copies (104c0f0) is cached in the static `box_ext_tmp_dir`, which a forked child inherits. When the child exits, its VM teardown removes the directory the parent still owns, so every later box-local extension require in the parent fails with `LoadError: can't prepare the extension file for Ruby Box (...): can't open the file to write`. A child that loads extensions itself is broken too: it reuses the parent's directory name and sequence counter. box = Ruby::Box.new box.require "digest/md5" pid = fork {} Process.wait(pid) Ruby::Box.new.require "digest/sha2" # LoadError Record the pid that created the directory, remove it at teardown only in that process, and discard the inherited path after fork so each process creates its own directory. This fixes `bundler/inline` under RUBY_BOX=1, which forks to run the install: 25 examples of bundler's spec/runtime/inline_spec.rb fail without this change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bumps the github-actions group with 1 update in the / directory: [taiki-e/install-action](https://github.com/taiki-e/install-action). Updates `taiki-e/install-action` from 2.87.0 to 2.87.1 - [Release notes](https://github.com/taiki-e/install-action/releases) - [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md) - [Commits](taiki-e/install-action@37f7c57...742a331) --- updated-dependencies: - dependency-name: taiki-e/install-action dependency-version: 2.87.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
When we run out of JIT memory, we don't compile the iseq, so jit_entry
will be forever NULL. This means that the JIT call counters (jit_entry_calls
and jit_exception_calls) will continue to increase on every call. This
is bad for copy-on-write because it causes the whole rb_iseq_constant_body
to be copied.
We can see an example using the following script (which can only be ran
on Linux) that allocates 200k methods, executes all the methods, then
forks a child process that also executes all the methods and outputs
the change in RSS and PSS:
SYMS = 200_000.times.map { |i| eval("def m#{i} = nil"); :"m#{i}" }
def stats = File.read("/proc/self/smaps_rollup").scan(/^(Rss|Pss):\s+(\d+)/).to_h
SYMS.each { |s| 3.times { send(s) } }
fork do
before = stats
SYMS.each { |s| send(s) }
after = stats
puts "RSS: #{before["Rss"]} -> #{after["Rss"]} kB"
puts "PSS: #{before["Pss"]} -> #{after["Pss"]} kB"
end
Process.wait
When running with YJIT (with `--yjit-call-threshold=1 --yjit-exec-mem-size=1`),
we can see that before this commit, the PSS increases significantly but
stays flat after this commit:
Before:
RSS: 210236 -> 210496 kB
PSS: 104676 -> 162300 kB
After:
RSS: 204308 -> 204692 kB
PSS: 101700 -> 101960 kB
We can see a similar result with ZJIT (with `--zjit-call-threshold=1 --zjit-exec-mem-size=1`):
Before:
RSS: 493564 -> 493732 kB
PSS: 246416 -> 390452 kB
After:
RSS: 493292 -> 493516 kB
PSS: 246286 -> 246428 kB
We may not know anything about the shape because the profiles are megamorphic, but we might know the class statically. Use that to optimize method lookup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )