Skip to content

gh-149718: Aggregate same stack frames in Tachyon in some collectors#149719

Merged
pablogsal merged 7 commits into
python:mainfrom
maurycy:remote-debugger-batching
May 12, 2026
Merged

gh-149718: Aggregate same stack frames in Tachyon in some collectors#149719
pablogsal merged 7 commits into
python:mainfrom
maurycy:remote-debugger-batching

Conversation

@maurycy
Copy link
Copy Markdown
Contributor

@maurycy maurycy commented May 12, 2026

Please see gh-149718 for more details.

On a MacBook M4, using test_asyncio since it has longer stacks (without --async-aware):

Sample rate (samples/sec)

Format main (c6fd7de) remote-debugger-batching (2864c275847)
--pstats 31,161.59 103,657.01
--collapsed 57,302.45 102,350.22
--flamegraph 33,490.73 100,641.14
--gecko 31,346.37 98,031.88
--heatmap 14,960.64 97,607.28
--jsonl 24,554.56 98,018.93

Error rate (%)

Format main (c6fd7de) remote-debugger-batching (2864c275847)
--pstats 1.94 0.54
--collapsed 1.47 0.78
--flamegraph 1.77 0.57
--gecko 1.92 0.57
--heatmap 2.33 0.44
--jsonl 2.06 0.54

Missed samples (%)

Format main (c6fd7de) remote-debugger-batching (2864c275847)
--pstats 90.65 68.90
--collapsed 82.81 69.29
--flamegraph 89.95 69.81
--gecko 90.60 70.59
--heatmap 95.51 70.72
--jsonl 92.63 70.59

Bench

mkdir -p /tmp/bench_results

git checkout c6fd7de
sudo -n ./python.exe -m profiling.sampling run -r 300khz --pstats     -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_pstats.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --collapsed  -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_collapsed.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --flamegraph -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_flamegraph.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --gecko      -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_gecko.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --heatmap    -o /tmp/heatmap_main    -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_heatmap.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --jsonl      -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_jsonl.txt

git checkout remote-debugger-batching
sudo -n ./python.exe -m profiling.sampling run -r 300khz --pstats     -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_pstats.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --collapsed  -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_collapsed.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --flamegraph -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_flamegraph.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --gecko      -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_gecko.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --heatmap    -o /tmp/heatmap_batching  -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_heatmap.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --jsonl      -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_jsonl.txt

Comment thread Lib/profiling/sampling/collector.py
Copy link
Copy Markdown
Member

@pablogsal pablogsal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fantastic! Thanks a lot @maurycy 🚀 Excellent job

@pablogsal pablogsal enabled auto-merge (squash) May 12, 2026 23:31
@pablogsal pablogsal added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 12, 2026
@pablogsal pablogsal merged commit 76f2285 into python:main May 12, 2026
58 checks passed
@miss-islington-app
Copy link
Copy Markdown

Thanks @maurycy for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented May 12, 2026

GH-149747 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 12, 2026
@maurycy maurycy deleted the remote-debugger-batching branch May 12, 2026 23:47
pablogsal pushed a commit that referenced this pull request May 13, 2026
…ectors (GH-149719) (#149747)

gh-149718: Aggregate same stack frames in Tachyon in some collectors (GH-149719)
(cherry picked from commit 76f2285)

Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants