Skip to content

perf bug: repeated decompression of runend ends #10126

Description

@tpoterba

What happened?

execute_scalar multiple times on a pco encoded array of ends:
find_slice_end_index in runend array:

root: vortex.runend(u16, len=524288) nbytes=13.97 kB (100.00%) 
  metadata: offset: 0                                          
  ends: vortex.pco(u32, len=27382) nbytes=8.29 kB (59.35%)     
    metadata: ptype: u32, nrows: 27382, slice: 0..27382        
    buffer: chunk_meta_0 host 76 B (align=1) (0.92%)           
    buffer: page_0 host 2.14 kB (align=1) (25.83%)             
    buffer: page_1 host 2.04 kB (align=1) (24.57%)             
    buffer: page_2 host 1.95 kB (align=1) (23.50%)             
    buffer: page_3 host 2.09 kB (align=1) (25.18%)             
  values: vortex.pco(u16, len=27382) nbytes=5.68 kB (40.65%)   
    metadata: ptype: u16, nrows: 27382, slice: 0..27382        
    buffer: chunk_meta_0 host 19 B (align=1) (0.33%)           
    buffer: page_0 host 1.45 kB (align=1) (25.48%)             
    buffer: page_1 host 1.38 kB (align=1) (24.34%)             
    buffer: page_2 host 1.38 kB (align=1) (24.34%)             
    buffer: page_3 host 1.45 kB (align=1) (25.50%)             

Steps to reproduce

See above

Environment

exists in latest

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA bug issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions