[fs] Optimize cloud file listing with per-page filtering and early termination in PinotFS#17847

KKcorps merged 2 commits intoapache:masterapache/pinot:masterfrom

anshul98ks123:preview-pagination-list-metadataanshul98ks123/pinot:preview-pagination-list-metadataCopy head branch name to clipboard