REST: Add pagination support for list_views

[ebyhr]

Rationale for this change

Follows REST catalog spec:

• /v1/{prefix}/views: https://github.com/apache/iceberg/blob/e7a5a87f26f9de5b200254155aa037368b13a29c/open-api/rest-catalog-open-api.yaml#L1525-L1562
• ListTablesResponse: https://github.com/apache/iceberg/blob/e7a5a87f26f9de5b200254155aa037368b13a29c/open-api/rest-catalog-open-api.yaml#L4210-L4219
• PageToken: https://github.com/apache/iceberg/blob/e7a5a87f26f9de5b200254155aa037368b13a29c/open-api/rest-catalog-open-api.yaml#L2233-L2256

Are these changes tested?

Yes, includes unit tests for paginated cases.

Are there any user-facing changes?

• Before: returned incomplete view list when server paginated (only first page)
• After: returns complete view list (fetches all pages)

[geruh]

Thanks for raising @ebyhr! I did a quick pass. Overall looks good, left a few comments.

[geruh]

Can we add some tests for an ommitted field and null for the next_page_token, and ensure the behavior is the same?

[ebyhr]

This test (test_list_views_paginated_200) verifies the omitted next-page-token field. Added another test with "next-page-token": None.

Let me know if you want me to add more tests.

[rambleraptor]

Open question for the community: do we want pagination to happen automatically? If you have hundreds of pages of views, calling list_views will now take hundreds of REST calls. Giving you controls on pagination is more annoying, but it gives you a lot more control over the number of REST calls you're making.

I'm personally in favor of having users manually do pagination and/or having a pagination flag somewhere (that defaults to False)

[ebyhr]

@rambleraptor I don't believe this is the most suitable place for discussing this topic. Implementing a pagination flag necessitates a modification to the REST catalog specification in my understanding. If you wish to continue the discussion, I suggest bringing it to the dev mailing list.

[rambleraptor]

I'm saying that PyIceberg users should have a way to call list_views with/without pagination automatically applied.

This wouldn't be a Catalog specification change. It's just an interface choice for PyIceberg.

[ebyhr]

@rambleraptor, I'm confused. The pagination is actually managed by the REST catalog server. PyIceberg is a client calling the endpoints.

What happens when pagination is set to false? Does increasing the page-size to the maximum integer value?