Cannot change `limit` in paginated.post_search() about globus-sdk-python

[sirosen]

Sorry about the delayed response here. I missed your initial issue coming in and am just reading this now.

I think one less hacky feeling approach would be to make the paginated call to get the paginator and then modify the limit parameter which it is using.
That is

paginator = sc.paginated.post_search(index_id, query_data)
paginator.limit = 300
for response in paginator:
    ...

I'm currently looking at the limit+offset pagination code to see if there are issues with this suggestion, and I believe it will work safely.
For now, that is what I would recommend for this customization.

---

More broadly, what you're asking about is how to control pagination parameters (rather than the parameters to the paginated call).

I need to think harder about what a good interface looks like. It might be that we need to recognize parameters which are valid for both purposes (like limit) and "lift" them into the paginator if appropriate, such that a paginated search with limit=300 "just works". But this model needs to work consistently for other pagination modes in order for us to confidently offer that as our interface.

For now, I think building a paginator and then adjusting its attributes is the best way. There are other possibilities for the future, but we'll need to do some analysis to figure out what's best for all cases.

EDIT: sorry, there was a typo in the original code sample I shared.

[nocollier]

That feels much better, many thanks.

[nocollier]

I have looked into this further and it seems to me that the limit keyword is not used in the paginated version of the method. If you look at this line you see that the post_search method has the paginated decorator where the page_size=100. This is used when the client is instantiated to build paginated versions of any routine with this decorator. However your page size is fixed. If I add:

sc.paginated.post_search._paginator_params['page_size'] = 300

just above my post_search then I get the expected pagination. But this is more of a hack, is there a more canonical method of selecting page size in paginated calls?