I’m new to working with APIs, especially with POST requests and pagination. I’ve managed to set up an API to pull data from Clarizen for Power BI M Query, but I’m hitting a wall with pagination.

I’m only getting 5,000 records even though I know there’s more data available. I tried increasing the limit to 8,000, but that didn’t work. Any advice on how to handle this? I’ve included some notes from the API documentation that might help.

Thanks a lot for any help you can offer!

Query 
let
    // Endpoint URL
    RequestUrl = "https://api2.clarizen.com/v2.0/services/data/query",

    // Request Headers as a record
    RequestHeaders = [
        #"Authorization" = "Session " & Login,
        #"Content-Type" = "application/json"
    ],

    // Request Body as binary
    RequestBody = Text.ToBinary("{
         ""q"": ""SELECT CreatedBy, CreatedOn, LastUpdatedBy, FROM timesheet 
         WHERE C_AdjustedReportedDate >= '09/01/2023' AND State = '/State/Approved'AND Duration > 0 LIMIT 8000 ""
    }"),

    // Send POST request
    ResponseMessage = Web.Contents(RequestUrl, [
        Headers = RequestHeaders,
        Content = RequestBody
    ]),

    // Parse JSON response
    ResponseJson = Json.Document(ResponseMessage),
    // Parse JSON response
    entities = ResponseJson[entities]
in
    entities

Pagine response

"paging": {
        "from": 100,
        "limit": 100,
        "hasMore": true
    }

Notes
https://api.clarizen.com/V2.0/services/data/EntityQuery

https://api.clarizen.com/V2.0/services/types/Paging

Paging information returned from this query. If paging.*hasMore* is true, this object should be 
passed as is, to the same query API in order to retrieve the next page

Paging information for a query

from -  integer -   The record number to start retrieve from
limit-  integer -   Number of records to retrieve
hasMore-    boolean -   When a query results is returned, 
indicates whether there are more records to fetch for this query

Paging was changed. Instead of transferring PageNumber and PageSize parameters, 
you now need to send the From and Limit parameters which allows more flexibility. If you followed the API 
best practices of taking the Paging object from a query result and pass it “as-is” to the next request,
then no changes are required