Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions adaptors/library/jobs/fetch-all-events-with-pagination.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// Fetching all events with pagination

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So why are there 2 export statements here?

One seems to pull all data in pages of 1000, the other then iterates over each page and pulls it again?

I'm not sure what the story is here but I think it needs explaining in comments

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a comment for both tracker.export

// Requires DHIS2 server v2.41+

// First call fetches page 1 (up to 10,000 records) and, via totalPages: true,
// returns pagination details (state.data.pager) describing how many pages remain.
tracker.export('events', { totalPages: true, pageSize: 1e4 });

fn(state => {
Comment thread
mtuchi marked this conversation as resolved.
state.results = state.data.events;
const { page, pageSize, pageCount, total } = state.data.pager;
const remainingPages = pageCount - page;

state.pages = Array.from({ length: remainingPages }, (_, i) => page + i + 1);
state.pageSize = pageSize;
return state;
});

// Second call uses the remaining page numbers computed above to fetch the
// rest of the records in batches, one tracker.export per page.
each(
$.pages,
tracker
.export('events', { pageSize: $.pageSize, page: $.data })
.then(state => {
state.results = state.results.concat(state.data.events);
return state;
})
);
4 changes: 4 additions & 0 deletions adaptors/library/jobs/fetch-all-events.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
// Fetching all events without pagination
// Be cautious: This can degrade performance if you have a lot of events, use with caution
// Requires DHIS2 server v2.41+, for older version `skipPaging: true`
tracker.export('events', { paging: false });
3 changes: 3 additions & 0 deletions adaptors/library/jobs/fetch-default-events.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
// DHIS2 Defaults: pageSize: 50
// Requires DHIS2 server v2.41+,
tracker.export('events');
20 changes: 19 additions & 1 deletion adaptors/library/staticExamples.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,22 @@
[
{
"expressionPath": "jobs/fetch-all-events",
"adaptor": "dhis2",
"adaptor_version": "v8.1.1",
"name": "Fetch all events without pagination"
},
{
"expressionPath": "jobs/fetch-default-events",
"adaptor": "dhis2",
"adaptor_version": "v8.1.1",
"name": "Fetch default events"
},
{
"expressionPath": "jobs/fetch-all-events-with-pagination",
"adaptor": "dhis2",
"adaptor_version": "v8.1.1",
"name": "Fetch all events with pagination"
},
{
"expressionPath": "jobs/commcare-asri-bulk-upload",
"adaptor": "commcare",
Expand Down Expand Up @@ -166,4 +184,4 @@
"adaptor": "kobotoolbox",
"name": "Extract submissions from KoboToolbox"
}
]
]