Implement pagination for Admin API raw queries to be consistent with built-in result set pagination

[s1monj]

While we're on the subject of pagination

• Listing observations and patients from the web console (Admin API) could be large and needs to use pagination
• Verify that the default pagination system for the Admin API endpoints uses DB limit/offset and not in-memory pagination
• If it does not use DB pagination update the default paginators for all Admin API endpoints (except for Organizations that uses a custom response) accordingly

[travis-sauer-oltech]

@s1monj, we can close this ticket. After thorough research, I found some resources I will include for future reference.

We're using GenericViewSets and mostly ModelViewSets.

Both classes use QuerySet slicing. Fetch to the DB happens, and then Django's ORM uses lazy QuerySets, which slice at the database level, effectively translating into SQL LIMIT and OFFSET clauses.

1. ListModelMixin:

GitHub

2. QuerySet methods:

Django Docs

and

Django Docs

[s1monj]

RE: https://stackoverflow.com/questions/5152984/django-pagination-and-rawqueryset#:~:text=Unfortunately%2C%20RawQuerySet,will%20hit%20the%20DB%20twice

• We will need to implement our own pagination for raw queries
• Params should be the same as default result set pagination params
• Similar to the pagination for FHIR Observations - it's up to you if you just want to pass along the params to the model function or implement a custom paginator - whatever makes most sense

[s1monj]

Also we will need to consider how to return the total - so probably best to run 2 queries again as per slack discussion

query1 = FROM core_patients WHERE…
query1_total = SELECT COUNT(*) ${query1}
query1_page = SELECT * ${query1} LIMIT ${limit} OFFSET ${offset}

so then when the request is made we execute query1_total and query1_page as two separate calls to the DB

[s1monj]

@travis-sauer-oltech I needed to revert DEFAULT_PAGINATION_CLASS in 8dd5e10 extended default pagination class for all admin apis because I think it's broken the UI and I'm not sure exactly why

[travis-sauer-oltech]

@s1monj

Refactored PRs: #57, #64, #65, and #66