Implement pagination for FHIR Observation REST API end point

[s1monj]

Background

• The JHE server should be able to support a large number of Observations
• Therefore it needs to support filtering and pagination options that are executed at the DB level (ie LIMIT and OFFSET etc)
• FHIR specifies the search param _count for the page size but does not specify a parameter for the page number (see https://www.hl7.org/fhir/search.html#table)
• Various servers have their own params

• We will use _page
• We also need to return the total number of results as bundle.total but we'll leave this for another ticket

TBD

1. Use Postman to create >30 Observations for testing - update the ID to use a random string (using Postman {{$randomPassword}} or similar so you can click the SEND button repeatedly without having to change the payload each time
2. Accept the parameter _count so that GET /fhir/r5/Observation?_count=10 displays only the first 10 results by adding LIMIT 10 to the end of the raw DB query
3. Accept the parameter _page so that GET /fhir/r5/Observation?_count=10&_page=2 displays results 11-20 by adding OFFSET (page-1)*10)
4. Lastly, add the link property to the response Bundle with self, previous and next as per FHIR spec - see example below.

// GET /fhir/r5/Observation?_count=1&_page=2
{
    "resourceType": "Bundle",
    "type": "searchset",
    "link": [ {
      "relation": "self",
      "url": "http://localhost:8000/Observation?_count=1&_page=2"
    }, {
      "relation": "previous",
      "url": "http://localhost:8000/Observation?_count=1&_page=1"
    },
   {
      "relation": "next",
      "url": "http://localhost:8000/Observation?_count=1&_page=3"
  } ],
    "entry": [
        {
            "resource": {
                "resourceType": "Observation",
                ...

Note:

• I don't know if this is best implemented in Django using a custom paginator (see https://www.django-rest-framework.org/api-guide/pagination/?ref=thetldr.tech#custom-pagination-styles) or by just passing the vars through and have the default page size set to int.max
• we don't want to be using any kind of in-memory result set pagination because that won't scale
• given that we are using raw sql queries for the JSON transformation simply passing extra params might be the best option

[travis-sauer-oltech]

@s1monj

PR: https://github.com/the-commons-project/jupyterhealth-exchange/pull/44/files#diff-9b084122d287fc8dc13f21f4c36a2871d4d7adc54b4f1f972039c1573ab2cea3L49

[s1monj]

@travis-sauer-oltech I should have been more clear - it appears this change has altered the Observations Admin API ❌ (in addition to the Observation FHIR API ✅ )

• This is the format/shape of data we're expecting from the Admin API response:

• But instead we are getting a FHIR response the same as the /fhir/observation endpoint

• Can you please revert the Observation Admin API to the original format/shape so this ticket only effects the FHIR API

[travis-sauer-oltech]

@s1monj -

Refactored PRs: #57, #60, and #62