[RFC] Subscription mode for SWR

[huozhi]

Define the Problem

currently useSWR and useSWRInfinite are more designed for endpoints or data sources that can directly access and patch proactively. The data flow is like below

useSWR <=== write and read ====> data source

But for some cases SWR doesn't have enough solid ability to interact with data sources such as observable data (states returned from an observable stream), subscribable data (data been sent from a remote websocket endpoint).

then the fetcher of SWR will lose meaning since they're not really accessible initiavely from user end. we can surly do a subscribe or observe call to attach on the source. however we cannot know the state of a certain time.

besides fetcher, the mutate, trigger and revalidate APIs are also kind f disabled due to the passive mode. I want to find a new way to think of the communication path.

Solution

propose to bring a new hook API useSWRSubscription to SWR to fit in the subscribable situation. unlike the basic useSWR hook, it won't return the manipulation APIs of useSWR.

API

function useSWRSubscription(key, subscribeFn: (onData, onError) => Disposable): {data?, error?};

Usage

// like the useSWRInfinite, we put it into an extra file
// or in the future we could make it import useSWRSubscription from 'swr/subscription'
import {useSWRSubscription} from 'swr'

const {data, error} = useSWRSubscription('wss://my.app/api/chat-messages', subscribeWs)

function subscribeWs(key, {onData, onError}) {
   const ws = new WebSocket(key)
   ws.onmessage = (event) => { 
      onData(event.data)
   }
   ws.onerror = (error) => {
      onError(event.data)   
   }
   return () => ws.close()
}

then in this case, if the key changed, SWR is able to close and re-subscribe to the source for new key;
the uncertain thing is the returned error might not always be valid since some obseravle won't throw error when the connection get closed;

Other Possible Solutions

make revalidate and returned trigger with no effect. but mutate is still able to patch states to that key

Pros & Cons

Pros

• support realtime data
• adaptive to any observable data source, user can also do customization upon it
• simple returned values to let user to leverage, no mind load on the meaning of revalidate/mutate/trigger

Cons

• if there's any new API can access the current data, like geo API is subscribable but getting current position of geo is also doable. might need to have some workaround for that...
• more APIs into SWR, might bring confusion to new users to choose API

[shuding]

Thank you for writing this amazing RFC!

ws.onmessage = (event) => { return event.data };

This doesn't solve the data problem right? Like how do I notify SWR that I got the data (or error).

const {data, error} =

I think we can probably also return a connecting or loading value, which indicates the initial connection state (before the first value arrives).

or in the future we could make it import useSWRObservable from 'swr/observable'

I'm 👍 on doing it.

[huozhi]

Thanks for the quick feedback, I updated the draft, providing some callbacks as second args

- function observeWebsocket(key) {
+ function observeWebsocket(key, callbacks) {
   const ws = new WebSocket(key)
   ws.onmessage = (event) => {
-       return event.data
+       callbacks.onSuccess(event.data)

   }
+  ws.onerror = (error) => {
+     callbacks.onError(event.data)   
+  }
   return () => ws.close()
}

I think we can probably also return a connecting or loading value, which indicates the initial connection state (before the first value arrives).

I guess it could be isConnected or isSubscribed? since you can know ws connection is connecting/open/closing/closed, but for other observable stream, we might not know the short connecting/pending state before the connection is established. just my current opinion, and open to discussion : )

[shuding]

@huozhi or a general status value (which can be any) that can be passed? Since some subscriptions have multiple states (e.g.: there're 4 states of WS: https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/readyState).

Regards callbacks, I think we might need to adjust the argument order since we can have multiple arguments. How about:

useSWRObservable([1, 2, 3], (onData, onError, onStateUpdate, 1, 2, 3) => {
  // ...
})

Hmmm I think we still need to find something simpler...

[huozhi]

yup, the basic event target like stuff and observable sounds like 2 scenarios, the first one is more simple.

Event Target Like Subscriptions

// event target (web defined)
eventTarget.addEventListener/removeEventListener(fn)

// event emitter
eventEmitter.on/off(fn)

// subscribable
close = subscribable.subscribe(fn)
close()

Observable

subscription = observable.subscribe(onNext, onError, onComplete)

// or
subscription = observable.subscribe({
  next() {...},
  error() {...},
  complete() {...},
})

and obserable also has some interesting standard from community like callbag

---

so in that case, I feel we should either find a united way to let user encapsulate their observable to fit swr, or we make a very simple API contract with only few callbacks and let user handle the complexity themselves to keep swr lightweight.

[huozhi]

just found use-subscription's API design for subscription case, to distinguish from observable case

[sergiodxa]

Just come to mention that my PR (#457) come from a different use-case than the idea of the RFC.

The useSWRObservable hook is to get data from an observable source without when you don't have a non-observable source. This is useful when your source of data is only WebSockets or something real-time so you can't use a fetcher, it makes total sense here.

My proposal to add a subscribe in the config comes from a different idea where you have an API you can fetch with a fetcher but you want to start a subscription to an observable-source to receive updates without using long-polling.

Imagine this flow, in your fetcher you do a GET /api/resource/1 to get the resource with ID 1, then your subscribe to /ws//resource/1 in a WS server, this way you get an initial up-to-date data for your resource and thank to the subscription if something change in the server you don't need to wait for SWR to revalidate it, the API will let you know, and the reason to pass mutate is that sometimes this WS servers can return the new data, sometimes they return a delta of what changed so you will need to call mutate to apply the update, and sometimes they return that something changed and you will need to trigger a revalidation against the HTTP API.

---

So, I think we should support both, useSWRObservable and the subscribe option in useSWR.

[o-alexandrov]

Could you please shred some light on the overall direction you plan to move forward with?
It'd be great to know, if you support Sergio's proposal, so only the following needs to be done to get it merged:

• Add subscription support to useSWR #457 (review)

[aboutlo]

I'm experimenting with a similar concept in a wrapper of SWR for Ethereum https://github.com/aboutlo/ether-swr
I'm not sure I really want to use a different hook to observe e.g. useSWRObservable I ended up to have a subscribe parameter that can handle the callback too:

const { data: balance, mutate } = useEthSWR([address, 'balanceOf', account], {
  subscribe: [
    // A filter from anyone to me
    {
      name: 'Transfer',
      topics: [null, account],
      on: (
        data: BigNumber,
        fromAddress: string,
        toAddress: string,
        amount: BigNumber,
        event: any
      ) => {
        console.log('receive', { event })
        const update = data.add(amount)
        mutate(update, false) // optimistic update skip re-fetch
      }
    },
  ]
})```

[huozhi]

So, I think we should support both, useSWRObservable and the subscribe option in useSWR.

agree with @sergiodxa that we'd better to support both. I'm trying to make the API more flexible in #913 that we could let user do a little customization to fit the API. observable is definitely kind of heavy, so I wanna find the balance between them and try to get closer to subscription shape.

I'm experimenting with a similar concept in a wrapper of SWR for Ethereum https://github.com/aboutlo/ether-swr

@aboutlo 👍 like the idea you build the customized subscription on top of core swr hook, awesome!