Control functions skip frames when frameloop = demand

[callumbooth]

Describe the bug

When using react three fiber with the frameloop set to demand and the Drei CameraControls. Dragging around works great however if you use controls functions like .rotate and .zoomTo the first few frames are skipped and the animation just jumps.

I've tried manually invalidating as per the r3f docs by adding an event listener to both the update and control event however neither fixed the issue.

To Reproduce

Steps to reproduce the behavior:

1. Go to https://codesandbox.io/p/sandbox/cameracontrols-basic-forked-tf9tct?file=%2Fsrc%2FApp.js%3A13%2C11
2. Click the rotate theta controls.
3. The first click (or anytime the frame is stationary) will skip frames, any clicks whilst the frameloop is running the animation will run correctly.

Code

No response

Live example

https://codesandbox.io/p/sandbox/cameracontrols-basic-forked-tf9tct?file=%2Fsrc%2FApp.js%3A13%2C11

Expected behavior

I'd expect the animation to run nicely regardless of if the frameloop is running or not

Screenshots or Video

No response

Device

Desktop

OS

MacOS

Browser

Chrome

[paulftw]

FYI, I'm not a maintainer, but your codesandbox link doesn't work for me - seems it's either deleted or marked private

[callumbooth]

FYI, I'm not a maintainer, but your codesandbox link doesn't work for me - seems it's either deleted or marked private

@paulftw ahh, didn't realise it was set to private by default. I've made it public now.

[yomotsu]

Thanks. reproduced the problem.
I think the following solution can address the problem.
https://docs.pmnd.rs/react-three-fiber/advanced/scaling-performance#triggering-manual-frames

However, this must be implemented within the R3F camera-controls wrapper.
You can report this issue at the following URL: https://github.com/pmndrs/drei.

[FrostKiwi]

Having exactly this problem. When demand resulted in rendering stopping, then multiple CameraControls features fail.
Scrollwheel doesn't lerp anymore, it skips instantly. I also make use of setLookAt and it's the same issue, CameraControls teleports to the target, instead of lerping. Essentially anything with time and smoothing breaks with CameraControls. I think it also affects normal camera rotation as well, introducing a small hitch that gets mostly unnoticed. I dug down and found the root issue to be specifically this:
pmndrs/drei#2005 (comment)

the controls.update function just ends up being called with a huge delta if no frames have been rendered in a while.

@yomotsu

I think the following solution can address the problem.
https://docs.pmnd.rs/react-three-fiber/advanced/scaling-performance#triggering-manual-frames

It doesn't, at least not universally. Not even the wrapper in pmndrs/drei#1398 by @sinedie solves this issue. We can call invalidate() when CameraControls does something, be it wake, transitionstart, etc. but at that point CameraControls already ingested the huge delta to calculate its smoothing, skipping the smoothing to its end, teleporting the camera to its target or introducing hitching. We are off-by-one frame for demand rendering to work.

Unless I'm missing something, there needs to be a deeper fix to guarantee that the frame delta time that CameraControls isn't garbage due to rendering stopping. That would solve this universally.

[abernier]

related https://github.com/pmndrs/drei/pull/2451#:~:text=🐛-,it%20also%20fixes%20a%20bug,-with%20frameloop%3D"demand