feat(web): implement xr 360 photo preview #14768

[andreymaznyak]

implement simple xr 360 preview
#14768

Description

I have a large library of 360 photos and videos from my Insta360 and drone, and I recently bought a Quest 3—because watching this kind of content in VR is truly an amazing experience.

I’m also experimenting with Apple Spatial Video and planning to implement rendering and playback in VR later on.

Starting small—for now, the Oculus browser doesn’t support proper 360 photo viewing, so I decided to implement this feature myself, since I have a lot of 360 photos.

I did some research on whether it’s possible to enable XR support using the PhotoSphereViewer library: mistic100/Photo-Sphere-Viewer#228 — unfortunately, PhotoSphereViewer doesn’t support VR.

Since we have the WebXR standard, I ended up creating a simple implementation for native 360 photo viewing using it.

Added support 360 video view

How Has This Been Tested?

I tested it in meta quest 3. Also you check toogle features with web xr polyfill, but it not working as needed. I put record of my quest screen.

Screenshots (if appropriate)

Checklist:

• I have performed a self-review of my own code
• I have made corresponding changes to the documentation if applicable
• I have no unrelated changes in the PR.
• I have confirmed that any new dependencies are strictly necessary.
• I have written tests for new code (if applicable)
• I have followed naming conventions/patterns in the surrounding code
• All code in src/services/ uses repositories implementations for database calls, filesystem operations, etc.
• All code in src/repositories/ is pretty basic/simple and does not have any immich specific logic (that belongs in src/services/)

[alextran1502]

Hello, do you mind including a sample asset in this PR?

[andreymaznyak]

I'm just start working with it for fun. 😄 Work in progress.
Thank you for good open source project - i migrating my assets to immich ❤️

I record small video how it works.

https://www.youtube.com/watch?v=eeLxnJTiCZk

[DeltaTango69]

Nice, I have also an Insta and Quest3. If I have to wait if this will be implemented, or what can I do to use it? Will this also work for videos?

[andreymaznyak]

Nice, I have also an Insta and Quest3. If I have to wait if this will be implemented, or what can I do to use it? Will this also work for videos?

I think, you can do it, if you want - you can checkout 😄 I check it only in photos. You can try to implement video support.

Do not forgot about web xr features works only with https - https://medium.com/@schezfaz/getting-started-with-webxr-development-for-the-oculus-browser-159339b2a7f3

And in the future, i think we should improve processing of insta360 files. Stack videos, and auto generate thumbs with instal360 sdk.

[andreymaznyak]

Hello, do you mind including a sample asset in this PR?

Good news, i researched how to implement rendering apple spatial videos in immich.

This this feature request for parse metadata from apple spatial video

I make some research and create repository with example transcoding mv-hevc video and rendering it web xr.
Repository here

I think we can combine this features.

And i think you should find some VR gadget for testing it 😉

[andreymaznyak]

I proceed work with implementation xr features, and research to showing apple spatial photos, results in same repository.

I think we are should create new feature flag and job for supporting web xr in immich. When flag is true, we should make preview photos for each eye for web xr view.

[orrd]

I would love to have the ability to view 360 photos with a Quest headset. Each time there is a new Immich release, I'm hoping they merged this pull request. Is there any reason they can't merge it as it is?