When transcoding a video fails, send it as a file

[jmartinesp]

Content

Changes:

• Make sure the 'at most' video dimensions are 1080p for non-compressed media and 720p for compressed media, matching iOS.
• Try avoiding transcoding when possible, and when it's not try matching as much properties of the original video as possible.
• If transcoding fails, send the original video as a file.

While reviewing you might want to skip/not thoroughly review commit 64ba34c since it's reworked shortly after.

Motivation and context

Fixes #4243, or at least adds a workaround.

Tests

I was able to make the transcoding fail with this video in a Xiaomi Mi 9T with Android 14.

• Try sending the video as an attachment in some room. Processing and transcoding the video will fail.
• Send the video. It will be sent as is, without a thumbnail, but it should still be playable on the other side. For some reason it's send as an m.video message instead of a m.file one, but it seems like this might need to be changed in the SDK.
• Try some other MP4 video, larger than 1080p if you have disabled media optimisation or 720p if it's enabled, the video will be resized.

Tested devices

• Physical
• Emulator
• OS version(s): 14

Checklist

• Changes have been tested on an Android device or Android emulator with API 24
• UI change has been tested on both light and dark themes
• Accessibility has been taken into account. See https://github.com/element-hq/element-x-android/blob/develop/CONTRIBUTING.md#accessibility
• Pull request is based on the develop branch
• Pull request title will be used in the release note, it clearly define what will change for the user
• Pull request includes screenshots or videos if containing UI changes
• You've made a self review of your PR

[github-actions]

📱 Scan the QR code below to install the build (arm64 only) for this PR.

If you can't scan the QR code you can install the build via this link: https://i.diawi.com/ifBAYU

[codecov]

Codecov Report

Attention: Patch coverage is 29.41176% with 48 lines in your changes missing coverage. Please review.

Project coverage is 79.96%. Comparing base (b636c69) to head (4c1b728).
Report is 64 commits behind head on develop.

Files with missing lines	Patch %	Lines
...roid/libraries/mediaupload/impl/VideoCompressor.kt	37.73%	29 Missing and 4 partials ⚠️
...aries/mediaupload/impl/AndroidMediaPreProcessor.kt	0.00%	15 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4257      +/-   ##
===========================================
- Coverage    79.99%   79.96%   -0.04%     
===========================================
  Files         2107     2107              
  Lines        55847    55895      +48     
  Branches      6973     6992      +19     
===========================================
+ Hits         44676    44696      +20     
- Misses        8774     8798      +24     
- Partials      2397     2401       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[bmarty]

Just a first comment, I do not see any MonotonicTimelineInterpolator in the changes. What am I missing?

[jmartinesp]

The title is a bit outdated, this PR was just added so we could test solutions to fix an issue when trying to transcode videos before sending them (the linked one above) in some Android versions (MIUI for Android 12?).

The original error said:

java.lang.IllegalStateException: Timestamps must be monotonically increasing: 0, 0

So I tried to fix that by using an interpolator that ensures the video timestamps always go forward, to see if that could solve the issue (it doesn't). This is either a problem with the codecs in the OS shipped by problematic vendors or an issue with the lib we're using to simplify the transcoding process.

So I guess our options are either allowing videos to be sent as is if transcoding fails (either as a file or as the actual media) or using a different lib or transcoder for this. I'm waiting for a decision from @mxandreas , but the latest commit tries to implement the first solution.

[mxandreas]

videos to be sent as is if transcoding fails (either as a file or as the actual media)

If "as is" means that the video size/quality will not be affected, but we can still remove the sensitive pieces (like geolocation tags), then I think that's fine. However, I would send them as "file" to have some indication that the transcoding was not applied.

[ganfra]

@jmartinesp what are we doing with that?

[jmartinesp]

@jmartinesp what are we doing with that?

It's now ready for review (I just remembered we had to add a mime-type change). If the video can't be sent as a video it'll be sent as a file, as the description says. I think it matches what Andreas said above.

[ganfra]

LGTM, maybe we should add some test on the VideoCompressor class?

[jmartinesp]

LGTM, maybe we should add some test on the VideoCompressor class?

Actually, that's going to be difficult since that's where we're actually performing the video transcoding. Maybe I can extract the transcoding configuration logic somewhere else and test that instead.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud