PERF: Use backend to load Note

[david-allison]

Purpose / Description

Match Anki Desktop's implementation, and get a 3x speedup (1m30 -> 30s to load 1MM notes)

❗️WARN: while unit testing, I had a failure of:

• [Bug] Unit Tests: java.lang.UnsatisfiedLinkError: 'byte[][] net.ankiweb.rsdroid.NativeMethods.openBackend(byte[])' when mixing RunWith(AndroidJUnit4::class) #14796

I have annotated the class which I believe is the problem, but this is not guaranteed to fix the issue. I have performed many retests and have been unable to reproduce the test failure

Approach

• try to use the backend, find there are type mismatches
• fix the mismatches
• use the backend
• performance test

How Has This Been Tested?

1m 27 -> 28s to load 1 million notes (~87 -> 27 micros)

    @Test
    fun notePerformance() {
        val note = addNoteUsingBasicModel("hello", "world")

        // 28s 271ms    - backend + no .toMutableList()
        // 28s 461ms    - backend + .toMutableList()
        // 1m 27s       - Cursor
        for (i in 0..1_000_000) {
            col.getNote(note.id)
        }
    }

Learning (optional, can help others)

https://github.com/ankitects/anki/blob/db93939ded947d74bb25bd8552a2b2356a096509/pylib/anki/notes.py#L54-L67

Backend Codegen is problematic: it exposes a java.util.List, but this is actually a com.google.protobuf.Internal.ProtobufList

Extends List to add the capability to make the list immutable and inspect if it is modifiable.

https://protobuf.dev/reference/java/api-docs/com/google/protobuf/Internal.ProtobufList

Checklist

• You have a descriptive commit message with a short title (first line, max 50 chars).
• You have commented your code, particularly in hard-to-understand areas
• You have performed a self-review of your own code
• UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)
• UI Changes: You have tested your change using the Google Accessibility Scanner

[BrayanDSO]

Pretty good

[mikehardy]

Looks good!

[mikehardy]

waiting for merge queue to empty so I don't thrash it with the squash merge from here then goes in

[mikehardy]

This one gave me a fit locally. 2 macOS machines ran tests with this PR rebased to main no problem but windows consistent failures. Stopped gradle daemons, gradle clean with --rerun-tasks then it was clean. Poisoned cache or something. Hate that "I'm missing something..." feeling that gives me.

[david-allison]

Was the failure:

• [Bug] Unit Tests: java.lang.UnsatisfiedLinkError: 'byte[][] net.ankiweb.rsdroid.NativeMethods.openBackend(byte[])' when mixing RunWith(AndroidJUnit4::class) #14796

If so, revert on the first sign of failure in CI

[mikehardy]

Nope, it's this, sometimes, I cannot get it to reproduce but I did just see it on a macOS machine so I've seen it cross-platform now. CI has zero instances, and locally I'm green again, I dunno

ReviewerTest > testLrnQueueAfterUndo FAILED
    java.lang.NoSuchMethodError: 'java.lang.String[] com.ichi2.libanki.Note.getFields()'
        at com.ichi2.anki.RobolectricTest.equalFirstField(RobolectricTest.kt:427)
        at com.ichi2.anki.ReviewerTest$testLrnQueueAfterUndo$1.invokeSuspend(ReviewerTest.kt:166)
        at com.ichi2.anki.ReviewerTest$testLrnQueueAfterUndo$1.invoke(ReviewerTest.kt)
        at com.ichi2.anki.ReviewerTest$testLrnQueueAfterUndo$1.invoke(ReviewerTest.kt)
        at com.ichi2.testutils.TestClass$runTest$1.invokeSuspend(TestClass.kt:154)
        at com.ichi2.testutils.TestClass$runTest$1.invoke(TestClass.kt)
        at com.ichi2.testutils.TestClass$runTest$1.invoke(TestClass.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$2$1$1.invokeSuspend(TestBuilders.kt:314)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
        at kotlinx.coroutines.test.TestDispatcher.processEvent$kotlinx_coroutines_test(TestDispatcher.kt:28)
        at kotlinx.coroutines.test.TestCoroutineScheduler.tryRunNextTaskUnless$kotlinx_coroutines_test(TestCoroutineScheduler.kt:103)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$2$1$workRunner$1.invokeSuspend(TestBuilders.kt:320)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:280)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersJvmKt.createTestResult(TestBuildersJvm.kt:13)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest-8Mi8wO0(TestBuilders.kt:308)
        at kotlinx.coroutines.test.TestBuildersKt.runTest-8Mi8wO0(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest-8Mi8wO0(TestBuilders.kt:166)
        at kotlinx.coroutines.test.TestBuildersKt.runTest-8Mi8wO0(Unknown Source)
        at com.ichi2.testutils.TestClass.runTest(TestClass.kt:152)
        at com.ichi2.testutils.TestClass.runTest$default(TestClass.kt:146)
        at com.ichi2.anki.ReviewerTest.testLrnQueueAfterUndo(ReviewerTest.kt:150)