🐛 [cloud_firestore] old data retrieved if get is called just after a transaction when a stream is active

[pldelattre]

Bug report

Describe the bug
Firestore doesn't retrieve the latest data when using 'get' on a document just after this document was modified through a transaction and a stream to listen on the same document is active

Steps to reproduce

Steps to reproduce the behavior:

1. Listen to a document
2. Update this document via a transaction
3. Call get method on this document just after transaction has succeeded.

Expected behavior

Content of the retrieved document in the 'Get' call should be updated but it is not !
This issue is not happening if we are not listening to the document OR if the document is not updated through a transaction.

Sample project

To reproduce the problem.

• Create a 'test' collection with a 'test' document. Add a 'population' field to this document with a number value of 1.
• Run the following widget.
• Issue will appear if you click 'Listen to the stream' and then 'Transact then Get'

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter/material.dart';

class TestBugWidget extends StatefulWidget {
  const TestBugWidget({super.key});

  @override
  State<TestBugWidget> createState() => _TestBugWidgetState();
}

class _TestBugWidgetState extends State<TestBugWidget> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Bug repro')),
      body: Column(children: [
        ListTile(
          title: const Text('Listen to the stream'),
          onTap: streamListen,
        ),
        ListTile(
          title: const Text('Update then Get'),
          onTap: updateThenGet,
        ),
        ListTile(
          title: const Text('Transact then Get'),
          onTap: transactThenGet,
        )
      ]),
    );
  }

  void streamListen() {
    FirebaseFirestore.instance
        .collection('test')
        .doc('test')
        .snapshots()
        .listen((event) {
      debugPrint('new test has arrived');
    });
  }

  Future<void> transactThenGet() async {
    final docRef = FirebaseFirestore.instance.collection("test").doc("test");
    var newPopulation;
    await FirebaseFirestore.instance.runTransaction((transaction) async {
      final snapshot = await transaction.get(docRef);
      final prevPopulation = snapshot.get("population");
      newPopulation = prevPopulation + 1;
      transaction.update(docRef, {"population": newPopulation});
    });
    Map<String, dynamic> updated = (await docRef.get()).data()!;
    debugPrint('Expecting : ${newPopulation.toString()}');
    debugPrint('Got : ${updated['population'].toString()}');
  }

  Future<void> updateThenGet() async {
    final docRef = FirebaseFirestore.instance.collection("test").doc("test");
    final snapshot = await docRef.get();
    final prevPopulation = snapshot.get("population");
    final newPopulation = prevPopulation + 1;
    await docRef.update({"population": newPopulation});
    Map<String, dynamic> updated = (await docRef.get()).data()!;
    debugPrint('Expecting : ${newPopulation.toString()}');
    debugPrint('Got : ${updated['population'].toString()}');
  }
}

---

Additional context

cloud_firestore: ^4.0.5
firebase_core: ^2.2.0

I tested both on Android and Web. Issue happens in both cases.

[darshankawar]

Thanks for the report @pldelattre
Using the details you provided, I ran on mobile platform (iOS) and observed below in the console:

flutter: new test has arrived
flutter: new test has arrived
flutter: Expecting : 2
flutter: Got : 2
flutter: new test has arrived
flutter: Expecting : 3
flutter: Got : 3

Is above an expected result you are looking for ? Or do you see any other output ?

I am unable to verify this on web due to throw FallThroughError(); error which is being discussed here already

[pldelattre]

A bit more investigation:

• Issue is consistently happening when using Firestore on GCP but not always when using emulator (50% of tries).
• A workaround that solve the issue is to add a "await Future.delayed(const Duration(milliseconds: 300));" in the transactThenGet() Future :

  Future<void> transactThenGet() async {
    final docRef = FirebaseFirestore.instance.collection("test").doc("test");
    var newPopulation;
    await FirebaseFirestore.instance.runTransaction((transaction) async {
      final snapshot = await transaction.get(docRef);
      final prevPopulation = snapshot.get("population");
      newPopulation = prevPopulation + 1;
      transaction.update(docRef, {"population": newPopulation});
    });
    await Future.delayed(const Duration(milliseconds: 300)); // <-- WORKAROUND
    Map<String, dynamic> updated = (await docRef.get()).data()!;
    debugPrint('Expecting : ${newPopulation.toString()}');
    debugPrint('Got : ${updated['population'].toString()}');
  }

[pldelattre]

@darshankawar
This is what I get when I click 'Listen to the stream' and then two times on 'Transact then Get'

new test has arrived
Expecting : 2
Got : 1
new test has arrived
Expecting : 3
Got : 2
new test has arrived

However if I reload the app and click only on 'Transact then Get', I get

Expecting : 4
Got : 4

It seems the problem is that the Firestore get action will retrieve the previous record until the stream has finished serving the new record. And that looks like a bug.

The issue is also happening on Android on my side. Maybe you can try to reproduce on this platform ?

[darshankawar]

Thanks for the feedback. Using above mentioned details and running on Android, seeing same behavior as reported. Keeping this issue open and labeling for team's attention on expected behavior.

[pldelattre]

@Lyokone Thanks for working on this. You mention that your PR close the issue, however, your code seem to only update IOS files.
This bug (#10153) happens on Android (tested by me and confirmed by @darshankawar) and Web platform (tested by me) but not on IOS (tested by @darshankawar), so your fix is possibly unrelated to this issue ?

[Lyokone]

Thanks for the precision. I was unable to reproduce on Android, I might have missed something I'll try again 👍

[pldelattre]

@Lyokone Have you tried on a firestore cloud instance ? It is is more difficult to reproduce with emulator.

[Lyokone]

I'm testing with with a Cloud Instance but not having any issue with Web 🤔

I feel like when using

    Map<String, dynamic> updated =
        (await docRef.get(const GetOptions(source: Source.server))).data()!;

I don't get the error anymore. Do you see an improvement with the source?

[pldelattre]

I updated to the latest versions :

• Flutter (Channel stable, 3.3.10, on macOS 12.6.2 21G320 darwin-x64, locale en)
• cloud_firestore: ^4.3.0
• firebase_core: ^2.4.0

And refactored the repro code to the bare minimum, everything in a single dart file, including the main() function. Also added the "const GetOptions(source: Source.server)" in the Get call :

import 'dart:async';
import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter/material.dart';
import 'package:firebase_core/firebase_core.dart';
import 'firebase_options.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp(
    options: DefaultFirebaseOptions.currentPlatform,
  );

  runApp(MaterialApp(
      title: 'Bug Repro',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const TestBugWidget()));
}


class TestBugWidget extends StatefulWidget {
  const TestBugWidget({super.key});

  @override
  State<TestBugWidget> createState() => _TestBugWidgetState();
}

class _TestBugWidgetState extends State<TestBugWidget> {
  StreamSubscription? _sub;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Bug repro')),
      body: Column(children: [
        ListTile(
          title: const Text('Listen to the stream'),
          onTap: streamListen,
        ),
        ListTile(
          title: const Text('Transact then Get'),
          onTap: transactThenGet,
        )
      ]),
    );
  }

  void streamListen() {
    debugPrint('*** User clicked on \'Listen to the stream\' ***');
    _sub?.cancel();
    debugPrint('Let\'s listen to test document!');
    _sub = FirebaseFirestore.instance
        .collection('test')
        .doc('test')
        .snapshots()
        .listen((event) {
      debugPrint('new test document has arrived');
    });
  }

  Future<void> transactThenGet() async {
    debugPrint('*** User clicked on \'Transact then Get\' ***');
    final docRef = FirebaseFirestore.instance.collection("test").doc("test");
    var newPopulation;
    await FirebaseFirestore.instance.runTransaction((transaction) async {
      final snapshot = await transaction.get(docRef);
      final prevPopulation = snapshot.get("population");
      newPopulation = prevPopulation + 1;
      debugPrint('updating population to : ${newPopulation.toString()}');
      transaction.update(docRef, {"population": newPopulation});
    });
    Map<String, dynamic> updated = (await docRef.get(const GetOptions(source: Source.server))).data()!;
    debugPrint('Expecting : ${newPopulation.toString()}');
    debugPrint('Got : ${updated['population'].toString()}');
  }

}

Unfortunately, it does not solve the issue for me.

Here is what I get when running on web :

*** User clicked on 'Listen to the stream' ***
Let's listen to test document!
new test document has arrived
*** User clicked on 'Transact then Get' ***
updating population to : 57
Expecting : 57
Got : 56
new test document has arrived
*** User clicked on 'Transact then Get' ***
updating population to : 58
Expecting : 58
Got : 57
new test document has arrived

Here is what I get when running on android

I/flutter ( 5669): *** User clicked on 'Listen to the stream' ***
I/flutter ( 5669): Let's listen to test document!
W/DynamiteModule( 5669): Local module descriptor class for com.google.android.gms.providerinstaller.dynamite not found.
I/DynamiteModule( 5669): Considering local module com.google.android.gms.providerinstaller.dynamite:0 and remote module com.google.android.gms.providerinstaller.dynamite:0
W/ProviderInstaller( 5669): Failed to load providerinstaller module: No acceptable module com.google.android.gms.providerinstaller.dynamite found. Local version is 0 and remote version is 0.
D/nativeloader( 5669): classloader namespace configured for unbundled product apk. library_path=/product/priv-app/PrebuiltGmsCore/lib/x86_64:/product/priv-app/PrebuiltGmsCore/PrebuiltGmsCore.apk!/lib/x86_64:/product/lib64:/system/product/lib64
V/NativeCrypto( 5669): Registering com/google/android/gms/org/conscrypt/NativeCrypto's 294 native methods...
W/eon.myapp.in( 5669): Accessing hidden method Ljava/security/spec/ECParameterSpec;->getCurveName()Ljava/lang/String; (unsupported, reflection, allowed)
I/ProviderInstaller( 5669): Installed default security provider GmsCore_OpenSSL
I/flutter ( 5669): new test document has arrived
W/eon.myapp.in( 5669): Accessing hidden field Ljava/net/Socket;->impl:Ljava/net/SocketImpl; (unsupported, reflection, allowed)
W/eon.myapp.in( 5669): Accessing hidden field Ljava/io/FileDescriptor;->descriptor:I (unsupported, JNI, allowed)
W/eon.myapp.in( 5669): Accessing hidden method Ljava/security/spec/ECParameterSpec;->setCurveName(Ljava/lang/String;)V (unsupported, reflection, allowed)
I/flutter ( 5669): new test document has arrived
D/EGL_emulation( 5669): app_time_stats: avg=1103.09ms min=17.37ms max=5156.58ms count=5
I/flutter ( 5669): *** User clicked on 'Transact then Get' ***
I/flutter ( 5669): updating population to : 59
I/flutter ( 5669): new test document has arrived
I/flutter ( 5669): Expecting : 59
I/flutter ( 5669): Got : 59
D/EGL_emulation( 5669): app_time_stats: avg=134.23ms min=6.90ms max=1313.40ms count=12
I/flutter ( 5669): *** User clicked on 'Transact then Get' ***
I/flutter ( 5669): updating population to : 60
I/flutter ( 5669): Expecting : 60
I/flutter ( 5669): Got : 59
I/flutter ( 5669): new test document has arrived
D/EGL_emulation( 5669): app_time_stats: avg=272.38ms min=6.09ms max=3581.75ms count=14
I/flutter ( 5669): *** User clicked on 'Transact then Get' ***
I/flutter ( 5669): updating population to : 61
I/flutter ( 5669): new test document has arrived
I/flutter ( 5669): Expecting : 61
I/flutter ( 5669): Got : 61
I/flutter ( 5669): *** User clicked on 'Transact then Get' ***
D/EGL_emulation( 5669): app_time_stats: avg=489.51ms min=74.10ms max=1857.45ms count=5
I/flutter ( 5669): updating population to : 62
I/flutter ( 5669): new test document has arrived
I/flutter ( 5669): Expecting : 62
I/flutter ( 5669): Got : 62
D/EGL_emulation( 5669): app_time_stats: avg=436.70ms min=74.24ms max=1064.18ms count=3
I/flutter ( 5669): *** User clicked on 'Transact then Get' ***
I/flutter ( 5669): updating population to : 63
I/flutter ( 5669): new test document has arrived
I/flutter ( 5669): Expecting : 63
I/flutter ( 5669): Got : 63
D/EGL_emulation( 5669): app_time_stats: avg=170.16ms min=43.96ms max=972.57ms count=8
I/flutter ( 5669): *** User clicked on 'Transact then Get' ***
I/flutter ( 5669): updating population to : 64
I/flutter ( 5669): Expecting : 64
I/flutter ( 5669): Got : 63
I/flutter ( 5669): new test document has arrived

We can notice that:

• it is always failing on Web but only from time to time on Android.
• When 'new test document has arrived' is written just after 'updating population', then Expecting and Got numbers will match. However, they won't match if 'new test document has arrived' is written after Expecting and Got.

I also tried to compile web on another computer with a fresh install of flutter to see if result would be different. Unfortunately it is not. And we have also @darshankawar that confirmed he was able to reproduce the issue on Android.

Another info that I didn't mention is that I am using EUR3 Firestore instance. Maybe it is worth trying to reproduce on that instance ?

[Lyokone]

After a lot of different tests, I was able to also reproduce it in the JS SDK with the following code:

import { initializeApp } from "firebase/app";
import {
  doc,
  getDoc,
  getFirestore,
  onSnapshot,
  runTransaction,
} from "firebase/firestore";

var config = {
...
};

// Initialize Firebase
var app = initializeApp(config);
const db = getFirestore(app);

const test_doc = doc(db, "test_10153", "test");

const unsub = onSnapshot(test_doc, (doc) => {
  console.log("Current data: ", doc.data());
});

for (let i = 0; i < 10; i++) {
  var newPopulation;

  await runTransaction(db, async (transaction) => {
    const sfDoc = await transaction.get(test_doc);
    if (!sfDoc.exists()) {
      throw "Document does not exist!";
    }

    newPopulation = sfDoc.data().population + 1;
    console.log("Updating to:", newPopulation);
    transaction.update(test_doc, { population: newPopulation });
  });

  const data = await getDoc(test_doc);

  console.log("Expected: ", newPopulation);
  console.log("Actual: ", data.data().population);

  console.log("---------");
}

unsub();

I'll post an issue to the JS SDK repository.

[pldelattre]

Thanks for following up on this @Lyokone and for opening an issue on the JS SDK repository.
Assuming they can fix the issue on the JS SDK, would that fix the issue only on Web or also Android ? Is Firestore Android implementation relying on the same SDK ?

[Lyokone]

@pldelattre I have opened the issue only on the JS SDK repository because I think this is not a bug but an expected behaviour (transaction taking some time to be applied on the backend properly). My guess is that the await runTransaction is just awaiting to send the transaction to the backend but does not await the transaction to be applied. If this is the case, it will be the same thing on iOS and Android.

If this end up being a bug, I will open issues on other repositories

[pldelattre]

@Lyokone
if your guess was correct, we would see the same behavior when there is no active listener. In my tests, when there is no active listener, I cannot reproduce the issue.

[pldelattre]

@Lyokone your theory and my last comment gave me the idea of a new test :

• Start a first chrome tab with the repro code and click listen to the document.
• Start the app in a second chrome tab and click on 'Transact then Get' without listening to the document.

In that context, I only see the issue in the first tab but not in the second tab. IMHO, this tends to prove that the issue is not with the backend (that would not be awaiting the transaction to be applied) but with the frontend.

[edhom]

@Lyokone @pldelattre We're experiencing a similar issue. In our case, we run a transaction where we delete a document and remove the reference to it in another. The stream listening to the parent collection emits two events after the transaction has run. The second one includes both changes. The first one however, includes the removed reference but the deleted document is still in the snapshot. Isn't the idea of transactions that exactly this doesn't happen, e.g. that the changes are atomic?

[Lyokone]

I'll close this as "will not fix" according to this response from a Googler: firebase/firebase-js-sdk#6915 (comment)