How can I prevent duplicate user processing in concurrent Firebase transactions?

I have two Firebase functions running hourly that process the same list of users. They share a batch document (this.batchId: MM-dd-yyyy-HH, unique per hour) and use transactions to coordinate processing. Each instance fetches a batch of users after the lastProcessedId and updates the batch document with:

• A new lastProcessedId
• A global totalCount increment
• Instance-specific counts and processed IDs

While the global totalCount is accurate, sometimes overlapping instance-specific fields indicate that both instances processed the same IDs (usually a batch of two).

I expected that once Thread 1 commits its transaction and updates the lastProcessedId, Thread 2 would see the new value and process the next batch, aborting if a conflict is detected. However, this isn't happening as anticipated.

Below is the code used within the transaction:

TYPESCRIPTCopied
private async getNextBatchTransaction(): Promise<{ userDocs: QueryDocumentSnapshot<DocumentData>[] | null, needsCleanup: boolean }> {
  return this.firestore.runTransaction(async (transaction) => {
    const batchRef = this.firestore.collection("batch_sequence").doc(this.batchId);
    const batchDoc = await transaction.get(batchRef);

    const data = (batchDoc.exists ? batchDoc.data() : {
      lastProcessedId: null,
      complete: false,
    }) as BatchDocument;

    if (data.complete) {
      return { userDocs: null };
    }

    let query = this.firestore
      .collection("users")
      .orderBy("__name__")
      .limit(this.batchSize);

    if (data.lastProcessedId) {
      query = query.startAfter(data.lastProcessedId);
    }

    const userSnapshot = await transaction.get(query);

    if (userSnapshot.empty) {
      transaction.set(
        batchRef,
        { complete: true },
        { merge: true }
      );
      return { userDocs: null };
    }

    const batchLength = userSnapshot.docs.length;
    const lastDoc = userSnapshot.docs[batchLength - 1];
    const processedIds = userSnapshot.docs.map(doc => doc.id);
    
    transaction.set(
      batchRef,
      {
        lastProcessedId: lastDoc.id,
        totalCount: FieldValue.increment(batchLength),
        [`instance.${this.instanceId}`]: FieldValue.increment(batchLength),
        [`processedIds.${this.instanceId}`]: FieldValue.arrayUnion(...processedIds),
      },
      { merge: true }
    );

    return { userDocs: userSnapshot.docs};
  });
}

Additionally, I reviewed the runTransaction documentation which explains that:

/*

• Executes the given updateFunction and commits the changes applied within
• the transaction.
• You can use the transaction object passed to 'updateFunction' to read and
• modify Firestore documents under lock. You have to perform all reads
• before you perform any write.
• Transactions can be performed as read-only or read-write transactions. By
• default, transactions are executed in read-write mode.
• A read-write transaction obtains a pessimistic lock on all documents that
• are read during the transaction. These locks block other transactions,
• batched writes, and other non-transactional writes from changing that
• document. Any writes in a read-write transactions are committed once
• 'updateFunction' resolves, which also releases all locks.
• If a read-write transaction fails with contention, the transaction is
• retried up to five times. The updateFunction is invoked once for each
• attempt.
• Read-only transactions do not lock documents. They can be used to read
• documents at a consistent snapshot in time, which may be up to 60 seconds
• in the past. Read-only transactions are not retried.
• Transactions time out after 60 seconds if no documents are read.
• Transactions that are not committed within than 270 seconds are also
• aborted. Any remaining locks are released when a transaction times out.
• @param updateFunction The function to execute within the transaction
• context.
• @param transactionOptions Transaction options.
• @return If the transaction completed successfully or was explicitly
• aborted (by the updateFunction returning a failed Promise), the Promise
• returned by the updateFunction will be returned here. Else if the
• transaction failed, a rejected Promise with the corresponding failure
• error will be returned.
  */

Why don't the transactions seem to prevent overlapping processing? Any insights or advice on ensuring that each instance processes a unique batch of users would be appreciated.