Skip to content

fix(cursor): don't use other operation's session for cloned cursor operation (NODE-3008) #2705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 29, 2021

Conversation

KonanMentor
Copy link
Contributor

Fixes NODE-3008.

Description

Please refer to NODE-3008 for more details.

I can try to write a test for this bug if someone give me a hint on an example of such tests.

Also we should check if this should be ported to v4.x

@emadum emadum requested review from nbbeeken, emadum and durran January 21, 2021 14:39
@durran
Copy link
Member

durran commented Jan 22, 2021

Hi @KonanMentor. Thank you so much for the related JIRA ticket and the PR. What I'm trying to understand is the use case for actually wanting to clone a cursor. Would it be possible for you to provide the reasoning for this and what you are trying to achieve with this functionality?

Once again, I thank you for your contribution and look forward to your response.

@KonanMentor
Copy link
Contributor Author

KonanMentor commented Jan 22, 2021

Hi @durran. My use case for using cursor.clone is about rendering progress of cursor iteration. By rendering I mean just a string in the following format: `${processed} / ${total}`. The progress string is rendered by listr in my case. I'll try to replicate the implementation quite close to the real thing, so there will be some RxJS.

I guess, the most crucial part is

const clonedCursor = cursor.clone(); // clone cursor in order to avoid execution of both `find` and `count` operations on the same cursor
const totalPromise = clonedCursor.count(true);
// An alternative way would be passing `query` + `collection` instead of `cursor`
// const totalPromise = collection.countDocuments(query);
// But it will become even more inconvenient if the cursor has `limit` and/or `skip` operators applied

Here is the full story:

const cursor = collection.find(QUERY);
const documents$ = asyncIterableToObservable(cursor) // this function treats cursor as asyncIterable and converts it to RxJS Observable
  .share(); // In order to avoid subscribing to the observable second time (i.e. iterating the same cursor twice) in addListrTask function
processDocuments(documents$);
addListrTask(cursor, documents$);

function processDocuments(documents$) {
  documents$
    .mergeMap(doc => doSomeWork(doc))
    .subscribe();
}

function addListrTask(cursor, documents$) {
  const clonedCursor = cursor.clone(); // clone cursor in order to avoid execution of both `find` and `count` operations on the same cursor
  const totalPromise = clonedCursor.count(true);
  // An alternative way would be passing `query` + `collection` instead of `cursor`
  // const totalPromise = collection.countDocuments(query);
  // But it will become even more inconvenient if the cursor has `limit` and/or `skip` operators applied

  // RxJS magic starts
  const processedCount$ = documents$
    .scan((processedCount) => processedCount + 1, 0)
    .startWith(0);
  const progressString$ = Observable.combineLatest(processedCount$, totalPromise)
    .map(([processed, total]) => `${processed} / ${total}`);
  // Add listr task, render progressString$
}

@durran
Copy link
Member

durran commented Jan 28, 2021

Ok I've tested this and have a test written for the suite as well. So I'll merge and then put a PR up with that test for it, and then a new PR with the port to 4.0. @emadum @nbbeeken Are you both ok with that?

Copy link
Contributor

@nbbeeken nbbeeken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@durran durran merged commit 8082c89 into mongodb:3.6 Jan 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants