Skip to content

Iterator does not return all values. #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wraithgar opened this issue Oct 23, 2021 · 3 comments · Fixed by #9
Closed

Iterator does not return all values. #8

wraithgar opened this issue Oct 23, 2021 · 3 comments · Fixed by #9

Comments

@wraithgar
Copy link

wraithgar commented Oct 23, 2021

It looks like there are two bugs with the iterator, first off the example gives a false sense of the data type that is returned.

iter.next() returns an array of even length, not just an array with two items. They can be larger, and will have look like value1, key1, ...., valuen, keyn]

This code snippet shows what actually works

  const db = new LevelDB(dbPath)
  await db.open()
  const iter = db.getIterator({ keys: true, values: true })
  while (entries = await iter.next()) {
    for (let i = 0; i < entries.length; i = i + 2) {
      const entry = { key: entries[i+1], value: entries[i] }
      // you now have a key/value pair you can act on
    }
  }
  await db.close()

This is the data type regardless of if you pass { keys: false } or { values: false }, all that does is leave empty Buffers (or strings if you asked for them) in those places in the entries array.

Second, the last set of entries is always discarded and is never returned from the iterator. This line is where that bug is:

return val.finished ? null : val.array

Even if the iterator returned val.finished = true, val.array may still have the last set of data in it and that should be returned. A subsequent call to the iterator will then return an empty val.array and of course finished will still be true.

The fix is to always return val.array. I don't think it can ever be empty. The last array of data will return with finished = true which will set this.finished in the iterator, and subsequent calls will short out on this line:

if (this.finished) return null

@extremeheat
Copy link
Owner

extremeheat commented Oct 24, 2021

Thanks for the report, looks like the cache logic was not ported correctly. Can you verify if #9 fixes?

As for the final check, should be fixed: if the DB is empty, per the C++ bindings we'd return empty array [] instead of going through the finished check, so it does need to be there. It looks like a similar issue is present for 1-entry DBs. So this should be addressed on the C++ side, but I think handling on the JS side is OK for now.

@wraithgar
Copy link
Author

I checked #9 out locally and tested it, it works. All of the entries are returned now, in the expected order.

Thank you.

@extremeheat
Copy link
Owner

Fix released as 1.1.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants