Key types don't match those used when querying

[mcfarljw]

I know that over the years the datastore key format has changed and we have been using it for nearly 10 years so we have entities spanning all these types. I've been noticing that it's quite hard to use some of these keys because sometimes they need to be either integers or strings. It seems that the node library just returns everything key related as a string.

Environment details

• OS: macOS High Sierra 10.13.2
• Node.js version: 8.9.3
• npm version: 5.6.8
• @google-cloud/datastore version: 1.3.3

Steps to reproduce

We have several entities that directly reference other entities by key.

When these entities are returned they do contained the referenced keys, but all id property have been converted to a string. Here are two examples using different id length formats.

// example 1
const key1 = Key {
 namespace: undefined,
 id: '47881200',
 kind: 'VocabListInfo',
 path: [ 'VocabListInfo', '47881200' ]
} 

// example 2
const key2 = Key {
 namespace: undefined,
 id: '6038270471438336',
 kind: 'VocabListInfo',
 path: [ 'VocabListInfo', '6038270471438336' ]
}

Now consider the following queries based on the referenced keys.

// won't return anything due to string key id value being used
const keyTest1 = datastore.key(['VocabListInfo', key1.id])
const list = await datastore.get(keyTest1)

// will return something after manually hacking the key format
const keyTest2 = datastore.key(['VocabListInfo', parseInt(key1.id, 10)])
const list = await datastore.get(keyTest2)

I guess my overall point is that the returned key information whether referenced or not should be in an immediately reusable state for querying.

[mcfarljw]

I think my workaround might also help solve the problem. It just makes the assumption that ids should always be integers and names strings. It seems to have solved the problem, but still feels a bit hacky.

export function getKeyFromEntity (entity) {
  const key = entity[datastore.KEY]

  if (!key) {
    throw new Error('EntityMissingKey')
  }

  return key.name ? key.name.toString() : parseInt(key.id, 10)
}

[stephenplusplus]

Unfortunately, this is more complicated than it seems like it needs to be. This started with this issue: googleapis/google-cloud-node#1412

Basically, the boundaries of an integer in JS cannot support the range of available IDs that are used by Datastore. We had to create a custom type, Datastore.int() and use it as the id property for every Key that uses a numeric ID.

Any place that Key object is used as a whole, we recognize that it's an ID and treat it as one. However, isolating the id property of a returned Key directly will return a simple string. At that point, you're somewhat off the path of our library and into your own application. Your solution using parseInt() is fine, or ds.int(key.id). Alternatively, you could override the kind property of the key, which will leave the ID in place.

I'm going to close, as I don't believe we have many options, given the requirements to support long IDs. However, please let me know if you can think of anything, and we can re-open and explore.