datastore: get stats / total / count?

[raitucarp]

how to get stats / total / count information in dataset?

[stephenplusplus]

@pcostell - are you able to help @raitucarp?

[pcostell]

Datastore has statistics which are generated on a daily basis. The Datastore documentation has information about all the different statistics about your dataset.

For example to programmatically get your total number of entities and their total size:

query = dataset.createQuery(['__Stat_Total__'])
dataset.runQuery(query, function(err, entities, endCursor) {
  if (err) {
    return;
  }
  console.log('Number of entities:', entities[0].data.count)
  console.log('Total entity size:', entities[0].data.entity_size)
});

You can also see this information in a UI by visiting:

https://console.developers.google.com/project/<YOUR-PROJECT-ID>/datastore/stats

[dhermes]

@pcostell Do you think it'd be worth adding some syntactic sugar around this and offering as a feature?

[pcostell]

I'm not totally sure, but my guess would be that this is not the sort of thing that is useful to most developers from an API perspective. My guess is that in general an application doesn't need to know about it, so it would only be useful if the developer wanted to have a different API than that provided by the Cloud Console.

[malditogeek]

@pcostell sorry for asking in the super old issue but do you know if the Datastore Emulator supports this stats queries?

[kunalvjti]

Seems like the stats are not generated on a daily basis. Both the API and the UI dashboard mentioned by @pcostell above throw the stale info. My dashboard says 'Last updated 2 days ago — May 5, 2017 at 12:42:27 AM UTC-7'. It also mentions 'Statistics are generated every 24–48 hours. In addition to the kinds used for your entities, statistics also include system kinds, such as those related to metadata. System kinds are not visible in your kinds menu. Note that statistics may not be available for very large datasets.'
What is a very large dataset ? I have loaded approx ~52k entities in Datastore and want to verify the actual count without waiting until the stats get regenerated again. How can i achieve this ?

[dgonzalez]

Well... this is super useful for developers. For example pagination... It is good to know how many pages you want to display.