Datastore Statistics

Firestore in Datastore mode maintains statistics about the data stored for an application, such as how many entities there are of a given kind, or how much space is used by property values of a given type. You can view these statistics in the Google Cloud console. Either open the Dashboard page or run a GQL query in the form of SELECT * FROM __Stat_Kind__ from the Entities page.

You can also access these values programmatically within the application by querying for specially named entities using the Datastore API. Each statistic is accessible as an entity whose kind name begins and ends with two underscores. For example, each app has exactly one entity of the kind __Stat_Total__ that represents statistics about all of the entities in Datastore mode in total. Each statistic entity has the following properties:

  • count, the number of items considered by the statistic (a long integer)
  • bytes, the total size of the items for this statistic (a long integer)
  • timestamp, the time of the most recent update to the statistic (a date-time value)

Some statistic kinds also have additional properties, listed below.

When the statistics system creates new statistic entities, it does not delete the old statistic entities right away. The best way to get a consistent view of the statistics is to query for the statistic entity with the most recent timestamp, then use that timestamp value as a filter when fetching other statistic entities.

The statistic entities are included in the calculated statistic values. Statistic entities take up space relative to the number of unique kinds and property names used by the application.

The statistics system will also create statistics specific to each namespace. Note that if an application does not use namespaces, then namespace specific statistics will not be created. Namespace specific stats are found in the namespace that they're specific to. The kind names for namespace specific stats are prefixed with __Stat_Ns_ and have the same corresponding suffix as application wide statistics kinds.

List of statistics

The complete list of available statistics is as follows:

Statistic Stat Entity Kind Description
all entities __Stat_Total__
Namespace specific entry:
__Stat_Ns_Total__
All entities. Additional properties:

entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in built-in index entries measured in bytes.
builtin_index_count: the count of built-in index entries.
composite_index_bytes: The storage in composite index entries measured in bytes.
composite_index_count: The count of composite index entries.
all entities in a namespace __Stat_Namespace__
Note that __Stat_Namespace__ entities are created for each namespace encountered and are only found in the empty string namespace.
All entities in a namespace.

subject_namespace, the namespace represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in built-in index entries measured in bytes.
builtin_index_count: the count of built-in index entries.
composite_index_bytes: The storage in composite index entries measured in bytes.
composite_index_count: The count of composite index entries.
all entries in application defined indexes __Stat_Kind_CompositeIndex__
Namespace specific entry: __Stat_Ns_Kind_CompositeIndex__
Entries in the composite index table; one stat entity for each kind of entity stored. Additional properties:

index_id, the index id.
kind_name, the name of the kind represented (a string)
entities of a kind __Stat_Kind__
Namespace specific entry:
__Stat_Ns_Kind__
Entities of a kind; one stat entity for each kind of entity stored. Additional properties:

kind_name, the name of the kind represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in built-in index entries measured in bytes.
builtin_index_count: the count of built-in index entries.
composite_index_bytes: The storage in composite index entries measured in bytes.
composite_index_count: The count of composite index entries.
root entities of a kind __Stat_Kind_IsRootEntity__
Namespace specific entry:
__Stat_Ns_Kind_IsRootEntity__
Entities of a kind that are entity group root entities (have no ancestor parent); one stat entity for each kind of entity stored. Additional properties:

kind_name, the name of the kind represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
non-root entities of a kind __Stat_Kind_NotRootEntity__
Namespace specific entry:
__Stat_Ns_Kind_NotRootEntity__
Entities of a kind that are not entity group root entities (have an ancestor parent); one stat entity for each kind of entity stored. Additional properties:

kind_name, the name of the kind represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
properties of a type __Stat_PropertyType__
Namespace specific entry:
__Stat_Ns_PropertyType__
Properties of a value type across all entities; one stat entity per value type. Additional properties:

property_type, the name of the value type (a string)
entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in built-in index entries measured in bytes.
builtin_index_count: the count of built-in index entries.
properties of a type per kind __Stat_PropertyType_Kind__
Namespace specific entry:
__Stat_Ns_PropertyType_Kind__
Properties of a value type across entities of a given kind; one stat entity per combination of property type and kind. Additional properties:

property_type, the name of the value type (a string)
kind_name, the name of the kind represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in the built-in index measured in bytes.
builtin_index_count: the count of built-in index entries.
properties with a name __Stat_PropertyName_Kind__
Namespace specific entry:
__Stat_Ns_PropertyName_Kind__
Properties with a given name across entities of a given kind; one stat entity per combination of unique property name and kind. Additional properties:

property_name, the name of the property (a string)
kind_name, the name of the kind represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in built-in index entries measured in bytes.
builtin_index_count: the count of built-in index entries.
properties of a type and with a name __Stat_PropertyType_PropertyName_Kind__
Namespace specific entry:
__Stat_Ns_PropertyType_PropertyName_Kind__
Properties with a given name and of a given value type across entities of a given kind; one stat entity per combination of property name, value type and kind that exists in the database. Additional properties:

property_type, the name of the value type (a string)
property_name, the name of the property (a string)
kind_name, the name of the kind represented (a string)
entity_bytes: The storage in the entities table measured in bytes.
builtin_index_bytes: The storage in built-in index entries measured in bytes.
builtin_index_count: the count of built-in index entries.

Some statistics refer to property value types by name, as strings. These names are as follows:

  • "Blob"
  • "BlobKey"
  • "Boolean"
  • "Category"
  • "Date/Time"
  • "Email"
  • "Float"
  • "GeoPt"
  • "IM"
  • "Integer"
  • "Key"
  • "Link"
  • "NULL"
  • "PhoneNumber"
  • "PostalAddress"
  • "Rating"
  • "ShortBlob"
  • "String"
  • "Text"
  • "User"

Statistics entities drop order

Applications with thousands of namespaces, kinds, or property names require a large number of statistics entities. To reduce the overhead of storing and updating the statistics, Firestore in Datastore mode databases progressively drop statistics entities according to the orders below.

Firestore in Datastore mode tracks the size of kind and non-kind statistics entities separately. The size of non-kind statistics entities does not affect when kind statistics are dropped.

The summary statistics entities __Stat_Kind_CompositeIndex__, __Stat_PropertyType__, and __Stat_Total__ are never dropped.

Drop order for kind statistics entities

Kind entity statistics are dropped in the following order:

  • per-namespace, per-kind statistics

    • __Stat_Ns_Kind__
    • __Stat_Ns_Kind_IsRootEntity__
    • __Stat_Ns_Kind_NotRootEntity__
    • __Stat_Ns_PropertyType_Kind__
  • per-kind statistics

    • __Stat_Kind__
    • __Stat_Kind_IsRootEntity__
    • __Stat_Kind_NotRootEntity__
    • __Stat_PropertyType_Kind__

Drop order for non-kind statistics entities

Non-kind entity statistics are dropped in the following order:

  • per-namespace, per-kind, and per-property statistics:

    • __Stat_Ns_PropertyName_Kind__
    • __Stat_Ns_PropertyType_PropertyName_Kind__
  • per-kind and per-property statistics

    • __Stat_PropertyName_Kind__
    • __Stat_PropertyType_PropertyName_Kind__
  • per-namespace statistics

    • __Stat_Namespace__
    • __Stat_Ns_Kind_CompositeIndex__
    • __Stat_Ns_PropertyType__
    • __Stat_Ns_Total__

See Also