Google Cloud Datastore

 Google Cloud Datastore

Google Cloud Datastore is a NoSQL document database primarily used for back end storage processes. It provides high performance and is highly scalable.  The databases are good at storing unstructured data.

Document Storage is a type of NoSQL database to store, retrieve, and manage un-structured information.

Characteristics

Datastore is highly scalable. It automatically handles sharding, the process that separates very large databases into smaller manageable parts. The performance is good because the reads are tied to the result set and not the entire data set.

Data is replicated  which ensures that data is protected by copying it to a different place. So availability of data is high and this minimizes the impact of hardware/network failures.

Seamless automatic scaling ensures that you will not run out of resources like disk space and traffic throughput as your application grows.

Using ACID transactions provides consistency and reliability of queries.  

Atomicity means that every step of the transaction is successful otherwise the entire transaction is rolled back.

Consistency means that a transaction adheres to data rules in place which and reject transactions violating these rules. takes the database from one.

Isolation prevents transactions from interfering with each other when simultaneous transactions occur and only handle them one at a time.

Durability ensures that when a transaction is complete it is committed and saved.

Schema flexibility allows for entities within the same kind not needed to share the same properties.

Data is secured by encrypting data at rest when stored on the Google cloud.

Horizontal Scalability allows for resources to be added to the pool as needed instead of dedicating them in advance.

Data Locality allows you to store related data near each other.

Access methods 

Methods to access data can be handled several ways.

By client methods via SDK’s for popular languages like Python, Node.js, Java and PHP. Provide objects and methods for access. 

Use REST APIs via HTTP GET/POST standard web request.

JSON API’s, human readable message returning JSON formatted data.

Saving data uses an asynchronous process to update 

When you save data, Datastore uses an asynchronous process to update indexes which means it updates in the background. This creates eventual consistency because it is run in the background updates will happen after the asynchronous process is complete.

Filters

Filters allow you to return data based on a criteria.

Can filter data based on the values of properties using” ==”,”<”,”>”, and “!=”.

You can combine two filters using AND or OR logic.

Access data via the unique ID via the key filter using the __key__ property.

Can sort data in ascending or descending order.

The IN and NOT_IN operators

When requesting data from Datastore you can use indexes and filters.

Some of the filters you can use are equity == to find data that matches request, less than and greater than for comparison, != for not equal to value and membership queries such as IN and NOT_IN.

Also, composite filters such as AND/OR that combine multiple filters.

Data can be sorted in ascending or descending order on a property. When using not equal filters data needs to be sorted first.

Data can be accessed via keys set up at creation of the kind. 

Cursors are a string that points to index the last query ended.

This allows you to jump to that spot without performance degradation.

Critical Limitations

Use of the “!=” or “NOT_IN” can only be used once per query.

Does not support partial matches that the SQL LIKE %match does.

Some Datastore limitations are you cannot join two different kinds like you can do in an RDBMS system. 

Also, if property is missing it will be excluded from the results.

Compared to Relational databases

Compared to Relational databases.

Datastore uses kind instead of tables.

Entity instead of Row or Record. 

And Property instead of Column or Field.

Individual data points (integers, strings, dates, etc.).

Key instead of Primary Key as unique identifier for an entity.

It is a collection of properties (integers, strings, dates, binary data, 

Keys used as a unique identifier either a string name or an auto-generated number.

Data Types

Below is some of the data types used when defining individual properties in a datastore kind

Integer - Numbers

Floating-point -  allows for decimal storing

Boolean - True or false values

String - Text, numbers, characters limited to 1500 bytes

Blob (Byte String) -  store large unstructured data

Text - up to 1 megabyte, can’t be indexed

Date and Time - stores date and time  in UTC

GQL (Google Query Language) 

GQL is an SQL like google developed language used to retrieve data from Datastore. 

The SELECT statement is similar to SQL 

SELECT (property) FROM (kind) WHERE (filters) 

Like SQL the * provides the full list of properties 

Filters can be added based upon specific criteria.

The ORDER BY command can be added to sort the output of the statement. Can be designated as ASC (ascending) or desc (descending)

You can use the wildcard(*) to select all entities.

LIMIT can be used to provide a maximum number of results you request.

The "Key Lookup" If you only have the unique ID and want to filter by it specifically.

WHERE __key__ = KEY(Task, 'high-priority-task-001')

Operations

These are the fundamental building blocks for managing data.

The 4 main operations are get(retrieves entities), put (writes/updates entities), delete(remove entities) and Query(return group of entities).

Datastore uses kind instead of tables and entities instead of Row or Record .

The put command writes records to the specified kind. If a key exists it will overwrite the entity with the new data. If a key doesn’t exist it will create one.

The Get command returns an entity leveraging the key for that kind. Very fast, bypasses search indexes and retrieves records directly.

The Delete command removes the entity related to the key provided.

The Query command returns entities based on a criteria.