Cheat Sheet for Amazon DynamoDB

What is Amazon DynamoDB?

Amazon DynamoDB on AWS is a fully managed, serverless, key-value NoSQL database that provides fast and predictable performance with seamless scalability.

With DynamoDB, you can create database tables that can store and retrieve any amount of data and serve any level of request traffic. You can scale up or scale down your tables' throughput capacity without downtime or performance degradation.

AWS DynamoDB provides on-demand backup capability. It allows you to create full backups of your tables for long-term retention and archival for regulatory compliance needs. DynamoDB also allows you to delete expired items from tables automatically, to help you reduce storage usage and the cost of storing data that is no longer relevant.

Amazon DynamoDB Advantages

Integration of DynamoDB with other AWS services provides multiple advantages that can make it more beneficial over other NoSQL databases like MongoDB.

• Setting up is similar to a severless database.
• Security for DynamoDB is governed by AWS Identity and Access Management (IAM). You can also use other AWS security features to enhance the controls.
• DynamoDB gives you the ability to auto-scale by tracking how close your usage is to the upper bounds. This can allow your system to adjust according to the amount of data traffic, helping you to avoid issues with performance while reducing costs.
• TTL or Time-to-Live is a process that allows you to set timestamps for deleting expired data from your tables. As soon as the timestamp expires, the data that is marked to expire is then deleted from the table. Through this functionality, developers can keep track of expired data and delete it automatically. This process also helps in reducing storage and cutting costs by reducing the need for manual deletion.
• Since Amazon DynamoDB is a fully managed service, you don't need to worry about hardware or software provisioning, setup and configuration, software patching, or partitioning data over multiple instances as you scale.

Amazon DynamoDB Disadvantages

• There is no support for on-premise deployment or hybrid cloud. DynamoDB is cloud-native, so it only runs on Amazon Web Services (AWS).
• DynamoDB is schemaless. Unlike relational databases, this database does not need to confirm any schema. So lack of any validation or convention might lead to a huge mess.
• Primary key has at most two attributes which limits query flexibility.
• It has limited data type support (number, string, binary, boolean only), which increases application complexity.
• DynamoDB is a NoSQL database. That means you can not do joins or complex queries on DynamoDB. Also it is ACID incompatible as it is not a relational database.
• There is a limitation of 400Kb object size for each attribute that you put into a DynamoDB table. So you can not write large objects (BLOBS) into DynamoDB .
• For bulk record updates, or change in your data model (splitting one field to multiple fields) you must update each record individually. This may not seem too bad at first, but with a large set of records it becomes very complex.

Core Components of Dynamo DB

In DynamoDB it's tables, items, and attributes that are the core components.

1. Table - Like relational database systems, DynamoDB stores data in tables.
   
2. Items - Each table contains zero or more items. An item is a group of attributes that is uniquely identifiable among all of the other items. Items in DynamoDB are similar in many ways to rows, records, or tuples in relational database systems. In DynamoDB, there is no limit to the number of items you can store in a table.
   
   Each item in the table has a unique identifier, or primary key, that distinguishes the item from all of the others in the table. In the Customer table, the primary key consists of one attribute (Customer ID).
   
3. Attributes- Each item is composed of one or more attributes. An 'attribute' is a fundamental/smallest data element. For example, an item in a Customer table contains attributes called CustomerID, LastName, FirstName.
   
   Attributes in DynamoDB are similar in many ways to fields or columns in other database systems. Most of the attributes are scalar, which means that they can have only one value. Strings and numbers are common examples of scalars.

Primary Key

Primary keys define where and how your data will be stored in partitions. DynamoDB supports two different kinds of primary keys:

• Partition Key (PK) also known as HASH - A simple primary key composed of one attribute known as the partition key. DynamoDB uses the partition key's value as input to an internal hash function. The output from the hash function determines the partition (physical storage internal to DynamoDB) in which the item will be stored.

In a table that has only a partition key, no two items can have the same partition key value.

• Partition Key and Sort Key (SK) also known as RANGE - This type of key is composed of two attributes, also referred as composite primary key. The first attribute is the partition key and the second attribute is the sort key. DynamoDB uses the partition key value as input to an internal hash function. The output from the hash function determines the partition (physical storage internal to DynamoDB) in which the item will be stored. All items with the same partition key value are stored together, in sorted order by sort key value.

Secondary Indexes

DynamoDB has two types of Indexes: Local Secondary Index (LSI) and Global Secondary Index (GSI).

LSI - Local Secondary Index

• An index that has the same partition key as the table, but a different sort key. Strongly or eventual consistency reads.
• Can only be created with initial table (cannot be modified and cannot be deleted unless also deleting the table)
• Only Composite
• 10GB or less per partition
• Share capacity units with base table
• Must share Partition Key (PK) with base table

GSI - Global Secondary Index

• An index with a partition key and sort key that can be different from those on the table
• Only Eventual consistency reads (cannot provide strong consistency)
• Can create, modify or delete at anytime
• Simple and Composite
• Can have whatever attributes as Primary Key (PK) or Secondary Key (SK)
• No size restriction per partition
• Has its own capacity settings (does not share with base table)

Creating Tables & Loading Data in DynamoDB

AWS Management Console is used to create tables in Amazon DynamoDB. You then load data into these tables using the AWS Command Line Interface (AWS CLI).

Open the DynamoDB console at https://console.aws.amazon.com/dynamodb/, then choose 'Create Table'.

• In the Table name box, enter "Music".
• For the Primary key, do the following:
  • In the Partition key box, enter "Artist". Set the data type to 'String'.
  • Choose 'Add' sort key.
  • In the 'Sort' key box, enter 'Datetime'. Set the data type to 'String'.
• In the Table settings section, clear Use default settings.
• In the Secondary indexes section, choose Add index.
• In the Add index window, do the following:
  • For the Primary key...
    • In the Partition key box, enter CustomerID. Set the data type to String.
    • Choose Add sort key.
    • In the Sort key box, enter createDate. Set the data type to String.
  • In the Index name box, enter CreateIndex
  • Set the Projected attributes to All.
  • Choose Add index.

When the settings are as you want them, choose 'Create table'.

Write Data Into Tables

1. Open the DynamoDB console at https://console.aws.amazon.com/dynamodb/.
2. In the navigation pane on the left side of the console, choose 'Tables'.
3. In the table list, choose the 'Music' table.
4. Click the 'Create Item' button, as shown below.

Choose 'Add new attribute' and its datatype to add new field to your document/items.

Read Data From Tables

Follow these steps to read data from the 'Music' table using the DynamoDB console.

• Open the DynamoDB console at https://console.aws.amazon.com/dynamodb/.
• In the navigation pane on the left side of the console, choose 'Tables'.
• Choose the 'Music' table from the table list.
• Select the 'View items.

Update an Item in DynamoDB

You can use the DynamoDB console to update data in the 'Music' table.

• Open the DynamoDB console at https://console.aws.amazon.com/dynamodb/.
• In the navigation pane on the left side of the console, choose 'Tables'.
• Choose the 'Music' table from the table list.
• Choose 'View items'.
• Clicking on a particular item, it will open item editor and that particular item can be altered

Dynamo DB CLI Commands

Create Table
aws dynamodb create-table \
--table-name TesttableName \
--attribute-definitions '[{"AttributeName":"id", "AttributeType":"S"},
{"AttributeName":"createdAt", "AttributeType":"S"}]' \
--key-schema '[{"AttributeName":"id", "KeyType":"HASH"}, {"AttributeName":"createdAt", "KeyType":"RANGE"}]' \
--provisioned-throughput '{"ReadCapacityUnits": 1, "WriteCapacityUnits": 1}'

Delete Table
aws dynamodb delete-table \
--table-name TesttableName

List Tables
aws dynamodb list-tables

Describe Table
aws dynamodb describe-table \
--table-name TesttableName

Backup Table
aws dynamodb create-backup \
--table-name TesttableName
--backup-name TestBackup

List Backup Tables
aws dynamodb list-backups

List Backups For a Specific Table Within a Date Range
(in epoch format)
aws dynamodb list-backups \
--table-name TesttableName \
--time-range-lower-bound 1577836800 \
--time-range-upper-bound 1583020800 \
--backup-type TESTUSER

Describe Backup
aws dynamodb describe-backup

Restore Table From Backup
aws dynamodb restore-table-from-backup \
--target-table-name NewTablefromBackup
--backup-arn ARNOfBackup

Get/Select All Items or Scan
aws dynamodb scan \
--table-name TestTableName

Filter Attribute Data
aws dynamodb scan \
--table-name TestTableName
--filter-expression "lastName = :name" \
--expression-attribute-values '{":name":{"S":"Cross"}}'

Get/Select Item
aws dynamodb get-item \
--table-name TestTableName \
--key '{"id": {"S": "100"}, "email": {"S": "David.cross@test.com"}}'

Put/Insert Item
aws dynamodb put-item \
--table-name TestTableName \
--item '{"id":"100"}'

Delete Item
aws dynamodb delete-item \
--table-name TestTableName \
--key '{"id": {"S": "100"}, "email": {"S": "David.cross@test.com"}}'

Delete Item With condition
aws dynamodb delete-item \
--table-name TestTableName \
--key '{"id": {"S": "100"}, "email": {"S": "David.cross@test.com"}}'
--condition-expression 'attribute_not_exists(updatedAt)'

Query Set of Items
aws dynamodb query\
--table-name TestTableName \
--key-condition-expression "id = :maid" \
--expression-attribute-values '{":v1": {"S": "my address"}}'
--filter-expression 'attribute_not_exists(updatedAt)'

Query With Sort
aws dynamodb query\
--table-name TestTableName \
--key-condition-expression "id = :testid" \
--expression-attribute-values '{":v1": {"S": "my address"}}'
--filter-expression 'attribute_not_exists(updatedAt)'
- -scan-index-forward

Update Item
aws dynamodb update-item \
--table-name TestTableName \
--key '{"id":{"S":"David-100"}}' \
--update-expression "SET Address = :h, DOB = :d" \
--expression-attribute-values '{":h": { "S": "Downtown Square 1" },":d": { "S": "987654321" }}' \
--return-values ALL_NEW

Query Pagination
To get all of the items matching query criteria, you should use 'Pagination'. In DynamoDB, pagination is consisting of two pieces:

1. NextToken: return value which tells when the Query operation last stopped.
2. Starting-token: parameter which tells where the next Query operation should start.

Putting 'LastEvaluatedKey' in the place of 'starting-token' will allow you to get all the data-matching key conditions.

aws dynamodb query \
--table-name TestTableName \
--key-condition-expression "id = :testId" \
--expression-attribute-values '{":v1": {"S": "My address"}}'
--filter-expression 'attribute_not_exists(updatedAt)'
--scan-index-forward
--starting-token '<VALUE_OF_NEXT_TOKEN_FROM_PREV_OPERATION>'

Identity & Access Management in Amazon DynamoDB

Access to Amazon DynamoDB requires credentials. Those credentials must have permissions to access AWS resources, such as an Amazon DynamoDBtable. The following sections describe how to manage permissions for Amazon DynamoDB.

Every AWS resource is owned by an AWS account, and permissions to create or access a resource are governed by permissions policies. An account administrator can attach permissions policies to IAM identities (that is, users, groups, and roles), and some services (such as AWS Lambda) also support attaching permissions policies to resources.

The following AWS managed policies, which you can attach to users in your account, are specific to DynamoDB and are grouped by use-case scenario:

• AmazonDynamoDBReadOnlyAccess: Grants read-only access to DynamoDBresources through the AWS Management Console.
• AmazonDynamoDBFullAccess: Grants full access to DynamoDB resources through the AWS Management Console.

You can review these AWS managed permissions policies by signing in to the IAM console and searching for specific policies there.

Wrap-up

As a non-relational database, DynamoDB is a reliable system that helps small, medium and large enterprises scale their applications. It comes with options to backup, restore and secure data, and is great for both mobile and web apps. With the exception of special services like financial transactions and healthcare, you can redesign almost any application with DynamoDB.

Zuar's products and services help you automate and manage your data pipeline. For example, we can help you connect DynamoDB to an analytics platform like Tableau. Visit here to learn more.