Dynasty.js

Dynasty is a clean and simple Amazon DynamoDB client for Node with baked in Promise support.

DynamoDB is Amazon's high performance, all SSD backed, fully managed NoSQL database offering. This library was built in an attempt to make it a bit more easy to work with.

Dynasty is open source software and is released under the developer and business-friendly MIT license.

Endorse victorquinn on Coderwall

Written by Victor Quinn

NPM

Or grab the latest version on Github.


Getting Started

node

Easy to install in Node.js

npm install dynasty

credentials

Amazon uses a 2 key system for access to its APIs. They are the Access Key Id and the Secret Access Key.

Simply create an object with at least these 2 keys and your specific keys as the values:

var credentials = {
    accessKeyId: '<YOUR ACCESS_KEY_ID>',
    secretAccessKey: '<YOUR_SECRET_ACCESS_KEY>'
};

Your accessKeyId and secretAccessKey can be obtained in the AWS console under the IAM (Identity Account Management) menu which has a green key as its icon.

Amazon recommends that you create a new User to limit access to DynamoDB if you haven’t already.

More info on getting started with credentials

Recommend storing these credentials in environment variables and loading them in a config file.

So rather than the text strings appearing in your code (and worse, getting committed to your repo!), set them as environment variables and load them in your code as follows:

var credentials = {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
};

Can optionally specify a region. If none is specified, us-east-1 is the default.

For example, to use the eu-west-1 region based in Ireland, use the following credentials:

var credentials = {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: 'eu-west-1'
};

Only this short string is necessary to specify the region, Dynasty will convert it into the full endpoint URL for you.

instantiate

Ok, so now that we’ve got it installed and we have the credentials sorted, let’s instantiate it so we can actually use it!

var dynasty = require('dynasty')(credentials);

That was easy! But how to query it?

primer

But first, a bit of background…

This is not a full DynamoDB primer, but it makes some sense to give a high level overview for the rest of this document to make sense.

Ideally, you should never have to visit the underlying DynamoDB docs in order to use it, Dynasty should provide everything you need. So this is not exhaustive, but should be enough to get you going.

Tables

DynamoDB stores items in Tables. A Table in Dynamo is analogous to a Table in SQL or a Collection in Mongo.

Though of course unlike a SQL table, DynamoDB is NoSQL so there is no Schema here.

It is the fundamental bucket into which items are pushed.

Each Table has a name and a Key Schema (explained below).

Example:

If you wanted to store data on a series of Users, it may make sense to create a UserData table to hold the data for each user.

Items

Items are put into Tables.

They are analogous to rows in SQL or documents in Mongo.

Each Item must at minimum have the keys/values specified in the Key Schema but they may have any number of other key/value pairs.

Aside from the Key Schema, there is no need to be consistent across items.

Example:

For user with username victorquinn you may want an item that looks like:

{
    'username': 'victorquinn',
    'first': 'Victor',
    'last': 'Quinn',
    'motorcycle': 'Harley'
}

whereas for someone else:

{
    'username': 'john',
    'first': 'John',
    'car': 'Ford'
}

Note how one has motorcycle, the other has car, one has a last name, the other doesn’t, this is fine as long as they both meet the Key Schema.

Key Schema

DynamoDB allows 2 different Key Schemas:

  • Hash Type Primary Key
  • Hash and Range Type Primary Key
Hash Type Primary Key

Should be used when you have a single unique identifier used in the Table to look up the Item.

It’s better to use a Hash and Range Type Primary Key when possible (as it can be better on performance, more below) but if you have only a single identifier a Hash Type Primary Key is perfect.

Example: A table of UserData, the username may be a Hash Type Primary Key. Given this username you can uniquely identify the user and retrieve their data.

Hash and Range Type Primary Key

A composite key. Made up of 2 keys, the so called Hash and Range keys.

This can be more performant as Amazon wil shard the database based on the Hash key making lookups in very large databases faster. (thankfully you never have to worry about any of this, it all happens magically in the background)

Example: A table of certifications of User by state. Hash key could be State, Range key could be Username. This will help as rather than looking through every record for the State/Username combo, DynamoDB can quickly narrow it down by 49 states.

For more examples, see Amazon’s list of examples. For more info generally on these topics, check out Amazon’s docs.

table object

Meanwhile, back at the ranch…

So assuming you’ve created a table in DynamoDB (either programmatically with Dynasty or manually in the AWS console) we can get an object representing that Table to put items into that table, get items from that table, and perform any other operations on that table.

So let’s say we have a DynamoDB table already created called UserData and we want to retrieve an item from it.

var dynasty = require('Dynasty')(credentials),
    // Retrieve an instance of the UserData Table
    users = dynasty.table('UserData');

Now we can query it:

// This will look in the UserData table, returning the item with a Hash key that
// matches 'victorquinn'
var promise = users.find('victorquinn');

Note, I’ve chosen to use a Mongo-style syntax here, rather than the DynamoDB names. This is because:

  1. I find it more natural
  2. Other developers coming to Dynamo will probably find it more natural
  3. If moving to/from DynamoDB this should ease the transition quite a bit.

Also note, we’re not passing it a callback. We can, but we don’t need to because Dynasty has promise support baked in!

Dynasty will also accept a callback function as an optional argument though!

promises

As shown above, we performed a query and set its return to a variable we called promise. Of course we could have called the variable anything, but we called it that for clarity because that thing it returned was a Promise.

Under the hood, we are using the Bluebird Promise library

Overview

In brief Promises are a way clean up the callback stack and provide a way for programs to pass the flow off as it sees fit rather than trying to jam it all into a single callback function. It offers an inversion of control so the code requesting the asyncronous operation need not relinquish it.

In Programming

In more concrete terms, a Promise is an object that has callbacks which will be called when the asynchronous action finishes.

Those callbacks do not need to be set or determined at the time the Promise is formed, so they can be added on later.

This means that a program can ask for something that will take time (such as going off into the intarwebs to hit DynamoDB and return a result) and from that request receive in return a Promise to which it can attach callbacks which will be called when the network call returns and the object is ready for use.

Real World Counter-example - Restaurant Pager

For something like this which is rather complex, I prefer a real world tangible example, or in this case a counter-example.

What follows is an example of a Callback and not a promise.

Imagine one of those flashy pagers that some restaurants will give you when they have a long wait for a table.

Restaurant Pager

You no longer have to stand and wait in a line, you can take that pager, go do something else, and eventually the pager will flash and vibrate you to alert you that your table is ready.

So this is already better than the blocking request most other systems use exclusively to handle the time spent waiting for a response.

This pager is the more analogous to a Callback in Node. You had to tell the maître d' up front that you wanted a table and for how many and all that.

The restaurant in this situation holds the control. You have to give them your request up front and when they’re ready they call you and you get what you asked for at the time they gave you the pager. If you changed your mind between the time you were handed the pager and the time you’re called you’re out of luck.

But what if you wanted to invert that control?

What if you could say to them, I will want something when you can handle me, I’m not sure exactly what I’ll want, but I can take my time figuring it out because you’re busy anyway.

Real World Example - Deli

For a real world example of a Promise, think of a deli.

In a deli, you grab a slip of paper which represents a Promise that when they have capacity, one of the workers will serve you.

Deli Number

You don’t have to tell them what you want up front. You could change your mind in the time between when you pulled that number, or you could ignore it completely when the number came up, or you could pass it off to a friend who could do what they’d like when the number comes up.

The applications and benefits of Promises in JavaScript may not be immediately noticeable but there are many and a lot of people are really excited about Promises.

You can certainly count me, the author of this library, among those who are excited :)

Learn More

If you want to learn more about Promises and async JavaScript programming generally I strongly recommend the book Async JavaScript: Build More Responsive Apps with Less Code by Trevor Burnham.

It does a fantastic job of explaining a lot of async JavaScript concepts and covers Promises in great detail in Chapter 3.

Code!

Enough of the Jibber Jabber though, let’s see an example:

// Get a promise back from the query command
var promise = users.find('victorquinn');

// Tell the promise what we want to do when it gets data back from DynamoDB
promise.then(function(user) {
    // Not doing much useful here, but you get the point, now we have the
    // user object so we can do cool things with it.
    console.log(user.first);
});

Above we just told the Promise what to do when it got data back immediately but we could have passed that promise off to some other method and it could have added that.

We also could have done other neat things with the Promise like chaining, error handling, and more, but that’s outside the scope of this brief intro to Dynasty.

combined

Ok, so putting all these snippets together, we have:

var credentials = {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
};

// Set up Dynasty with the AWS credentials
var dynasty = require('dynasty')(credentials);

// Get the Dynasty table object representing the table you want to work with
var users = dynasty.table('UserData');

// Fire off the query, putting its result in the promise
var promise = users.find('victorquinn');

// Add a promise success handler for when the call returns
promise.then(function(user) {
    console.log(user.first);
});

And that’s it! Many more examples below for each specific method.

General

table

dynasty.table(<tablename>) ⇒ <Table Object>

Retrieve a Table object for the table matching the supplied name.

To retrieve an object for the table named Lands:

var lands = dynasty.table('Lands');

This table object can then be used to perform other queries.

Table

table overview

These methods act on Tables.

As such, for all the following examples, we’ll assume the setup has been, so there’s a dynasty object with the credentials

In other words, we’re going to assume the following code has been run already:

var credentials = {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
};

var dynasty = require('Dynasty')(credentials);

Further, all of these Table methods operate one of two ways:

  1. You can create and build a Table object, then call the method on that table object to enact the method.

    – OR –

  2. You can call them directly on the dynasty object, passing in as an argument an object which represents the attributes of the Table on which you’d like to perform the operation.

We’ll show examples of both modes of operation below.

alter

dynasty.alter('Lands', {throughput: {read: 25, write: 50}}) ⇒ promise

Alter a table.

Note: The only use case Amazon supports for altering is changing the throughput. There is no way, at current, to rename a table or alter its key schema.

dynasty
    .alter('Lands', { throughput: { read: 25, write: 50 } })
    .then(function(resp) {
        // Throughput has been updated!
    });

Optionally specify a callback function

dynasty.alter('Lands', {throughput: {read: 25, write: 50}}), function(err, resp) {
    if (err) // Something went wrong!
    
    // Your table has been created!
    console.log(resp);
});

create

dynasty.create('Lands', {key_schema: {hash: ['name', 'string']}, ...}) ⇒ promise

Create a table.

Simplest case, the first argument is the name of the table, the second is an object with at least a key_schema key.

dynasty
    .create('Lands', { key_schema: { hash: ['name', 'string'] } })
    .then(function(resp) {
        // Your table has been created!
    });

In this simplest case, we’ll default the throughput for you at:

5 write units/10 read units

Optionally specify the throughput:

var table_options = {
    key_schema: { hash: ['name', 'string'] },
    throughput: { write: 5, read: 10 }
};

dynasty
    .create('Lands', table_options)
    .then(function(resp) {
        // Your table has been created!
    });

Optionally specify a callback function:

dynasty.create('Lands', { key_schema: { hash: ['name', 'string'] } }, function(err, resp) {
    if (err) // Something went wrong!
    
    // Your table has been created!
    console.log(resp);
});

Optionally specify a range key when creating the table:

dynasty
    .create('Counties', { key_schema: {
        hash: ['country', 'string'],
        range: ['county', 'string']
    } })
    .then(function(resp) {
        // Your table has been created!
    });

describe

dynasty.describe('Lands') ⇒ promise

Describe a table.

Call it from the dynasty object

dynasty
    .describe('Lands')
    .then(function(resp) {
        // Log the description
        console.log(resp);
    });

Or create a table object and call it on that table object

var lands = dynasty.table('Lands');

lands
    .describe()
    .then(function(resp) {
        // Log the description
        console.log(resp);
    });

The return value for describe() is unaltered from the format Amazon sends.

Look in their docs for more details.

drop

dynasty.drop('Lands') ⇒ promise

Drop a table.

Simplest case, just provide the name of the table to be dropped.

dynasty
    .drop('Lands')
    .then(function(resp) {
        // Your table has been dropped!
    });

Optionally specify a callback function

dynasty.drop('Lands', function(err, resp) {
    if (err) // Something went wrong!
    
    // Your table has been created!
    console.log(resp);
});

list

dynasty.list() ⇒ promise
dynasty.list(<table name>) ⇒ promise
dynasty.list({limit: <limit>}) ⇒ promise
dynasty.list({start: <table name>, limit: <limit>}) ⇒ promise
dynasty.list({start: <table name>, limit: <limit>}, callback) ⇒ promise
dynasty.list(callback) ⇒ promise

List tables

Simplest case, just call it.

dynasty.list()
    .then(function(resp) {
        // List tables
        console.log(resp.TableNames);
    });

Optionally specify the name of a table to start the list. This is useful for paging.

If you had previously done a list() command and there were more tables than a response can handle, you would have received a LastEvaluatedTableName with the response which was the last table it could return. Pass this back in a subsequent request to start the list where you left off.

dynasty.list('Lands')
    .then(function(resp) {
        // List tables
        console.log(resp.TableNames);
    });

Optionally specify a limit which is the max number of table names to return.

Useful for paging.

dynasty.list({limit: 3})
    .then(function(resp) {
        // List 3 tables
        console.log(resp.TableNames);

        // Name of Last Table Returned, to be used if following up with another
        // request so you can start where you left off.
        console.log(resp.LastEvaluatedTableName);
    });

Optionally specify a callback function with or without other arguments.

dynasty.list(function(err, resp) {
    if (err) // Something went wrong!
    
    // List tables
    console.log(resp.TableNames);
});

Item

item overview

These methods act on Items within a Table.

As such, for all the following examples, we’ll assume:

  1. That the setup has been, so there’s a dynasty object with the credentials
  2. That the tables we’re accessing have already been created, unless otherwise specified
  3. That there is a table object already created for each table

In other words, we’re going to assume the following code has been run already:

var credentials = {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
};

var dynasty = require('Dynasty')(credentials);

// A table of countries with a hash key of *name*
var lands = dynasty.table('Lands');

// A table of counties with a hash key of *land* and a range key of *name*
var counties = dynasty.table('Counties');

We will use lands and counties as examples below.

This top-level overview is to help cut down on the repetition below.

batchFind

states.batchFind([<hashkey>, <hashkey>]) ⇒ promise
counties.batchFind([{hash: <hashkey>, range: <rangekey>}, {hash: <hashkey>, range: <rangekey>}, ... ]) ⇒ promise

Find multiple items in batch via a single call. That is find all the items in a table associated with a hash key.

Promise resolves with an array including all items matching your search.

// Simplest case, table with only hash keys:
var promise = states.batchFind(['Virginia', 'Maryland', 'New York']);

promise.then(function(states) {
    states.forEach(function(state) {
        console.log(state.population);
    });
});

// More complex case, table with hash and range keys
counties
    .batchFind([{hash: 'Ireland', range: 'Cork'}, {hash: 'Ireland', range: 'Galway'}])
    .then(function(result) {
        result.forEach(function(county) {
            console.log(county);
        });
    });

find

lands.find(<hashkey>) ⇒ promise
lands.find({hash: <hashkey>}) ⇒ promise
lands.find(<hashkey>, <callback>) ⇒ promise
counties.find({hash: <hashkey>, range: <rangekey>}) ⇒ promise

Find items in a table.

Simplest case, just supply a hash key, everything else defaults.

lands
    .find('France')
    .then(function(land) {
        console.log(land);
    });

We’re incredibly flexible here though, so we can take all manners of input.

A series of examples below:

// an object with a key of hash which represents the hash key
var promise = lands.find({hash: 'France'})

// an object with a key of hash and a key of range which represents the hash key
// and range key, for getting items from tables with a hash and range type
// primary key
var promise = counties.find({hash: 'Ireland', range: 'Cork'})

// an object with a string hash key and a callback function
lands.find('China', function(err, result) {
    console.log(result);
});

// and so on...

findAll

counties.findAll(<hashkey>) ⇒ promise

Find items in a table with a range key via the hash key. That is find all the items in a table associated with a hash key.

counties
    .findAll('Virginia')
    .then(function(counties) {
        counties.forEach(console.log);
    });

insert

lands.insert(<object>) ⇒ promise

Insert an item into a table.

Simplest case, just supply the item, everything else defaults.

lands
    .insert({name: 'China'})
    .then(function(resp) {
        console.log(resp);
    });

remove

lands.remove(<hashkey>) ⇒ promise
lands.remove({hash: <hashkey>}) ⇒ promise
lands.remove(<hashkey>, <callback>) ⇒ promise
counties.remove({hash: <hashkey>, range: <rangekey>}) ⇒ promise

Remove items.

Same flexible format as find() above, can take any series of: string/object, object/callback, callback

Simplest case, just supply a hash key, everything else defaults.

lands
    .remove('Russia')
    .then(function(land) {
        console.log(land.name + ' has been deleted');
    });

scan

lands.scan() ⇒ promise
lands.scan({ExclusiveStartKey: <startkey>}) = promise

Scans and returns all items in a table as an array of objects.

You can optionally pass in the LastEvaluatedKey from a previously executed scan operation as the ExclusiveStartKey to implement paginated scans (when you have more than 1 MB of date and can’t get it all in one go).

lands
    .scan()
    .then(function(allLands) {
        // Iterate through allLands here and do stuff with it.
    });

update

lands.update(<object>) ⇒ promise

Update an item in a table.

Simplest case, just supply the item, everything else defaults.

lands
    .update({name: 'China'})
    .then(function(resp) {
        console.log(resp);
    });

Change Log

Acknowledgements & Thanks

Big thanks to Mike Atkins who has been a huge contributor to this codebase!

This documentation is based on the layout of the Zepto.js which itself is based on the layout of the Backbone.js documentation, which is released under the MIT license.

This library is built on top of @teleportd's node-dynamodb library.

And of course a monster thank you to all of the Contributors to this project!

© 2013 Victor Quinn
Like this? Also check out Chance
Endorse victorquinn on Coderwall
Dynasty and this documentation are released under the terms of the MIT license.