Building an AWS Trading System - Basic DynamoDB Operations (Part 6) (Python Tutorial)

Dec 17, 2022

∙ Paid

Title image created with Cacoo.com

This post is part of a series of tutorials with the subject of building a scalable algorithmic trading system on the Amazon Web Services (AWS) cloud platform. Bear with me, as I’m learning AWS myself.

The advantage of using AWS is that it offers dozens of powerful, integrated software services that can be used to build a scalable, high-performance, low-cost production trading system.

In this post we are going to over basic operations on how to use the AWS DynamoDB No-SQL database, how to create a table, load news articles, and query for the news articles.

This story is solely for general information purposes, and should not be relied upon for trading recommendations or financial advice. Source code and information is provided for educational purposes only, and should not be relied upon to make an investment decision. Please review my full cautionary guidance before continuing.

Previous Tutorials

The following posts are suggested reading for this post to get you started:

Trade Ideas provides AI stock suggestions, AI alerts, scanning, automated trading, real-time stock market data, charting, educational resources, and more. Get a 15% discount with promo code ‘BOTRADING15’.

What is AWS DynamoDB?

AWS DynamoDB is Amazon’s serverless, scalable No-SQL database. This feature-rich service come with built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools.

This type database can store less structured information such as news articles or social media posts in JSON format, but you can also store price information in DynamoDB.

Instead of using the SQL (Structured Query Language) for querying for items in a table, this type of database uses keys and attributes to access to document records stored in a table.

Learn more about AWS DynamoDB here.

Structuring DynamoDB data

One major difference between DynamoDB and a relational database such as MySQL for example is that DynamoDB data cannot be structured in database silos. All data is stored in one ‘database’ under one AWS account but in different tables. Since Table names have to be unique in each region, it is advisable to take a minute and think about a naming convention to organize your data.

For example, one could structure news articles in tables with the following naming convention:

news_<country_code>_<news source>_<news category>

For example: e.g. news_us_cnn_business_news.

For price information, we could use the follow this naming convention:

prices_<data provider>_<instrument>_<interval>_<symbol>,

For example: prices_eod_stocks_15m_tlsa.

Using a naming convention for tables like this avoids the issue of name collision as you add more and more data to your trading system.

DynamoDB Partitioning & Sorting

Partitioning is a database practice to transparently spread data across multiple hosts for performance, scalability and availability reasons.

The DynamoDB partition key is used to spread data across partitions for scalability. It’s important to choose an attribute with a wide range of values and that is likely to have evenly distributed access patterns.

A sort key can be specified to be used to sort the data in the table that shares the same partition key. The sort key will become part of the primary table key.

Tutorial Outline

In this tutorial we will perform the following operations related to DynamoDB:

Create a table
Describe table
Load data with business news
Query all items in a table
Query a single item from the table
Delete the table.

B/O Trading Blog