by

Read Data from DynamoDB using AWS Lambda function

In this tutorial, you will learn how to read data from Dynamo DB using the AWS Lambda function. This tutorial mainly talks about scan and query operations on a DynamoDB table.

First, let’s have a quick overview of the DynamoDB table.

DynamoDB Table

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. DynamoDB lets you create database tables that can store and retrieve any amount of data and serve any level of request traffic. With DynamoDB, tables’ throughput capacity can be scaled up or down without downtime or performance degradation.

Now let’s create a DynamoDB table for our example.

  • First, log in to your AWS account and type DynamoDB in the search bar.
  • Then click on Create table.
  • Next, provide the Table name, Partition key, and Sort key (optional).
  • Under Settings, select Customize settings.
  • Under Read/write capacity settings, select Provisioned.
  • Turn off AutoScaling for Read Capacity and Write Capacity and set Provisioned capacity units to 1.
  • Leave the remaining settings to default and click on Create table.

The following image shows a table Journal that contains data from a Publishing company that has various research journals. Every journal has different research papers.

 

Working with Scans

A Scan operation in Amazon DynamoDB reads every item in a table or a secondary index. A Scan operation returns all of the data attributes for every item in the table or index by default. Note that the ProjectionExpression parameter can be used so that Scan only returns some of the attributes, rather than all of them. When you do a Scan operation, it always returns a result set. The result set is empty if no matching items are found.

Let’s learn to scan through an example. Now we will create a lambda function and perform a scan operation on the DynamoDB table. We will also create an API and call the lambda function through API Gateway. Note that the lambda function should have DynamoDB access along with the lambda basic execution role.

Scan Example

Let’s suppose you want to get data of all the research papers from every journal of the Publishing company. This is how you are going to implement the aforementioned case.

  • In the Handler class, create a method that initializes the DynamoDB client.
  • In the constructor, set the region where the DynamoDB table resides and call the above-mentioned method. This way every time the constructor is called, the DynamoDB client will automatically be initiated.
  • In the handleRequest method, call the service that returns research papers’ data. We will talk about this service later in the tutorial.
  • Finally return the data that the service provides.

Handler.java

import java.util.List;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

public class Handler implements RequestHandler<Object, Object> {

    DynamoDB dynamoDb;
    Regions dynamoDbRegion;


    public Handler() {
        
        super();
        this.dynamoDbRegion=Regions.US_EAST_1;
        initDynamoDbClient();
        
    }


    public Object handleRequest(Object input, Context context) {     
        List<Response> responseList = JournalService.getPapers(dynamoDb);
        return responseList;
    }

    private void initDynamoDbClient() {
        AmazonDynamoDBClient client = new AmazonDynamoDBClient();
        client.setRegion(Region.getRegion(dynamoDbRegion));
        dynamoDb = new DynamoDB(client);
    }

}

JournalService.java

Here comes the main part which is the implementation of the scan operation.

  • In the getPapers() method, first, create a table object by getting table Journal from the DynamoDB object.
  • Then create a ScanSpec object with Projection Expression. ProjectionExpression specifies the attributes you want in the scan result. Here it is set to author and title of the research paper.
  • Pass scanSpec object to the scan method.
  • scan method reads every item in the entire Journal table and returns all the data in the table. You can provide an optional filter_expression so that only the items matching your criteria are returned. However, the filter is applied only after the entire table has been scanned.
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.ScanOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.ScanSpec;

public class JournalService {
    
    static String tableName = "Journal";
    static String projectionExpression = "author,title";

    public static List<Response> getPapers(DynamoDB dynamoDb) {

        Table table = dynamoDb.getTable(tableName);
        ScanSpec scanSpec = new ScanSpec().withProjectionExpression(projectionExpression);

        List<Response> responseList = new ArrayList<>();

        try {
            ItemCollection<ScanOutcome> items = table.scan(scanSpec);

            Iterator<Item> iter = items.iterator();
            while (iter.hasNext()) {
                Item item = iter.next();
                Response res = new Response();
                
                res.setAuthor(item.getString("author"));
                res.setTitle(item.getString("title"));
                responseList.add(res);
            }

        } catch (Exception e) {
            System.err.println("Unable to scan the table:");
            System.err.println(e.getMessage());
        }

        return responseList;
    }
    
    

}

Create API

Now let’s create an API using AWS API Gateway.

  • First, log in to your AWS account and type API Gateway in the search bar.
  • After that, click on Create API.
  • Then click on Build under Rest API.
  • Provide API name and description and leave the rest of the setting to default. Then click on Create API.
  • In the Actions dropdown, click on Create Resource.
  • Set the resource name to papers and click on Create Resource.
  • In the Actions dropdown, click on Create Method for /papers.
  • Then select GET and tick the checkmark.
  • Now select the lambda function for the API and click on Save.
  • In the Actions dropdown, click on Deploy API.
  • Create Deployment stage and click on Deploy.

 

 

Testing

Let’s test our Lambda function that scans data from the DynamoDB table. After selecting the GET method under /papers, click on Test. Now you can read Data from Dynamo DB using AWS Lambda function by performing scan operation.

 

Working with Queries

The Query operation in Amazon DynamoDB finds items based on primary key values. You must provide the name of the partition key attribute and a single value for that attribute. When you perform query operation on a DynamoDB table, it returns all items with that partition key value. Apart from that you can also provide a sort key attribute and use a comparison operator to refine the search results but that is optional.

Let’s learn to query through an example. Now we will create a lambda function and perform a query operation on the DynamoDB table. We will also create an API and call the lambda function through API Gateway.

Query Example

Let’s suppose you want to get data of all the research papers but now of a particular journal. This is how you are going to implement the aforementioned case.

  • In the Handler class, create a method that initializes the DynamoDB client.
  • In the constructor, set the region where the DynamoDB table resides and call the above-mentioned method. This way every time the constructor is called, the DynamoDB client will automatically be initiated.
  • In the handleRequest method, call the service that returns research papers’ data. Pass the journalID along with the DynamoDB object while calling the service. We will talk about this service later in the tutorial.
  • Finally return the data that the service provides.

Handler.java

import java.util.List;
import java.util.Map;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

public class Handler implements RequestHandler<Map<String,String>, Object> {

    DynamoDB dynamoDb;
    Regions dynamoDbRegion;


    public Handler() {
        super();
        this.dynamoDbRegion=Regions.US_EAST_1;
        initDynamoDbClient();
    }


    public Object handleRequest(Map<String,String> req, Context context) {
        String journalId = req.get("journalid");
        List<Response> responseList = JournalService.getPapers(dynamoDb,journalId);
        return responseList;
    }

    private void initDynamoDbClient() {
        AmazonDynamoDBClient client = new AmazonDynamoDBClient();
        client.setRegion(Region.getRegion(dynamoDbRegion));
        dynamoDb = new DynamoDB(client);
    }

}

JournalService.java

Now let’s look at the JournalService that uses the query method.

  • In the getPapers method, first, create a table object by getting table Journal from the DynamoDB object.
  • Then create the querySpec object, which describes the query parameters. Later, pass this object to the query method.
  • KeyConditions are the selection criteria for a Query operation. For a query on a table, you can have conditions only on the table’s primary key attributes.
  • Here valueMap provides value substitution to define the actual values for the KeyConditionExpression placeholders.
  • ProjectionExpression specifies the attributes you want in the query result. Here it is set to author and title.
  • The query method returns data based on the journalID provided.
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec;
import com.amazonaws.services.dynamodbv2.document.utils.ValueMap;

public class JournalService {
    
    static String tableName = "Journal";
    static String projectionExpression = "author,title";
    static String keyConditionExpression = "journal_id = :journalId";

    public static List<Response> getPapers(DynamoDB dynamoDb, String journalId) {

        Table table = dynamoDb.getTable(tableName);
        QuerySpec querySpec = new QuerySpec().withKeyConditionExpression(keyConditionExpression)
                .withValueMap(new ValueMap().withString(":journalId", journalId))
                .withProjectionExpression(projectionExpression);

        List<Response> responseList = new ArrayList<>();

        try {
            ItemCollection<QueryOutcome> items = table.query(querySpec);

            Iterator<Item> iter = items.iterator();
            while (iter.hasNext()) {
                Item item = iter.next();
                Response res = new Response();
                res.setAuthor(item.getString("author"));
                res.setTitle(item.getString("title"));
                responseList.add(res);
            }

        } catch (Exception e) {
            System.err.println("Unable to scan the table:");
            System.err.println(e.getMessage());
        }

        return responseList;
    }
    
    

}

Create API

Now let’s get back to the API we created earlier.

  • Select /papers and click on Create Resource.
  • Set the resource name to journal-id. In the Resource Path, add braces to journal-id i.e {journal-id} and click on Create Resource.
  • In the Actions dropdown, click on Create Method for /{journal-id}.
  • Then select GET and tick the checkmark.
  • Now select the lambda function for the API and click on Save.
  • Click on Integration Request then expand Mapping Templates.
  • Select When there are no templates defined, then click on Add mapping template.
  • Define Content-Type to be application/json.
  • Select Empty in Generate template and get journalid from the path parameter.
  • In the Actions dropdown, click on Deploy API.
  • Create Deployment stage and click on Deploy.

 

Testing

Let’s test our Lambda function that queries data from the DynamoDB table. After selecting the GET method under /{journal-id}, click on Test. Now you can read Data from Dynamo DB using AWS Lambda function by performing query operation.

 

Conclusion

With this, we have come to the end of our tutorial. In this tutorial, we learn to perform query and scan operations and read data from Dynamo DB using the AWS Lambda function.

Stay tuned for some more informative tutorials coming ahead.

Happy learning!


Related Posts:

Powered by Contextual Related Posts