Skip to main content

Runtime - AWS Lambda

Now that you have a understanding of the runtime, lets continue with the use case example. At the end of the tutorial setup, you should have had a config file that contains all your configuration values, feature flags, and business rules.

Let's now clone the github repo for the use case example.

Prerequisites​

This tutorial integrates a number of AWS services. You will need to have an AWS account and have some understanding of the following services:

  • AWS Amplify
  • AWS Cognito
  • AWS Lambda
  • AWS API Gateway
  • AWS DynamoDB
  • AWS S3
  • AWS Secrets Manager
  • AWS Systems Manager Parameter Store
  • AWS CloudFormation
  • AWS CDK

If you are not familiar with any of these services, do not worry. We will be guiding you step by step, and you can always refer to the AWS documentation for more information. This will be a fun and exciting journey, so let's get started!

Sample Project​

git clone https://github.com/flexicious/aws-node-rules-engine-lambda-config

This repo contains a sample project that uses the Lambda Genie runtime. The repo contains 2 folders:

API Project​

  • api - This folder contains a CDK project that deploys the following resources:
    • A Cognito User Pool
    • A Cognito User Pool Client
    • A Lambda Function that uses the Lambda Genie runtime to access the configuration values, feature flags, and business rules.
    • An API Gateway that exposes the Lambda Function as an API endpoint, and uses Cognito User Pool for authentication.
    • A DynamoDB table that demonstrates how to use the runtime to load configuration values from DynamoDB.
    • A Parameter Store value that demonstrates how to use the runtime to load configuration values from Parameter Store.
    • A Secrets Manager secret that demonstrates how to use the runtime to load configuration values from Secrets Manager.
    • IAM Roles and Policies that allow the Lambda Function to access the Cognito User Pool, DynamoDB, Parameter Store, and Secrets Manager, as well as the S3 bucket that contains the Lambda Genie configuration file.
AWS Resources

The DynamoDB, Parameter Store and Secret Manager Resources are not used specifically for this use case example. They are there to demonstrate how to use the runtime to load configuration values from these services. We will use them in a followup tutorial.

UI Project​

  • ui - This folder contains a React project that demonstrates how to use the API endpoint to access the configuration values, feature flags, and business rules. It contains:
    • A login page that uses the Cognito User Pool to authenticate the user. This page uses the Auth module from the Amplify library to authenticate the user.
    • A page that demonstrates how to use the API endpoint to access the configuration values, feature flags, and business rules. This page uses the API module from the Amplify library to call the API endpoint.

Deploy the Config File​

In the setup step, you downloaded the config file for your use case example. You will need to upload this file to a location that you lambda can read from, in this case, we are using AWS S3. Go ahead and create a bucket in S3 and upload the config file to the bucket. You will need the bucket name and the key of the config file in the next step.

Update the bucket name and key​

In the API project, open the file /api/lambda/utils/config-utils.ts and update the following values:

const bucketParams = {
    Bucket: "lambda-accelerator-rules", // Replace with your bucket name
    Key: "config.json", // Replace with your config file name
};

Lambda Permissions to read from S3​

In the API project, open the file //api/lib/cdk-stack.ts and update the following values:

const bucket = s3.Bucket.fromBucketName(this, 
    "TestBucket", "lambda-accelerator-rules"); // Replace with your bucket name
    bucket.grantRead(configHandler);
    const getProductsHandler = new NodejsFunction(this, "GetProductsHandler", {
      entry: "lambda/getProducts.ts",
      handler: "handler",
      logRetention: cdk.aws_logs.RetentionDays.ONE_DAY,
    });

Deployment​

  1. Deploy the API project
cd api
npm run watch
  1. Gather output values from the API project Once the API project is deployed, you will need to gather the following from the output:
  • DemoStack.ProductsApiUrl
  • DemoStack.UserPoolId
  • DemoStack.UserPoolClientId
  1. Configure awsExports.js in the UI project Once the API project is deployed, you will need to configure the UI project to use the API. In the UI project, open the file src/aws-exports.js and update the following values:
  • userPoolId - Replace the value with the Cognito User Pool ID from the API project output from step 2
  • userPoolWebClientId - Replace the value with the Cognito User Pool Client ID from the API project output from step 2
  1. Create a proxy configuration file In the UI project, create a file called src/setupProxy.js and add the following content:
    const { createProxyMiddleware } = require('http-proxy-middleware');

    module.exports = function(app) {
    app.use(
        '/api',
        createProxyMiddleware({
        target: 'https://<API Gateway ID>.execute-api.<AWS Region>.amazonaws.com/<Stage Name>',
        changeOrigin: true,
        secure: false,
            pathRewrite: {
                '^/api': ''
            }
        })
    );
    };

Replace the target value with the API Gateway URL from the API project output from step 2

  1. Deploy the UI project
cd ui
npm start
  1. Open the UI project

Open a browser and navigate to http://localhost:3000. You should see the login page and a create account page. Since we need to test multiple users, please create multiple accounts with different email addresses, genders, and ages.

Cognito will prompt you to confirm the email, but since these are not real email addresses, you can just go into cognito, and manually confirm the email addresses.

  1. Login to the UI project

Once you have created the accounts, login to the UI project using the credentials for the account you created. You should be able to see the configuration values, feature flags, and business rules for the user.

Final Product

Using Lambda Genie in your project​

Now that you have deployed the sample project, let's understand how to use Lambda Genie in your project. To start using Lambda Genie in your project, you need to install the Lambda Genie runtime. The runtime is a Node.js module that you can install in your project. The runtime can take your Lambda Genie configuration, and expose api methods that you can use in your code to access the configuration values, feature flags, and execute business rules against your entities. Before we dive into the use case examples, let's first understand how the runtime works.

Installation​

To install the Lambda Genie runtime, run the following command in your project:

If you are using this in a AWS Lambda project, you can install the runtime using npm:

npm install @euxdt/node-rules-engine  @euxdt/lambda-config

Usage​

To use the Lambda Genie runtime, you need to import the module and initialize it with your Lambda Genie configuration. The Lambda Genie configuration is a JSON object that contains the configuration for your Lambda Genie project. This is the config file you downloaded in the previous step. The runtime does not require that you use a specific mechanism to load the configuration. You can load the configuration from S3, Parameter Store, Dynamo, or any other mechanism you prefer. In the below example, we are loading the configuration from S3.

import { GetObjectCommand, HeadObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { ConfigJson } from "@euxdt/node-rules-engine";
import { Readable } from "stream";
import { getLambdaConfigApi, streamToString,LambdaConfigApi } from "@euxdt/lambda-config";
const s3Client = new S3Client({  });
const bucketParams = {
    Bucket: "lambda-accelerator-rules", // Replace with your bucket name
    Key: "config.json", // Replace with your config file name
};
//This is where we load the config rules from S3. 
//The config rules are stored in a JSON file in S3.
//we check to see if the config rules have been updated since the last time we loaded them.
//If they have, we load the new ones.
//If they haven't, we use the cached version. 
//Config rules are published from the lambda genie console. 

export const loadConfigApi = async (lambdaName:string):Promise<LambdaConfigApi> => {
    const configApi = await getLambdaConfigApi(
        {
            lambdaName,
            loadConfig: async (lastRefreshed?:Date, existingConfig?:ConfigJson) => {
                if(lastRefreshed && existingConfig){
                    const head = await s3Client.send(new HeadObjectCommand(bucketParams));
                    if(lastRefreshed && head.LastModified && lastRefreshed.getTime() >= head.LastModified.getTime()){
                        console.log("Config is up to date");
                        return existingConfig;  
                    }
                }
                console.log("Loading Config");
                const file = await s3Client.send(new GetObjectCommand(bucketParams));
                const body = await streamToString(file.Body as Readable);
                console.log("Config Json",  body);
                const configJson = JSON.parse(body);
                console.log("Config Json", { configJson });
                return configJson as unknown as ConfigJson;
            },
            log: (level, message, extra) => {
                console.log(level, message, extra);
            }
        }
    );
    return configApi;
};

The loadConfig function is called the first time you call the loadConfigApi function. The loadConfig function takes the last time the config was refreshed and the existing config. If the config file has not changed since the last time it was refreshed, you can return the existing config. This will prevent unnecessary calls to the config file. The getConfigApi takes a parameter object of type LambdaConfigApiOptions:

export interface LambdaConfigApiOptions {
    lambdaName: string;
    loadConfig : (lastRefreshed?:Date, lastConfig?:ConfigJson) => Promise<ConfigJson>;
    log?: (level:LogLevel, message:string, extra:any)=>void;
    cacheDurationSeconds?: number;
    disableEnvironmentVariables?: boolean;
    throwOnFailure?: boolean;
}

The lambdaName is the name of the lambda function. This is used to identify the lambda function in the config file. The loadConfig function is the function that loads the config file. The log function is used to log messages. The cacheDurationSeconds is the duration in seconds to cache the config file. The disableEnvironmentVariables is a boolean that indicates whether to disable environment variables. By default, lambda genie will store the values of config variables in environment variables of the same name. The throwOnFailure is a boolean that indicates whether to throw an error if the config file is not found, or in case of AWS specific configurations, if the underlying AWS service throws an error. (DynamoDB, S3, Parameter Store, Secrets Manager, etc.)

API​

The Lambda Genie runtime exposes the following methods:

export interface LambdaConfigApi {
    getConfigValue(key: string, environment:string): Promise<string>;
    getAllConfigValues(environment:string): Promise<Record<string, string>>;
    lastRefreshed: Date;
    configJson: ConfigJson;
}

getConfigValue​

The getConfigValue method takes a key and an environment, and returns the value for the key in the specified environment. The key is the name of the configuration value in the config file. The environment is the name of the environment in the config file. The environment can be dev, qa, prod, or any other environment you have defined in the config file. The getConfigValue method returns a promise that resolves to the value of the configuration value.

getAllConfigValues​

The getAllConfigValues method takes an environment, and returns all the configuration values for the specified environment. The environment is the name of the environment in the config file. The environment can be dev, qa, prod, or any other environment you have defined in the config file. The getAllConfigValues method returns a promise that resolves to a record of all the configuration values for the specified environment. This is useful if you want to load all the configuration values into a single object in parallel. For example, if you have a configuration value for the database username and password that are stored in Secrets Manager, Host and Port that are stored in Parameter Store, you can load all the configuration values in parallel using the getAllConfigValues method.

lastRefreshed​

The lastRefreshed property is a date object that indicates the last time the config file was refreshed.

configJson​

The configJson property is the config file as a JSON object.

Using Lambda Genie in your Lambda​

Once you have initialized the Lambda Genie runtime, you can use it in your Lambda function. The following is an example of a Lambda function that uses Lambda Genie to load dynamic configuration, and execute rules.



export const handler = async (event:APIGatewayEvent) => {
    //Ensure the user is logged in
    const user = getUserInformation(event);
    if (!user) {
        return {
            statusCode: 401,
            body: "Unauthorized"
        };
    }

    //This would come from environment variables, but for the sake of the demo, we'll hard code it
    const environment = "dev";

    //Load the config rules
    const configApi = await loadConfigApi("GET_PRODUCTS");

    //Get the slot names from the dynamic config we defined in the lambda genie console
    const slotNames = JSON.parse(await configApi.getConfigValue(CONFIG.LAMBDA_CONFIGS.GET_PRODUCTS.SLOT_NAMES,environment)) as SlotNames;

    //Default the slot names to some values if they aren't defined in the dynamic config
    const slot1 = [slotNames.slot1] ||["Home & Kitchen"];
    const slot2 = [slotNames.slot2] ||  [ "Clothing, Shoes & Jewelry"];
    const slot3 = [slotNames.slot3] || ["Learning & Education"];
    const slot4 = [slotNames.slot4] ||  ["Hobbies"];
    const maxProducts = slotNames.maxProducts ||  4;

    //Get the featured products from the dynamic config we defined in the lambda genie console
    const featuredProducts = JSON.parse(await configApi.getConfigValue(CONFIG.LAMBDA_CONFIGS.GET_PRODUCTS.FEATURED_PRODUCTS,environment)) as FeaturedProduct[];

    //Get the slot 2 rules from the rule set we defined in the lambda genie console, which allow 
    //us to dynamically change the slot 2 category based on the user's age and gender 
    const slot2Rules= configApi.configJson.ruleSets.find((ruleSet) => ruleSet.name === CONFIG.RULE_SETS.HOME_PAGE_PERSONALIZATION);
    const userInfo= {
        ...user,
        age: user.birthdate ? Math.floor((new Date().getFullYear() - new Date(user.birthdate).getFullYear())) : 25
    };
    if(slot2Rules){
        //Execute the rules and get the result
        const slot2RuleResult = executeRule(slot2Rules,
        configApi.configJson.predefinedLists, userInfo,environment);
        if(slot2RuleResult && slot2RuleResult.result){
            //this will be one of Men, Women, Boys, or Girls [As defined in the rule set in the lambda genie console]
            slot2.push(String(slot2RuleResult.result));
        }
    }
    //Now, lets get the next gen feature flag from the rule set we defined in the lambda genie console
    let nextGenFeature = false;
    const featureFlagRules= configApi.configJson.ruleSets.find((ruleSet) => ruleSet.name === CONFIG.RULE_SETS.NEXT_GEN_FEATURE);
    if(featureFlagRules){
        //Execute the rules and get the result
        const featureFlagRuleResult = executeRule(featureFlagRules,
        configApi.configJson.predefinedLists, userInfo,environment);
        nextGenFeature = featureFlagRuleResult.result ? true : false;
        console.log("featureFlagRuleResult", featureFlagRuleResult);
    }                
    //For now, we are just loading the products from a local file, but this could be replaced with a call to a database or an API
    const getProducts = (slot: string[]) => {
        return PRODUCTS.filter((product) => {
            return slot.every((category) => {
                return product.categories.includes(category);
            });
        }).slice(0, maxProducts);
    };

    //Get the products for each slot
    const slot1Products = getProducts(slot1);
    const slot2Products = getProducts(slot2);
    const slot3Products = getProducts(slot3);
    const slot4Products = getProducts(slot4);

    //Return the full result
    return {
        statusCode: 200,
        body: JSON.stringify({
            slot1: slot1[0],
            slot2: slot2[0],
            slot3: slot3[0],
            slot4: slot4[0],
            slot1Products,
            slot2Products,
            slot3Products,
            slot4Products,
            featuredProducts,
            nextGenFeature
        }),
    };
};

As you can see, the Lambda function uses the getConfigValue method to get the slot names and featured products from the dynamic config. It also uses the executeRule method to execute the rules in the rule set to get the slot 2 category and the next gen feature flag. The executeRule method takes a rule set, predefined lists, and user information, and returns the result of the rule set.

Word Wrap

There is a word wrap button in the top right corner of the code blocks. If you are having trouble reading the code, click the word wrap button to make the code easier to read.