Merge pull request #395 from dmitchsplunk/main

Add code for lambda workshop and update instructions

Author: dmitchsplunk

content/en/ninja-workshops/6-lambda-kinesis/1-setup.md

@@ -9,7 +9,8 @@ weight: 1
 ## Prerequisites
 
 ### Observability Workshop Instance
-The Observability Workshop is most often completed on a Splunk-issued and preconfigured EC2 instance running Ubuntu.
+The Observability Workshop uses the `Splunk4Ninjas - Observability` workshop template in Splunk Show, 
+which provides a pre-configured EC2 instance running Ubuntu. 
 
 Your workshop instructor will provide you with the credentials to your assigned workshop instance.
 
@@ -66,19 +67,14 @@ Your Splunk-issued workshop instance should already have **terraform** installed
   sudo apt update && sudo apt install terraform
   ```
 
-### Workshop Directory (o11y-lambda-workshop)
-The Workshop Directory `o11y-lambda-workshop` is a repository that contains all the configuration files and scripts to complete both the auto-instrumentation and manual instrumentation of the example Lambda-based application we will be using today.
+### Workshop Directory (lambda)
+The Workshop Directory `lambda` is a repository that contains all the configuration files and scripts to complete both the auto-instrumentation and manual instrumentation of the example Lambda-based application we will be using today.
 
 - Confirm you have the workshop directory in your home directory:
   ```bash
-  cd && ls
-  ```
-    - _The expected output would include **o11y-lambda-workshop**_
-
-- If the **o11y-lambda-workshop** directory is not in your home directory, clone it with the following command:
-  ```bash
-  git clone https://github.com/gkono-splunk/o11y-lambda-workshop.git
+  cd ~/workshop && ls
   ```
+    - _The expected output would include **lambda**_
 
 ### AWS & Terraform Variables
 
@@ -92,7 +88,7 @@ The AWS CLI requires that you have credentials to be able to access and manage r
     - _This command should provide a prompt similar to the one below:_
       ```bash
       AWS Access Key ID [None]: XXXXXXXXXXXXXXXX
-      AWS Secret Acces Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+      AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
       Default region name [None]: us-east-1
       Default outoput format [None]:
       ```
@@ -102,6 +98,11 @@ The AWS CLI requires that you have credentials to be able to access and manage r
   aws configure
   ```
 
+> Note to the workshop instructor:  create a new user in the target AWS account called `lambda-workshop-user`. 
+> Ensure it has full permissions to perform the required actions via Terraform.  Create an access token for the `lambda-workshop-user`
+> user and share the Access Key ID and Secret Access Key with the workshop participants.  Delete the user 
+> when the workshop is complete. 
+
 #### Terraform
 Terraform supports the passing of variables to ensure sensitive or dynamic data is not hard-coded in your .tf configuration files, as well as to make those values reusable throughout your resource definitions.
 
@@ -118,7 +119,7 @@ We will be using a combination of _**variables.tf**_ and _**terraform.tfvars**_
 
 - Using either **vi** or **nano**, open the _**terraform.tfvars**_ file in either the **auto** or **manual** directory
   ```bash
-  vi ~/o11y-lambda-workshop/auto/terraform.tfvars
+  vi ~/workshop/lambda/auto/terraform.tfvars
   ```
 - Set the variables with their values. Replace the **CHANGEME** placeholders with those provided by your instructor.
   ```bash
@@ -128,22 +129,23 @@ We will be using a combination of _**variables.tf**_ and _**terraform.tfvars**_
   prefix            = "CHANGEME"
   ```
   - _Ensure you change only the placeholders, leaving the quotes and brackets intact, where applicable._
+  - _For the **otel_lambda_layer**, use the value for **us-east-1** found [here](https://github.com/signalfx/lambda-layer-versions/blob/main/splunk-apm/splunk-apm.md)
   - _The _**prefix**_ is a unique identifier you can choose for yourself, to make your resources distinct from other participants' resources. We suggest using a short form of your name, for example._
-  - _Also, please only lowercase letters for the **prefix**. Certain resouces in AWS, such as S3, would through an error if you use uppercase letters._
+  - _Also, please only lowercase letters for the **prefix**. Certain resources in AWS, such as S3, would through an error if you use uppercase letters._
 - Save your file and exit the editor.
 - Finally, copy the _**terraform.tfvars**_ file you just edited to the other directory.
   ```bash
-  cp ~/o11y-lambda-workshop/auto/terraform.tfvars ~/o11y-lambda-workshop/manual
+  cp ~/workshop/lambda/auto/terraform.tfvars ~/workshop/lambda/manual
   ```
   - _We do this as we will be using the same values for both the autoinstrumentation and manual instrumentation protions of the workshop_
  
 ### File Permissions
 
 While all other files are fine as they are, the **send_message.py** script in both the `auto` and `manual` will have to be executed as part of our workshop. As a result, it needs to have the appropriate permissions to run as expected. Follow these instructions to set them.
 
-- First, ensure you are in the `o11y-lambda-workshop` directory:
+- First, ensure you are in the `lambda` directory:
   ```bash
-  cd ~/o11y-lambda-workshop
+  cd ~/workshop/lambda
   ```
 
 - Next, run the following command to set executable permissions on the `send_message.py` script:

content/en/ninja-workshops/6-lambda-kinesis/2-auto-instrumentation.md

@@ -7,12 +7,12 @@ weight: 2
 The first part of our workshop will demonstrate how auto-instrumentation with OpenTelemetry allows the OpenTelemetry Collector to auto-detect what language your function is written in, and start capturing traces for those functions.
 
 ### The Auto-Instrumentation Workshop Directory & Contents
-First, let us take a look at the `o11y-lambda-workshop/auto` directory, and some of its files. This is where all the content for the auto-instrumentation portion of our workshop resides.
+First, let us take a look at the `workshop/lambda/auto` directory, and some of its files. This is where all the content for the auto-instrumentation portion of our workshop resides.
 
 #### The `auto` Directory
-- Run the following command to get into the **o11y-lambda-workshop/auto** directory:
+- Run the following command to get into the **workshop/lambda/auto** directory:
   ```bash
-  cd ~/o11y-lambda-workshop/auto
+  cd ~/workshop/lambda/auto
   ```
 
 - Inspect the contents of this directory:
@@ -114,11 +114,11 @@ In order to deploy the resources defined in the `main.tf` file, you first need t
   ```bash
   pwd
   ```
-  - _The expected output would be **~/o11y-lambda-workshop/auto**_
+  - _The expected output would be **~/workshop/lambda/auto**_
 
 - If you are not in the `auto` directory, run the following command:
   ```bash
-  cd ~/o11y-lambda-workshop/auto
+  cd ~/workshop/lambda/auto
   ```
 
 - Run the following command to initialize Terraform in this directory
@@ -172,11 +172,11 @@ To start getting some traces from our deployed Lambda functions, we would need t
   ```bash
   pwd
   ```
-  - _The expected output would be **~/o11y-lambda-workshop/auto**_
+  - _The expected output would be **~/workshop/lambda/auto**_
 
 - If you are not in the `auto` directory, run the following command
   ```bash
-  cd ~/o11y-lambda-workshop/auto
+  cd ~/workshop/lambda/auto
   ```
 
 The `send_message.py` script is a Python script that will take input at the command line, add it to a JSON dictionary, and send it to your `producer-lambda` function's endpoint repeatedly, as part of a while loop.

content/en/ninja-workshops/6-lambda-kinesis/3-lambdas-in-splunk.md

@@ -86,12 +86,12 @@ Please follow these steps to destroy your resources:
   pwd
   ```
 
-  - _The expected output would be **~/o11y-lambda-workshop/auto**_
+  - _The expected output would be **~/workshop/lambda/auto**_
 
 - If you are not in the `auto` directory, run the following command:
 
   ```bash
-  cd ~/o11y-lambda-workshop/auto
+  cd ~/workshop/lambda/auto
   ```
 
 - Destroy the Lambda functions and other AWS resources you deployed earlier:

content/en/ninja-workshops/6-lambda-kinesis/4-manual-instrumentation.md

@@ -8,14 +8,14 @@ The second part of our workshop will focus on demonstrating how manual instrumen
 
 ### The Manual Instrumentation Workshop Directory & Contents
 
-Once again, we will first start by taking a look at our operating directory, and some of its files. This time, it will be `o11y-lambda-workshop/manual` directory. This is where all the content for the manual instrumentation portion of our workshop resides.
+Once again, we will first start by taking a look at our operating directory, and some of its files. This time, it will be `workshop/lambda/manual` directory. This is where all the content for the manual instrumentation portion of our workshop resides.
 
 #### The `manual` directory
 
-- Run the following command to get into the `o11y-lambda-workshop/manual` directory:
+- Run the following command to get into the `workshop/lambda/manual` directory:
 
   ```bash
-  cd ~/o11y-lambda-workshop/manual
+  cd ~/workshop/lambda/manual
   ```
 
 - Inspect the contents of this directory with the `ls` command:
@@ -42,23 +42,23 @@ Let's make sure that all these files that LOOK the same, are actually the same.
 - Compare the `main.tf` files in the `auto` and `manual` directories:
 
   ```bash
-  diff ~/o11y-lambda-workshop/auto/main.tf ~/o11y-lambda-workshop/manual/main.tf
+  diff ~/workshop/lambda/auto/main.tf ~/workshop/lambda/manual/main.tf
   ```
 
   - There is no difference! _(Well, there shouldn't be. Ask your workshop facilitator to assist you if there is)_
 
 - Now, let's compare the `producer.mjs` files:
 
   ```bash
-  diff ~/o11y-lambda-workshop/auto/handler/producer.mjs ~/o11y-lambda-workshop/manual/handler/producer.mjs
+  diff ~/workshop/lambda/auto/handler/producer.mjs ~/workshop/lambda/manual/handler/producer.mjs
   ```
 
   - There's quite a few differences here!
 
 - You may wish to view the entire file and examine its content
 
   ```bash
-  cat ~/o11y-lambda-workshop/handler/producer.mjs
+  cat ~/workshop/lambda/manual/handler/producer.mjs
   ```
 
   - Notice how we are now importing some OpenTelemetry objects directly into our function to handle some of the manual instrumentation tasks we require.
@@ -75,7 +75,7 @@ Let's make sure that all these files that LOOK the same, are actually the same.
 - Finally, compare the `consumer.mjs` files:
 
   ```bash
-  diff ~/o11y-lambda-workshop/auto/handler/consumer.mjs ~/o11y-lambda-workshop/manual/handler/consumer.mjs
+  diff ~/workshop/lambda/auto/handler/consumer.mjs ~/workshop/lambda/manual/handler/consumer.mjs
   ```
 
   - Here also, there are a few differences of note. Let's take a closer look

content/en/ninja-workshops/6-lambda-kinesis/5-redeploy-lambdas.md

@@ -16,12 +16,12 @@ Seeing as we're in a new directory, we will need to initialize Terraform here on
   pwd
   ```
 
-  - _The expected output would be **~/o11y-lambda-workshop/manual**_
+  - _The expected output would be **~/workshop/lambda/manual**_
 
 - If you are not in the `manual` directory, run the following command:
 
   ```bash
-  cd ~/o11y-lambda-workshop/manual
+  cd ~/workshop/lambda/manual
   ```
 
 - Run the following command to initialize Terraform in this directory
@@ -72,12 +72,12 @@ Once more, we will send our `name` and `superpower` as a message to our endpoint
   pwd
   ```
 
-  - _The expected output would be **~/o11y-lambda-workshop/manual**_
+  - _The expected output would be **~/workshop/lambda/manual**_
 
 - If you are not in the `manual` directory, run the following command:
 
   ```bash
-  cd ~/o11y-lambda-workshop/manual
+  cd ~/workshop/lambda/manual
   ```
 
 - Run the `send_message.py` script as a background process:

content/en/ninja-workshops/6-lambda-kinesis/6-updated-lambdas.md

@@ -93,12 +93,12 @@ Please follow these steps to destroy your resources:
   pwd
   ```
 
-  - _The expected output would be **~/o11y-lambda-workshop/manual**_
+  - _The expected output would be **~/workshop/lambda/manual**_
 
 - If you are not in the `manual` directory, run the following command:
 
   ```bash
-  cd ~/o11y-lambda-workshop/manual
+  cd ~/workshop/lambda/manual
   ```
 
 - Destroy the Lambda functions and other AWS resources you deployed earlier:

workshop/lambda/auto/.gitignore

@@ -0,0 +1 @@
+!terraform.tfvars

workshop/lambda/auto/handler/consumer.mjs

@@ -0,0 +1,23 @@
+// Lambda Consumer App for Kinesis Stream
+
+// Lambda Consumer
+const consumer = async( event ) => {
+  try{
+	for( const record of event.Records ) {
+      const payload = record.kinesis;
+      const message = Buffer.from(payload.data, 'base64').toString();
+
+      console.log(
+        `Kinesis Message:
+        partition key: ${payload.partitionKey}
+        sequence number: ${payload.sequenceNumber}
+        kinesis schema version: ${payload.kinesisSchemaVersion}
+        data: ${message}`
+      );
+    }
+  } catch ( error ) {
+    console.log(error);
+  }
+};
+
+export { consumer };

workshop/lambda/auto/handler/producer.mjs

@@ -0,0 +1,47 @@
+// Lambda Producer App for Kinesis Stream
+import { KinesisClient, PutRecordCommand } from "@aws-sdk/client-kinesis";
+
+
+// Lambda Producer
+const producer = async( event ) => {
+  const kinesis = new KinesisClient({});
+
+  let statusCode = 200;
+  let message;
+
+  if (!event.body) {
+    return {
+      statusCode: 400,
+      body: JSON.stringify({
+        message: "No message body found",
+      }),
+    };
+  }
+
+  const streamName = process.env.KINESIS_STREAM;
+
+  try {
+    await kinesis.send(
+      new PutRecordCommand({
+        StreamName: streamName,
+        PartitionKey: "1234",
+        Data: event.body,
+      }),
+
+      message = `Message placed in the Event Stream: ${streamName}`
+	)
+  } catch ( error ) {
+    console.log(error);
+    message = error;
+    statusCode = 500;
+  }
+
+  return {
+    statusCode,
+    body: JSON.stringify({
+      message,
+    }),
+  };
+};
+
+export { producer };