Tip
To deploy this project using GUI-based flow, navigate to console
- Real-time event streaming and analytics with Kinesis.
- Send events (pageviews, clicks) to a Kinesis stream, consume them in batches, and aggregate counts by type and hour in DynamoDB.
- Query aggregated analytics via the REST API.
- This project includes a pre-configured stacktape.yml configuration. The configured infrastructure is described in the stack description section
-
AWS account. If you don't have one, create new account here.
-
Stacktape account. If you don't have one, create new account here.
-
Stacktape installed.
Install on Windows (Powershell)
iwr https://installs.stacktape.com/windows.ps1 -useb | iexInstall on Linux
curl -L https://installs.stacktape.com/linux.sh | shInstall on MacOS
curl -L https://installs.stacktape.com/macos.sh | shInstall on MacOS ARM (Apple silicon)
curl -L https://installs.stacktape.com/macos-arm.sh | shTo initialize the project, use
stacktape init --starterId kinesis-realtime-analyticsThe deployment will take ~5-15 minutes. Subsequent deploys will be significantly faster.
Deploy from local machine
The deployment from local machine will build and deploy the application from your system. This means you also need to have:
- Docker. To install Docker on your system, you can follow this guide.- Node.js installed.
To perform the deployment, use the following command:
stacktape deploy --projectName <<project-name>> --stage <<stage>> --region <<region>>stage is an arbitrary name of your environment (for example staging, production or dev-john)
region is the AWS region, where your stack will be deployed to. All the available regions are listed below.
projectName is the name of your project. You can create it in the console or interactively using CLI.
| Region name & Location | code |
|---|---|
| Europe (Ireland) | eu-west-1 |
| Europe (London) | eu-west-2 |
| Europe (Frankfurt) | eu-central-1 |
| Europe (Milan) | eu-south-1 |
| Europe (Paris) | eu-west-3 |
| Europe (Stockholm) | eu-north-1 |
| US East (Ohio) | us-east-2 |
| US East (N. Virginia) | us-east-1 |
| US West (N. California) | us-west-1 |
| US West (Oregon) | us-west-2 |
| Canada (Central) | ca-central-1 |
| Africa (Cape Town) | af-south-1 |
| Asia Pacific (Hong Kong) | ap-east-1 |
| Asia Pacific (Mumbai) | ap-south-1 |
| Asia Pacific (Osaka-Local) | ap-northeast-3 |
| Asia Pacific (Seoul) | ap-northeast-2 |
| Asia Pacific (Singapore) | ap-southeast-1 |
| Asia Pacific (Sydney) | ap-southeast-2 |
| Asia Pacific (Tokyo) | ap-northeast-1 |
| China (Beijing) | cn-north-1 |
| China (Ningxia) | cn-northwest-1 |
| Middle East (Bahrain) | me-south-1 |
| South America (São Paulo) | sa-east-1 |
Deploy using AWS CodeBuild pipeline
Deployment using AWS CodeBuild will build and deploy your application inside AWS CodeBuild pipeline. To perform the deployment, use
stacktape codebuild:deploy --stage <<stage>> --region <<region>> --projectName <<project-name>>stage is an arbitrary name of your environment (for example staging, production or dev-john)
region is the AWS region, where your stack will be deployed to. All the available regions are listed below.
projectName is the name of your project. You can create it in the console or interactively using CLI.
| Region name & Location | code |
|---|---|
| Europe (Ireland) | eu-west-1 |
| Europe (London) | eu-west-2 |
| Europe (Frankfurt) | eu-central-1 |
| Europe (Milan) | eu-south-1 |
| Europe (Paris) | eu-west-3 |
| Europe (Stockholm) | eu-north-1 |
| US East (Ohio) | us-east-2 |
| US East (N. Virginia) | us-east-1 |
| US West (N. California) | us-west-1 |
| US West (Oregon) | us-west-2 |
| Canada (Central) | ca-central-1 |
| Africa (Cape Town) | af-south-1 |
| Asia Pacific (Hong Kong) | ap-east-1 |
| Asia Pacific (Mumbai) | ap-south-1 |
| Asia Pacific (Osaka-Local) | ap-northeast-3 |
| Asia Pacific (Seoul) | ap-northeast-2 |
| Asia Pacific (Singapore) | ap-southeast-1 |
| Asia Pacific (Sydney) | ap-southeast-2 |
| Asia Pacific (Tokyo) | ap-northeast-1 |
| China (Beijing) | cn-north-1 |
| China (Ningxia) | cn-northwest-1 |
| Middle East (Bahrain) | me-south-1 |
| South America (São Paulo) | sa-east-1 |
Deploy using Github actions CI/CD pipeline
- If you don't have one, create a new repository at https://github.com/new
- Create Github repository secrets: https://docs.stacktape.com/user-guides/ci-cd/#2-create-github-repository-secrets
- Replace
<<stage>>and<<region>>in the .github/workflows/deploy.yml file. git init --initial-branch=maingit add .git commit -m "setup stacktape project"git remote add origin git@github.com:<<namespace-name>>/<<repo-name>>.gitgit push -u origin main- To monitor the deployment progress, navigate to your github project and select the Actions tab
stage is an arbitrary name of your environment (for example staging, production or dev-john)
region is the AWS region, where your stack will be deployed to. All the available regions are listed below.
projectName is the name of your project. You can create it in the console or interactively using CLI.
| Region name & Location | code |
|---|---|
| Europe (Ireland) | eu-west-1 |
| Europe (London) | eu-west-2 |
| Europe (Frankfurt) | eu-central-1 |
| Europe (Milan) | eu-south-1 |
| Europe (Paris) | eu-west-3 |
| Europe (Stockholm) | eu-north-1 |
| US East (Ohio) | us-east-2 |
| US East (N. Virginia) | us-east-1 |
| US West (N. California) | us-west-1 |
| US West (Oregon) | us-west-2 |
| Canada (Central) | ca-central-1 |
| Africa (Cape Town) | af-south-1 |
| Asia Pacific (Hong Kong) | ap-east-1 |
| Asia Pacific (Mumbai) | ap-south-1 |
| Asia Pacific (Osaka-Local) | ap-northeast-3 |
| Asia Pacific (Seoul) | ap-northeast-2 |
| Asia Pacific (Singapore) | ap-southeast-1 |
| Asia Pacific (Sydney) | ap-southeast-2 |
| Asia Pacific (Tokyo) | ap-northeast-1 |
| China (Beijing) | cn-north-1 |
| China (Ningxia) | cn-northwest-1 |
| Middle East (Bahrain) | me-south-1 |
| South America (São Paulo) | sa-east-1 |
Deploy using Gitlab CI pipeline
- If you don't have one, create a new repository at https://gitlab.com/projects/new
- Create Gitlab repository secrets: https://docs.stacktape.com/user-guides/ci-cd/#2-create-gitlab-repository-secrets
- replace
<<stage>>and<<region>>in the .gitlab-ci.yml file. git init --initial-branch=maingit add .git commit -m "setup stacktape project"git remote add origin git@gitlab.com:<<namespace-name>>/<<repo-name>>.gitgit push -u origin mainTo monitor the deployment progress, navigate to your gitlab project and select CI/CD->jobs
stage is an arbitrary name of your environment (for example staging, production or dev-john)
region is the AWS region, where your stack will be deployed to. All the available regions are listed below.
projectName is the name of your project. You can create it in the console or interactively using CLI.
| Region name & Location | code |
|---|---|
| Europe (Ireland) | eu-west-1 |
| Europe (London) | eu-west-2 |
| Europe (Frankfurt) | eu-central-1 |
| Europe (Milan) | eu-south-1 |
| Europe (Paris) | eu-west-3 |
| Europe (Stockholm) | eu-north-1 |
| US East (Ohio) | us-east-2 |
| US East (N. Virginia) | us-east-1 |
| US West (N. California) | us-west-1 |
| US West (Oregon) | us-west-2 |
| Canada (Central) | ca-central-1 |
| Africa (Cape Town) | af-south-1 |
| Asia Pacific (Hong Kong) | ap-east-1 |
| Asia Pacific (Mumbai) | ap-south-1 |
| Asia Pacific (Osaka-Local) | ap-northeast-3 |
| Asia Pacific (Seoul) | ap-northeast-2 |
| Asia Pacific (Singapore) | ap-southeast-1 |
| Asia Pacific (Sydney) | ap-southeast-2 |
| Asia Pacific (Tokyo) | ap-northeast-1 |
| China (Beijing) | cn-north-1 |
| China (Ningxia) | cn-northwest-1 |
| Middle East (Bahrain) | me-south-1 |
| South America (São Paulo) | sa-east-1 |
After a successful deployment, some information about the stack will be printed to the terminal (URLs of the deployed services, links to logs, metrics, etc.).
-
Send an event to the stream:
curl -X POST <API_GATEWAY_URL>/events \ -H "Content-Type: application/json" \ -d '{"type": "pageview", "page": "/home", "userId": "user-1"}'
-
Query aggregated analytics (wait a few seconds for batch processing):
curl <API_GATEWAY_URL>/analytics
To run functions in the development mode (remotely on AWS), you can use the
dev command. For example, to develop and debug lambda function processEvents, you can use
stacktape dev --region <<your-region>> --stage <<stage>> --resourceName processEventsThe command will:
- quickly re-build and re-deploy your new function code
- watch for the function logs and pretty-print them to the terminal
The function is rebuilt and redeployed, when you either:
- type
rs + enterto the terminal - use the
--watchoption and one of your source code files changes
-
Stacktape deployments use AWS CloudFormation under the hood. It brings a lot of guarantees and convenience, but can be slow for certain use-cases.
-
To speed up the deployment, you can use the
--hotSwapflag which avoids using Cloudformation. -
Hotswap deployments work only for source code changes (for lambda function, containers and batch jobs) and for content uploads to buckets.
-
If the update deployment is not hot-swappable, Stacktape will automatically fall back to using a Cloudformation deployment.
stacktape deploy --hotSwap --stage <<stage>> --region <<region>> --projectName <<project-name>>- If you no longer want to use your stack, you can delete it.
- Stacktape will automatically delete every infrastructure resource and deployment artifact associated with your stack.
stacktape delete --stage <<stage>> --region <<region>>Stacktape uses a simple stacktape.yml configuration file to describe infrastructure resources, packaging, deployment
pipeline and other aspects of your project.
You can deploy your project to multiple environments (stages) - for
example production, staging or dev-john. A stack is a running instance of an project. It consists of your application
code (if any) and the infrastructure resources required to run it.
The configuration for this project is described below.
- Every resource must have an arbitrary, alphanumeric name (A-z0-9).
- Stacktape resources consist of multiple underlying AWS or 3rd party resources.
API Gateway receives HTTP requests and routes them to the API function.
CORS is enabled for convenience.
resources:
apiGateway:
type: http-api-gateway
properties:
cors:
enabled: trueA real-time data stream for ingesting events. Kinesis handles scaling and durability automatically.
eventStream:
type: kinesis-streamStores aggregated analytics with a composite key — partition key for event type, sort key for the hour bucket.
Atomic counters (ADD) ensure accurate counts even under concurrent writes.
analyticsTable:
type: dynamo-db-table
properties:
primaryKey:
partitionKey:
name: pk
type: string
sortKey:
name: sk
type: stringConsumes records from the Kinesis stream in batches. Decodes base64 records, groups them by event type and hour,
and writes aggregated counts to DynamoDB using atomic ADD operations.
- Batch size - up to 100 records per invocation.
- Max batch window - waits up to 10 seconds to collect a full batch.
- Starting position -
LATESTto only process new events.
processEvents:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: ./src/process-events.ts
memory: 512
timeout: 60
connectTo:
- analyticsTable
events:
- type: kinesis-stream
properties:
kinesisStreamName: eventStream
batchSize: 100
startingPosition: LATEST
maxBatchWindowSeconds: 10Hono-based HTTP API for sending events to the Kinesis stream and querying aggregated analytics from DynamoDB.
api:
type: function
properties:
packaging:
type: stacktape-lambda-buildpack
properties:
entryfilePath: ./src/index.ts
memory: 512
connectTo:
- eventStream
- analyticsTable
events:
- type: http-api-gateway
properties:
httpApiGatewayName: apiGateway
path: /
method: '*'
- type: http-api-gateway
properties:
httpApiGatewayName: apiGateway
path: /{proxy+}
method: '*'