|
| 1 | +# AWS severless fraud detection experiment |
| 2 | + |
| 3 | +This is a simple serverless application experiment that, |
| 4 | +- Builds and pushes Images of specified sub directories to ECR for lambda |
| 5 | +- Uses cloudformation and github actions to deploy the serverless Infra |
| 6 | +- contains Jupyter Notebooks on possible ML solutions for fraud detection problem |
| 7 | +- and notes on the problem |
| 8 | + |
| 9 | + |
| 10 | +## Prerequisites |
| 11 | +- [AWS Vault](https://github.com/99designs/aws-vault) |
| 12 | +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) |
| 13 | +- [uv] (https://docs.astral.sh/uv/getting-started/installation/) |
| 14 | + |
| 15 | + |
| 16 | +## Steps to run the experiment |
| 17 | + |
| 18 | +### Setting up the project |
| 19 | +- Clone the repository |
| 20 | +- Install the Pre-requisites |
| 21 | +- Run `uv run pytest` this will automatically create an environment and install all dependencies mentioned in the pyproject.toml file. |
| 22 | + |
| 23 | +### Deploy the application |
| 24 | +- Create a GitHub Actions workflow that will trigger on every push to the main branch. |
| 25 | + |
| 26 | +### Deploying to your own AWS account |
| 27 | +You can deploy from local by running github actions locally, |
| 28 | +- Setup [nektos/act](https://github.com/nektos/act) this helps you run GitHub Actions locally. |
| 29 | +- Install Docker, nektos/act uses docker to run your workflows. |
| 30 | +- And if you are using vscode you can install [Github Local Actions Docs](https://sanjulaganepola.github.io/github-local-actions-docs/) for a helpful UI to run this. (this is how I was testing my workflows locally) |
| 31 | + |
| 32 | +Alternatively, you can run with your own github actions |
| 33 | +- Fork the repo and add in `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to Actions secrets. |
| 34 | +- Run the workflow using your github actions. |
| 35 | + |
| 36 | + |
| 37 | +## What it will deploy? |
| 38 | +- A cloud formation application `fraud-detection` with the following resources: |
| 39 | + |
| 40 | + |
| 41 | + |
| 42 | + |
| 43 | +## Testing the cloudformation application |
| 44 | +You can use `utils/simulate_events.py` to simulate events and test the cloudformation application. |
| 45 | + |
| 46 | +- Activate the environment with `source .venv/bin/activate`, if you don't have a virtualenv, create one with `uv run pytest`. |
| 47 | +- Add AWS accont information to the session using aws-vault `aws-vault exec <profile name> --region <region name>` this for me turns out to be `aws-vault exec personal --region ca-central-1` personal being my profile and ca-central-1 being my region. |
| 48 | +- Now run `uv run python utils/simulate_events.py --stream-name fraud-detection-TransactionIngestion-mHXRRNzZAWJV --num-transactions 1000` |
| 49 | + |
| 50 | +The simulate.py pushes transactions events random some with error in the transaction-id format and some in the correct format, you should observe data being pushed into both your valid and invalid s3 buckets. |
0 commit comments