Skip to content
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions .github/workflows/setup-e2e-environment/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@ runs:
steps:
- name: Wait for database to be ready
run: |
until docker exec ${{ job.services.db.id }} pg_isready -U nest_user_e2e -d nest_db_e2e; do
echo "Waiting for database..."
sleep 5
done
timeout 1m bash -c '
until docker exec ${{ job.services.db.id }} pg_isready -U nest_user_e2e -d nest_db_e2e; do
echo "Waiting for database..."
sleep 5
done
'
shell: bash

- name: Install PostgreSQL client
Expand Down Expand Up @@ -43,6 +45,7 @@ runs:
--env-file backend/.env.e2e.example \
--network host \
-p 9000:9000 \
-e DJANGO_DB_HOST=localhost \
owasp/nest:test-backend-e2e-latest \
sh -c '
gunicorn wsgi:application --bind 0.0.0.0:9000
Expand All @@ -51,9 +54,11 @@ runs:

- name: Waiting for the backend to be ready
run: |
until wget --spider http://localhost:9000/a; do
echo "Waiting for backend..."
sleep 5
done
timeout 1m bash -c '
until wget --spider http://localhost:9000/a; do
echo "Waiting for backend..."
sleep 5
done
'
echo "Backend is up!"
shell: bash
28 changes: 28 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,34 @@ Follow these steps to setup your e2e testing environment:

**Please note that you only need to do these steps once.**

#### Updating e2e Test Data
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My data file update comment was not about the instructions for the data loading. It was about how to maintain the nest-e2e..sql.gz file.


If you need to update the e2e test data, follow these steps:

1. Delete the e2e db container from `Docker Desktop Dashboard` or with the following command:

```bash
docker rm -f e2e-nest-db
```

2. Delete the e2e db volume from `Docker Desktop Dashboard` or with the following command:

```bash
docker volume rm nest-e2e_e2e-db-data
```

3. Re-run the e2e backend instance with the following command:

```bash
make run-backend-e2e
```

4. Load the updated data into the e2e db with the following command (in another terminal session):

```bash
make load-data-e2e
```

### Test Coverage

- There is a **minimum test coverage requirement** for the **backend** code -- see [pyproject.toml](https://github.com/OWASP/Nest/blob/main/backend/pyproject.toml).
Expand Down
2 changes: 1 addition & 1 deletion backend/.env.e2e.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ DJANGO_AWS_ACCESS_KEY_ID=None
DJANGO_AWS_SECRET_ACCESS_KEY=None
DJANGO_SETTINGS_MODULE=settings.e2e
DJANGO_CONFIGURATION=E2E
DJANGO_DB_HOST=None
DJANGO_DB_HOST=db
DJANGO_DB_NAME=nest_db_e2e
DJANGO_DB_USER=nest_user_e2e
DJANGO_DB_PASSWORD=nest_user_e2e_password
Expand Down
4 changes: 4 additions & 0 deletions backend/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ dump-data-e2e:
@echo "Dumping Nest e2e data"
@CMD="pg_dumpall -U nest_user_e2e --clean | gzip -9 > backend/data/nest-e2e.sql.gz" $(MAKE) exec-db-command-e2e

dump-data-local:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's look at this one more time. Are we going to use our full data snapshot for e2e testing (e.g. the entire data from nest.json.gz)? If so I'd prefer to keep it simple and have just a single data file -- without nest-e2e.sql.gz introduction.

Copy link
Collaborator Author

@ahmedxgouda ahmedxgouda Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can use nest.sql.gz instead of nest.json.gz. As it is extremely faster to load.

@echo "Dumping Nest local data"
@CMD="pg_dumpall -U nest_user_dev --clean | gzip -9 > backend/data/nest-e2e.sql.gz" $(MAKE) exec-db-command-it

enrich-data: \
github-enrich-issues \
owasp-enrich-chapters \
Expand Down
2 changes: 2 additions & 0 deletions backend/apps/api/decorators/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ def cache_response(
def decorator(view_func):
@wraps(view_func)
def _wrapper(request, *args, **kwargs):
if settings.IS_E2E_ENVIRONMENT:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why e2e needs this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We didn't setup cache for e2e, so when it tries to access cache it gives 500 internal error. I think there is an option to setup redis cache in CI/CD. Maybe we can do that or keep it simple.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, let's add cache service for e2e instead. The closer to production architecture the better -- for both local and CI/CD cases.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I will add the cache in another PR after this one.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's configure the cache backed for e2e via Django settings (locmem for now)

return view_func(request, *args, **kwargs)
if request.method not in ("GET", "HEAD"):
return view_func(request, *args, **kwargs)

Expand Down
2 changes: 1 addition & 1 deletion backend/apps/api/rest/v0/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
}

api_settings_customization = {}
if settings.IS_LOCAL_ENVIRONMENT:
if settings.IS_LOCAL_ENVIRONMENT or settings.IS_E2E_ENVIRONMENT:
api_settings_customization = {
"auth": None,
"servers": [
Expand Down
2 changes: 1 addition & 1 deletion backend/settings/e2e.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ class E2E(Base):
"""End-to-end testing configuration."""

APP_NAME = "OWASP Nest E2E Testing"
SITE_URL = "http://localhost:9000"

ALLOWED_ORIGINS = (
"http://frontend:3000", # NOSONAR
Expand All @@ -17,7 +18,6 @@ class E2E(Base):
CORS_ALLOWED_ORIGINS = ALLOWED_ORIGINS
CSRF_TRUSTED_ORIGINS = ALLOWED_ORIGINS

DEBUG = False
IS_E2E_ENVIRONMENT = True
LOGGING = {}
PUBLIC_IP_ADDRESS = values.Value()