Merge all services under one server

[marcoacierno]

What

The first attempt here: b289a84, but we couldn't get the NAT instance to work again. (It worked once) and we had to revert it before the morning to avoid ticketing not working today.

This should 4$ of compute and more on storage costs. We should be able to save more once we migrate Pretix to ARM.

Services

• PyCon workers (celery and beat)
• Pretix
• NAT instance
• Redis

[marcoacierno]

Some notes:

It is quite difficult to get NAT instance + Docker working. Docker applies some iptables rules that make the NAT instance stop working.
If we can't merge the nat instance with the rest of the services, it might be harder to justify the cost of switching to t3a.medium.

We could still see some savings by using a custom ECS image with a lower block storage requirement (we don't need 30GB of storage for all those servers): https://github.com/aws/amazon-ecs-ami

Pretix is quite memory-heavy. When deployed on a t3.small with redis, backend beat, and backend worker, it seems like it fails to start the web process and gets killed signal 9. I need to investigate a bit better, memory usage is high, but even after adding some swap space, it doesn't help.

Current setup:

nat instance	t4g.nano 8GB
production-pretix-instance	t3.small 30gb EBS / 20GB EBS
pythonit-production-worker	t4g.micro 30gb EBS 
pythonit-production-redis	t4g.nano 30gb EBS / 10GB EBS

All GP3.

Things to look to continue into this

• Understand how much savings we will get by lowering the storage from 30 GB to 8 GB. Last month cost was ~10$ of "EC2 other"
• Can we merge the Redis server with the worker server at least?
• How much would it cost us to have t3a.medium + t4g.nano?
• How hard is it to move Pretix to ARM? How much could we save with a single ARM server? Would performance be better?
• We deploy Pretix in a single container that internally starts the multiple processes (web server, celery, etc). Would we get more control over performance if we split them in multiple containers? Could we move the web server to lambda?
• Can we move the worker to t4g.nano?

We also need to keep an eye on credits:

Amazon Elastic Compute Cloud T3ACPUCredits
USD 0.02
$0.05 per vCPU-Hour of T3A CPU Credits
0.473 vCPU-Hours	USD 0.02

Amazon Elastic Compute Cloud T4GCPUCredits
USD 0.00
$0.04 per vCPU-Hour of T4G CPU Credits
0.037 vCPU-Hours	USD 0.00

[marcoacierno]

Done in:

• Migrate to a single server setup with ECS #4164
• Deploy the Frontend on ECS #4198
• Move clamav on ECS #4190