Laravel shut downs the Ubuntu EC2 instances on AWS.

[Eugene-Melbourne]

How do you call when computer is being unplugged from the power without processing the shut down functions? I call it "hard rest" down the text.

I have a few EC2 instances, they are put into autoscaling group and some of them are getting "hard rest" sometimes. Once in a couple of days. It is especially painful when it is happening in the middle of the cron job. I don't know what is causing it for sure, but I know it is not consistent. I also know that it comes with CPU usage spikes up to 100% and it takes about 10 seconds to instance to get into the "hard rest". Even the load balancer's health checks has no time to fail once.

I started to log things.

My latest idea was to blame the S3, I check S3 every minute to see if new files got there. It is a very short check, but my records show that this procedure not finished a few times. It is a good chance that it causes the whole problem.

Plus, recently I noticed that a few of my AWS code deploy deployments failed on "download bundle" stage. I guess S3 is not stable in Ohio this days.

It brought me to conclusion that it is possible for the S3 to be "not available" or "too slow" sometimes. And it makes the PHP request to get into "panic mode" and heat up CPU to 100% and even "hard rest" the instance. Maybe it was not the PHP itself, maybe it was some Ubuntu functions that were called by PHP.

Does any of you experience something similar? Could you reproduce it? Is there a workaround? Do you know what can cause such problems in Laravel code?

This is my code that work with S3

    $files = Storage::files($directory);
    foreach ($files as $fileName) {
        $content = Storage::get($fileName);
        $this->processFile($content);
        $newFileName = $archiveDirectory . basename($fileName);
        Storage::move($fileName, $newFileName);
        usleep(100);
    }

I use PHP 7.4 and Laravel 8 on Ubuntu.