Symfony cold starts

[eddiejan]

Hi all,

I noticed that the Symfony cold starts tend to get quite slow when following the base setup from here.

I found major improvements in cold starts (3500ms to 1650ms) by warming up the cache before deployment.

This is how I've accomplished it:
Step 1:
Change the getCacheDir to the following function in the Kernel class

    public function getCacheDir()
    {
        return $this->getProjectDir().'/var/cache/'.$this->environment;
    }

Step 2:
Make sure that no cache adapters are using the file system. In my case I had to disable cache for Twig (which doesn't matter because I'm using API-platform anyway) and let the Doctrine cache drive on APCu.

Step 3:
Run these commands for deployment

$ composer install --prefer-dist --optimize-autoloader --no-dev --no-scripts
$ php bin/console cache:clear --env=prod
$ composer dump-env prod --empty
$ serverless deploy --stage=prod

If there is enough intrest I might add a more brief description to the documentation.

[mnapoli]

FYI in Symfony 5.2 the cache directory has been split in 2: cache and build directory. That allows more easily pre-building the container (in the readonly build directory) before deploying, and moving the read/write "cache" directory to /tmp.

I haven't had time to document or write an example yet, but from 5.2 and up that would be the best solution (fastest + most compatible with other bundles).

[eddiejan]

I do wonder how it's going to work though.

Problem is that when I'm warming up the cache, in the container the kernel.cache_dir is set to my local environment. I'm struggling overwriting this parameter with the correct settings for the Lambda service. This setting seems to be, logically, used by quite some Symfony components/bundles to decide where to store the cache.

You already have some kind of reference on how to accomplish this?

[eddiejan]

I made it work, but it's probably not the cleanest way to do it. It suits the needs though. If anyone wants to benefit from it. Extra methods in Kernel.php:

    public function getLogDir()
    {
        // When on the lambda only /tmp is writeable
        if (getenv('LAMBDA_TASK_ROOT') !== false) {
            return '/tmp/log/';
        }

        return parent::getLogDir();
    }

    public function getBuildDir(): string
    {
        return $this->getProjectDir().'/var/build/'.$this->environment;
    }

    public function getCacheDir()
    {
        // When on the lambda only /tmp is writeable
        if (getenv('LAMBDA_TASK_ROOT') !== false) {
            return '/tmp/cache/'.$this->environment;
        }

        return parent::getCacheDir();
    }

    protected function getKernelParameters()
    {
        if($this->environment == 'prod') {
            $bundles = [];
            $bundlesMetadata = [];

            foreach ($this->bundles as $name => $bundle) {
                $bundles[$name] = \get_class($bundle);
                $bundlesMetadata[$name] = [
                    'path' => $bundle->getPath(),
                    'namespace' => $bundle->getNamespace(),
                ];
            }

            return [
                'kernel.project_dir' => realpath($this->getProjectDir()) ?: $this->getProjectDir(),
                'kernel.environment' => $this->environment,
                'kernel.runtime_environment' => '%env(default:kernel.environment:APP_RUNTIME_ENV)%',
                'kernel.debug' => $this->debug,
                'kernel.build_dir' => $this->getBuildDir(),
                'kernel.cache_dir' => '../../../../../tmp/cache/' . $this->environment,
                'kernel.logs_dir' => '../../../../../tmp/log/',
                'kernel.bundles' => $bundles,
                'kernel.bundles_metadata' => $bundlesMetadata,
                'kernel.charset' => $this->getCharset(),
                'kernel.container_class' => $this->getContainerClass(),
            ];
        }

        return parent::getKernelParameters();
    }

[wysow]

Hello @eddiejan and thanks for sharing this!

Looks like there is a typo on the line that says 'kernel.build_dir' => $this->getCacheDir(),, no? Should be 'kernel.build_dir' => $this->getBuildDir(), instead, right?

Also another tip is to construct all the cache with env variables already injected in with this:

    protected function build(ContainerBuilder $container): void
    {
        if ($this->environment === 'prod') {
            $container->addCompilerPass(new ResolveEnvPlaceholdersPass(), PassConfig::TYPE_AFTER_REMOVING, -1000);
        }
    }

So this way before constructing the zip file for lambda your prod cache could be already filled with all it needs.

[eddiejan]

Thanks for the catch. I've editted the comment.

Personally I don't like compiling .env variables in the container, because in my application I'm using the serverless.yml to setup the production variables. Some off them are dynamic, for example the database url which is reffering to created databases by CloudFormation.

[wysow]

You're welcome!

Sure I understand pre-compiling production cache is not doable in all use-cases for sure, but this is a good trick to know anyway 😉