Performance of zero-downtime deployment with many hosts?

[nathan-io]

I'm considering Deployer as a solution in a case where we want to:

1. prepare the release on a build server (including composer and npm)
2. zip up the release
3. run some php artisan commands
4. simultaneously sync the .zip to our application servers, have them unzip it, then update the symlink

There will be 10-20 application servers starting off, but we may need more eventually.

Assuming the approach described above is doable and is not a terrible idea, my question is: Around what number of application servers would performance break down with this approach?

To my knowledge, there's no limit on the number of outgoing SSH connections our build server could have. What I'm concerned about is the resource utilization on the build server, trying to simultaneously transfer the archive to that many servers.

The build server is in the same private network as the application servers, so transfer speed should be good.

Has anyone tested ZDD with a lot of application servers? Any perspective or guidance is appreciated!

[antonmedv]

I think doable. I saw a project with ~550 application servers. And a three build servers.

For transferring archive to 20 server there is no obstacles. Of course, it’s possible to upload in smaller batches (with ->limit(5)), for example, of 5 parallel at a time.

Updating symlinks in batches is also a good idea. Maybe even adding sone sleep time between batches.

[lexjwa]

@antonmedv I have prepared the release build on build server, and zip it up and uploaded it on an application server with a upload task. I am trying to update the symlink on the application server, but as I deploy:prepare on the build server to prepare release build, I can not run 'deploy:prepare', 'deploy:release', on the application server. It start fetching from the repo or if I start with 'deploy:release it then can not find .dep directory on the application server. And do not symlink on the application server.

Can you point me to right direction if I am missing something?

task('initiate_build', function () {
    set('deploy_path', __DIR__ . '/.build');
    invoke('deploy:prepare');
})->local();

task('composer', function () {
    cd('{{release_or_current_path}}');
    run('composer install');
})->local();

task('artisan', function () {
    cd('{{release_or_current_path}}');
    run('php artisan storage:link');
    run('php artisan view:clear');
    run('php artisan config:cache');
    run('php artisan config:clear');
    run('php artisan route:cache');
    run('php artisan clear-compiled');
    run('php artisan route:clear');
    run('php artisan optimize');
})->local();

task('npm', function () {
    cd('{{release_or_current_path}}');
    run('npm install');
    run('npm run dev');
})->local();

task('unlock', function () {
    cd('{{release_or_current_path}}');
    invoke('deploy:unlock');
})->local();

task('build', [
    'initiate_build',
    'composer',
    'artisan',
    'npm',
    'unlock'
]); 

task('zip', function () {
    cd('.build/');
    run('rm -rf release.zip');
    run('zip -r release.zip release -x release/node_modules/**\*  release/public/css/**\* release/public/js/**\* release/public/vendor/**\* ');
})->local();

task('pre_upload', function () {
    cd('.build/');
    run('mkdir upload');
    run('mv release.zip upload');
})->local();

task('release', [
    'deploy:prepare', //I am not sure if I should prepare when I deploy from build server, but it will not sylink  on the application server if I do not run this
    'deploy:release', //I am not sure if I should start deploy:release when I deploy, but it will not sylink  on the application server without it.
    'upload',
    'unzip',
    'deploy:shared',
    'deploy:writable',
    'deploy:symlink',
]);

task('upload', function () {
    upload(__DIR__ . "/.build/upload/", '{{release_or_current_path}}');
});

task('unzip', function () {
    cd('{{release_or_current_path}}');
    run('unzip release.zip');
    run('mv release/*  {{release_or_current_path}}');
    run('rm -rf release.zip');
    run('rm -rf release');
});

task('clean_upload', function () {
    cd('.build/');
    run('rm -rf release.zip');
    run('rm -rf upload');
})->local();

// Deploy to application servers
task('deploy', [
    'zip',
    'pre_upload',
    'release',
    'clean_upload'
]);

Basically I am trying to achieve zero down time when deploying from build server to multiple application servers.

[antonmedv]

No need to deploy:release separatly. See https://deployer.org/docs/7.x/recipe/common#deployprepare
Just override deploy:update_code to download zip from build server.