deploy returns 'Operation not permitted' with chgrp on shared_dirs

[memen45]

A while after the first deploy, I wanted to update, so did a deploy again. In the mean time, the web user (www-data) added some files to some of the shared_dirs, also making some subdirectories. These files and directories are created with owner www-data:www-data and this should not be a problem.

Now, trying to deploy, the dep deploy returns an extended list of all those files and directories with e.g.
chgrp: changing group of 'pub/media/tmp/design/file': Operation not permitted
The deploy process is ran from the user 'deploy' with the following settings:

set('writable_mode', 'chgrp');
set('writable_recursive', true);
set('http_group', 'www-data');

Effectively, the command that is executed by the deploy user is chgrp -H -R www-data writable_dirs, however, chgrp is not allowed on files or directories that are not owned by the user calling it. Since the new files are created by www-data:www-data, this command fails, even thought the group actually already is www-data. So, the complete deploy fails, while nothing is wrong with the file permissions.

How can this be solved? Is this caused by the overlap of shared_dirs and writable_dirs? I can imagine that permissions do not have to be repeatedly set for shared_dirs, as those are not created again for each deploy. So only the first deploy should set permissions for shared_dirs AND writable_dirs, but subsequent deploys should only set permissions for writable_dirs that are NOT shared_dirs (with one exception, in case the shared_dirs have changed, the new directories have to be created and permissions have te be set).

[memen45]

Current procedure in writable.php is to

• try and create all directories from writable_dirs (mkdir -p $dirs)
• apply permission on all writable_dirs

This only works if all the directories and files are created by the deploy user. This is not the case for shared directories, as these might contain directories/files that are created by the web user (www-data). The correct procedure should be

• create only new shared_dirs
  • if shared_dir exists: nothing to do
  • if shared_dir does not exist: create it
• create all writable_dirs, NOT in shared_dirs
• apply permissions on created dirs only

Another simple solution could be to just follow the current procedure, but add the -f option to chgrp/chown commands, as this will silence these errors and the deploy will at least not break unnecessarily.

Hope this clarifies a little. I am not sure why this is moved to discussion @antonmedv. This is more a bug report or issue and is highly related to the code itself, not so much about discussion of the usage of depoyer in general.

[antonmedv]

I see now. Was too quick with decision. Let’s try to fix it. Can you create a PR with test what is failing to show this error?

[memen45]

Is there a test already for the writable.php that can be altered? Would be nice to have some example, as I am not so aware of the coding practices for php or this repository, so I am not sure where to start.

[memen45]

As far as I can tell, the writable_dirs are not part of any tests yet, is that correct?

[antonmedv]

Yes, no test covers it.

[memen45]

Thanks for your response. So how would you go about testing permissions of folders and files?

• Is it possible to change permission of some folder during a test?
• Is it possible to write tests for second deploy rather than first?
• Is it possible to define overlapping shared_dirs and writable_dirs for the test?

Hopefully you can help. I will see when I have time to work on this, I think it would save a lot of time if the deploy process could work flawlessly every time (allows for automated deploy on git change in the future).

[memen45]

I looked into it some more, and I see two possible solutions.

1. First solution is very simple, add -f --silent --quiet option to the chgrp command in writable.php
   run("$sudo chgrp -H -f $recursive $httpGroup $dirs");
   This will ensure the deploy is not interrupted by this command when www-data:www-data group is owner already.

2. The other possible solution requires some more work. First in writable.php we have to change the chgrp command to something as suggested here.

find $dirs \
   -path $shared_dir1 -prune \
   -o -path $shared_dir2 -prune \
   -o -path $shared_dir3 -prune \
   -o -exec chgrp -H $httpGroup {} +

This will cause shared_dirs to have deploy:deploy owner, while we need deploy:www-data ownership. Therefore, permissions have to be fixed at the moment of creation:
https://github.com/memen45/deployer/blob/851ce772a07ebba670190ac42ee495a5dedbc754/recipe/deploy/shared.php#L29
Instead of run("mkdir -p $sharedPath/$dir"); the following could be used:

$httpUser = get('http_user', false);
$useroption = $httpUser ? '-o ' . $httpUser : '';
$httpGroup = get('http_group', false);
$groupoption = $httpGroup ? '-g ' . $httpGroup : '';
run("install -d $useroption $groupoption $sharedPath/$dir");

• not sure how this permission fits in the writable_modes that are available. I would think it is independent, but not sure.
• not sure how to make the chgrp command easier with variable shared_dirs to exclude?

Looking forward to hear some feedback!

[antonmedv]

Looks good. What about PR with the simple solution?

[memen45]

Thanks, see #2648