Adding new module doesn't remove not existing directories & files

[ihor-sviziev]

Pre-requirements: You have installed some module wo magento composer installer (as example just unzip to magento directory) and added some files in module subfolder, as example some folder with images in skin/...

Actual result: When you try install this module via composer installer - these images are present.
Expected result: When you try install this module via composer installer - these images shouldn't be present. Should be present only files, that present in module repository

[Flyingmana]

Problem with solving this:
how to recognize files not belonging to the module, which should not get removed,
or files which get created from the module during the usage, like it happens to media and var

I see this problem as unresolvable, or not resolvable without manual action.

How do you think this could be solved?