feature symfony#59020 [AssetMapper] add support for assets pre-compression (dunglas)

This PR was squashed before being merged into the 7.3 branch.

Discussion
----------

[AssetMapper] add support for assets pre-compression

| Q             | A
| ------------- | ---
| Branch?       | 7.3
| Bug fix?      | no
| New feature?  | yes
| Deprecations? | no
| Issues        | n/a
| License       | MIT

Adds support for assets and files precompression using Zstandard, Brotli, and gzip (this will be part of [my talk at SymfonyCon next week](https://live.symfony.com/2024-vienna-con/schedule#http-compression-in-symfony-apps)).

When calling `bin/console asset-map:compile` and if the new option `asset_mapper.precompress` is enabled, all files matching the configured extensions will be compressed in Brotli, Zstandard, and gzip (zopfli when available) at maximal compression (defaults to the list supported by Caddy and Cloudflare). The PHP extension is used if available, otherwise the Unix command is used as a fallback.

The compressed files are created with the same name as the original asset but with the `.br`, `.zst`, or `.gz` extension appended.
This allows native compatibility with web servers supporting precompression such as [Caddy and FrankenPHP](https://caddyserver.com/docs/caddyfile/directives/file_server#precompressed) or [NGINX with the Brotli module](https://github.com/google/ngx_brotli).
Apache can also use these files but will require [some extra config](https://stackoverflow.com/questions/16883241/how-to-host-static-content-pre-compressed-in-apache).

This PR also adds an `assets:compress` command that can be used to compress files not managed by AssetMapper (e.g. `robots.txt`).

Finally, the new `asset_mapper.compressor` service can be used to precompress files uploaded by the user (among various other use cases).

TODO:

* [x] add tests

Commits
-------

5fa3e92 [AssetMapper] add support for assets pre-compression

Author: fabpot

.github/workflows/unit-tests.yml

@@ -33,6 +33,9 @@ jobs:
             mode: low-deps
           - php: '8.3'
           - php: '8.4'
+            # brotli and zstd extensions are optional, when not present the commands will be used instead,
+            # we must test both scenarios
+            extensions: amqp,apcu,brotli,igbinary,intl,mbstring,memcached,redis,relay,zstd
             #mode: experimental
       fail-fast: false
 
@@ -53,6 +56,12 @@ jobs:
           extensions: "${{ matrix.extensions || env.extensions }}"
           tools: flex
 
+      - name: Install optional commands
+        if: matrix.php == '8.4'
+        run: |
+          sudo apt-get update
+          sudo apt-get install zopfli
+
       - name: Configure environment
         run: |
           git config --global user.email ""

src/Symfony/Bundle/FrameworkBundle/CHANGELOG.md

@@ -1,6 +1,11 @@
 CHANGELOG
 =========
 
+7.3
+---
+
+ * Add support for assets pre-compression
+
 7.2
 ---
 

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php

@@ -17,6 +17,7 @@
 use Symfony\Bundle\FullStack;
 use Symfony\Component\Asset\Package;
 use Symfony\Component\AssetMapper\AssetMapper;
+use Symfony\Component\AssetMapper\Compressor\CompressorInterface;
 use Symfony\Component\Cache\Adapter\DoctrineAdapter;
 use Symfony\Component\Config\Definition\Builder\ArrayNodeDefinition;
 use Symfony\Component\Config\Definition\Builder\NodeBuilder;
@@ -924,6 +925,29 @@ private function addAssetMapperSection(ArrayNodeDefinition $rootNode, callable $
                             ->info('The directory to store JavaScript vendors.')
                             ->defaultValue('%kernel.project_dir%/assets/vendor')
                         ->end()
+                        ->arrayNode('precompress')
+                            ->info('Precompress assets with Brotli, Zstandard and gzip.')
+                            ->canBeEnabled()
+                            ->fixXmlConfig('format')
+                            ->fixXmlConfig('extension')
+                            ->children()
+                                ->arrayNode('formats')
+                                    ->info('Array of formats to enable. "brotli", "zstandard" and "gzip" are supported. Defaults to all formats supported by the system. The entire list must be provided.')
+                                    ->prototype('scalar')->end()
+                                    ->performNoDeepMerging()
+                                    ->validate()
+                                        ->ifTrue(static fn (array $v) => array_diff($v, ['brotli', 'zstandard', 'gzip']))
+                                        ->thenInvalid('Unsupported format: "brotli", "zstandard" and "gzip" are supported.')
+                                    ->end()
+                                ->end()
+                                ->arrayNode('extensions')
+                                    ->info('Array of extensions to compress. The entire list must be provided, no merging occurs.')
+                                    ->prototype('scalar')->end()
+                                    ->performNoDeepMerging()
+                                    ->defaultValue(interface_exists(CompressorInterface::class) ? CompressorInterface::DEFAULT_EXTENSIONS : [])
+                                ->end()
+                            ->end()
+                        ->end()
                     ->end()
                 ->end()
             ->end()

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

@@ -32,6 +32,7 @@
 use Symfony\Component\Asset\PackageInterface;
 use Symfony\Component\AssetMapper\AssetMapper;
 use Symfony\Component\AssetMapper\Compiler\AssetCompilerInterface;
+use Symfony\Component\AssetMapper\Compressor\CompressorInterface;
 use Symfony\Component\BrowserKit\AbstractBrowser;
 use Symfony\Component\Cache\Adapter\AdapterInterface;
 use Symfony\Component\Cache\Adapter\ArrayAdapter;
@@ -1372,6 +1373,26 @@ private function registerAssetMapperConfiguration(array $config, ContainerBuilde
             ->replaceArgument(3, $config['importmap_polyfill'])
             ->replaceArgument(4, $config['importmap_script_attributes'])
         ;
+
+        if (interface_exists(CompressorInterface::class)) {
+            $compressors = [];
+            foreach ($config['precompress']['formats'] as $format) {
+                $compressors[$format] = new Reference("asset_mapper.compressor.$format");
+            }
+
+            $container->getDefinition('asset_mapper.compressor')->replaceArgument(0, $compressors ?: null);
+
+            if ($config['precompress']['enabled']) {
+                $container
+                    ->getDefinition('asset_mapper.local_public_assets_filesystem')
+                    ->addArgument(new Reference('asset_mapper.compressor'))
+                    ->addArgument($config['precompress']['extensions'])
+                ;
+            }
+        } else {
+            $container->removeDefinition('asset_mapper.compressor');
+            $container->removeDefinition('asset_mapper.assets.command.compress');
+        }
     }
 
     /**

src/Symfony/Bundle/FrameworkBundle/Resources/config/asset_mapper.php

@@ -17,6 +17,7 @@
 use Symfony\Component\AssetMapper\AssetMapperInterface;
 use Symfony\Component\AssetMapper\AssetMapperRepository;
 use Symfony\Component\AssetMapper\Command\AssetMapperCompileCommand;
+use Symfony\Component\AssetMapper\Command\CompressAssetsCommand;
 use Symfony\Component\AssetMapper\Command\DebugAssetMapperCommand;
 use Symfony\Component\AssetMapper\Command\ImportMapAuditCommand;
 use Symfony\Component\AssetMapper\Command\ImportMapInstallCommand;
@@ -28,6 +29,11 @@
 use Symfony\Component\AssetMapper\Compiler\CssAssetUrlCompiler;
 use Symfony\Component\AssetMapper\Compiler\JavaScriptImportPathCompiler;
 use Symfony\Component\AssetMapper\Compiler\SourceMappingUrlsCompiler;
+use Symfony\Component\AssetMapper\Compressor\BrotliCompressor;
+use Symfony\Component\AssetMapper\Compressor\ChainCompressor;
+use Symfony\Component\AssetMapper\Compressor\CompressorInterface;
+use Symfony\Component\AssetMapper\Compressor\GzipCompressor;
+use Symfony\Component\AssetMapper\Compressor\ZstandardCompressor;
 use Symfony\Component\AssetMapper\Factory\CachedMappedAssetFactory;
 use Symfony\Component\AssetMapper\Factory\MappedAssetFactory;
 use Symfony\Component\AssetMapper\ImportMap\ImportMapAuditor;
@@ -254,5 +260,20 @@
         ->set('asset_mapper.importmap.command.outdated', ImportMapOutdatedCommand::class)
             ->args([service('asset_mapper.importmap.update_checker')])
             ->tag('console.command')
+
+        ->set('asset_mapper.compressor.brotli', BrotliCompressor::class)
+        ->set('asset_mapper.compressor.zstandard', ZstandardCompressor::class)
+        ->set('asset_mapper.compressor.gzip', GzipCompressor::class)
+
+        ->set('asset_mapper.compressor', ChainCompressor::class)
+            ->args([
+                abstract_arg('compressor'),
+                service('logger'),
+            ])
+        ->alias(CompressorInterface::class, 'asset_mapper.compressor')
+
+        ->set('asset_mapper.assets.command.compress', CompressAssetsCommand::class)
+            ->args([service('asset_mapper.compressor')])
+            ->tag('console.command')
     ;
 };

src/Symfony/Bundle/FrameworkBundle/Resources/config/schema/symfony-1.0.xsd

@@ -206,6 +206,7 @@
             <xsd:element name="excluded-pattern" type="xsd:string" minOccurs="0" maxOccurs="unbounded" />
             <xsd:element name="extension" type="asset_mapper_extension" minOccurs="0" maxOccurs="unbounded" />
             <xsd:element name="importmap-script-attribute" type="asset_mapper_attribute" minOccurs="0" maxOccurs="unbounded" />
+            <xsd:element name="precompress" type="asset_mapper_precompress" minOccurs="0" maxOccurs="1" />
         </xsd:sequence>
         <xsd:attribute name="enabled" type="xsd:boolean" />
         <xsd:attribute name="exclude-dotfiles" type="xsd:boolean" />
@@ -230,6 +231,16 @@
         <xsd:attribute name="key" type="xsd:string" use="required" />
     </xsd:complexType>
 
+    <xsd:complexType name="asset_mapper_precompress">
+        <xsd:sequence>
+            <xsd:element name="format" type="xsd:string" minOccurs="0" maxOccurs="unbounded" />
+            <xsd:element name="extension" type="xsd:string" minOccurs="0" maxOccurs="unbounded" />
+        </xsd:sequence>
+
+        <xsd:attribute name="enabled" type="xsd:boolean" />
+    </xsd:complexType>
+
+
     <xsd:simpleType name="missing-import-mode">
         <xsd:restriction base="xsd:string">
             <xsd:enumeration value="strict" />

src/Symfony/Bundle/FrameworkBundle/Tests/DependencyInjection/ConfigurationTest.php

@@ -15,6 +15,7 @@
 use PHPUnit\Framework\TestCase;
 use Symfony\Bundle\FrameworkBundle\DependencyInjection\Configuration;
 use Symfony\Bundle\FullStack;
+use Symfony\Component\AssetMapper\Compressor\CompressorInterface;
 use Symfony\Component\Cache\Adapter\DoctrineAdapter;
 use Symfony\Component\Config\Definition\Exception\InvalidConfigurationException;
 use Symfony\Component\Config\Definition\Processor;
@@ -141,6 +142,11 @@ public function testAssetMapperCanBeEnabled()
             'vendor_dir' => '%kernel.project_dir%/assets/vendor',
             'importmap_script_attributes' => [],
             'exclude_dotfiles' => true,
+            'precompress' => [
+                'enabled' => false,
+                'formats' => [],
+                'extensions' => interface_exists(CompressorInterface::class) ? CompressorInterface::DEFAULT_EXTENSIONS : [],
+            ],
         ];
 
         $this->assertEquals($defaultConfig, $config['asset_mapper']);
@@ -847,6 +853,11 @@ protected static function getBundleDefaultConfig()
                 'vendor_dir' => '%kernel.project_dir%/assets/vendor',
                 'importmap_script_attributes' => [],
                 'exclude_dotfiles' => true,
+                'precompress' => [
+                    'enabled' => false,
+                    'formats' => [],
+                    'extensions' => interface_exists(CompressorInterface::class) ? CompressorInterface::DEFAULT_EXTENSIONS : [],
+                ],
             ],
             'cache' => [
                 'pools' => [],

src/Symfony/Component/AssetMapper/CHANGELOG.md

@@ -1,6 +1,11 @@
 CHANGELOG
 =========
 
+7.3
+---
+
+ * Add support for pre-compressing assets with Brotli, Zstandard, Zopfli, and gzip
+
 7.2
 ---
 

src/Symfony/Component/AssetMapper/Command/CompressAssetsCommand.php

@@ -0,0 +1,63 @@
+<?php
+
+/*
+ * This file is part of the Symfony package.
+ *
+ * (c) Fabien Potencier <[email protected]>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Symfony\Component\AssetMapper\Command;
+
+use Symfony\Component\AssetMapper\Compressor\CompressorInterface;
+use Symfony\Component\Console\Attribute\AsCommand;
+use Symfony\Component\Console\Command\Command;
+use Symfony\Component\Console\Input\InputArgument;
+use Symfony\Component\Console\Input\InputInterface;
+use Symfony\Component\Console\Output\OutputInterface;
+use Symfony\Component\Console\Style\SymfonyStyle;
+
+/**
+ * Pre-compresses files to serve through a web server.
+ *
+ * @author Kévin Dunglas <[email protected]>
+ */
+#[AsCommand(name: 'assets:compress', description: 'Pre-compresses files to serve through a web server')]
+final class CompressAssetsCommand extends Command
+{
+    public function __construct(
+        private readonly CompressorInterface $compressor,
+    ) {
+        parent::__construct();
+    }
+
+    protected function configure(): void
+    {
+        $this
+            ->addArgument('paths', InputArgument::IS_ARRAY | InputArgument::REQUIRED, 'The files to compress')
+            ->setHelp(<<<'EOT'
+The <info>%command.name%</info> command compresses the given file in Brotli, Zstandard and gzip formats.
+This is especially useful to serve pre-compressed files through a web server.
+
+The existing file will be kept. The compressed files will be created in the same directory.
+The extension of the compression format will be appended to the original file name.
+EOT
+            );
+    }
+
+    protected function execute(InputInterface $input, OutputInterface $output): int
+    {
+        $io = new SymfonyStyle($input, $output);
+
+        $paths = $input->getArgument('paths');
+        foreach ($paths as $path) {
+            $this->compressor->compress($path);
+        }
+
+        $io->success(\sprintf('File%s compressed successfully.', \count($paths) > 1 ? 's' : ''));
+
+        return Command::SUCCESS;
+    }
+}

src/Symfony/Component/AssetMapper/Compressor/BrotliCompressor.php

@@ -0,0 +1,48 @@
+<?php
+
+/*
+ * This file is part of the Symfony package.
+ *
+ * (c) Fabien Potencier <[email protected]>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Symfony\Component\AssetMapper\Compressor;
+
+use Symfony\Component\Process\Process;
+
+/**
+ * Compresses a file using Brotli.
+ *
+ * @author Kévin Dunglas <[email protected]>
+ */
+final class BrotliCompressor implements SupportedCompressorInterface
+{
+    use CompressorTrait;
+
+    private const WRAPPER = 'compress.brotli';
+    private const COMMAND = 'brotli';
+    private const PHP_EXTENSION = 'brotli';
+    private const FILE_EXTENSION = 'br';
+
+    public function __construct(
+        ?string $executable = null,
+    ) {
+        $this->executable = $executable;
+    }
+
+    /**
+     * @return resource
+     */
+    private function createStreamContext()
+    {
+        return stream_context_create(['brotli' => ['level' => BROTLI_COMPRESS_LEVEL_MAX]]);
+    }
+
+    private function compressWithBinary(string $path): void
+    {
+        (new Process([$this->executable, '--best', '--force', "--output=$path.".self::FILE_EXTENSION, '--', $path]))->mustRun();
+    }
+}