[Bug] “sendgrid+api” dsn scheme is not supported after installing Symfony SendGrid mailer

[Insannik]

Describe the bug

I'm trying to install the Symfony SendGrid mailer plugin as described here.

We run Mautic v5.2.5 in Docker and since the documentation for docker-mautic states the following:

Currently this image has no easy way to extend Mautic (e.g. adding extra composer dependencies or installing extra plugins or themes).
This is an ongoing effort we hope to support in an upcoming 5.x release.
For now, please build your own images based on the official ones to add the needed dependencies, plugins and themes.

I've tried to slightly extend the Dockerfile in order to get composer and SendGrid mailer to work.

After trying to configure Mautic to use the SendGrid API, I get the following error in the UI:

Steps to reproduce

Extend the Dockerfile as shown below.

RUN cd /opt && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer create-project mautic/recommended-project:${MAUTIC_VERSION} mautic --no-interaction && \
    rm -rf /opt/mautic/var/cache/js && \
    find /opt/mautic/node_modules -mindepth 1 -maxdepth 1 -not \( -name 'jquery' -or -name 'vimeo-froogaloop2' \) | xargs rm -rf

Becomes:

# Install Mautic using composer, install Symfony Sengrid Mailer and clean cache
RUN cd /opt && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer create-project mautic/recommended-project:${MAUTIC_VERSION} mautic --no-interaction && \
    cd /opt/mautic && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer require symfony/sendgrid-mailer:* && \
    rm -rf /opt/mautic/var/cache/js && \
    find /opt/mautic/node_modules -mindepth 1 -maxdepth 1 -not \( -name 'jquery' -or -name 'vimeo-froogaloop2' \) | xargs rm -rf

This builds without errors. Inspecting the files by running an interactive shell shows that the composer.json file now references the sendgrid-mailer plugin and the vendor folder contains the necessary files. The full Dockerfile can be found below.

Deploy the container as normal and try to configure the SendGrid API using the documentation here.

Expected behavior

I expect to be able to use the sendgrid+api scheme when configuring Email Settings via the UI.

Actual behavior

After configuring the Email Settings to use the sendgrid+api scheme as described here and saving the configuration, the frontend shows an error message saying “sendgrid+api” dsn scheme is not supported.

Docker Image Tag

Based on: mautic/mautic:5.2.5-apache

Server Architecture

AMD64 (Intel or AMD processors)

Docker Image Digest (Optional but helpful)

No response

Dockerfile (if applicable)

FROM php:8.1-apache as builder

LABEL vendor="Mautic"
LABEL maintainer="Mautic core team <>"

# Install PHP extensions
RUN apt-get update && apt-get install --no-install-recommends -y \
    ca-certificates \
    build-essential  \
    git \
    curl \
    libcurl4-gnutls-dev \
    libc-client-dev \
    libkrb5-dev \
    libmcrypt-dev \
    libssl-dev \
    libxml2-dev \
    libzip-dev \
    libjpeg-dev \
    libmagickwand-dev \
    libpng-dev \
    libgif-dev \
    libtiff-dev \
    libz-dev \
    libpq-dev \
    imagemagick \
    graphicsmagick \
    libwebp-dev \
    libjpeg62-turbo-dev \
    libxpm-dev \
    libaprutil1-dev \
    libicu-dev \
    libfreetype6-dev \
    libonig-dev \
    librabbitmq-dev \
    unzip \
    nodejs \
    npm

RUN curl -L -o /tmp/amqp.tar.gz "https://github.com/php-amqp/php-amqp/archive/refs/tags/v2.1.1.tar.gz" \
    && mkdir -p /usr/src/php/ext/amqp \
    && tar -C /usr/src/php/ext/amqp -zxvf /tmp/amqp.tar.gz --strip 1 \
    && rm /tmp/amqp.tar.gz

RUN docker-php-ext-configure gd --with-freetype --with-jpeg \
    && docker-php-ext-configure imap --with-kerberos --with-imap-ssl \
    && docker-php-ext-configure opcache --enable-opcache \
    && docker-php-ext-install intl mbstring mysqli curl pdo_mysql zip bcmath sockets exif amqp gd imap opcache \
    && docker-php-ext-enable intl mbstring mysqli curl pdo_mysql zip bcmath sockets exif amqp gd imap opcache

# Install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin --filename=composer

RUN echo "memory_limit = -1" > /usr/local/etc/php/php.ini

# Define Mautic version by package tag
ARG MAUTIC_VERSION=5.2.5

# Install Mautic using composer, install Symfony Sengrid Mailer and clean cache
RUN cd /opt && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer create-project mautic/recommended-project:${MAUTIC_VERSION} mautic --no-interaction && \
    cd /opt/mautic && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer require symfony/sendgrid-mailer:* && \
    rm -rf /opt/mautic/var/cache/js && \
    find /opt/mautic/node_modules -mindepth 1 -maxdepth 1 -not \( -name 'jquery' -or -name 'vimeo-froogaloop2' \) | xargs rm -rf

FROM php:8.1-apache

COPY --from=builder /usr/local/lib/php/extensions /usr/local/lib/php/extensions
COPY --from=builder /usr/local/etc/php/conf.d/ /usr/local/etc/php/conf.d/

COPY --from=builder --chown=www-data:www-data /opt/mautic /var/www/html

# Install PHP extensions requirements and other dependencies
RUN apt-get update && apt-get install --no-install-recommends -y \
    unzip libwebp-dev libzip-dev libfreetype6-dev libjpeg62-turbo-dev libpng-dev libc-client-dev librabbitmq4 \
    mariadb-client supervisor cron \
    && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
    && rm -rf /var/lib/apt/lists/* \
    && rm /etc/cron.daily/*

# Setting PHP properties
ENV PHP_INI_VALUE_DATE_TIMEZONE='UTC' \
    PHP_INI_VALUE_MEMORY_LIMIT=512M \
    PHP_INI_VALUE_UPLOAD_MAX_FILESIZE=512M \
    PHP_INI_VALUE_POST_MAX_FILESIZE=512M \
    PHP_INI_VALUE_MAX_EXECUTION_TIME=300

COPY ./common/php.ini /usr/local/etc/php/php.ini

ENV APACHE_DOCUMENT_ROOT=/var/www/html/docroot

RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
RUN sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf

# Enable Apache Rewrite Module
RUN a2enmod rewrite

COPY ./common/docker-entrypoint.sh /entrypoint.sh
COPY ./common/entrypoint_mautic_web.sh /entrypoint_mautic_web.sh
COPY ./common/entrypoint_mautic_cron.sh /entrypoint_mautic_cron.sh
COPY ./common/entrypoint_mautic_worker.sh /entrypoint_mautic_worker.sh

# Apply necessary permissions
RUN ["chmod", "+x", "/entrypoint.sh", "/entrypoint_mautic_web.sh", "/entrypoint_mautic_cron.sh", "/entrypoint_mautic_worker.sh"]

# Setting worker env vars
ENV DOCKER_MAUTIC_WORKERS_CONSUME_EMAIL=2 \
    DOCKER_MAUTIC_WORKERS_CONSUME_HIT=2 \
    DOCKER_MAUTIC_WORKERS_CONSUME_FAILED=2

COPY ./common/supervisord.conf /etc/supervisor/conf.d/supervisord.conf

# Define Mautic volumes to persist data
VOLUME /var/www/html/config
VOLUME /var/www/html/var/logs
VOLUME /var/www/html/docroot/media

WORKDIR /var/www/html/docroot

ENV DOCKER_MAUTIC_ROLE=mautic_web \
    DOCKER_MAUTIC_RUN_MIGRATIONS=false \
    DOCKER_MAUTIC_LOAD_TEST_DATA=false

ENTRYPOINT ["/entrypoint.sh"]

CMD ["apache2-foreground"]

Docker Compose (if applicable)

We do not use Docker Compose. Instead, the 3 docker containers (web, worker, cron) are deployed using Ansible. Our Mautic setup has been functional for a few years now and has recently been updated to v5.2.5 without issues.

Additional context (Optional)

Please note that I'm not 100% sure this is a bug even though I suspect it is.

It could very well be I'm taking the wrong approach trying to expand the docker image to support the SendGrid API.

Any help or advice is appreciated.

Acknowledgements

• I agree to follow this project's Code of Conduct
• This issue is NOT a security vulnerability. (Security vulnerabilities should be reported via the Security tab.)

[cibero42]

Hey @Insannik ,

Sorry for the delay - could you try doing your Dockerfile based on the official container? Just so that we can start diag this.

Example

[Insannik]

Hey @cibero42,

I've taken a look at your proposed solution but I don't think the use of the Sendgrid API is classified as installing "plugin". According to the docs here, it requires the installation of a "package" using composer.

If I understand correctly, "plugins" are the ones you can install through the frontend (integrations -> plugins). However, the symfony/sendgrid-mailer package needs to be installed in a default installation (not using Docker) by using SSH to connect to the server and running a few commands.

So, I could be wrong but I don't think it can be downloaded and added as a "plugin" in the container. It has to be installed using composer like I tried initially.

Thanks again for the help.

Kind regards,
Insannik

[cibero42]

Hello @Insannik ,

Yes, you're correct. I'll have a better look at your issue next week - I'll check if it is viable to provide this support by default.

[Insannik]

Hi again @cibero42,

I hope we're able to figure something out. In any case, thanks in advance for the help.

Kind regards,
Insannik

[Insannik]

Hi,

I think we may have figured it out. The issue was that the cache wasn't being cleared as I suspected it would've been when a container is launched from scratch. Apparently the cache is stored as part of the file system and is copied to the final container from the builder container because of the following line in the Dockerfile:

COPY --from=builder --chown=www-data:www-data /opt/mautic /var/www/html

I've now changed the Dockerfile I posted above to include the php bin/console cache:clear and php bin/console cache:warmup command after the installation of the sendgrid-mailer package as follows:

# Install Mautic using composer, install Symfony Sengrid Mailer and clean cache. Note that the cache:clear command is required to load the newly installed sendgrid-mailer package.
RUN cd /opt && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer create-project mautic/recommended-project:${MAUTIC_VERSION} mautic --no-interaction && \
    cd /opt/mautic && \
    COMPOSER_ALLOW_SUPERUSER=1 COMPOSER_PROCESS_TIMEOUT=10000 composer require symfony/sendgrid-mailer:* && \
    rm -rf /opt/mautic/var/cache/js && \
    find /opt/mautic/node_modules -mindepth 1 -maxdepth 1 -not \( -name 'jquery' -or -name 'vimeo-froogaloop2' \) | xargs rm -rf && \
    php bin/console cache:clear && \
    php bin/console cache:warmup

After launching the newly built container, I'm able to set the email scheme to sendgrid+api without errors.

I've now requested the team to test their findings. I'll update this issue as soon as I know more but I suspect this will be the fix to our issue.

Kind regards,
Insannik

[cibero42]

Hey @Insannik ,

I'm sorry, but I ended up not having time last week to focus on this.

I've now requested the team to test their findings. I'll update this issue as soon as I know more but I suspect this will be the fix to our issue.

Did they validate it?

[Insannik]

Hey @cibero42,

I haven't heard back from them yet but from testing myself I can only conclude that it seems to be working. I'll update here if I hear anything but it might take a while because of the summer holidays and all that.