Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/release-latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:

strategy:
matrix:
target: [misp-core, misp-modules, misp-core-slim, misp-modules-slim]
target: [core, modules, core-slim, modules-slim]

permissions:
contents: read
Expand All @@ -32,7 +32,7 @@ jobs:
run: |
sed -e '/^[[:space:]]*$/d' -e '/[#@]/d' -e 's/\"//g' -e 's/\(^[^=]*\)=\(.*\)/\1="\2"/' template.env > env.hcl
echo "COMMIT_HASH=`echo '${{ github.sha }}' | cut -c 1-7`" >> "$GITHUB_ENV"
echo "NAMESPACE=ghcr.io/misp/misp-docker" >> "$GITHUB_ENV"
echo "NAMESPACE=ghcr.io/misp" >> "$GITHUB_ENV"

- name: Log in to the container registry
uses: docker/login-action@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-build-latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:

strategy:
matrix:
target: [misp-core, misp-modules, misp-core-slim, misp-modules-slim]
target: [core, modules, core-slim, modules-slim]

steps:
- name: Checkout repository
Expand Down
28 changes: 14 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
A production ready Docker MISP image (formerly hosted at https://github.com/ostefano/docker-misp, now deprecated) loosely based on CoolAcid and DSCO builds, with nearly all logic rewritten and verified for correctness and portability.

Notable features:
- MISP and MISP modules are split into two different Docker images, `misp-core` and `misp-modules`
- MISP and MISP modules are split into two different Docker images, `core` and `modules`
- Docker images are pushed regularly, no build required
- Lightweigth Docker images by using multiple build stages and a slim parent image
- Rely on off the shelf Docker images for Exim4, Redis, and MariaDB
Expand All @@ -17,7 +17,7 @@ Notable features:
- Fix MISP modules loading of faup library
- Fix MISP modules loading of gl library
- Add support for new background job [system](https://github.com/MISP/MISP/blob/2.4/docs/background-jobs-migration-guide.md)
- Add support for building specific MISP and MISP-modules commits
- Add support for building specific MISP and modules commits
- Add automatic configuration of syncservers (see `configure_misp.sh`)
- Add automatic configuration of authentication keys (see `configure_misp.sh`)
- Add direct push of docker images to GitHub Packages
Expand Down Expand Up @@ -51,7 +51,7 @@ The `docker-compose.yml` file allows further configuration settings:
"MYSQL_USER=misp"
"MYSQL_PASSWORD=example" # NOTE: This should be AlphaNum with no Special Chars. Otherwise, edit config files after first run.
"MYSQL_DATABASE=misp"
"MISP_MODULES_FQDN=http://misp-modules" # Set the MISP Modules FQDN, used for Enrichment_services_url/Import_services_url/Export_services_url
"MISP_MODULES_FQDN=http://modules" # Set the MISP Modules FQDN, used for Enrichment_services_url/Import_services_url/Export_services_url
"WORKERS=1" # Legacy variable controlling the number of parallel workers (use variables below instead)
"NUM_WORKERS_DEFAULT=5" # To set the number of default workers
"NUM_WORKERS_PRIO=5" # To set the number of prio workers
Expand All @@ -66,13 +66,13 @@ New options are added on a regular basis.

Set environment variables in .env to configure settings instead of in docker-compose.yml where possible. Setting the variables in .env will allow you to pull updates from Github without issues caused by a modified docker-compose.yml file, should there be an update for it.

Environment variable driven settings are enforced every time the misp-core container starts. This means that if you change the config.php file or database for a setting that has a set environment variable, it will be changed to the environment variable value upon next container start. Empty environment variables may have a safe default which is enforced instead.
Environment variable driven settings are enforced every time the `core` container starts. This means that if you change the config.php file or database for a setting that has a set environment variable, it will be changed to the environment variable value upon next container start. Empty environment variables may have a safe default which is enforced instead.

If you push a change to add or remove an environment variable, please look in "core/files/etc/misp-docker/" for json files with "envars" in the name and adjust there.

#### Unset safe default settings behaviour

The misp-core container has definitions for minimum safe default settings which are set if needed each time the container starts. They will only be set if there is no existing entry in the config.php file or database for these settings. If you specify a custom value for any of these settings it will be respected. See the definitions of these in "core/files/etc/misp-docker" where the filenames contain the word "defaults".
The `core` container has definitions for minimum safe default settings which are set if needed each time the container starts. They will only be set if there is no existing entry in the config.php file or database for these settings. If you specify a custom value for any of these settings it will be respected. See the definitions of these in "core/files/etc/misp-docker" where the filenames contain the word "defaults".

#### Storing system settings in the DB

Expand All @@ -88,7 +88,7 @@ While storing system settings in the DB works as expected most of the time, you

If you are trying to accomplish something and the above behaviours get in the way, please let us know as this is not intended.

To override these behaviours edit the docker-compose.yml file's misp-core volume definitions to enable the "customize_misp.sh" behaviour (see the bottom of the Production section for details). The "customize_misp.sh" script triggers after the above behaviours complete and is an appropriate place to override a setting. It is suggested that you use the "/var/www/MISP/app/cake Admin setSetting" command to override a setting, as this tool is config.php file and database setting aware.
To override these behaviours edit the docker-compose.yml file's `core` volume definitions to enable the "customize_misp.sh" behaviour (see the bottom of the Production section for details). The "customize_misp.sh" script triggers after the above behaviours complete and is an appropriate place to override a setting. It is suggested that you use the "/var/www/MISP/app/cake Admin setSetting" command to override a setting, as this tool is config.php file and database setting aware.

#### Adding a new setting and unsure what files to edit?

Expand Down Expand Up @@ -131,12 +131,12 @@ Using a slow disk as the mounted volume or a volume with high latency like NFS,

## Installing custom root CA certificates

Custom root CA certificates can be mounted under `/usr/local/share/ca-certificates` and will be installed during the `misp-core` container start.
Custom root CA certificates can be mounted under `/usr/local/share/ca-certificates` and will be installed during the `core` container start.

**Note:** It is important to have the .crt extension on the file, otherwise it will not be processed.

```yaml
misp-core:
core:
# ...
volumes:
- "./configs/:/var/www/MISP/app/Config/"
Expand Down Expand Up @@ -197,10 +197,10 @@ The process is *NOT* battle-tested, so it is *NOT* to be followed uncritically.

## Versioning

A GitHub Action builds both `misp-core` and `misp-modules` images automatically and pushes them to the [GitHub Package registry](https://github.com/orgs/MISP/packages). We do not use tags inside the repository; instead we tag images as they are pushed to the registry. For each build, `misp-core` and `misp-modules` images are tagged as follows:
- `misp-core:${commit-sha1}[0:7]` and `misp-modules:${commit-sha1}[0:7]` where `${commit-sha1}` is the commit hash triggering the build
- `misp-core:latest` and `misp-modules:latest` in order to track the latest builds available
- `misp-core:${CORE_TAG}` and `misp-modules:${MODULES_TAG}` reflecting the underlying version of MISP and MISP modules (as specified inside the `template.env` file at build time)
A GitHub Action builds both `core` and `modules` images automatically and pushes them to the [GitHub Package registry](https://github.com/orgs/MISP/packages). We do not use tags inside the repository; instead we tag images as they are pushed to the registry. For each build, `core` and `modules` images are tagged as follows:
- `core:${commit-sha1}[0:7]` and `modules:${commit-sha1}[0:7]` where `${commit-sha1}` is the commit hash triggering the build
- `core:latest` and `modules:latest` in order to track the latest builds available
- `core:${CORE_TAG}` and `modules:${MODULES_TAG}` reflecting the underlying version of MISP and MISP modules (as specified inside the `template.env` file at build time)

## Podman (experimental)

Expand Down Expand Up @@ -248,8 +248,8 @@ Start services:
systemctl --user start mail.service
systemctl --user start db.service
systemctl --user start redis.service
systemctl --user start misp-core.service
systemctl --user start misp-modules.service
systemctl --user start core.service
systemctl --user start modules.service
```

Wait a bit and check your service at `https://<IP>:10443`.
Expand Down
2 changes: 1 addition & 1 deletion core/files/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ export SMTP_FQDN=${SMTP_FQDN:-mail}

export ADMIN_EMAIL=${ADMIN_EMAIL:[email protected]}
export GPG_PASSPHRASE=${GPG_PASSPHRASE:-passphrase}
export MISP_MODULES_FQDN=${MISP_MODULES_FQDN:-http://misp-modules}
export MISP_MODULES_FQDN=${MISP_MODULES_FQDN:-http://modules}
export ATTACHMENTS_DIR=${ATTACHMENTS_DIR:-/var/www/MISP/app/files}

export AUTOCONF_GPG=${AUTOCONF_GPG:-true}
Expand Down
24 changes: 12 additions & 12 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -72,17 +72,17 @@ variable "PHP_VER" {

group "default" {
targets = [
"misp-modules",
"misp-modules-slim",
"misp-core",
"misp-core-slim",
"modules",
"modules-slim",
"core",
"core-slim",
]
}

target "misp-modules" {
target "modules" {
context = "modules/."
dockerfile = "Dockerfile"
tags = flatten(["${NAMESPACE}/misp-modules:latest", "${NAMESPACE}/misp-modules:${COMMIT_HASH}", MODULES_TAG != "" ? ["${NAMESPACE}/misp-modules:${MODULES_TAG}"] : []])
tags = flatten(["${NAMESPACE}/modules:latest", "${NAMESPACE}/modules:${COMMIT_HASH}", MODULES_TAG != "" ? ["${NAMESPACE}/modules:${MODULES_TAG}"] : []])
args = {
"MODULES_TAG": "${MODULES_TAG}",
"MODULES_COMMIT": "${MODULES_COMMIT}",
Expand All @@ -91,10 +91,10 @@ target "misp-modules" {
platforms = "${PLATFORMS}"
}

target "misp-modules-slim" {
target "modules-slim" {
context = "modules/."
dockerfile = "Dockerfile"
tags = flatten(["${NAMESPACE}/misp-modules:latest-slim", "${NAMESPACE}/misp-modules:${COMMIT_HASH}-slim", MODULES_TAG != "" ? ["${NAMESPACE}/misp-modules:${MODULES_TAG}-slim"] : []])
tags = flatten(["${NAMESPACE}/modules:latest-slim", "${NAMESPACE}/modules:${COMMIT_HASH}-slim", MODULES_TAG != "" ? ["${NAMESPACE}/modules:${MODULES_TAG}-slim"] : []])
args = {
"MODULES_TAG": "${MODULES_TAG}",
"MODULES_COMMIT": "${MODULES_COMMIT}",
Expand All @@ -103,10 +103,10 @@ target "misp-modules-slim" {
platforms = "${PLATFORMS}"
}

target "misp-core" {
target "core" {
context = "core/."
dockerfile = "Dockerfile"
tags = flatten(["${NAMESPACE}/misp-core:latest", "${NAMESPACE}/misp-core:${COMMIT_HASH}", CORE_TAG != "" ? ["${NAMESPACE}/misp-core:${CORE_TAG}"] : []])
tags = flatten(["${NAMESPACE}/core:latest", "${NAMESPACE}/core:${COMMIT_HASH}", CORE_TAG != "" ? ["${NAMESPACE}/core:${CORE_TAG}"] : []])
args = {
"CORE_TAG": "${CORE_TAG}",
"CORE_COMMIT": "${CORE_COMMIT}",
Expand All @@ -125,10 +125,10 @@ target "misp-core" {
platforms = "${PLATFORMS}"
}

target "misp-core-slim" {
target "core-slim" {
context = "core/."
dockerfile = "Dockerfile"
tags = flatten(["${NAMESPACE}/misp-core:latest-slim", "${NAMESPACE}/misp-core:${COMMIT_HASH}-slim", CORE_TAG != "" ? ["${NAMESPACE}/misp-core:${CORE_TAG}-slim"] : []])
tags = flatten(["${NAMESPACE}/core:latest-slim", "${NAMESPACE}/core:${COMMIT_HASH}-slim", CORE_TAG != "" ? ["${NAMESPACE}/core:${CORE_TAG}-slim"] : []])
args = {
"CORE_TAG": "${CORE_TAG}",
"CORE_COMMIT": "${CORE_COMMIT}",
Expand Down
12 changes: 7 additions & 5 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
name: misp

services:
# This is capable to relay via gmail, Amazon SES, or generic relays
# See: https://hub.docker.com/r/ixdotai/smtp
Expand Down Expand Up @@ -51,8 +53,8 @@ services:
start_period: 30s
start_interval: 5s

misp-core:
image: ghcr.io/misp/misp-docker/misp-core:${CORE_RUNNING_TAG:-latest}
core:
image: ghcr.io/misp/core:${CORE_RUNNING_TAG:-latest}
cap_add:
- AUDIT_WRITE
build:
Expand All @@ -77,7 +79,7 @@ services:
condition: service_healthy
db:
condition: service_healthy
misp-modules:
modules:
condition: service_healthy
healthcheck:
test: curl -ks ${BASE_URL:-https://localhost}/users/heartbeat > /dev/null || exit 1
Expand Down Expand Up @@ -246,8 +248,8 @@ services:
- "X_FRAME_OPTIONS=${X_FRAME_OPTIONS}"
- "CONTENT_SECURITY_POLICY=${CONTENT_SECURITY_POLICY}"

misp-modules:
image: ghcr.io/misp/misp-docker/misp-modules:${MODULES_RUNNING_TAG:-latest}
modules:
image: ghcr.io/misp/modules:${MODULES_RUNNING_TAG:-latest}
build:
context: modules/.
args:
Expand Down
6 changes: 3 additions & 3 deletions experimental/podman-systemd/misp-core.container
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ Requires=redis.service

[Container]
AutoUpdate=registry
ContainerName=misp-core
Image=ghcr.io/misp/misp-docker/misp-core:latest
ContainerName=core
Image=ghcr.io/misp/core:latest
PublishPort=10443:443
Network=misp-net
PodmanArgs=--network-alias misp-core
PodmanArgs=--network-alias core
Volume=conf:/var/www/MISP/app/Config/
Volume=logs:/var/www/MISP/app/tmp/logs/
Volume=files:/var/www/MISP/app/files/
Expand Down
6 changes: 3 additions & 3 deletions experimental/podman-systemd/misp-modules.container
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ Requires=redis.service

[Container]
AutoUpdate=registry
ContainerName=misp-modules
Image=ghcr.io/misp/misp-docker/misp-modules:latest
ContainerName=modules
Image=ghcr.io/misp/modules:latest
Network=misp-net
PodmanArgs=--network-alias misp-modules
PodmanArgs=--network-alias modules
EnvironmentFile=vars.env

[Service]
Expand Down
10 changes: 5 additions & 5 deletions modules/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@ EOF

RUN <<-EOF
if [ ! -z ${MODULES_COMMIT} ]; then
git clone https://github.com/MISP/misp-modules.git /srv/misp-modules && cd /srv/misp-modules && git checkout ${MODULES_COMMIT}
git clone https://github.com/MISP/modules.git /srv/modules && cd /srv/modules && git checkout ${MODULES_COMMIT}
else
git clone --branch ${MODULES_TAG} --depth 1 https://github.com/MISP/misp-modules.git /srv/misp-modules
git clone --branch ${MODULES_TAG} --depth 1 https://github.com/MISP/modules.git /srv/modules
fi
EOF

WORKDIR /srv/misp-modules
WORKDIR /srv/modules
RUN pip install poetry
RUN sed -i "s/^requires-python = .*/requires-python = \"$(python -c 'import platform; print(platform.python_version())')\"/" pyproject.toml
RUN poetry lock
Expand All @@ -53,7 +53,7 @@ EOF
RUN poetry build --output /wheels/

WORKDIR /srv/
RUN rm -rf /srv/misp-modules
RUN rm -rf /srv/modules


FROM "${DOCKER_HUB_PROXY}python:3.12-slim-bookworm"
Expand All @@ -79,4 +79,4 @@ EOF
RUN pip uninstall -y pip
RUN mkdir -p /custom/{action_mod,expansion,export_mod,import_mod}

ENTRYPOINT [ "/usr/local/bin/misp-modules", "-l", "0.0.0.0", "-c", "/custom/"]
ENTRYPOINT [ "/usr/local/bin/modules", "-l", "0.0.0.0", "-c", "/custom/"]