fix: address review feedback on indexer integration

Fix critical network ID capitalization bugs (preview/qanet/testnet-02), standardize on .secret file extension for all sensitive data, consolidate all services into single compose.yml with Docker Compose profiles, and remove legacy compose files as requested.

Author: cosmir17

.envrc

@@ -22,10 +22,10 @@ export POSTGRES_USER="postgres"
 
 # A random password is used for your safety. Docker (but not podman) exposes
 # ports to the internet by default. This needs to be unguessable.
-if [ ! -f postgres.password ]; then
-    uuidgen | tr -d '-' | head -c 16 > postgres.password
+if [ ! -f postgres.secret ]; then
+    uuidgen | tr -d '-' | head -c 16 > postgres.secret
 fi
-export POSTGRES_PASSWORD="$(cat ./postgres.password)"
+export POSTGRES_PASSWORD="$(cat ./postgres.secret)"
 
 export POSTGRES_DB="cexplorer"
 
@@ -37,11 +37,11 @@ export APPEND_ARGS="--allow-private-ip --pool-limit 10 --trie-cache-size 0 --pro
 
 # Node Key:
 # Generate a unique node key for P2P networking
-if [ ! -f midnight-node.privatekey ]; then
+if [ ! -f midnight-node.secret ]; then
   # Generate node key using parity/subkey
-  DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run --rm docker.io/parity/subkey:latest generate-node-key > midnight-node.privatekey
+  DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run --rm docker.io/parity/subkey:latest generate-node-key > midnight-node.secret
 fi
-export NODE_KEY="$(cat ./midnight-node.privatekey)"
+export NODE_KEY="$(cat ./midnight-node.secret)"
 
 # Indexer Values:
 if [ ! -f indexer.secret ]; then

.envrc.preview

@@ -1,7 +1,7 @@
 # overrides for preview
 export MIDNIGHT_NODE_IMAGE="ghcr.io/midnight-ntwrk/midnight-node:0.17.0-rc.4"
 export INDEXER_IMAGE="ghcr.io/midnight-ntwrk/indexer-standalone:3.0.0-alpha.5"
-export INDEXER_NETWORK_ID="Preview"
+export INDEXER_NETWORK_ID="preview"
 
 # Mock mode configuration
 export MOCK_REGISTRATIONS_FILE="res/mock-bridge-data/default-registrations.json"

.envrc.qanet

@@ -1,7 +1,7 @@
 # overrides for qanet
 export MIDNIGHT_NODE_IMAGE="ghcr.io/midnight-ntwrk/midnight-node:0.12.0-rc.3"
 export INDEXER_IMAGE="ghcr.io/midnight-ntwrk/indexer-standalone:2.1.4"
-export INDEXER_NETWORK_ID="QANet"
+export INDEXER_NETWORK_ID="qanet"
 
 # Mock mode configuration
 export MOCK_REGISTRATIONS_FILE="res/mock-bridge-data/qanet-mock.json"

.envrc.testnet-02

@@ -1,7 +1,7 @@
 # overrides for testnet-02
 export MIDNIGHT_NODE_IMAGE="midnightnetwork/midnight-node:0.12.1"
 export INDEXER_IMAGE="ghcr.io/midnight-ntwrk/indexer-standalone:2.1.4"
-export INDEXER_NETWORK_ID="TestNet"
+export INDEXER_NETWORK_ID="testnet"
 
 # Mock mode configuration
 export MOCK_REGISTRATIONS_FILE="res/mock-bridge-data/testnet-02-mock.json"

.github/workflows/ci.yaml

@@ -27,9 +27,13 @@ jobs:
       - name: Check compose files
         run: |
           source ./.envrc
-          docker compose -f compose.yml config -q
-          docker compose -f compose-partner-chains.yml config -q
-          docker compose -f proof-server.yml config -q
+          # Test main compose file
+          docker compose config -q
+          # Test profiles
+          docker compose --profile cardano config -q
+          docker compose --profile ogmios config -q
+          docker compose --profile proof-server config -q
+          docker compose --profile cardano --profile ogmios --profile proof-server config -q
 
       - name: Run Super-linter
         uses: github/super-linter@b807e99ddd37e444d189cfd2c2ca1274d8ae8ef1  #v7

.gitignore

@@ -1,6 +1,4 @@
 /cardano-data/*
 /data
-*.privatekey
-*.password
 *.secret
 ogmios_client.log

README.md

@@ -21,34 +21,41 @@ This allows for easy orchestration of the Midnight Node service.
 
 4. Run `docker-compose up`
 
-The `.envrc` file will automatically create a random private key and save it as `midnight-node.privatekey` and a random secret for the indexer saved as `indexer.secret`.
+The `.envrc` file will automatically create random secrets and save them as `postgres.secret` (database password), `midnight-node.secret` (node private key), and `indexer.secret` (indexer app secret).
 
-The default `compose.yml` now includes both the Midnight Node and Indexer Standalone, providing a GraphQL API at `http://localhost:8088`.
+All services are defined in a single `compose.yml` file. Use Docker Compose profiles to control which services run:
 
-Choose which compose files to use:
+**Available profiles:**
 
-- `compose.yml` for Midnight Node + Indexer
-- `compose-partner-chains.yml` for Cardano + DB Sync
-- `proof-server.yml` for Local Proof Server
+- (no profile) - Midnight Node only
+- `cardano` - Adds Cardano stack (cardano-node, postgres, cardano-db-sync) and Indexer with GraphQL API at `http://localhost:8088`
+- `ogmios` - Adds Ogmios service at `http://localhost:1337`
+- `proof-server` - Adds local Proof Server at `http://localhost:6300`
 
-One can use one or multiple compose files at once.
+**Usage examples:**
 
-For example, to run the Midnight Node, you can do:
+Run Midnight Node only:
 
 ```shell
 docker compose up -d
 ```
 
-or to run the Midnight Node and Cardano DB Sync, you can do:
+Run Midnight Node + Cardano stack + Indexer:
 
 ```shell
-docker compose -f ./compose-partner-chains.yml -f ./compose.yml up -d
+docker compose --profile cardano up -d
 ```
 
-or to run the Midnight Node, Cardano DB Sync and a local Proof Server, you can do:
+Run with Cardano stack + Ogmios:
 
 ```shell
-docker compose -f ./compose-partner-chains.yml -f ./compose.yml -f ./proof-server.yml up -d
+docker compose --profile cardano --profile ogmios up -d
+```
+
+Run everything (Cardano + Ogmios + Proof Server):
+
+```shell
+docker compose --profile cardano --profile ogmios --profile proof-server up -d
 ```
 
 🚀 That's it.
@@ -65,8 +72,8 @@ If you're using `midnight-node smartcontract` or `midnight-node wizards` that ne
 To restart from fresh, run:
 
 ```sh
-docker compose -f ./compose-partner-chains.yml -f ./compose.yml -f ./proof-server.yml down -v
-docker compose -f ./compose-partner-chains.yml -f ./compose.yml -f ./proof-server.yml kill
+docker compose --profile cardano --profile ogmios --profile proof-server down -v
+docker compose --profile cardano --profile ogmios --profile proof-server kill
 rm -R ./cardano-data
 docker volume rm midnight-node-docker_midnight-data-testnet
 ```

compose-partner-chains.yml

@@ -19,6 +19,7 @@ volumes:
   postgres-data: {}
   cardano-ipc: {}
   db-sync-data: {}
+  ogmios-data: {}
 
 services:
   postgres:
@@ -34,7 +35,7 @@ services:
     volumes:
       - postgres-data:/var/lib/postgresql/data
     ports:
-      - "${POSTGRES_PORT}:5432"
+      - "${POSTGRES_PORT}:${POSTGRES_PORT}"
     healthcheck:
       test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
       interval: 5s
@@ -127,3 +128,34 @@ services:
         condition: service_healthy
     volumes:
       - indexer-data:/data
+
+  cardano-ogmios:
+    profiles: ["ogmios"]
+    image: cardanosolutions/ogmios:v6.11.0
+    platform: linux/amd64
+    container_name: cardano-ogmios
+    restart: unless-stopped
+    environment:
+      - DATA_DIR=/data
+    ports:
+      - "1337:1337"
+    volumes:
+      - cardano-ipc:/ipc
+      - ogmios-data:/data
+    command:
+      - --node-socket
+      - /ipc/node.socket
+      - --node-config
+      - /config/${CARDANO_NETWORK}/cardano-node/config.json
+      - --host
+      - 0.0.0.0
+
+  proof-server:
+    profiles: ["proof-server"]
+    image: midnightnetwork/proof-server:4.0.0
+    container_name: proof-server
+    ports:
+      - "6300:6300"
+    command: "'midnight-proof-server --network testnet'"
+    healthcheck:
+      test: ["CMD", "/bin/bash", "-c", ":> /dev/tcp/127.0.0.1/6300 || exit 1"]