Populate by git

Configuration - WIP

Repository Settings

• consul.git.uri: Git repository URI (e.g., https://github.com/user/repo.git) (Required)
• consul.git.branch: Branch to clone and track. Defaults to main (Optional)
• consul.git.local-path: Local directory path where the repository will be cloned. Defaults to /tmp (Optional)
• consul.git.ssl-verify: Whether to verify SSL certificates. Defaults to true. Set to false for self-signed certificates (Optional)

Authentication

Use one of the following authentication methods:

Token Authentication (Recommended)

• consul.git.token: Personal access token for authentication

Basic Authentication

• consul.git.username: Username for basic authentication
• consul.git.password: Password for basic authentication

File Import Settings

• consul.git.files.root-path: Path within the repository to root directory (Required)
• consul.git.files.target: Subdirectory used to override root configurations (Required)
• consul.git.files.format: (YAML | JSON | PROPERTIES) Supported format of the files (Required)

See Populate by files for directory structure details.

Polling (Server Mode)

Polling allows the server to periodically check for repository changes and automatically update Consul.

• consul.git.polling.enabled: Whether to enable polling. Defaults to false (Optional)
• consul.git.polling.delay: Delay between polling attempts. Defaults to 5m (Optional)

Webhook

Webhooks allow git providers to notify the server when changes are pushed, triggering an immediate update.

• consul.git.webhook.type: Git provider type (GITHUB | BITBUCKET | CUSTOM). Defaults to GITHUB (Required if using webhooks)
• consul.git.webhook.secret: Secret key for webhook signature verification (Optional but recommended)

Supported Providers

Provider	Type	Event Header	Signature Header
GitHub	GITHUB	X-GitHub-Event	X-Hub-Signature-256
Bitbucket	BITBUCKET	X-Event-Key	X-Hub-Signature
Custom	CUSTOM	User-defined	User-defined

Custom Webhook Handler

For providers other than GitHub or Bitbucket, set consul.git.webhook.type=CUSTOM and provide your own WebhookPayloadHandler bean:

import com.frogdevelopment.consul.populate.git.endpoint.handlers.webhook.WebhookPayloadHandler;

@Singleton
public class GitLabWebhookPayloadHandler implements WebhookPayloadHandler {

    @Override
    public String getEventHeaderKey() {
        return "X-Gitlab-Event";
    }

    @Override
    public String getExpectedEventValue() {
        return "Push Hook";
    }

    @Override
    public String getSignatureHeaderKey() {
        return "X-Gitlab-Token";
    }

    @Override
    public Optional<String> extractBranchName(JsonNode payload) {
        var refNode = payload.get("ref");
        if (refNode == null || refNode.isNull()) {
            return Optional.empty();
        }
        return Optional.of(refNode.asText());
    }
}

If type=CUSTOM is configured but no handler bean is provided, the webhook endpoint returns 501 Not Implemented.

Cleanup

• consul.git.clean-up: Whether to delete the cloned repository on shutdown. Defaults to false (Optional)

Management Endpoints

The server exposes management endpoints under /git:

Endpoint	Method	Description
/git/force-pull	POST	Manually trigger a pull and update
/git/webhook	POST	Receive webhook events from git provider
/git/toggle-poll	POST	Toggle polling on/off
/git	GET	Get current git status and summary

Usage

Server Mode with Polling

services:
  consul-populate-server:
    image: frogdevelopment/consul-populate-server:{version}
    environment:
      CONSUL_HOST: consul
      CONSUL_PORT: 8500
      CONSUL_KV_PREFIX: config/frog/my_app
      CONSUL_GIT_URI: https://github.com/myorg/config-repo.git
      CONSUL_GIT_TOKEN: ${GIT_TOKEN}
      CONSUL_GIT_BRANCH: main
      CONSUL_GIT_FILES_FORMAT: YAML
      CONSUL_GIT_FILES_TARGET: dev
      CONSUL_GIT_FILES_ROOT_PATH: configurations
      CONSUL_GIT_POLLING_ENABLED: true
      CONSUL_GIT_POLLING_DELAY: 5m

Server Mode with GitHub Webhook

services:
  consul-populate-server:
    image: frogdevelopment/consul-populate-server:{version}
    ports:
      - "8080:8080"
    environment:
      CONSUL_HOST: consul
      CONSUL_PORT: 8500
      CONSUL_KV_PREFIX: config/frog/my_app
      CONSUL_GIT_URI: https://github.com/myorg/config-repo.git
      CONSUL_GIT_TOKEN: ${GIT_TOKEN}
      CONSUL_GIT_BRANCH: main
      CONSUL_GIT_FILES_FORMAT: YAML
      CONSUL_GIT_FILES_TARGET: dev
      CONSUL_GIT_FILES_ROOT_PATH: configurations
      CONSUL_GIT_WEBHOOK_TYPE: GITHUB
      CONSUL_GIT_WEBHOOK_SECRET: ${WEBHOOK_SECRET}

Configure your GitHub repository webhook to:

• Payload URL: http://your-server:8080/git/webhook
• Content type: application/json
• Secret: Same as CONSUL_GIT_WEBHOOK_SECRET
• Events: Just the push event

CLI Mode (One-shot)

For CI/CD pipelines, use the CLI with git configuration:

java -jar consul-populate-cli-{version}.jar \
    --consul.host=localhost \
    --consul.port=8500 \
    --consul.kv.prefix=config/frog/my_app \
    --consul.git.uri=https://github.com/myorg/config-repo.git \
    --consul.git.token=${GIT_TOKEN} \
    --consul.git.branch=main \
    --consul.git.files.format=YAML \
    --consul.git.files.target=dev \
    --consul.git.files.root-path=configurations