Syncthing with Elastic monitoring

Prototype with two Syncthing nodes and Elastic to monitor them. Based on this getting started from Elastic.

Getting started

Starting services

Since Metricbeat needs the API keys of the Syncthing instances, we first start those before starting the Elasticsearch stack.

# start syncthing instances
docker compose up syncthing01 syncthing02

# getting the API keys
SYNCTHING01_API_KEY="$(docker compose exec syncthing01 syncthing cli config gui apikey get)"
SYNCTHING02_API_KEY="$(docker compose exec syncthing02 syncthing cli config gui apikey get)"
sed -i "/^SYNCTHING01_API_KEY=.*/ s//SYNCTHING01_API_KEY=$SYNCTHING01_API_KEY/" .env
sed -i "/^SYNCTHING02_API_KEY=.*/ s//SYNCTHING01_API_KEY=$SYNCTHING02_API_KEY/" .env

# start elasticsearch stack
docker compose up es01 kibana metricbeat01

Use the Rust script to monitor the synchronization

# copy the test root certificate of elastic
docker cp syncthing-elastic-es01-1:/usr/share/elasticsearch/config/certs/ca/ca.crt /tmp/.
# run rust script
cd rust-cli; cargo run

Sample output

syncthing01:
  localFiles   : 277
  localBytes   : 27400019.0
  state        : "idle"
  stateChanged : "2025-01-14T16:27:58Z"
  error        : ""

syncthing02:
  localFiles   : 277
  localBytes   : 27400019.0
  state        : "idle"
  stateChanged : "2025-01-14T16:28:09Z"
  error        : ""

Troubleshooting

# test whether the certificate was well copied
curl --cacert /tmp/ca.crt -u elastic:changeme https://localhost:9200
# test a call to the syncthing API
curl -s -H "X-API-Key: $SYNCTHING01_API_KEY" "http://127.0.0.1:8384/rest/db/status?folder=default"
# doc: https://docs.syncthing.net/rest/db-status-get.html

Script to simulate some file activity

for i in $(seq 1 10);
do
    echo "$i"
    for j in $(seq 1 100);
    do
        echo " $j"
        sleep 1
        head -c 100000 /dev/urandom > "syncthing01_data/Sync/dummyfile$i$j"
    done
    sleep 20
done

# delete the files created
rm syncthing01_data/Sync/dummyfile*

Project state

• two instances
• make them sync something
• feed folder status
• add some data/file periodically to provoke syncs
• get syncthing api key and set it in metricbeat config
• kibana dashboard with overview
• add connection between instances via cli

Saving kibana dashboard:

curl -X GET -u elastic:changeme http://localhost:5601/api/saved_objects/dashboard/33396680-d293-11ef-928f-f70dc0e26cc3