new docker compose deployment running idle, no workload, fills up hard drive disk to 100% full capacity, consuming using all available disk space.

[chrislawso]

Using TrustGraph Configuration Builder docker compose deployment with no modifications to compose yaml, after several days the host machine's hard disk drive fills to 100% full capacity. some applications or processes related to the trustgraph containers eagerly consume all available disk space, when providing more disk space trustgraph continues to grow consuming actual disk space seemingly infinite.

iotop shows a running process ie
3562426 be/4 10000 113.07 M 0.00 B java -Dlog4j.shutdownHookEnabled=false -cp /pulsar/c~BrokerStarter --broker-conf /pulsar/conf/broker.conf

Performing a docker compose stop stops all related containers and releases hundreds of gigs of disk space. Need to perform further investigation into which trustgraph container is writing profusely and consuming all available disk space.

container images:
trustgraph/ddg-mcp-server:0.1.0 3f7e1e80557b 221MB 53.5MB
trustgraph/trustgraph-flow:2.4.11 6a8a70bc262f 1.37GB 307MB
trustgraph/trustgraph-unstructured:2.4.11 ce7cfb22149f 4.07GB 941MB
trustgraph/workbench-ui:1.8.2 4dc9ce3943de 119MB 29.8MB
apachepulsar/pulsar:4.1.0 188277e85d72 772MB 313MB
cassandra:5.0.6 b494afb347bf 544MB 172MB
qdrant/qdrant:v1.15.4 6ac4807063bb 259MB 70.8MB
qdrant/qdrant:v1.16.0 1005201498cf 288MB 79.4MB

How to mitigate this issue?

[cybermaggedon]

Those container image sizes look about right, so it's container volume size I guess.

TrustGraph 2.4.11 is not stable at the moment. 2.4.20 is about 'alpha' quality. Probably fine for test/eval but the e2e test changes are at 70% feature coverage.

I think there were pub/sub disconnect problems early on and there is still a little too much log noise, it's possible the loki storage is full of spurious noise.

[cybermaggedon]

2.3 is the last stable version, FYI

[Matheus-Garbelini]

Hi, @cybermaggedon is there any way to run this latest repo directly in docker compose? Currently i can't get this project to run default recommended configuration from the npx @trustgraph/config tool

[cybermaggedon]

@Matheus-Garbelini yes that's possible. You have to build the containers yourself if you want to go straight from a repo.

[cybermaggedon]

The process would be something like...

• Build / create a docker compose file from config-ui or using the template generator
• Run make containers VERSION=x.y.z from the repo

What's the problem you're seeing?

[Matheus-Garbelini]

But does it work out of the box or requires changing the docker compose file or env variables besides API keys? Will try build and report back if I get any errors

[cybermaggedon]

It will work out of the box. The docker compose file changes in accordance with settings you provide e.g. which LLM you're using, so there isn't a standard docker compose.

Can you describe what you're seeing in terms of errors?