Skip to content

HugeGraph Distributed (pd‐store) Version Deployment Guide‐EN

Jump to bottom
YangJiaqi edited this page Mar 10, 2025 · 5 revisions

Note: You can refer to the [distributed cluster]configuration in the Docker-Compose repository for similar setup.

Prerequisites

  • Linux, macOS (Windows environment has not been tested yet)
  • Java version ≥ 11
  • Maven version ≥ 3.5.0

Build and Start from Source

  1. Clone the master branch of the source code
git clone https://github.com/apache/hugegraph.git
  1. Build in the project root directory
cd hugegraph
mvn clean install -DskipTests=true

If the build is successful, the build artifacts for PD, Store, and Server modules will be stored in:

  • PD: hugegraph-pd/apache-hugegraph-pd-incubating-1.5.0
  • Store: hugegraph-store/apache-hugegraph-store-incubating-1.5.0
  • Server: hugegraph-server/apache-hugegraph-server-incubating-1.5.0 
.
├── hugegraph-pd
│  ├── apache-hugegraph-pd-incubating-1.5.0
│  └── apache-hugegraph-pd-incubating-1.5.0.tar.gz
├── hugegraph-server
│  ├── apache-hugegraph-server-incubating-1.5.0
│  └── apache-hugegraph-server-incubating-1.5.0.tar.gz
└─── hugegraph-store
   ├── apache-hugegraph-store-incubating-1.5.0
   └── apache-hugegraph-store-incubating-1.5.0.tar.gz

These paths will be used as the working directories when running PD, Store, and Server.

  1. Start PD
cd hugegraph-pd/apache-hugegraph-pd-incubating-1.5.0
./bin/start-hugegraph-pd.sh

If the startup is successful, you can find the following log in hugegraph-pd/apache-hugegraph-pd-incubating-1.5.0/logs/hugegraph-pd-stdout.log:

2024-04-08 15:15:45 [main] [INFO] o.a.h.p.b.HugePDServer - Started HugePDServer in 3.879 seconds (JVM running for 5.149)
  1. Start Store
cd hugegraph-store/apache-hugegraph-store-incubating-1.5.0
./bin/start-hugegraph-store.sh

If the startup is successful, you can find the following log in hugegraph-store/apache-hugegraph-store-incubating-1.5.0/logs/hugegraph-store-server.log:

2024-04-08 15:16:29 [main] [INFO] o.a.h.s.n.StoreNodeApplication - Started StoreNodeApplication in 4.794 seconds (JVM running for 6.21)
  1. Start Server
cd hugegraph-server/apache-hugegraph-server-incubating-1.5.0
./bin/start-hugegraph.sh -p true

Passing -p true imports the example graph, as described in:

https://hugegraph.apache.org/docs/quickstart/hugegraph-server/#517-create-an-example-graph-when-startup

Simple verification:

> curl http://localhost:8080/graphs
{"graphs":["hugegraph"]}
  1. Stop Server, Store, and PD

In order, execute the following commands in the respective directories:

./bin/stop-hugegraph.sh
./bin/stop-hugegraph-store.sh
./bin/stop-hugegraph-pd.sh

Start Using Downloaded tar Packages

Download the PD, Store, and Server tar packages from [this page] and extract them. After extraction, you will get the following folders:

  • apache-hugegraph-pd-incubating-1.5.0
  • apache-hugegraph-store-incubating-1.5.0
  • apache-hugegraph-server-incubating-1.5.0

The subsequent process is the same as described above.

Multi-Node Configuration Reference

  • 3 PD nodes
    • Raft ports: 8610, 8611, 8612
    • RPC ports: 8686, 8687, 8688
    • REST ports: 8620, 8621, 8622
  • 3 Store nodes
    • Raft ports: 8510, 8511, 8512
    • RPC ports: 8500, 8501, 8502
    • REST ports: 8520, 8521, 8522
  • 3 Server nodes (disable auth + use distributed scheduler)
    • REST ports: 8081, 8082, 8083
    • RPC ports: 8091, 8092, 8093
    • Gremlin ports: 8181, 8182, 8183

Server Configuration

hugegraph.properties

backend=hstore
serializer=binary
task.scheduler_type=distributed

# PD service addresses, multiple addresses separated by commas ⚠️ Use the PD RPC ports
pd.peers=127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

rest-server.properties

Each server has its own configuration:

restserver.url=http://127.0.0.1:8081
gremlinserver.url=http://127.0.0.1:8181

rpc.server_host=127.0.0.1
rpc.server_port=8091

server.id=server-1
server.role=master
restserver.url=http://127.0.0.1:8082
gremlinserver.url=http://127.0.0.1:8182

rpc.server_host=127.0.0.1
rpc.server_port=8092

server.id=server-2
server.role=worker
restserver.url=http://127.0.0.1:8083
gremlinserver.url=http://127.0.0.1:8183

rpc.server_host=127.0.0.1
rpc.server_port=8093

server.id=server-3
server.role=worker

gremlin-server.yaml

host: 127.0.0.1
port: 8181
host: 127.0.0.1
port: 8182
host: 127.0.0.1
port: 8183

[Download multi-server.zip for complete Server node configuration files.

PD Configuration

Modify application.yml for each PD node. Example:

spring:
  application:
    name: hugegraph-pd

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

logging:
  config: 'file:./conf/log4j2.xml'
license:
  verify-path: ./conf/verify-license.json
  license-path: ./conf/hugegraph.license
grpc:
  # 集群模式
  port: 8686 # ⚠️本地测试要配不同端口
  host: 127.0.0.1 

server:
  # rest 服务端口号
  port: 8620 # ⚠️本地测试要配不同端口

pd:
  # 存储路径
  data-path: ./pd_data # ⚠️本地测试要配不同路径
  # 自动扩容的检查周期，定时检查每个 store 的分区数量，自动进行分区数量平衡
  patrol-interval: 1800
  # 初始 store 列表，在列表内的 store 自动激活
  initial-store-count: 1
  # grpc IP:grpc port 
  # store 的配置信息
  initial-store-list: 127.0.0.1:8500,127.0.0.1:8501,127.0.0.1:8502

raft:
  # 集群模式
  address: 127.0.0.1:8610 # ⚠️本地测试要配不同端口
  peers-list: 127.0.0.1:8610,127.0.0.1:8611,127.0.0.1:8612

store:
  # store 下线时间。超过该时间，认为 store 永久不可用，分配副本到其他机器，单位秒
  max-down-time: 172800
  # 是否开启 store 监控数据存储
  monitor_data_enabled: true
  # 监控数据的间隔，minute (默认), hour, second
  # default: 1 min * 1 day = 1440
  monitor_data_interval: 1 minute
  # 监控数据的保留时间 1 天; day, month, year
  monitor_data_retention: 1 day
  initial-store-count: 1

partition:
  # 默认每个分区副本数
  default-shard-count: 1
  # 默认每机器最大副本数，初始分区数 = store-max-shard-count * store-number / default-shard-count
  store-max-shard-count: 12

[Download multi-pd.zip](https://github.com/hugegraph/hugegraph/releases/download/pd-store-tmp/multi-pd.zip) for full PD node configuration files.

Store Configuration

Modify application.yml for each Store node. Example:

pdserver:
  # pd 服务地址，多个 pd 地址用逗号分割 ⚠️配置 pd 的 rpc 端口
  address: 127.0.0.1:8686,127.0.0.1:8687,127.0.0.1:8688

management:
  metrics:
    export:
      prometheus:
        enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

grpc:
  # grpc 的服务地址
  host: 127.0.0.1
  port: 8500 # ⚠️本地测试要配不同端口
  netty-server:
    max-inbound-message-size: 1000MB
raft:
  # raft 缓存队列大小
  disruptorBufferSize: 1024
  address: 127.0.0.1:8510 # ⚠️本地测试要配不同端口
  max-log-file-size: 600000000000
  # 快照生成时间间隔，单位秒
  snapshotInterval: 1800
server:
  # rest 服务地址
  port: 8520 # ⚠️本地测试要配不同端口

app:
  # 存储路径，支持多个路径，逗号分割
  data-path: ./storage # ⚠️本地测试要配不同路径
  #raft-path: ./storage

spring:
  application:
    name: store-node-grpc-server
  profiles:
    active: default
    include: pd

logging:
  config: 'file:./conf/log4j2.xml'
  level:
    root: info

[Download multi-store.zip]for full Store node configuration files.

Documentation license here.

Clone this wiki locally