Skip to content

Commit f92c5a4

Browse files
koi2000imbajin
andauthored
docs(pd): update test commands and improve documentation clarity (#2893)
* docs(pd): update test commands and improve documentation clarity * Update README.md --------- Co-authored-by: imbajin <jin@apache.org>
1 parent 126885d commit f92c5a4

File tree

2 files changed

+7
-244
lines changed

2 files changed

+7
-244
lines changed

hugegraph-pd/README.md

Lines changed: 1 addition & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,6 @@
33
[![License](https://img.shields.io/badge/license-Apache%202-0E78BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
44
[![Version](https://img.shields.io/badge/version-1.7.0-blue)](https://github.com/apache/hugegraph)
55

6-
> **Note**: From revision 1.5.0, the HugeGraph-PD code has been adapted to this location.
7-
86
## Overview
97

108
HugeGraph PD (Placement Driver) is a meta server that provides cluster management and coordination services for HugeGraph distributed deployments. It serves as the central control plane responsible for:
@@ -15,7 +13,7 @@ HugeGraph PD (Placement Driver) is a meta server that provides cluster managemen
1513
- **Node Scheduling**: Intelligent scheduling and load balancing of graph operations
1614
- **Health Monitoring**: Continuous health checks and failure detection via heartbeat mechanism
1715

18-
PD uses [Apache JRaft](https://github.com/sofastack/sofa-jraft) for Raft consensus and RocksDB for persistent metadata storage, ensuring high availability and consistency in distributed environments.
16+
PD uses [SOFA-jraft](https://github.com/sofastack/sofa-jraft) for Raft consensus and RocksDB for persistent metadata storage, ensuring high availability and consistency in distributed environments.
1917

2018
## Architecture
2119

@@ -256,21 +254,9 @@ PD exposes metrics via REST API at:
256254

257255
## Community
258256

259-
- **Website**: https://hugegraph.apache.org
260257
- **Documentation**: https://hugegraph.apache.org/docs/
261258
- **GitHub**: https://github.com/apache/hugegraph
262-
- **Mailing List**: dev@hugegraph.apache.org
263259

264260
## Contributing
265261

266262
Contributions are welcome! Please read our [Development Guide](docs/development.md) and follow the Apache HugeGraph contribution guidelines.
267-
268-
## License
269-
270-
HugeGraph PD is licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
271-
272-
---
273-
274-
**Status**: BETA (from v1.5.0+)
275-
276-
For questions or issues, please contact the HugeGraph community via GitHub issues or mailing list.

hugegraph-pd/docs/development.md

Lines changed: 6 additions & 229 deletions
Original file line numberDiff line numberDiff line change
@@ -181,26 +181,26 @@ mvn test jacoco:report
181181

182182
```bash
183183
# Core module tests
184-
mvn test -pl hg-pd-test -am -P pd-core-test
184+
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-core-test
185185

186186
# Client module tests
187-
mvn test -pl hg-pd-test -am -P pd-client-test
187+
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-client-test
188188

189189
# Common module tests
190-
mvn test -pl hg-pd-test -am -P pd-common-test
190+
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-common-test
191191

192192
# REST API tests
193-
mvn test -pl hg-pd-test -am -P pd-rest-test
193+
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-rest-test
194194
```
195195

196196
#### Single Test Class
197197

198198
```bash
199199
# Run specific test class
200-
mvn test -pl hg-pd-test -am -Dtest=PartitionServiceTest
200+
mvn -pl hugegraph-pd/hg-pd-test test -Dtest=PartitionServiceTest -DfailIfNoTests=false
201201

202202
# Run specific test method
203-
mvn test -pl hg-pd-test -am -Dtest=PartitionServiceTest#testSplitPartition
203+
mvn -pl hugegraph-pd/hg-pd-test test -Dtest=PartitionServiceTest#testSplitPartition -DfailIfNoTests=false
204204
```
205205

206206
#### Test from IDE
@@ -227,236 +227,13 @@ open hg-pd-test/target/site/jacoco/index.html
227227
- Utility classes: >70%
228228
- Generated gRPC code: Excluded from coverage
229229

230-
### Integration Tests
231-
232-
Integration tests start embedded PD instances and verify end-to-end functionality.
233-
234-
```bash
235-
# Run integration test suite
236-
mvn test -pl hg-pd-test -am -Dtest=PDCoreSuiteTest
237-
```
238-
239230
**What Integration Tests Cover**:
240231
- Raft cluster formation and leader election
241232
- Partition allocation and balancing
242233
- Store registration and heartbeat processing
243234
- Metadata persistence and recovery
244235
- gRPC service interactions
245236

246-
## Development Workflows
247-
248-
### Adding a New gRPC Service
249-
250-
#### 1. Define Protocol Buffers
251-
252-
Create or modify `.proto` file in `hg-pd-grpc/src/main/proto/`:
253-
254-
```protobuf
255-
// example_service.proto
256-
syntax = "proto3";
257-
258-
package org.apache.hugegraph.pd.grpc;
259-
260-
service ExampleService {
261-
rpc DoSomething(DoSomethingRequest) returns (DoSomethingResponse);
262-
}
263-
264-
message DoSomethingRequest {
265-
string input = 1;
266-
}
267-
268-
message DoSomethingResponse {
269-
string output = 1;
270-
}
271-
```
272-
273-
#### 2. Generate Java Stubs
274-
275-
```bash
276-
cd hugegraph-pd
277-
mvn clean compile -pl hg-pd-grpc
278-
279-
# Generated files location:
280-
# hg-pd-grpc/target/generated-sources/protobuf/java/
281-
# hg-pd-grpc/target/generated-sources/protobuf/grpc-java/
282-
```
283-
284-
**Note**: Generated files are excluded from source control (`.gitignore`)
285-
286-
#### 3. Implement Service
287-
288-
Create service implementation in `hg-pd-service`:
289-
290-
```java
291-
// ExampleServiceImpl.java
292-
package org.apache.hugegraph.pd.service;
293-
294-
import io.grpc.stub.StreamObserver;
295-
import org.apache.hugegraph.pd.grpc.ExampleServiceGrpc;
296-
297-
public class ExampleServiceImpl extends ExampleServiceGrpc.ExampleServiceImplBase {
298-
299-
@Override
300-
public void doSomething(DoSomethingRequest request,
301-
StreamObserver<DoSomethingResponse> responseObserver) {
302-
String output = processInput(request.getInput());
303-
304-
DoSomethingResponse response = DoSomethingResponse.newBuilder()
305-
.setOutput(output)
306-
.build();
307-
308-
responseObserver.onNext(response);
309-
responseObserver.onCompleted();
310-
}
311-
312-
private String processInput(String input) {
313-
// Business logic here
314-
return "Processed: " + input;
315-
}
316-
}
317-
```
318-
319-
#### 4. Register Service
320-
321-
Register service in gRPC server (in `hg-pd-service`):
322-
323-
```java
324-
// In GrpcServerInitializer or similar
325-
ExampleServiceImpl exampleService = new ExampleServiceImpl();
326-
grpcServer.addService(exampleService);
327-
```
328-
329-
#### 5. Add Tests
330-
331-
Create test class in `hg-pd-test`:
332-
333-
```java
334-
// ExampleServiceTest.java
335-
package org.apache.hugegraph.pd.service;
336-
337-
import org.junit.Test;
338-
import static org.junit.Assert.*;
339-
340-
public class ExampleServiceTest extends BaseTest {
341-
342-
@Test
343-
public void testDoSomething() {
344-
ExampleServiceImpl service = new ExampleServiceImpl();
345-
// Test service logic...
346-
}
347-
}
348-
```
349-
350-
#### 6. Update Documentation
351-
352-
Document the new API in `docs/api-reference.md`.
353-
354-
### Modifying Partition Logic
355-
356-
Partition logic is in `hg-pd-core/.../PartitionService.java` (2000+ lines).
357-
358-
**Key Methods**:
359-
- `splitPartition()`: Partition splitting
360-
- `balancePartitions()`: Auto-balancing
361-
- `updatePartitionLeader()`: Leader changes
362-
- `getPartitionByCode()`: Partition routing
363-
364-
**Development Process**:
365-
366-
1. **Understand Current Logic**:
367-
```bash
368-
# Read relevant methods
369-
# File: hg-pd-core/src/main/java/.../PartitionService.java
370-
```
371-
372-
2. **Make Changes**:
373-
- Modify partition allocation algorithm
374-
- Update balancing logic
375-
- Add new partition operations
376-
377-
3. **Test Changes**:
378-
```bash
379-
# Run partition service tests
380-
mvn test -pl hg-pd-test -am -Dtest=PartitionServiceTest
381-
382-
# Run integration tests
383-
mvn test -pl hg-pd-test -am -Dtest=PDCoreSuiteTest
384-
```
385-
386-
4. **Submit Raft Proposals**:
387-
All partition metadata changes must go through Raft:
388-
```java
389-
// Example: Update partition metadata via Raft
390-
KVOperation operation = KVOperation.put(key, value);
391-
raftTaskHandler.submitTask(operation, closure);
392-
```
393-
394-
### Adding a New Metadata Store
395-
396-
Metadata stores extend `MetadataRocksDBStore` (in `hg-pd-core/.../meta/`).
397-
398-
**Example**: Creating `GraphMetaStore`:
399-
400-
```java
401-
package org.apache.hugegraph.pd.meta;
402-
403-
public class GraphMetaStore extends MetadataRocksDBStore {
404-
405-
private static final String GRAPH_PREFIX = "@GRAPH@";
406-
407-
public GraphMetaStore(PDConfig config) {
408-
super(config);
409-
}
410-
411-
public void saveGraph(String graphName, Graph graph) throws PDException {
412-
String key = GRAPH_PREFIX + graphName;
413-
byte[] value = serialize(graph);
414-
put(key.getBytes(), value);
415-
}
416-
417-
public Graph getGraph(String graphName) throws PDException {
418-
String key = GRAPH_PREFIX + graphName;
419-
byte[] value = get(key.getBytes());
420-
return deserialize(value, Graph.class);
421-
}
422-
423-
public List<Graph> listGraphs() throws PDException {
424-
List<Graph> graphs = new ArrayList<>();
425-
String startKey = GRAPH_PREFIX;
426-
String endKey = GRAPH_PREFIX + "\uffff";
427-
428-
scan(startKey.getBytes(), endKey.getBytes(), (key, value) -> {
429-
Graph graph = deserialize(value, Graph.class);
430-
graphs.add(graph);
431-
return true; // Continue scanning
432-
});
433-
434-
return graphs;
435-
}
436-
437-
private byte[] serialize(Object obj) {
438-
// Use Hessian2 or Protocol Buffers
439-
}
440-
441-
private <T> T deserialize(byte[] bytes, Class<T> clazz) {
442-
// Deserialize bytes to object
443-
}
444-
}
445-
```
446-
447-
**Testing**:
448-
```java
449-
@Test
450-
public void testGraphMetaStore() {
451-
GraphMetaStore store = new GraphMetaStore(config);
452-
453-
Graph graph = new Graph("test_graph", 12);
454-
store.saveGraph("test_graph", graph);
455-
456-
Graph retrieved = store.getGraph("test_graph");
457-
assertEquals("test_graph", retrieved.getName());
458-
}
459-
```
460237

461238
### Debugging Raft Issues
462239

0 commit comments

Comments
 (0)