Skip to content

Commit 0332faa

Browse files
sleipnirAdriano Santospolvalente
authored
[Feat]: separating client and server into two distinct packages (#477)
* chore: separating client and server into two distinct packages (wow commit) * remove root deps * remove some files * format * chore: adjustments in interop script and formatting * feat: create automation to publish hex packages * fix: download deps * bump to v1.0.0-alpha * chore: test publish script * chore: remove stray dirs * chore: remove release scripts * remove release pipeline * remove release scripts * removing unnecessary things * Update grpc_core/README.md Co-authored-by: Paulo Valente <[email protected]> * remove stray text * fix broken links * update version * remove unused things * removed unused module * refactor benchmark to use mix tasks * docs: update readmes * feat: refactor benchmark to use OTP Application with Mix tasks - Created Mix.Tasks.Benchmark.Test for automated benchmarking - Created Mix.Tasks.Benchmark.Worker for distributed testing - Added GRPC.Client.Supervisor to Application startup - Fixed Protobuf syntax (.new() -> %Module{}) - Fixed NIF syscall.so loading from priv_dir - Updated documentation with usage examples - Fixed grpc_server missing protobuf dependency - Benchmark achieving ~2000 req/s with 0.5ms avg latency * remove elixir_tools again * remove unused files * use user agent version * fix tests * fix interop tests * fix hardcoded version * formatting * fix: remove pipeline validation for hex --------- Co-authored-by: Adriano Santos <[email protected]> Co-authored-by: Paulo Valente <[email protected]>
1 parent 2fe10b2 commit 0332faa

File tree

233 files changed

+10139
-925
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

233 files changed

+10139
-925
lines changed

‎.formatter.exs‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1 @@
1-
[
2-
inputs: [
3-
"{mix,.formatter}.exs",
4-
"{config,lib,test}/**/*.{ex,exs}",
5-
"examples/*/{config,lib,priv}/*.ex"
6-
],
7-
import_deps: [:protobuf],
8-
locals_without_parens: [rpc: 3, intercept: 1, intercept: 2, run: 1, run: 2],
9-
export: [
10-
locals_without_parens: [rpc: 3, intercept: 1, intercept: 2, run: 1, run: 2]
11-
]
12-
]
1+
[subdirectories: ["grpc_core", "grpc_server", "grpc_client"]]

‎.github/workflows/ci.yml‎

Lines changed: 2 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ jobs:
5050
path: deps
5151
key: v1-${{ matrix.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-mix-${{ hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
5252
- name: Install Dependencies
53-
run: mix deps.get 1>/dev/null
53+
run: mix setup 1>/dev/null
5454
- name: Run Tests
5555
run: mix test --warnings-as-errors
5656
interop-tests:
@@ -104,27 +104,4 @@ jobs:
104104
working-directory: ./interop
105105
- name: Run interop tests
106106
run: mix run script/run.exs
107-
working-directory: ./interop
108-
109-
check_release:
110-
runs-on: ubuntu-latest
111-
name: Check release
112-
needs: check_format
113-
steps:
114-
- uses: actions/checkout@v3
115-
- uses: erlef/setup-beam@v1
116-
with:
117-
otp-version: 26.1.x
118-
elixir-version: 1.15.x
119-
- name: Retrieve dependencies cache
120-
uses: actions/cache@v3
121-
id: mix-cache # id to use in retrieve action
122-
with:
123-
path: deps
124-
key: v1-${{ matrix.os }}-${{ matrix.otp }}-${{ matrix.elixir }}-mix-${{ hashFiles(format('{0}{1}', github.workspace, '/mix.lock')) }}
125-
- name: Install Dependencies
126-
run: mix deps.get 1>/dev/null
127-
- name: Build hex
128-
run: mix hex.build
129-
- name: Generate docs
130-
run: mix docs
107+
working-directory: ./interop

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,4 +24,6 @@ erl_crash.dump
2424

2525
.elixir_ls
2626

27+
.elixir_tools
28+
2729
grpc-*.tar

‎Makefile‎

Lines changed: 0 additions & 22 deletions
This file was deleted.

‎benchmark/.formatter.exs‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
# Used by "mix format"
21
[
3-
inputs: ["mix.exs", "{config,lib,test}/**/*.{ex,exs}"],
4-
import_deps: [:protobuf, :grpc]
2+
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
53
]

‎benchmark/.gitignore‎

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,20 +7,17 @@
77
# The directory Mix downloads your dependencies sources to.
88
/deps/
99

10-
# Where 3rd-party dependencies like ExDoc output generated docs.
10+
# Where third-party dependencies like ExDoc output generated docs.
1111
/doc/
1212

13-
# Ignore .fetch files in case you like to edit your project deps locally.
14-
/.fetch
15-
1613
# If the VM crashes, it generates a dump, let's ignore it too.
1714
erl_crash.dump
1815

1916
# Also ignore archive artifacts (built via "mix archive.build").
2017
*.ez
2118

2219
# Ignore package tarball (built via "mix hex.build").
23-
benchmark-*.tar
20+
benchmarkg-*.tar
2421

25-
/syscall.so
26-
/syscall.so.*
22+
# Temporary files, for example, from tests.
23+
/tmp/

‎benchmark/README.md‎

Lines changed: 63 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,73 @@
1-
# Benchmark
1+
# gRPC Benchmark
22

3-
Benchmark implementation followed by [official spec](https://grpc.io/docs/guides/benchmarking/)
3+
Benchmark tools for testing gRPC performance.
44

55
## Usage
66

7+
This project provides Mix tasks for running benchmarks:
8+
9+
### Starting a Worker Server
10+
11+
```bash
12+
mix benchmark.worker --port=10000
13+
```
14+
15+
Options:
16+
- `--port` - Port to listen on (required)
17+
18+
The worker will run until you send it a quit signal.
19+
20+
### Running Benchmark Tests
21+
22+
```bash
23+
# Use defaults (port 10000, 1000 requests)
24+
mix benchmark.test
25+
26+
# Custom port
27+
mix benchmark.test --port=9999
28+
29+
# Custom number of requests
30+
mix benchmark.test --requests=5000
31+
32+
# Both options
33+
mix benchmark.test --port=9999 --requests=5000
734
```
8-
$ git clone https://github.com/elixir-grpc/grpc.git
9-
$ cd grpc && git checkout -t origin/elixir-bench
10-
$ git submodule update --init
11-
$ export ELIXIR_GRPC_PATH= # elixir-grpc path
12-
$ python tools/run_tests/run_performance_tests.py -l elixir
35+
36+
Options:
37+
- `--port` - Server port (default: 10000)
38+
- `--requests` - Number of requests to send (default: 1000)
39+
40+
### Legacy Scripts
41+
42+
The original `bin/` scripts are still available for backward compatibility:
43+
44+
```bash
45+
# Start worker
46+
elixir bin/worker.exs --port=10000
47+
48+
# Run test
49+
elixir bin/test.exs
1350
```
1451

15-
## Result
52+
## Project Structure
53+
54+
This project follows standard Elixir/OTP conventions:
1655

1756
```
18-
# Elixir
19-
# Branch improve-perf
20-
I0707 13:04:20.222163000 4618786240 report.cc:82] QPS: 3070.4
21-
I0707 13:04:20.224064000 4618786240 report.cc:122] QPS: 3070.4 (255.9/server core)
22-
I0707 13:04:20.224079000 4618786240 report.cc:127] Latencies (50/90/95/99/99.9%-ile): 305.0/406.1/469.6/733.8/1730.8 us
23-
I0707 13:04:20.224088000 4618786240 report.cc:137] Server system time: 4.72%
24-
I0707 13:04:20.224097000 4618786240 report.cc:139] Server user time: 47.32%
25-
I0707 13:04:20.224104000 4618786240 report.cc:141] Client system time: 7.93%
26-
I0707 13:04:20.224112000 4618786240 report.cc:143] Client user time: 94.19%
27-
I0707 13:04:20.224119000 4618786240 report.cc:148] Server CPU usage: 0.00%
28-
I0707 13:04:20.224126000 4618786240 report.cc:153] Client Polls per Request: 0.00
29-
I0707 13:04:20.224134000 4618786240 report.cc:155] Server Polls per Request: 0.00
30-
I0707 13:04:20.224142000 4618786240 report.cc:160] Server Queries/CPU-sec: 5899.79
31-
I0707 13:04:20.224149000 4618786240 report.cc:162] Client Queries/CPU-sec: 3006.81
32-
33-
# Go
34-
# It seems there are some problems with Go's benchmark because there's no CPU related metrics
35-
I0707 12:36:06.782124000 4488484288 report.cc:82] QPS: 9525.8
36-
I0707 12:36:06.783670000 4488484288 report.cc:122] QPS: 9525.8 (793.8/server core)
37-
I0707 12:36:06.783688000 4488484288 report.cc:127] Latencies (50/90/95/99/99.9%-ile): 93.8/118.7/139.0/247.9/526.0 us
38-
I0707 12:36:06.783698000 4488484288 report.cc:137] Server system time: 0.00%
39-
I0707 12:36:06.783707000 4488484288 report.cc:139] Server user time: 0.00%
40-
I0707 12:36:06.783750000 4488484288 report.cc:141] Client system time: 0.00%
41-
I0707 12:36:06.783758000 4488484288 report.cc:143] Client user time: 0.00%
42-
I0707 12:36:06.783764000 4488484288 report.cc:148] Server CPU usage: 0.00%
43-
I0707 12:36:06.783771000 4488484288 report.cc:153] Client Polls per Request: 0.00
44-
I0707 12:36:06.783777000 4488484288 report.cc:155] Server Polls per Request: 0.00
45-
I0707 12:36:06.783783000 4488484288 report.cc:160] Server Queries/CPU-sec: inf
46-
I0707 12:36:06.783790000 4488484288 report.cc:162] Client Queries/CPU-sec: inf
57+
benchmarkg/
58+
├── lib/
59+
│ ├── benchmark/ # Core benchmark modules
60+
│ │ ├── application.ex # OTP Application supervisor
61+
│ │ ├── client_manager.ex
62+
│ │ ├── server_manager.ex
63+
│ │ └── ...
64+
│ └── mix/tasks/ # Mix tasks
65+
│ ├── benchmark.worker.ex
66+
│ └── benchmark.test.ex
67+
├── bin/ # Legacy scripts (preserved)
68+
├── proto/ # Protocol buffer definitions
69+
└── mix.exs
4770
```
71+
72+
See [REFACTORING.md](REFACTORING.md) for details on the migration from scripts to Mix tasks.
73+

‎benchmark/config/config.exs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@ import Config
22

33
config :logger, level: :info
44

5-
import_config "#{Mix.env}.exs"
5+
import_config "#{Mix.env()}.exs"

‎benchmark/config/test.exs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import Config
2+
3+
config :logger, level: :warning

‎benchmark/lib/benchmark.ex‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
11
defmodule Benchmark do
2+
@moduledoc """
3+
Documentation for `Benchmark`.
4+
"""
25
end

0 commit comments

Comments
 (0)