nats bench should publish random data, not zeroes?

[alexbozhenko]

Proposed change

We publish zeroes here:

natscli/cli/bench_command.go

Line 1757 in f479ef0

buffer := make([]byte, msgSize)

This may skew the results because of compression usage, etc.
Should probably use https://pkg.go.dev/crypto/rand#Read

Also related, when --payload is used:

natscli/cli/bench_command.go

Lines 1749 to 1752 in f479ef0

	buffer, err := os.ReadFile(c.payloadFilename)
	if err != nil {
	return nil, fmt.Errorf("reading the payload file: %w", err)
	}

It problably makes sense to read the file only once, not once per client, to make this work:

nats bench pub hello --clients=30 --msgs=30 --payload <(head --bytes=1M /dev/urandom)

Otherwise, the named pipe that bash creates is "split" across N goroutines reading it concurrently. So total published size will be 1M, not 30M in this case.

Use case

Benchmarks that are closer resemble real workloads.

Contribution

No response

[jnmoyne]

You mean get the random bytes for the payload once, not every time a message is published, right? (as that would really affect performance I would assume)

[alexbozhenko]

@jnmoyne yes, I think when both generating a random payload and reading payload from a file, we should do it only once per run, not once per client.

[ripienaar]

Publishing the same data over and over will also compress really well.

[alexbozhenko]

Ok in that case --size= probably should do random data in every message, and for those who need control over the payload, they can do it with a file using --payload= flag?