Fetch method not returning more than 100 messages

[robertmircea]

I am using the following code for testing the performance of a pull subscriber using Fetch method:

    var opts = GetNatsOpts();
    var connectionFactory = new ConnectionFactory();
    using var connection = connectionFactory.CreateConnection(opts);
    var jetStreamManagementContext = connection.CreateJetStreamManagementContext();
    var jetStreamContext = connection.CreateJetStreamContext();
    
    var cc = ConsumerConfiguration.Builder()
        .WithDurable(Guid.NewGuid().ToString("N"))
        .WithMaxAckPending(100)
        .WithAckWait(30000)
        .WithMaxDeliver(3)
        .WithReplayPolicy(ReplayPolicy.Instant)
        .WithFilterSubject(subjectName)
        .WithDeliverPolicy(DeliverPolicy.All)
        .Build();
    jetStreamManagementContext.AddOrUpdateConsumer(streamName, cc);

    Console.WriteLine("Starting subscriber...");
    var psob = new PullSubscribeOptions.PullSubscribeOptionsSubscribeOptionsBuilder()
        .WithConfiguration(cc);
    using var sub = jetStreamContext.PullSubscribe(subjectName, psob.Build());
    while (!cts.IsCancellationRequested)
    {
        var msgs = subscription.Fetch(200, 1000);
        foreach (var msg in msgs)
        {
            msg.Ack();
[...display/measure logic omitted...]
        }

    }

During my tests, I've noticed that subscription.Fetch(200, 1000) will not bring more than 100 messages in a batch no matter how high is parameter batchSize. Is there a hard limit set somewhere? Can it be raised?

Which is the most performant way to fetch & consume messages from server?

Using latest nats-server 2.9 and nats.net 1.x client.

[scottf]

If you want a continuous flow and most performant, right now I would use reader https://github.com/nats-io/nats.java/blob/main/src/main/java/io/nats/client/impl/NatsJetStreamPullSubscription.java#L331

There is no hard limit. As far as only getting 100 messages, it could be a couple things.

• The 1000 ms timeout.
• The WithMaxAckPending 100.
• The time it takes you to process each message.

Of these I'm inclined to believe that the acks pending is the most likely culprit, but they all contribute.

[robertmircea]

It was the WithMaxAckPending == 100. As I understand it, the server will stop sending more messages to my consumer because I will breach the max ack pending parameter even if the batch size is greater. This means that I need to correlate and have MaxAckPending set to a value at least batch size all the time. Very hard to correlate, thanks for helpful guide.