Index out of bounds when sending notification

[michael-getsignapps]

Hi
I am getting the following error which I cannot seem to solve.

java.lang.IndexOutOfBoundsException: readerIndex(0) + length(10) exceeds writerIndex(0): PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 30)

I am using boring ssl version 2.0.50.Final and netty 4.1.74.Final running on ubuntu.
I am managing to get it to run on my local machine (ubuntu) but when it hits the server it blows out.

Any suggestions?

Log:

DEBUG  [15:32:10.801] [main] i.n.u.i.NativeLibraryLoader -  -Dio.netty.native.workdir: /tmp (io.netty.tmpdir) 
DEBUG  [15:32:10.802] [main] i.n.u.i.NativeLibraryLoader -  -Dio.netty.native.deleteLibAfterLoading: true 
DEBUG  [15:32:10.802] [main] i.n.u.i.NativeLibraryLoader -  -Dio.netty.native.tryPatchShadedId: true 
DEBUG  [15:32:10.802] [main] i.n.u.i.NativeLibraryLoader -  -Dio.netty.native.detectNativeLibraryDuplicates: true 
DEBUG  [15:32:10.836] [main] i.n.u.i.NativeLibraryLoader -  Successfully loaded the library /tmp/libnetty_tcnative_linux_x86_645463013217351450111.so 
DEBUG  [15:32:10.837] [main] i.n.u.i.NativeLibraryLoader -  Loaded library with name 'netty_tcnative_linux_x86_64' 
DEBUG  [15:32:10.837] [main] i.n.h.s.OpenSsl -  Initialize netty-tcnative using engine: 'default' 
DEBUG  [15:32:10.838] [main] i.n.h.s.OpenSsl -  netty-tcnative using native library: BoringSSL 
DEBUG  [15:32:10.932] [main] i.n.u.ResourceLeakDetector -  -Dio.netty.leakDetection.level: simple 
DEBUG  [15:32:10.932] [main] i.n.u.ResourceLeakDetector -  -Dio.netty.leakDetection.targetRecords: 4 
DEBUG  [15:32:10.940] [main] i.n.b.AbstractByteBuf -  -Dio.netty.buffer.checkAccessible: true 
DEBUG  [15:32:10.940] [main] i.n.b.AbstractByteBuf -  -Dio.netty.buffer.checkBounds: true 
DEBUG  [15:32:10.941] [main] i.n.u.ResourceLeakDetectorFactory -  Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@286b39c2 
DEBUG  [15:32:10.952] [main] i.n.u.i.InternalThreadLocalMap -  -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 
DEBUG  [15:32:10.952] [main] i.n.u.i.InternalThreadLocalMap -  -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 
DEBUG  [15:32:10.955] [main] i.n.b.PooledByteBufAllocator -  -Dio.netty.allocator.numHeapArenas: 4 
DEBUG  [15:32:10.955] [main] i.n.b.PooledByteBufAllocator -  -Dio.netty.allocator.numDirectArenas: 4 
DEBUG  [15:32:10.955] [main] i.n.b.PooledByteBufAllocator -  -Dio.netty.allocator.pageSize: 8192 
--
INFO   [15:32:11.002] [main] c.e.p.a.ApnsClientBuilder -  Native SSL provider is available; will use native provider.

[jchambers]

Can you provide more context? What are you trying to do when you hit that exception? Is there a stack trace or any indication of when/where it's happening?

[michael-getsignapps]

Its when I am calling apple with a simple notification I get this as an exception as a response. I am running this from within Dropwizard which also uses Netty. I think when I upgraded Dropwizard started having these issues. Could be the handshake failing?

Debug before
DEBUG [16:24:17.670] [nioEventLoopGroup-2-1] i.n.h.s.SslHandler - [id: 0x6f08b265, L:/10.0.1.60:42856 - R:api.sandbox.push.apple.com/17.188.168.149:443] HANDSHAKEN: protocol:TLSv1.3 cipher suite:TLS_AES_128_GCM_SHA256
DEBUG [16:24:17.768] [nioEventLoopGroup-2-1] c.e.p.a.ApnsClientHandler - Received settings from APNs gateway: {HEADER_TABLE_SIZE=4096, MAX_CONCURRENT_STREAMS=1000, MAX_FRAME_SIZE=16384, MAX_HEADER_LIST_SIZE=8000}
DEBUG [16:24:17.768] [nioEventLoopGroup-2-1] c.e.p.a.ApnsChannelPool - Created channel [id: 0x6f08b265, L:/R:api.sandbox.push.apple.com/17.188.168.149:443]

The error and the stacktrace

Apple push notification failed for user xxxxx - java.lang.IndexOutOfBoundsException: readerIndex(0) + length(10) exceeds writerIndex(0): PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 30)
DEBUG [16:24:17.770] [nioEventLoopGroup-2-1] i.n.h.c.h.Http2ConnectionHandler - [id: 0x6f08b265, L:/10.0.1.60:42856 - R:api.sandbox.push.apple.com/17.188.168.149:443] Sent GOAWAY: lastStreamId '0', errorCode '2', debugData ''. Forcing shutdown of the connection.

[jchambers]

As a debugging step, could you please try without netty-tcnative/boringssl?

[michael-getsignapps]

tried without and same issue

[jchambers]

Okay; I hate to be picky, but I'm afraid that what you've provided isn't exactly a stack trace. A stack trace identifies the specific line where the problem happened and the specific calls that led to that point. Reading between the lines, I think you're doing something like this:

try {
    // Send a notification
    // ...
} catch (Exception e) {
    System.out.println("Apple push notification failed for user ... - " + e.getMessage());
}

…but what we really need is the full stack trace:

try {
    // Send a notification
    // ...
} catch (Exception e) {
    System.out.println("Apple push notification failed for user ... - ");
    e.printStackTrace();
}

If you're using slf4j, you can print the stack trace by including the exception as an additional argument to the logging statement:

try {
    // Send a notification
    // ...
} catch (Exception e) {
    log.warn("Apple push notification failed for user {}", user.getId(), e);
}

If you can provide a complete stack trace, we'll be much better-equipped to figure out what's going on here.

[michael-getsignapps]

Ok it seems the native boring ssl is causing an issue for some reason and I deployed on another box and ran it using open jdk ssl and it seems to work.