Make sure we return the OOM error condition when closing containers

Usually, if you ignore the OOM error, the encoder API may start
returning success, even though no buffer overrun happens. The way to
check if an OOM happened is with cbor_encoder_get_extra_bytes_needed,
which will return non-zero if a larger buffer is returned.

That said, the API was mostly already returning the OOM error
consistently, due to almost everything ending up calling
append_to_buffer(). This case was an exception.

Signed-off-by: Thiago Macieira <[email protected]>

Author: thiagomacieira

src/cborencoder.c

@@ -534,6 +534,8 @@ CborError cbor_encoder_close_container(CborEncoder *encoder, const CborEncoder *
     encoder->end = containerEncoder->end;
     if (containerEncoder->flags & CborIteratorFlag_UnknownLength)
         return append_byte_to_buffer(encoder, BreakByte);
+    if (encoder->end)
+        return CborErrorOutOfMemory;    /* keep the state */
     return CborNoError;
 }