fix sentinel comparison

[uwezkhan]

fix(cgif_rgb): add NULL checks after malloc/realloc, fix sentinel comparison

• Add NULL check on colhash struct and all 5 sub-allocations in
  init_col_hash_table(); free all on any failure.
• Use temp pointers in resize_col_hash_table() to avoid losing original
  pointers on realloc failure; return early on any allocation failure.
• Add NULL guards in hash_to_dense(), new_tree_node(),
  quantize_and_dither(), cgif_rgb_newgif(), and cgif_rgb_addframe().
• Fix integer overflow in float buffer: cast to size_t before multiply.
• Check get_color_histogram() return value in cgif_rgb_addframe();
  free pNewBef and fConfig.pImageData on failure.
• Fix uint32_t h == -1 always-false comparison in get_color_histogram()
  and quantize_and_dither() by changing h to int32_t.

Fixes: CWE-476 (NULL Pointer Dereference), CWE-190 (Integer Overflow),
CWE-570 (Expression Always False)

[dloebl]

Hey @uwezkhan,
please refer to the contributing guidelines before opening any new pull requests or issues. Thanks.