target: fix unsigned computation in 'monitor profile'

The implementation of command 'monitor profile' has few
issues:
- the address_space is a signed int, so cannot wrap-around on
  space over INT_MAX;
- max address is incremented without check for overflow;
- assert() used on errors instead of returning error codes;
- only handles 32 bits PC;
- output file created and left empty on error.

This patch fixes the first two issues, as a wider fix would be too
invasive and should be postponed in a following series.

Change-Id: Id8ead3f6db0fd5730682a0d1638f11836d06a632
Signed-off-by: Antonio Borneo <[email protected]>
Fixes: https://sourceforge.net/p/openocd/tickets/370/
Reviewed-on: https://review.openocd.org/c/openocd/+/7394
Tested-by: jenkins

Author: borneoa

src/target/target.c

@@ -4251,10 +4251,11 @@ static void write_gmon(uint32_t *samples, uint32_t sample_num, const char *filen
 
 		/* max should be (largest sample + 1)
 		 * Refer to binutils/gprof/hist.c (find_histogram_for_pc) */
-		max++;
+		if (max < UINT32_MAX)
+			max++;
 	}
 
-	int address_space = max - min;
+	uint32_t address_space = max - min;
 	assert(address_space >= 2);
 
 	/* FIXME: What is the reasonable number of buckets?