Misleading "namespace doesn't exist" error when in fact catalog doesn't exist

[sfc-gh-npuka]

CREATE TABLE test (a int) USING iceberg WITH (catalog = 'rest');

ERROR:  HTTP request failed (HTTP 404)
DETAIL:  Namespace does not exist: 
HINT:  The rest catalog returned error type: NoSuchNamespaceException

After debugging, we realized with @sfc-gh-abozkurt that the namespace URL actually showed us that it was using a non-existent catalog name: https://.../polaris/api/catalog/v1/non_existing_catalog_name/namespace_name. This was a mistake from my part, as I had used a different name for the REST catalog.

The error, however, is quite misleading. When the GET for the namespace returns 404, the code assumes "namespace doesn't exist" and tries to create it. But if the catalog doesn't exist, Polaris also returns 404, and the subsequent CreateNamespaceOnRestCatalog call also fails with a 404 "Namespace does not exist" which is misleading. See function RegisterNamespaceToRestCatalog. We should check whether the catalog exists first.

[sfc-gh-npuka]

Claude suggestion:

The fix should go in the 404 handler. When the namespace GET returns 404, before assuming the namespace doesn't exist, we should check if the catalog itself exists by listing its namespaces. If that also returns 404, the catalog is the problem.

https://github.com/Snowflake-Labs/pg_lake/blob/main/pg_lake_iceberg/src/rest_catalog/rest_catalog.c#L259-L274

	char	   *getUrl =
		psprintf(REST_CATALOG_NAMESPACE_NAME,
				 RestCatalogHost, URLEncodePath(catalogName),
				 URLEncodePath(namespaceName));
	HttpResult	httpResult = SendRequestToRestCatalog(HTTP_GET, getUrl, NULL, GetHeadersWithAuth());

	switch (httpResult.status)
	{
			/* namespace not found */
		case 404:
			{
                /*
                 * For debugging purposes
                 */
				ReportHTTPError(httpResult, DEBUG2);

				/*
-                 * Does not exists, we'll create it.
+				 * A 404 could mean the namespace doesn't exist, or the
+				 * catalog itself doesn't exist. Verify the catalog exists
+				 * by listing its namespaces before trying to create one.
				 */
+				char	   *catalogUrl =
+					psprintf(REST_CATALOG_NAMESPACE, RestCatalogHost,
+							 URLEncodePath(catalogName));
+				HttpResult	catalogResult =
+					SendRequestToRestCatalog(HTTP_GET, catalogUrl, NULL,
+											 GetHeadersWithAuth());
+
+				if (catalogResult.status == 404)
+					ereport(ERROR,
+							(errcode(ERRCODE_EXTERNAL_ROUTINE_EXCEPTION),
+							 errmsg("catalog \"%s\" does not exist in the rest catalog server",
+									catalogName),
+							 errhint("Create the catalog in the rest catalog server, "
+									 "or check pg_lake_iceberg.rest_catalog_host and "
+									 "the database name (used as the catalog name).")));
+				else if (catalogResult.status != 200)
+					ReportHTTPError(catalogResult, ERROR);
+
				CreateNamespaceOnRestCatalog(catalogName, namespaceName);
				break;
			}