Should IcebergCatalogProvider::try_new not short-circuit on the first failed IcebergSchemaProvider::try_new?

[krishnakeshan]

I've been working with a GCS bucket that has some invalid schemas that cause IcebergSchemaProvider::try_new to fail. When building these schemas with IcebergCatalogProvider::try_new, I'm unable to get the catalog provider for valid schemas due to some corrupted schemas. I modified the method like below and it works fine for me:

impl IcebergCatalogProvider {
  pub async fn try_new(client: Arc<dyn Catalog>) -> Result<Self> {
    let schema_names: Vec<_> = client
        .list_namespaces(None)
        .await?
        .iter()
        .flat_map(|ns| ns.as_ref().clone())
        .collect();
    
    // changed try_join_all to join_all and manually collected them into valid IcebergSchemaProviders
    let schema_providers = join_all(
        schema_names
            .iter()
            .map(|name| async {
                (
                    name.clone(),
                    IcebergSchemaProvider::try_new(
                        client.clone(),
                        NamespaceIdent::new(name.clone()),
                    )
                    .await,
                )
            })
            .collect::<Vec<_>>(),
    )
    .await;
    
    let mut schemas = HashMap::new();
    for (name, provider) in schema_providers {
        if let Ok(provider) = provider {
            schemas.insert(name, Arc::new(provider) as Arc<dyn SchemaProvider>);
        }
    }
    
    Ok(IcebergCatalogProvider { schemas })
  }
}

I'm not mentioning the error currently as my question is about whether such a method makes sense. I can add error details if those would help make a decision

[liurenjie1024]

I have concerns about this change, which makes the error hidden from user. I think a more reasonable approach is to load schema/table lazily, e.g. only load them when necessary, and reports the error when loading a corrupted schema/table. But due to the limitation of datafusion's trait definition, we can't do it for now.