This is probably really easy but I'm putting it here so it won't be forgotten.
Currently, vcpkg still builds Debug and Release configurations for all dependencies, even if cmake is set to build Release, which just more or less doubles initial build time.