After #7 cargo rail release check --all will skip those marked publish = false. However, adding publish = false to a crate after cargo rail release check --all has already been run will continue to try to check the crate until I rm target/cargo-rail/metadata.json. It would be nice if the relevant part of metadata.json were automatically recomputed.