It appears that packages that have their access changed from public to private remain listed on yarnpkg.
Looking through the code, as far as I can tell, there isn't a mechanism for detecting changes in access level, which would mean the private package will remain indexed indefinitely.