Auto merge of rust-lang#147486 - petrochenkov:optpriv, r=lcnr

privacy: Introduce some caching to type visiting in `DefIdVisitorSkeleton`

The caching fixes compilation speed issues in special cases like rust-lang#145741, without introducing too much overhead in general cases.

I tried to cache more, but it caused regressions from the caching overhead, like it can be seen from benchmark runs below.

Inspired by rust-lang#146128.
Closes rust-lang#145741.

Author: bors