Fix import performance regression and update test

[dweindl]

Unfortunately, #3030 doubled the import time for our performance test model, but the check there was too lax to catch it.

Sometimes, smart_subs_dict is applied to individual expressions, sometimes to complete matrices. #3030 was aiming at the latter case. For individual expressions, this is too costly.

Fix test tolerances. Fix import time.