Test on julia 1 and lts

[VictorVanthilt]

No description provided.

[Yue-Zhengyuan]

Note that rtol is already relaxed in #106. We need to actually look into the test log to see whether things would fail under the old rtol.

[VictorVanthilt]

Im very confused

[Yue-Zhengyuan]

Oh, I remember that you're using iterative method to solve for eigenvalues of the transfer matrix in spec, which uses a random initialization. I suggest you set a random seed in the tests for reproducible output. Some random initializations may just be very bad.

[Yue-Zhengyuan]

I examined the logs. Looking at the 0.125 scaling dimension produced by the tests:

shape	[1,4,1]	[sqrt(2), 2 * sqrt(2), 0]	[1, 8, 1]	[4 / sqrt(10), 2 * sqrt(10), 2 / sqrt(10)]
macOS 1.12	0.12503601544406606	0.1250286849045871	0.12532260340963547	0.12531494312906244
macOS 1.10	0.12500584642486906	0.12500727785146276	0.12524027989071687	0.12522876261130136
ubuntu 1.12	0.12500948326440822	0.12501257295404752	0.1253964153300084	0.12534551426346335
ubuntu 1.10	0.12500948326440806	0.12501257295404794	0.12539641533000528	0.12534551426346427
Windows 1.12	0.125014305592712	0.1250171241887859	0.12541428493609857	0.12534250745684256
Windows 1.10	0.1250143055927122	0.1250171241887857	0.12541428493609905	0.12534250745684244

You can see on ubuntu and Windows, the results are almost the same (change is smaller than 1e-14) for Julia 1.12 and 1.10.10. But on macOS the result changed by 1e-5~1e-4. There's definitely something going on in the macOS version of Julia 1.12.

[VictorVanthilt]

@lkdvos @Jutho Are you guys having issues with this as well?

[lkdvos]

Can confirm I saw weird behavior here: QuantumKitHub/MatrixAlgebraKit.jl#75, also specifically for macOS v1.12. In that case, it seemed to be related to floating point errors, and not the rng.

Some things to keep in mind:

• I'm pretty sure @testset already fixes the rng seed for reproducible results.
• The default rng does not promise stable streams under different versions, even with the same seed. For that, see StableRNGs.jl

So my first guess here would be that something in the random number generator changed, causing that specific version to have different results, and only then I'd investigate if there are floating point weirdnesses

[Yue-Zhengyuan]

@VictorVanthilt I see that in cft_data!, the function area_term still uses a random number here:

TNRKit.jl/src/utility/cft.jl

Line 79 in 31a6a91

x0 = rand(a_in ⊗ b_in)

Maybe this is the cause?

[VictorVanthilt]

Yeah I figured, fixing that now.

[Yue-Zhengyuan]

If this fixes the tests, how about we reset the rtol to the old value?

[VictorVanthilt]

Yes

[VictorVanthilt]

Ok this makes no sense. I'm very skeptical about why the initial guess should matter that much in the end result (on the order of 1% relative difference). Should we revisit the CFT calculation method?

Now ubuntu is screwing up. Make it make sense.

[Yue-Zhengyuan]

Did you simply rerun the test, then ubuntu is fixed?

[VictorVanthilt]

Yes, so extra investigation is necessary. This should not happen.

[VictorVanthilt]

If these tests pass I will merge this and get to the other PRs