[WIP] Belief propagation gauge fixing

[lkdvos]

This is an initial implementation for gauge fixing a PEPS based on the algorithm described here

• Tindall, Joseph, and Matthew Fishman. “Gauging Tensor Networks with Belief Propagation.” SciPost Physics 15, no. 6 (December 1, 2023): 222. https://doi.org/10.21468/SciPostPhys.15.6.222.

From what I can tell it is mostly functional, although it requires some tests etc to actually see how well it performs in practice.
The idea would be to attempt to stabilize some gradient methods by re-gauging the state every N steps.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 235 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/algorithms/contractions/bp_contractions.jl	0.00%	93 Missing ⚠️
src/algorithms/bp/beliefpropagation.jl	0.00%	85 Missing ⚠️
src/environments/bp_environments.jl	0.00%	57 Missing ⚠️

Files with missing lines	Coverage Δ
src/PEPSKit.jl	100.00% <ø> (ø)
src/environments/bp_environments.jl	0.00% <0.00%> (ø)
src/algorithms/bp/beliefpropagation.jl	0.00% <0.00%> (ø)
src/algorithms/contractions/bp_contractions.jl	0.00% <0.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Yue-Zhengyuan]

I remember that BP is equivalent to simple update (PhysRevResearch.3.023073), although I haven't looked at the details (such as whether the bond messages are guaranteed to be real diagonal, and thus already represented by SUWeight). We may use this equivalence to avoid introducing redundant structs, or generalize the existing simple update algorithm.

BTW, YASTN people are also working on BP, and with NNN Hamiltonian support (in one of the branches) as well.

[lkdvos]

I looked into trying to reuse some of that functionality, and at least for the BPEnv struct there really are substantial differences: note how there are 2 messages per edge, one going in each direction, which are connected between bra and ket, instead of between the virtual edges of the PEPS. That's why I ended up modeling this more like the CTMRGEnv, where I'm effectively storing a CTMRG environment with chi = 1 trivial legs. (I'll try add a converter/promoter from BPEnv to CTMRGEnv to at some point.)

I do agree that it would be nice to join the simple update part, but for now I was actually mostly interested in affecting the gauge of a PEPS, and we can look into extending this to actually do updates in the future.

It's nice to hear that YASTN also are finding this helpful! I don't think adding NNN support would be too hard, the contractions with these rank 1 environments are a lot more manageable in general.
Something I would like to look into at some point is improving the accuracy with loop corrections, see https://arxiv.org/pdf/2409.03108

[pbrehmer]

Thanks for getting this started! I'd be happy to join in on this since I was recently talking to Bram about PEPS optimization, how well it might perform and how that might be related to gauge freedoms. Specifically, I was looking at optimization runs that get stuck in linesearching at some point - e.g. when choosing a CTMRG environment dimension that is too low. Around these iterations where the optimizer gets stuck I computed cuts through the energy landscape along the current gradient directions and I was getting all sorts of weird curves with cusps or discontinuities (inspired by Appendix B of the periodic PEPS paper: https://arxiv.org/pdf/2411.12731).

For example, a Heisenberg model at $D=3$ and $\chi=4$ will look like this ($i$ being the LBFGS iteration, $g$ the gradient at iteration $i$):

At $D=3$ and $\chi=8$ it looks a bit more wild:

In these cases you can really see that a low environment dimension will first get you smooth convex functions that become more closely centered around $\alpha=0$ and then cusps and discontinuities may appear. In these cases the linesearching algorithms of course run into trouble and eventually may spit out unphysical optimization steps. Also, this process of breaking down might stretch over many LBFGS iterations.

I seemed that gauging by symmetrizing the PEPS spatially did improve the energy landscapes but I haven't properly investigated that. In any case, I would be quite interested to see how BP gauging could improve optimization convergence and perhaps stabilize more complicated optimization runs.

[Yue-Zhengyuan]

Something I would like to look into at some point is improving the accuracy with loop corrections, see https://arxiv.org/pdf/2409.03108

I also have wanted to try it out for quite a while, and see how it can systemically improve over SU.

To suggest some tests for the current PR, we can first check that if the bond weights produced by SU (with identity gates) are consistent with the BP message tensors.

[lkdvos]

@pbrehmer these are some really cool plots! This was indeed one of the primary use-cases I had in mind, so it would be cool to see how this alters these gradient plots.

As a side note, at such low bond dimensions I would also be slightly cautious about picking bond dimensions that necessarily break the symmetry, for example in a U1 case with charge conjugation I would expect that you might need to check for each D whether or not it gives a valid combination. This is motivated by similar problems for MPS simulations, where certain bond dimensions don't work because they have to cut inbetween multiplets.

[GlebFedorovich]

@pbrehmer Indeed, we have seen this a lot for periodic PEPS, and pushing $$\chi$$ and imposing some rotation and mirror symmetries help to stabilize the optimization and avoid these discontinuous points. However, there are many cases, beyond ising and heisenberg models, when imposing spatial symmetries leads to higher energies compared let's say with just SU (or with AD fixed-point one before the optimization arrives at "bad" point :) )

[pbrehmer]

However, there are many cases, beyond ising and heisenberg models, when imposing spatial symmetries leads to higher energies compared let's say with just SU (or with AD fixed-point one before the optimization arrives at "bad" point :) )

Yes I have also noticed that recently in some cases! Good to know that that seems to be consistent with your findings :)

[Yue-Zhengyuan]

Since adding BP may need some revisions for SU and InfiniteWeightPEPS, I want to ask a few things to get more familiar with it.

[Yue-Zhengyuan]

Could you please add a unicode diagram illustrating the convention?

[Yue-Zhengyuan]

In the original BP gauging paper, the message tensors are chosen to be positive definite, and their square root is calculated with eigen-decomposition. Here I see you changes it to SVD, and the BPEnv are initialized simply with randn without enforcing positiveness. I'm curious whether this is valid.