RegularizedMeanField sometimes fails

[artemmaksov]

I am trying to run RMF solver with synthetic data, but sometimes it results in an error. Here is the traceback:

File "python3.7/site-packages/coniii/solvers.py", line 1964, in solve solution = minimize_scalar(func)
File "python3.7/site-packages/scipy/optimize/_minimize.py", line 770, in minimize_scalar return _minimize_scalar_brent(fun, bracket, args, **options)
File "python3.7/site-packages/scipy/optimize/optimize.py", line 2141, in _minimize_scalar_brent brent.optimize()
File "python3.7/site-packages/scipy/optimize/optimize.py", line 1925, in optimize xa, xb, xc, fa, fb, fc, funcalls = self.get_bracket_info()
File "python3.7/site-packages/scipy/optimize/optimize.py", line 1899, in get_bracket_info xa, xb, xc, fa, fb, fc, funcalls = bracket(func, args=args)
File "python3.7/site-packages/scipy/optimize/optimize.py", line 2324, in bracket fa = func(*(xa,) + args)
File "python3.7/site-packages/coniii/solvers.py", line 1935, in func isingSamples = samples(J)
File "python3.7/site-packages/coniii/solvers.py", line 1917, in samples self.multipliers = np.concatenate([J.diagonal(), squareform(mean_field_ising.zeroDiag(-J))])
File "python3.7/site-packages/scipy/spatial/distance.py", line 2193, in squareform is_valid_dm(X, throw=True, name='X')
File "python3.7/site-packages/scipy/spatial/distance.py", line 2269, in is_valid_dm 'symmetric.') % name)
ValueError: Distance matrix 'X' must be symmetric.

[eltrompetero]

Thanks for pointing this out.

My first guess is that there is some numerical precision error because the matrix J is symmetrized in the preceding line. Do you have a sense for what range of values your coupling matrix will be? For example, do you have nearly perfectly correlated spins?

[artemmaksov]

Thanks for the quick response! That's something I haven't really looked into. I have been using randomly generated matrix following an example in one of the notebooks and just changing value of n: h,J = np.random.normal(scale=.1,size=n),np.random.normal(scale=.1,size=n*(n-1)//2)

[eltrompetero]

I'm guessing that this problem occurs more frequently at smaller n. If that's the case, you may want to regularize the problem by disallowing correlations that are close to -1 or 1. That's where the parameters diverge.

Probably we should include a warning for such situations.

[bcdaniels]

Yes, I'm guessing that there's an issue with loss of precision or probably nans. Note squareform gives the same error for symmetric nans:

In [20]: scipy.spatial.distance.squareform([[0,1],[1,0]])
Out[20]: array([1])

In [21]: scipy.spatial.distance.squareform([[0,scipy.nan],[scipy.nan,0]])
ValueError: Distance matrix 'X' must be symmetric.

Maybe we need to check for nans in the J matrix?

[eltrompetero]

@bcdaniels The RMF code is a bit opaque to me right now. Could you point out where nans could appear?