[firedrake] firedrake broken on ARCHER after update

[Lawrence Mitchell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 26/04/15 13:53, Eike Mueller wrote:
> Hi,
> 
>>> At higher order, I don’t see to be able to run the matrix-free 
>>> solver in the original configuration any more: it works on one 
>>> core (with a reduced problem size), but not on a full node. I
>>> did check on my laptop that I can still run in parallel, but
>>> the results of a 1 core run and a 4 core run are not exactly 
>>> identical at higher order, so something is not quite right 
>>> there.
> Hang on, thinking about this again, if I use BJACOBI+ILU as the 
> preconditioner for the velocity mass inverse, isn’t the result 
> naturally dependent on the number of processors, since by default
> a block corresponds to the dofs stored on one process? At lowest
> order I use the inverse of the diagonal lumped mass matrix, so
> there I don’t see this.

Yes, that's right, bjacobi+ilu first decouples the processes and then
does ILU on the local block, so it will be different on different
processes.

> However, if I change BJACOBI+ILU -> JACOBI I still get a
> difference between the results on 1 and on 4 processors, so I must
> be missing something else.

In this situation I would recommend checking that the operator is
indeed the same on 1 and 4 processors.  Computing some of the leading
eigenvalues is probably good enough, you can do this with SLEPc:

pip install git+https://wence@bitbucket.org/slepc/slepc.git#egg=slepc
pip install
git+https://wence@bitbucket.org/slepc/slepc4py.git#egg=slepc4py

A = assemble(form)

petsc_mat = A.M.handle

from slepc4py import SLEPc

eps = SLEPc.EPS().create()

eps.setOperators(petsc_mat)

eps.setWhichEigenpairs(eps.Which.LARGEST_MAGNITUDE)

eps.solve()

nev = eps.getConverged()

for i in range(nev):
    evalue = eps.getEigenvalue(i)
    print evalue

Cheers,

Lawrence
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVPf+JAAoJECOc1kQ8PEYv650H/3ZGkCy/6G/BivFhSMyKNaCz
HS3P4KO/sOcNEJSGum1YSktWvWa5YkRWX9xACZWRisO3TWKgTpBiqhS4aGuOn2Xh
QrO3Uhyoj5dx1S5o69qQiZppsMjdPViRnw4/O3YEAw7xUiKbbJqBmb30yP2fLGKe
q2Ce/kwPG8DfVf0Xl6Z4muKHHquG38YwqDCSMc6s4QbIQwi8PQI+fAWcBwbZ2Wag
vuB9TOIJUXmBMMpdlk8YPr903iu1Qd4koxbP8kN0kHsORdJJzxOsYfotEEk5Fdm4
de0UjSUSKFP1Kj45ZbZLbXFrJFIrFlSaWRFQmQEmpck2H8AMYDsF8TMviH4dE4w=
=RMZi
-----END PGP SIGNATURE-----