[firedrake] parallel weirdness

[Colin Cotter]

Dear all,
We have a rather weird parallel bug, and Hiroe and I would appreciate 
some suggestions. The relevant code is in 
bitbucket.org/colinjcotter/slicemodels/branch/periodic_parallel revision 
622fb24.
We were observing different results on 1 and 2 cores, which Hiroe fixed 
by moving some code, but it makes no sense to us!
In line 270, you can see an extra term,
- (-dt*div(w)*pbar)*dx,
which was previously added to the RHS in line 139
so, this changes implementation but not the algorithm
either, this extra term is solved into a contribution of self.ures in 
L142, in which case it pops up as part of self.ures in l270, or we write 
it explicitly in the form there in general, we don't wish to write it 
explicitly in the form though, and I want to understand what is going on!

Some further investigation showed that the problem seems to occur in the 
pressure solve on line 393, resulting in a nonsense value of self.Deltap 
occurring. This is just inverting a DG mass system so it ought to be 
straightforward, and the inputs are (approximately) the same.

Do let me know if you have any ideas,

cheers
--cjc