[firedrake] (Even more) progress on installing PyOP2/firedrake on ARCHER

[Florian Rathgeber]

On 15/09/14 09:20, Eike Mueller wrote:
> Hi Florian,
> 
> I'm having another go at this now.
> 
>>>
>>> export
>>> PYTHONPATH=${FDRAKE_DIR}/petsc4py/lib/python2.7/site-packages:${PYTHONPATH}
>>> export
>>> PYTHONPATH=${FDRAKE_DIR}/ufl/lib/python2.7/site-packages/:${PYTHONPATH}
>>> export
>>> PYTHONPATH=${FDRAKE_DIR}/fiat/lib/python2.7/site-packages/:${PYTHONPATH}
>>> export
>>> PYTHONPATH=${FDRAKE_DIR}/ffc/lib/python2.7/site-packages/:${PYTHONPATH}
>>>
> 
> I added this to the template file on the wiki.

This *should* be added by loading fdrake-python-env

>> * The https cloning issue I believe is resolved, at least I can't
>> reproduce it anymore.
> 
> All seems to work fine for me now.
> 
>>
>> * The PyOP2 installation issue related to NumPy is a bug [1] in
>> setuptools 2.2 which comes with anaconda. For now, the best thing to do
>> is to remove/comment line 138 in PyOP2/setup.py which declares the setup
>> dependency on NumPy.
>>
> 
> I commented out the line  'setup_requires=setup_requires,' in setup.py
> and the PyOP2 installation goes through without problems.
> 
>> * The anaconda/environment modules conflict is resolved: I have removed
>> ANACONDA_LIB from the LD_LIBRARY_PATH and added it to LD_RUN_PATH
>> instead. This means Python extension modules get an RPATH to that
>> directory baked in and therefore can find the right libpython2.7.so
>> without any LD_LIBRARY_PATH being set (which is preferable anyway).
>>
> 
> Works for me now, I can run the module command even after loading the
> firedrake environment.
> 
>> Unfortunately, Firedrake's setup.py manually sets an RPATH for PETSc and
>> therefore the LD_RUN_PATH is ignored. This patch is required:
>>
>> diff --git a/setup.py b/setup.py
>> index 00838f8..350d51d 100644
>> --- a/setup.py
>> +++ b/setup.py
>> @@ -70,6 +70,7 @@ setup(name='firedrake',
>>                              include_dirs=include_dirs,
>>                              libraries=["petsc"],
>>                              extra_link_args=["-L%s/lib" % d for d in
>> petsc_dirs] +
>> -                             ["-Wl,-rpath,%s/lib" % d for d in
>> petsc_dirs]),
>> +                             ["-Wl,-rpath,%s/lib" % d for d in
>> petsc_dirs] +
>> +                             ["-Wl,-rpath,%s/lib" % sys.prefix]),
>>                    Extension('evtk.cevtk', evtk_sources,
>>                              include_dirs=[np.get_include()])])
>>
> 
> Can you send me that patch as a separate text file, please? When I
> copied it out of your email it didn't work.

I have pushed this to firedrake master.

> Still something not quite right with the python. When I run a 'Hello
> World!' test, i.e.
> 
> import sys
> print 'Hello World!'
> sys.exit(0)
> 
> I still get
>  
> python: error while loading shared libraries: libpython2.7.so.1.0:
> cannot open shared object file: No such file or directory

That works for me. Not sure why this would fail since you're not loading
an extension module. Is this on the login node with anaconda?

> but I guess this is due to the unresolved issue with anaconda that you
> reported to the helpdesk, so I will wait for what comes back from them.

Indeed, as reported to helpdesk, it was premature thinking the
LD_LIBRARY_PATH was no longer required. I'm currently stuck on mpi4py
even though all the RPATHs are correctly set afaict. For now you'll need
to add $ANANCONDA_LIBS to LD_LIBRARY_PATH in your firedrake.tpl

Florian

> Thanks,
> 
> Eike
> 
>> For FFC it's more tricky, still trying to figure out what's needed.
>>
>> Keep us posted on how things are progressing!
>>
>> Florian
>>
>>> I will go ahead and edit the wiki.
>>>
>>> Eike
>>>
>>>
>>> On 8 Sep 2014, at 15:53, Eike Mueller <E.Mueller at bath.ac.uk
>>> <mailto:E.Mueller at bath.ac.uk>
>>> <mailto:E.Mueller at bath.ac.uk>> wrote:
>>>
>>>> Hi Florian,
>>>>
>>>> thanks a lot for your reply and for updating the wiki, I got a bit
>>>> further now.
>>>>
>>>>>
>>>>> You're exactly right again: it seems that anaconda contains a
>>>>> version of
>>>>> the TCL libraries which are incompatible with those use by the
>>>>> environment modules. I'm not sure how to work around this, since we
>>>>> need
>>>>> the anaconda libs (as you figured out later) - will open a ticket.
>>>>
>>>> ok, I will use the environment script as it is and not worry about the
>>>> fact that the anaconda library breaks the module command at the
>>>> moment. Let's see if the support can figure something out.
>>>>
>>>>>
>>>>> I've changed the wiki page to clone via https, thanks!
>>>>
>>>> It works with the updated instructions on the webpage. Strangely, if I
>>>> clone a repository into a directory, delete this directory and then
>>>> try to clone again with exactly the same command, nothing happens.
>>>>
>>>>> This seems to happen because (for reasons I can't figure out)
>>>>> setuptools
>>>>> doesn't detect that NumPy is already installed and builds it from
>>>>> source. The primary cause of the error you're seeing is this:
>>>>> https://stackoverflow.com/a/21621493
>>>>
>>>> I will ignore this then, and just call the setup.py script twice in
>>>> the PyOP2 directory.
>>>>
>>>>> This is because you've removed the anaconda lib path, which also
>>>>> contains the Python libraries themselves, from LD_LIBRARY_PATH
>>>>
>>>> I tried a simpler hello-world script where I swap python -> anaconda
>>>> and run a python script. This does work.
>>>> Still trying to run the poisson example from the benchmark repository,
>>>> but the ARCHER queues seem to be terribly slow, even if I just run on
>>>> one node on the debug queue :-(
>>>>
>>>>>
>>>>> Absolutely, feel free to edit the wiki if you find more errors or
>>>>> things
>>>>> that are unclear.
>>>>>
>>>>
>>>> I think I already have access to the repository, so will edit if
>>>> necessary.
>>>>
>>>> Thanks a lot,
>>>>
>>>> Eike
>>>>
>>>>> Florian
>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Eike
>>>>>>
>>>>>> *** content of firedrake.env ***
>>>>>>
>>>>>> module swap PrgEnv-cray PrgEnv-gnu
>>>>>> module unload python
>>>>>> module add anaconda
>>>>>>
>>>>>> # Add fdrake module path
>>>>>> export FDRAKE_DIR=/work/y07/y07/fdrake
>>>>>> module use "$FDRAKE_DIR/modules"
>>>>>>
>>>>>> # Load build environment
>>>>>> module load fdrake-build-env
>>>>>> module load fdrake-python-env
>>>>>>
>>>>>> LD_LIBRARY_PATH=`echo $LD_LIBRARY_PATH | sed
>>>>>> 's/\/work\/y07\/y07\/cse\/anaconda\/1.9.2\/lib//g'`
>>>>>>
>>>>>> export
>>>>>> PYTHONPATH=$WORK/firedrake-bench:$WORK/firedrake-bench/pybench:$WORK/firedrake:$WORK/PyOP2:$PYTHONPATH

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2980 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mailman.ic.ac.uk/pipermail/firedrake/attachments/20140915/d7aa0884/attachment.p7s>