[firedrake] (Even more) progress on installing PyOP2/firedrake on ARCHER
Eike Mueller
e.mueller at bath.ac.uk
Mon Sep 15 09:20:55 BST 2014
Hi Florian,
I'm having another go at this now.
>>
>> export
>> PYTHONPATH=${FDRAKE_DIR}/petsc4py/lib/python2.7/site-packages:${PYTHONPATH}
>> export
>> PYTHONPATH=${FDRAKE_DIR}/ufl/lib/python2.7/site-packages/:${PYTHONPATH}
>> export
>> PYTHONPATH=${FDRAKE_DIR}/fiat/lib/python2.7/site-packages/:${PYTHONPATH}
>> export
>> PYTHONPATH=${FDRAKE_DIR}/ffc/lib/python2.7/site-packages/:${PYTHONPATH}
>>
I added this to the template file on the wiki.
>
> * The https cloning issue I believe is resolved, at least I can't
> reproduce it anymore.
All seems to work fine for me now.
>
> * The PyOP2 installation issue related to NumPy is a bug [1] in
> setuptools 2.2 which comes with anaconda. For now, the best thing to do
> is to remove/comment line 138 in PyOP2/setup.py which declares the setup
> dependency on NumPy.
>
I commented out the line 'setup_requires=setup_requires,' in setup.py and the PyOP2 installation goes through without problems.
> * The anaconda/environment modules conflict is resolved: I have removed
> ANACONDA_LIB from the LD_LIBRARY_PATH and added it to LD_RUN_PATH
> instead. This means Python extension modules get an RPATH to that
> directory baked in and therefore can find the right libpython2.7.so
> without any LD_LIBRARY_PATH being set (which is preferable anyway).
>
Works for me now, I can run the module command even after loading the firedrake environment.
> Unfortunately, Firedrake's setup.py manually sets an RPATH for PETSc and
> therefore the LD_RUN_PATH is ignored. This patch is required:
>
> diff --git a/setup.py b/setup.py
> index 00838f8..350d51d 100644
> --- a/setup.py
> +++ b/setup.py
> @@ -70,6 +70,7 @@ setup(name='firedrake',
> include_dirs=include_dirs,
> libraries=["petsc"],
> extra_link_args=["-L%s/lib" % d for d in
> petsc_dirs] +
> - ["-Wl,-rpath,%s/lib" % d for d in
> petsc_dirs]),
> + ["-Wl,-rpath,%s/lib" % d for d in
> petsc_dirs] +
> + ["-Wl,-rpath,%s/lib" % sys.prefix]),
> Extension('evtk.cevtk', evtk_sources,
> include_dirs=[np.get_include()])])
>
Can you send me that patch as a separate text file, please? When I copied it out of your email it didn't work.
Still something not quite right with the python. When I run a 'Hello World!' test, i.e.
import sys
print 'Hello World!'
sys.exit(0)
I still get
python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
but I guess this is due to the unresolved issue with anaconda that you reported to the helpdesk, so I will wait for what comes back from them.
Thanks,
Eike
> For FFC it's more tricky, still trying to figure out what's needed.
>
> Keep us posted on how things are progressing!
>
> Florian
>
>> I will go ahead and edit the wiki.
>>
>> Eike
>>
>>
>> On 8 Sep 2014, at 15:53, Eike Mueller <E.Mueller at bath.ac.uk
>> <mailto:E.Mueller at bath.ac.uk>> wrote:
>>
>>> Hi Florian,
>>>
>>> thanks a lot for your reply and for updating the wiki, I got a bit
>>> further now.
>>>
>>>>
>>>> You're exactly right again: it seems that anaconda contains a version of
>>>> the TCL libraries which are incompatible with those use by the
>>>> environment modules. I'm not sure how to work around this, since we need
>>>> the anaconda libs (as you figured out later) - will open a ticket.
>>>
>>> ok, I will use the environment script as it is and not worry about the
>>> fact that the anaconda library breaks the module command at the
>>> moment. Let's see if the support can figure something out.
>>>
>>>>
>>>> I've changed the wiki page to clone via https, thanks!
>>>
>>> It works with the updated instructions on the webpage. Strangely, if I
>>> clone a repository into a directory, delete this directory and then
>>> try to clone again with exactly the same command, nothing happens.
>>>
>>>> This seems to happen because (for reasons I can't figure out) setuptools
>>>> doesn't detect that NumPy is already installed and builds it from
>>>> source. The primary cause of the error you're seeing is this:
>>>> https://stackoverflow.com/a/21621493
>>>
>>> I will ignore this then, and just call the setup.py script twice in
>>> the PyOP2 directory.
>>>
>>>> This is because you've removed the anaconda lib path, which also
>>>> contains the Python libraries themselves, from LD_LIBRARY_PATH
>>>
>>> I tried a simpler hello-world script where I swap python -> anaconda
>>> and run a python script. This does work.
>>> Still trying to run the poisson example from the benchmark repository,
>>> but the ARCHER queues seem to be terribly slow, even if I just run on
>>> one node on the debug queue :-(
>>>
>>>>
>>>> Absolutely, feel free to edit the wiki if you find more errors or things
>>>> that are unclear.
>>>>
>>>
>>> I think I already have access to the repository, so will edit if
>>> necessary.
>>>
>>> Thanks a lot,
>>>
>>> Eike
>>>
>>>> Florian
>>>>
>>>>> Cheers,
>>>>>
>>>>> Eike
>>>>>
>>>>> *** content of firedrake.env ***
>>>>>
>>>>> module swap PrgEnv-cray PrgEnv-gnu
>>>>> module unload python
>>>>> module add anaconda
>>>>>
>>>>> # Add fdrake module path
>>>>> export FDRAKE_DIR=/work/y07/y07/fdrake
>>>>> module use "$FDRAKE_DIR/modules"
>>>>>
>>>>> # Load build environment
>>>>> module load fdrake-build-env
>>>>> module load fdrake-python-env
>>>>>
>>>>> LD_LIBRARY_PATH=`echo $LD_LIBRARY_PATH | sed
>>>>> 's/\/work\/y07\/y07\/cse\/anaconda\/1.9.2\/lib//g'`
>>>>>
>>>>> export
>>>>> PYTHONPATH=$WORK/firedrake-bench:$WORK/firedrake-bench/pybench:$WORK/firedrake:$WORK/PyOP2:$PYTHONPATH
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Dr Eike Hermann Mueller
>>>>> Research Associate (PostDoc)
>>>>>
>>>>> Department of Mathematical Sciences
>>>>> University of Bath
>>>>> Bath BA2 7AY, United Kingdom
>>>>>
>>>>> +44 1225 38 5803
>>>>> e.mueller at bath.ac.uk
>>>>> <mailto:e.mueller at bath.ac.uk> <mailto:e.mueller at bath.ac.uk>
>>>>> http://people.bath.ac.uk/em459/
>>>>
>>>> _______________________________________________
>>>> firedrake mailing list
>>>> firedrake at imperial.ac.uk <mailto:firedrake at imperial.ac.uk>
>>>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>>>
>>> _______________________________________________
>>> firedrake mailing list
>>> firedrake at imperial.ac.uk <mailto:firedrake at imperial.ac.uk>
>>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>>
>>
>>
>> _______________________________________________
>> firedrake mailing list
>> firedrake at imperial.ac.uk
>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>>
>
> _______________________________________________
> firedrake mailing list
> firedrake at imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-------------- next part --------------
HTML attachment scrubbed and removed
More information about the firedrake
mailing list