[firedrake] Hardware counters for Firedrake

[Justin Chang]

First option works wonderfully for me, but now I am wondering how I would
employ the second option.

Specifically, I want to profile SNESSolve()

I would prefer to circumvent profiling of the DMPlex distribution because
it seems that is a major bottleneck for multiple processes at the moment.

How would I do this?

Thanks,
Justin

On Thu, Jul 9, 2015 at 3:02 PM, Justin Chang <jychang48 at gmail.com> wrote:

> Lawrence, I will give this a try, thanks.
>
> On Thu, Jul 9, 2015 at 7:41 AM, Lawrence Mitchell <
> lawrence.mitchell at imperial.ac.uk> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi Justin,
>>
>> On 09/07/15 09:18, Justin Chang wrote:
>> > Hi everyone,
>> >
>> > I am very new to Firedrake, i recently switched over from FEniCS,
>> > so pardon me if this is a newbie question.
>> >
>> > If I wanted to run hardware performance counters from PAPI to
>> > diagnose my firedrake application how would I go about doing this?
>> > In FEniCS, my application was written in C++ and I included the
>> > PAPI directives and libraries in the CMakefile. With this I was
>> > able to specify where in the source code I wanted to start and stop
>> > the counters.
>> >
>> > Is there a way to somehow replicate this if my code is written
>> > using Firedrake?
>>
>> This is somewhat tricky, since we generate all code to spin over the
>> mesh for assembly at runtime, and the calls into PETSc happen purely
>> from petsc4py.  There are two approaches which will have slightly
>> different measurement characteristics:
>>
>>
>> 1. Measure from Python
>>
>> I wrote some bindings for the high-level PAPI API that are callable
>> from Python.  They seem to be broadly functional.  See
>>
>> https://github.com/firedrakeproject/PyPAPI
>>
>> To use these you do:
>>
>> import pypapi
>> import numpy as np
>>
>> events = np.asarray([pypapi.Event.TOT_CYC], dtype=pypapi.EventType)
>>
>> counts = np.zeros(1, dtype=pypapi.CountType)
>>
>> pypapi.start_counters(events)
>> # Do your computation (for example, assembling an operator)
>> ...
>> pypapi.stop_counters(counts)
>>
>> # counts now contains the number of cycles elapsed between start
>> # and stop
>>
>> Note that this approach includes cycle/flop counts and timings for the
>> full stack of computation: that is, it counts work in the python
>> interpreter as well as the low-level C that does the assembly loop.
>> In some sense, this is the fairest thing to report (since that's the
>> work your code is actually doing).
>>
>> 2. Measure in the generated code
>>
>> This gives you finer-grained control over where you want to actually
>> count things, and gets you much closer to the metal.  However, it's
>> (much) trickier to do.
>>
>> Firedrake defers to PyOP2 to do computation on meshes, and it is PyOP2
>> which generates the code to do this.  If you want to insert PAPI calls
>> here there are two things to consider:
>>
>> 1. Where exactly you want to measure.  I guess you'll want to measure
>> around a complete assembly over the mesh, rather than the execution of
>> an individual kernel.
>>
>> 2. How to get the information back out again so you can inspect it.
>>
>> As to the former, I think it should be possible to extend the code
>> generation to optionally insert the correct PAPI calls.  Returning the
>> information is also probably not too difficult, we can stuff the
>> results in a Python dict (or similar) that you can then query from PyOP2.
>>
>>
>> My recommendation is to try the pure Python approach first.  If you
>> want finer-grained control, we can provide guidance and/or help on how
>> to best do that.
>>
>> Cheers,
>>
>> Lawrence
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1
>>
>> iQEcBAEBAgAGBQJVnmvxAAoJECOc1kQ8PEYvSqYH/iL2eE6Vz5OF9yG8yIFRAghN
>> h6VOYZ5GgA420U8k6sjP1fBdHwidAUV8gtVLhBuj7UskTWf/HwSVvfUwf1KbPUie
>> fAzopYfuDXVolR3uYnUX5GDYDin1gkCCRRbOrk0wU98bSOyaRmcgmg0l3aDp4VTO
>> zHruPKzxyhgqoaWqcq5y7HFTCa3TYPBfdoJ3mu3zK4U8AWKMnuhKq24BEA6pBZfV
>> fSf4MvlCMKSmxOFTHYeOuPExmtM3m4y049Me88uxRi0gS3fTb1Yhhuhi9zF9DRXC
>> YIt1xKjZ+kudwiII22MA35rlXYjqxtShqMDuPC2HZi3xuIuANEWIGAE9uqhx4aY=
>> =HJsh
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> firedrake mailing list
>> firedrake at imperial.ac.uk
>> https://mailman.ic.ac.uk/mailman/listinfo/firedrake
>>
>
>
-------------- next part --------------
HTML attachment scrubbed and removed