[Nektar-users] memory usage and computation time in Taylor-Green problem

[Vishal Saini]

Hi all,

After obtaining exciting error vs. computational cost results from
relatively heavy 2D computations, I've been setting up the 3D Taylor Green
vortex (TGV) problem using Nektar++ Incompressible solver. The Reynolds
number used is 1600 (same as in the TGV tutorial from Nektar++ team). A 3D
grid of 64^3 elements with NumModes=5 was used to obtain a 256^3 simulation
(close to a DNS). I ran this problem on Midlands+ Tier 2 machine using 4
nodes (112 cores, 512 GB RAM in total).
http://www.hpc-midlands-plus.ac.uk/about/system-description/

The memory consumption during the run was ~450GB and it took ~330
wall-clock-minutes for 1000 time-steps (with dt=1e-4). For a reference, I
ran a 256^3 TGV simulation of the same case in OpenFOAM using same
resources. The memory consumption was ~40GB and it took ~20 minutes for
1000 time-steps.

So for this case, I observe high resource consumption in terms of memory
and computation time. I was not expecting this on the basis of my
experience with Nektar++ on 2D simulations, at least in terms of
computation time.

Is this normal? In addition to my set-up file, I'm suspecting my
installation of Nektar++ on the aforementioned cluster. Could anyone please
try and run a few hundred time-steps on their machines using the files
attached? FYI, I'm partitioning on-the-fly. The calculation seems to scale
well on 8 nodes in terms of wall clock time while increasing the memory
consumption by ~10%.

Any input is highly appreciated.

Best regards,
Vishal


---
*Vishal SAINI*
Master of Research,
University of Cambridge.
-------------- next part --------------
HTML attachment scrubbed and removed
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nektar_user_tgv.tar.gz
Type: application/gzip
Size: 14028126 bytes
Desc: not available
URL: <https://mailman.ic.ac.uk/mailman/private/nektar-users/attachments/20180319/89c4d3ee/attachment.bin>