Emil M.
Constantinescu
Computational Mathematician
emconsta[at]mcs.anl.gov
Office #: 2151
• LANS
• Mathematics and Computer Science
• Argonne National Laboratory

• Scientist at large at CASE
• University of Chicago

Efficient High-Order Time-Integrators for Local High-Order Discretization Methods

Funding: Computational Mathematics AFOSR

Synopsis: This project is concerned with the development of high-order accurate and highly efficient time-integration methods for local high-order spatial discretizations of hyperbolic partial differential equations typically found in fluid flow problems. The focus is on the numerical solution of the shallow water, Euler, and compressible Navier-Stokes equations. For this class of differential equations, local high-order methods such as finite volumes, finite elements, and finite differences have shown to give excellent solution accuracy while scaling well on the current class of multi-core parallel computers. These efforts will be implemented and tested on the Non-hydrostatic Unified Model of the Atmosphere (NUMA)

Simulation of the rising thermal bubble hitting the top wall:

Papers, Conference Presentations, and Proceedings

• Hong Zhang, Zhengyu Liu, Emil M. Constantinescu, and Robert Jacob, "Stability analysis of interface conditions for ocean-atmosphere coupling." Submitted, 2019. [https://arxiv.org/abs/1909.00916]

   

• Simone Marras, Michal A. Kopera, Emil M. Constantinescu, Jenny Suckale, and Francis X. Giraldo "A residual-based shock capturing scheme for the continuous/discontinuous spectral element solution of the 2D shallow water equations." Vol. 114, Pages 45-63, Advances in Water Resources, 2018. [http://arxiv.org/abs/1607.04547]

   

• Debojyoti Ghosh* and Emil M. Constantinescu, "Nonlinear compact finite-difference schemes with semi-implicit time stepping."Springer's Lecture Notes in Computational Science and Engineering (LNCSE) Series, ICOSAHOM 2014, Vol. 106, Pages 237-245, 2015.

   

• Debojyoti Ghosh* and Emil M. Constantinescu, "Semi-implicit time integration of atmospheric flows with characteristic-based flux partitioning." SIAM Journal on Scientific Computing (SISC), Vol. 38(3), Pages A1848-A1875, 2016. [Preprint # ANL/MCS-P5417-1015; http://arxiv.org/abs/1510.05751].

   

• Debojyoti Ghosh* and Emil M. Constantinescu, and Jed Brown, "A well-balanced conservative finite-difference algorithm for atmospheric flows." AAIA Journal, Accepted, 2015. [Preprint # ANL/MCS-P5371-0615].

   

• Francis X Giraldo, James F. Kelly, and Emil Constantinescu, "Implicit-explicit formulations of a three-dimensional nonhydrostatic unified model of the atmosphere (NUMA)." SIAM Journal on Scientific Computing (SISC), Vol 35(5), pp B1162-B1194, 2013. [Preprint # ANL/MCS-P2083-0512]

   

Building NUMA3dCG with PETSc:

Configure & build PETSc with debug options and optimized (gFortran on Mac):

Downloading PETSc:

[w/ sshkey] git clone [email protected]:petsc/petsc.git petsc-dev
[w/o ssh key] git clone https://bitbucket.org/petsc/petsc petsc-dev

Building PETSc:

cd petsc-dev

Building NetCDF's Fortran API:

<move to the NUMA repository and get netcdf-fortran-xx download it, and read instructions or follow>
[e.g.] wget http://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-fortran-4.2.tar.gz
tar -xzvf netcdf-fortran-4.2.tar.gz
cd netcdf-fortran-4.2
export FC=$PETSC_DIR/$PETSC_ARCH/bin/mpif90
export CC=$PETSC_DIR/$PETSC_ARCH/bin/mpicc
export LD_LIBRARY_PATH=$PETSC_DIR/$PETSC_ARCH/lib:$LD_LIBRARY_PATH
export CPPFLAGS="-DgFortran -I$PETSC_DIR/$PETSC_ARCH/include"
export LDFLAGS=-L$PETSC_DIR/$PETSC_ARCH/lib
./configure --prefix=$PETSC_DIR/$PETSC_ARCH

make
make check
make install

Building NUMA:

<move to numa3dCG_PETSc>

<define Metis and NetCDF paths e.g., in .bashrc>
export METIS_DIR=$PETSC_DIR/$PETSC_ARCH
export NETCDF_DIR=$PETSC_DIR/$PETSC_ARCH

make mcs_petsc_debug

<make sure /lib was created otherwise create it and re-run make>

Running NUMA:

<here's a config file used in preliminary testing for case 9 numa3dCG.in and for case 21 numa3dCG.in>

<here's an example run command: (from the run folder) >

$PETSC_DIR/$PETSC_ARCH/bin/mpiexec -np 6 ../bin/numa3dCG -{ts,ksp,snes}_monitor -ts_adapt_type none -use-petsc true -snes_mf

[Top]

Configure & build PETSc with debug options (gFortran on Linux):

Downloading PETSc:

[w/ sshkey] git clone [email protected]:petsc/petsc.git petsc-dev
[w/o ssh key] git clone https://bitbucket.org/petsc/petsc petsc-dev

Building PETSc:

cd petsc-dev

Building NetCDF's Fortran API:

<move to the NUMA repository and get netcdf-fortran-xx download it, and read instructions or follow>
[e.g.] wget http://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-fortran-4.2.tar.gz
tar -xzvf netcdf-fortran-4.2.tar.gz
cd netcdf-fortran-4.2
export FC=$PETSC_DIR/$PETSC_ARCH/bin/mpif90
export CC=$PETSC_DIR/$PETSC_ARCH/bin/mpicc
export LD_LIBRARY_PATH=$PETSC_DIR/$PETSC_ARCH/lib:$LD_LIBRARY_PATH
export CPPFLAGS="-DgFortran -I$PETSC_DIR/$PETSC_ARCH/include"
export LDFLAGS=-L$PETSC_DIR/$PETSC_ARCH/lib
./configure --prefix=$PETSC_DIR/$PETSC_ARCH

make
make check
make install

Building NUMA:

<move to numa3dCG_PETSc>

<define Metis and NetCDF paths e.g., in .bashrc>
export METIS_DIR=$PETSC_DIR/$PETSC_ARCH
export NETCDF_DIR=$PETSC_DIR/$PETSC_ARCH

make mcs_petsc_debug

<make sure /lib was created otherwise create it and re-run make>

Running NUMA:

<here's a config file used in preliminary testing for case 9 numa3dCG.in and for case 21 numa3dCG.in>

<here's an example run command: (from the run folder) >

$PETSC_DIR/$PETSC_ARCH/bin/mpiexec -np 6 ../bin/numa3dCG -{ts,ksp,snes}_monitor -ts_adapt_type none -use-petsc true -snes_mf

Configure & build PETSc with debug options for the Fusion machine:

edit .soft to include:
+cmake
+mercurial
+git
@default

./configure --PETSC_ARCH=arch-linux2-c-debug --with-hdf5-include=/soft/hdf5/1.8.6-parallel/include --with-hdf5-lib="-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs" --with-mpi-dir=/soft/mvapich2/1.4.1-intel-11.1.064 --with-shared-libraries=0 --with-netcdf-include=/soft/netcdf/4.1.1-parallel/include --with-netcdf-lib="-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz" --download-metis=yes

make PETSC_DIR=$PWD PETSC_ARCH=arch-linux2-c-debug all test

<move to numa3dCG_PETSc>

make mcs_fusion_petsc_debug

<make sure /lib was created otherwise create it and re-run make>

<here's a config file used in preliminary testing numa3dCG.in>

Configure & build PETSc with debug options for the Beagle machine:

module swap PrgEnv-cray PrgEnv-pgi;
module swap PrgEnv-pgi PrgEnv-cray;
module load cmake

./configure --PETSC_ARCH=arch-cray-c-debug --download-metis=yes --with-cc=cc --with-clanguage=C++ --with-clib-autodetect=0 --with-cxx=CC --with-cxxlib-autodetect=0 --with-fc=ftn --with-fortranlib-autodetect=0 --with-x=0 --FFLAGS='-F -em' --LIBS='-L/opt/cray/cce/8.1.4/CC/x86-64/lib/x86-64/ -lcray-c++-rts -lcraystdc++ -lsupc++ -lgcc_eh' --with-shared-libraries=0

make PETSC_DIR=$PWD PETSC_ARCH=arch-cray-c-debug all test