Periscope Tuning Framework

Periscope Tuning Framework Logo

The Periscope Tuning Framework is a suite of tools designed to assist the HPC application developer in the optimization of their application. The current version is based on the Score-P monitoring infrastructure. Score-P provides performance analysis data and allows to dynamically configure tuning paramters of the application and the execution environment.

Using Periscope, an online instrumentation and measurement tool, the developers can more easily identify the bottlenecks of their application. Periscopes develops hypotheses about the bottlenecks of an application automatically and verifies them during run-time.

Using Periscope’s various tuning-plugins, it is possible to automatically find the optimal combination of settings such as compiler flags, MPI settings, the number of OpenMP threads in each parallel section, etc.

Using Pathway, the whole process of optimizing an application, including for example running jobs on the HPC system, adjusting the job’s settings and recording data can be formalized and partially automated.

Installation

Requirements

The build procedure for the READEX version of PTF requires the following tools to be already installed:

  • Score-P extension for READEX as described [here].
  • Intel compiler version 2017.2.174/2018.1.163 or GCC (G++ and GFortran) version 6.3.0/7.1.0. Other Intel or GCC compiler versions can also be used, but have not been explicitly tested by the READEX developers.
  • PAPI version 5.5.1.
  • Boost version 1.62.0.
  • Cereal version 1.2.1.
  • Bison version 3.0.4.
  • Python version 3.6 or higher.
  • Ace version 6.3.3.
  • Flex version 2.5.39.
  • Score-P developer tools containing patched Libtool version 2.4.6, Autoconf version 2.69, Automake version 1.13.4, Doxygen version 1.8.10 and M4 version 1.4.16.

Please make sure that the Score-P version is also compiled with the same compiler as the one used for PTF.

Download

Please download the READEX branch of PTF and Score-P development tools from the following locations, and unpack them:

wget -c http://www.readex.eu/wp-content/uploads/2018/08/Periscope_READEX.tar.gz
tar -xzvf Periscope_READEX.tar.gz
wget -c http://www.readex.eu/wp-content/uploads/2018/04/scorep-dev-06.tar.gz
tar -xzvf scorep-dev-06.tar.gz

Installing scorep-dev

cd scorep-dev-06
./install-scorep-dev.06.sh --prefix=<Desired path for scorep-dev installation>

Check version of Autotools

These have to be Libtool 2.4.6, Autoconf 2.69 and Automake 1.13.4.

autoconf --version
automake --version
libtool --version

Preparing the PTF directory

Please bootstrap PTF as follows:

cd ../PTF
./bootstrap
mkdir build
cd build

Configuring and installing PTF

Add the paths to the libraries and executables of the Score-P developer tools to PATH and LD LIBRARY PATH environment variables:

export PATH=<path to Score-P developer tools>/bin:$PATH version
export LD_LIBRARY_PATH=<path to Score-P developer tools>/lib:$LD_LIBRARY_PATH

You may use the following naming scheme for “–prefix”:

<Desired path for PTF installation>/ptf/ptf_readex_<mpi version>_<compiler version>_with_scorep/
<day of build>: build date in the form YYYY-MM-DD
<compiler>: for example: intel2017.2.174
<mpi version>: for example: intelmpi2017.2.174

To configure and install PTF please now do:

../configure '--prefix=<Desired path for PTF installation>/ptf/ptf_readex_<mpi version>_<compiler version>_with_scorep/'
--enable-developer-mode \
--with-starter=<slurm|superMUC> \
--with-ace-include=<path to ACE include> \
--with-ace-lib=<path to ACE lib> \
--with-boost-include=<path to Boost include> \
--with-boost-lib=<path to Boost lib> \
--with-cube-include=<path to Cube include> \
--with-cube-lib=<path to Cube lib>' \
--with-scorep-include=<path to Score-P include> \
--with-scorep-lib=<path to Score-P lib> \
--with-cereal-include=<path to Cereal include>
make -j
make install

If your system is not supported by one of the available starter plugins, please contact contact the Periscope developers.

If you want to use the Intel compiler to compile PTF, please add the following to “../configure” :

--with-compiler-suite=intel

Please be aware that there is no special cube module needed, if you are using the same compiler for Score-P and PTF. Therefore the options for “–with-cube-include” and “–with-cube-lib” are the same as for “–with-scorep-include” and “–with-scorep-lib”.

For more details on installing PTF, refer to Section 2.3 in [http://periscope.in.tum.de/releases/latest/pdf/PTF_Installation_Guide.pdf].

Usage in READEX

In READEX, we use Periscope during Design Time Analysis. To do so, the application must be prepared and compiled as in our usage guide given here. Before it can be applied, you need a configuration file (readex_config.xml) that describes the tuning step. The configuration file is created by the readex-dyn-detect tool that is part of the READEX Periscope installation. Based on its output, you can select different tuning plugins, e.g., readex_intraphase and readex_interphase.

Once you prepared everything, start the Periscope frontend to run the tuning process. This will create the file tuning_model.json, which will be used during Runtime Application Tuning.

psc_frontend --phase="<Loop-phase-name>"
--apprun="<binary>" --mpinumprocs=<nr processes>
--ompnumthreads=<nr threads> --tune=readex_intraphase
--config-file=readex_config.xml

Sources

[Website] http://periscope.in.tum.de
[GitHub] https://github.com/readex-eu/readex-ptf
[Download tarball]

[1] A. Sikora, E. César, I. Compres, M. Gerndt, “Autotuning of MPI applications Using PTF” Workshop Proceedings of HPDC 2016, International Workshop on Software Engineering Methods for HPC (SEM4HPC), Kyoto, Japan, 2016.
doi: 10.1145/2916026.2916028