READEX Runtime Library

The READEX Runtime Library (RRL) is written as part of the READEX project. It performs the detection of the system scenarios at runtime, applies the Application Tuning Model generated at design time by adjusting tuning parameters and an efficient run-time scenario calibration. The design goal of the RRL was to create an easy to modify and easy to extend library which provides low-overhead switching capabilities in order to keep the impact of the library on application performance as low as possible. The RRL is implemented as a Score-P substrate plugin to avoid direct integration of the tuning functionality into the Score-P code base. The RRL performs the detection of runtime situations, the classification of runtime situations into scenarios, the selection of configurations and the application of these configurations through Parameter Control Plugins. Moreover, it includes the interaction between the tuning model, the Score-P substrate plugin interface (used for the detection of runtime situations), the Online Access interface (used at design time to control parameter settings), and the calibration mechanism. It also include the support for applying Application Tuning Parameters (ATP) tuning both at design time at runtime.

Installation

Requirements

The build procedure for RRL requires the following tools to be already installed:

  • Score-P extension for READEX as described [here].
  • Intel compiler version 2017.2.174/2018.1.163 or GCC (G++ and GFortran) version 6.3.0/7.1.0. Other Intel or GCC compiler versions can also be used, but have not been explicitly tested by the READEX developers.
  • CMake version 3.11 or higher.
  • PAPI version 5.5.1.

Please make sure that the Score-P version is also compiled with the same compiler as the one used for RRL.

Download

Please download RRL from the following location and unpack it:

wget -c http://www.readex.eu/wp-content/uploads/2018/08/RRL.tar.gz
tar -xzvf RRL.tar.gz

Preparing the RRL directory

Please switch to the RRL directory of the downloaded repository:

cd RRL
mkdir build
cd build

Configuring and installing the RRL

You may use the following naming scheme for “–DCMAKE_INSTALL_PREFIX”:

<Desired path for RRL installation>/readex-rrl/rrl_readex_<mpi version>-<compiler version>
<day of build>: build date in the form YYYY-MM-DD
<compiler>: for example: intel2017.2.174
<mpi version>: for example: intelmpi2017.2.174

To build the RRL, you will need x86 adapt [Link]. Please be sure to have the path to libx86_adapt.so in your LD_LIBRARY_PATH.


export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to libx86_adapt.so>

Now, to build the RRL please do:

cmake ../ --prefix=<Desired path for Score-P installation>/readex-rrl/rrl_readex_<mpi version>-<compiler version> -DDISABLE_CALIBRATION=ON
make -j
make install

NOTE: You may omit x86_adapt and PAPI from the build process if you choose to use RRL without the calibration feature.

Usage

The following steps describe how to use RRL to tune the application during its production run.

  1. If Application Tuning Parameters are exploited in the application then the ATP related instrumentation functions should remain in the code.
  2. For the application run tuned with RRL, use the application built for analysis with PTF as described in Section 3 (page 6) of the usage guide. If the readex_interphase tuning plugin was used for DTA, perform the following steps before proceeding with RAT:
    • To perform runtime tuning after the inter-phase analysis, the cluster prediction library must be invoked. The SCOREP_USER_PARAMETER macro must be added to the source code, with a call to the predict_cluster function of the cluster prediction library.
      For C/C++ applications:
      #include
      // loop starts
      SCOREP_USER_OA_PHASE_BEGIN( REGION_HANDLE, "PHASE_REGION_NAME", SCOREP_USER_REGION_TYPE_COMMON )
      SCOREP_USER_PARAMETER_INT64( "Cluster", predict_cluster() )
      // loop body (phase region)
      SCOREP_USER_OA_PHASE_END( REGION_HANDLE )
      // loop ends
      For Fortran applications:
      SCOREP_USER_REGION_DEFINE( REGION_HANDLE )
      SCOREP_USER_PARAMETER_DEFINE(cluster)
      // loop starts
      SCOREP_USER_OA_PHASE_BEGIN( REGION_HANDLE, "PHASE_REGION_NAME", SCOREP_USER_REGION_TYPE_COMMON )
      SCOREP_USER_PARAMETER_INT64( "Cluster", predict_cluster() )
      // loop body (phase region)
      SCOREP_USER_OA_PHASE_END( REGION_HANDLE )
      // loop ends
    • Add the linker flags in the Makefile and rebuild the application.
  3. Set the number of nodes to run the application, and allocate enough memory to fit the application (lines 1-6). Here, the number of nodes required is the same as the number of nodes on which to run the application.
  4. For the RRL-tuned run of the application (lines 68-106) perform the following steps:
    1. Disable Score-P profiling and tracing (lines 13 and 14), set the Score-P substrate plugins to rrl, RRL plugins to the tuning plugins to use and the tuning model to the file generated by PTF (lines 15-17).
    2. Depending on whether the calibration mechanism is enabled or disabled for RRL, set the environment variable SCOREP_FORCE_CFG_FILES to true or false, respectively, to enable or disable generation of Score-P configuration file for use by RRL (line 23).
    3. Run the RRL-tuned version of the application (line 26) using the application executable built for PTF.

1 ### Batch system (SLURM, PBS, ...) directives ###
2 ### number of nodes requested for application run
3 ### ...
4 ### memory requested for application run
5 ### ...
6 ###
7
8 ###
9 ### Load READEX tool suite modules, or set paths to executables and libraries
10 ###
11
12 ### start RRL-tuned run
13 export SCOREP_ENABLE_PROFILING="false"
14 export SCOREP_ENABLE_TRACING="false"
15 export SCOREP_SUBSTRATE_PLUGINS="rrl"
16 export SCOREP_RRL_PLUGINS=
17 export SCOREP_RRL_TMM_PATH=
18 export SCOREP_MPI_ENABLE_GROUPS=ENV
19
20 ### If RRL calibration mechanism is enabled, then set SCORE_FORCE_CFG_FILES to true.
21 ### This will create Score-P configuration files in a Score-P directory for the application run.
22 ### If RRL calibration mechanish is disabled, then set SCORE_FORCE_CFG_FILES to false.
23 export SCOREP_FORCE_CFG_FILES=false
24
25 ### run RRL-tuned application
26 # code to start application run with command-line arguments
27 ### end RRL-tuned run

Sources

[GitHub] https://github.com/readex-eu/readex-rrl
[Download tarball]