Parameter Control Plugins

Figure 1: Trace of CFS and UFS in Vampir.

Parameter Control Plugins (PCPs) are written as part of the READEX project. They are used for changing different hardware and software parameter values. A common interface is provided for each PCP, which contains the name of a parameter, a set function, an unset function, and a get current configuration function. RRL communicates with PCPs using this interface to set and unset the parameter values and also to retrieve the current configuration for the available parameters.

 

Installation

Requirements

The build procedure for the PCPs requires the following tools to be already installed:

  • READEX Runtime Library (RRL) as described [here].
  • Intel compiler version 2017.2.174/2018.1.163 or GCC (G++ and GFortran) version 6.3.0/7.1.0. Other Intel or GCC compiler versions can also be used, but have not been explicitly tested by the READEX developers.

Please make sure that the RRL version is also compiled with the same compiler as the one used for the PCPs.

Download

Please download the PCP from the following location and unpack it:

wget -c http://www.readex.eu/wp-content/uploads/2018/08/PCPs.tar.gz
tar -xzvf PCPs.tar.gz 

Configuring and installing the PCP’s

You may use the following naming scheme for the argument of the build script:

<Desired path for PCPs installation>/parameter_control_plugins/pcp_readex_<mpi version>_<compiler version>
<day of build>: build date in the form YYYY-MM-DD
<compiler>: for example: intel2017.2.174

To build the PCPs please now do:

cd PCPs
./build.sh <Desired path for PCPs installation>/parameter_control_plugins/pcp_readex_<mpi version>_<compiler version>

CPU Frequency Scaling (CFS) and Uncore Frequency Scaling (UFS)

READEX supports the tuning of core and uncore frequencies using various interfaces. To do so, we use libfreqgen, which we developed during the project.

Possible interfaces for libfreqgen are:

  1. msr and msr-safe, provided by the msr and msr-safe kernel module, available for Intel processors
  2. x86-adapt, provided by the x86_adapt kernel module (if found during installation), available for Intel processors
  3. likwid, provided by likwid and the cpufreq and msr / msr-safe kernel modules (if found during installation), for core and uncore tuning, respectively
  4. sysfs cpufreq, provided by the cpufreq kernel module. Only available for CPU frequency, but not for uncore frequency. Make sure that intel_pstate is deactivated if you want to use this option. Furthermore, check whether you can read and write

Usage in READEX

You can increase the verbosity of the plugins via the following environment variables:

SCOREP_TUNING_[UNCORE|CPU]_FREQ_PLUGIN_VERBOSE=[DEBUG|INFO|WARN|VERBOSE]

The uncore plugin checks whether you actually use all threads of a processor. Only if you use ALL of them, uncore frequency scaling will be tuned. The reason for this is that you might share the processor with other applications that do not like it when you play around with the uncore frequency. However, you can override this concern and tune UFS nevertheless by setting

CHECK_IF_NODE_FULLY_OCCUPIED=0

During Design Time Analysis with Periscope

Register the PCP plugins via

SCOREP_SUBSTRATE_RRL_PLUGINS=cpu_freq_plugin,uncore_freq_plugin

Make sure that your readex_config.xml contains the following under <Configuration> ... <tuningParameter>:

<frequency>
<min_freq>1300</min_freq>
<max_freq>2500</max_freq>
<freq_step>200</freq_step>
<default>2500</default>
</frequency>
<uncore>
<min_freq>1400</min_freq>
<max_freq>3000</max_freq>
<freq_step>400</freq_step>
<default>3000</default>
</uncore>

Here for each plugin, you can specify a minimum, a maximum, and a step size for the search (in MHz), make sure the frequencies fit your system.

During Runtime Application Tuning

Register the PCP plugins via

SCOREP_SUBSTRATE_RRL_PLUGINS=cpu_freq_plugin,uncore_freq_plugin

 

OpenMP

We implemented a parameter control plugin and a tuning plugin to tune OpenMP parameters. The tuning plugin allows the user to specify the number of threads that he/she wants to search.

Moreover, the plugin itself changes the scheduler policy according to the available policies specified in the OpenMP specification. Here the scheduler policy defines how OpenMP schedules loop iterations. Finally, the tuning plugin is able to change the chunk size which is associated to the scheduler. Each thread will take a set number of iterations, called a “chunk”, execute it, and then be assigned a new chunk when it is done. The chunk size specifies the number of iterations in a chunk which each thread gets assigned.

Usage in READEX

During Design Time Analysis with Periscope

Register the PCP plugins via

SCOREP_SUBSTRATE_RRL_PLUGINS=OpenMPTP

Make sure that your readex_config.xml contains the following under <Configuration> ... <tuningParameter>:

<openMPThreads>
<lower_value>1</lower_value>
<step>2</step>
</openMPThreads>
</tuningParameter>

Here for each plugin, you can specify a minimum, a maximum, and a step size for the the number of threads to search (in MHz). Make sure they fit your system.

During Runtime Application Tuning

Register the PCP plugin via

SCOREP_SUBSTRATE_RRL_PLUGINS=OpenMPTP

Sources

[GitHub] https://github.com/readex-eu/PCPs
[Download tarball]

[1] Robert Schöne and Daniel Molka. Integrating performance analysis and energy efficiency optimizations in a unified environment. Computer Science – R&D, 29(3-4):231–239, 2014.
DOI: 10.1007/s00450-013-0243-7.