benchbuild.utils package

Submodules

benchbuild.utils.actions module

This defines classes that can be used to implement a series of Actions.

class benchbuild.utils.actions.Any(actions)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Just run all actions, no questions asked.'

NAME = 'ANY'

class benchbuild.utils.actions.Build(project)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Build the project'

NAME = 'BUILD'

class benchbuild.utils.actions.Clean(project_or_experiment, action_fn=None, check_empty=False)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Cleans the build directory'

NAME = 'CLEAN'

class benchbuild.utils.actions.CleanExtra(project_or_experiment, action_fn=None)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Cleans the extra directories.'

NAME = 'CLEAN EXTRA'

class benchbuild.utils.actions.Configure(project)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Configure project source files'

NAME = 'CONFIGURE'

class benchbuild.utils.actions.Download(project)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Download project source files'

NAME = 'DOWNLOAD'

class benchbuild.utils.actions.Echo(message)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Print a message.'

NAME = 'ECHO'

class benchbuild.utils.actions.Experiment(experiment, actions)

Bases: benchbuild.utils.actions.Any

DESCRIPTION = 'Run a experiment, wrapped in a db transaction'

NAME = 'EXPERIMENT'

begin_transaction()

end_transaction(experiment, session)

class benchbuild.utils.actions.MakeBuildDir(project_or_experiment, action_fn=None)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Create the build directory'

NAME = 'MKDIR'

class benchbuild.utils.actions.Prepare(project)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Prepare project build folder'

NAME = 'PREPARE'

class benchbuild.utils.actions.RequireAll(actions)

Bases: benchbuild.utils.actions.Step

class benchbuild.utils.actions.Run(project)

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Execute the run action'

NAME = 'RUN'

class benchbuild.utils.actions.Step(project_or_experiment, action_fn=None)

Bases: object

DESCRIPTION = None

NAME = None

onerror()

class benchbuild.utils.actions.StepClass

Bases: abc.ABCMeta

class benchbuild.utils.actions.StepResult

Bases: enum.Enum

An enumeration.

ERROR = 2

OK = (1,)

benchbuild.utils.actions.log_before_after(name, desc)

benchbuild.utils.actions.to_step_result(f)

benchbuild.utils.bootstrap module

Helper functions for bootstrapping external dependencies.

benchbuild.utils.bootstrap.check_uchroot_config()

benchbuild.utils.bootstrap.find_package(binary)

benchbuild.utils.bootstrap.install_package(pkg_name)

benchbuild.utils.bootstrap.install_uchroot()

benchbuild.utils.bootstrap.linux_distribution_major()

benchbuild.utils.bootstrap.provide_package(pkg_name)

benchbuild.utils.bootstrap.provide_packages(pkg_names)

benchbuild.utils.compiler module

Helper functions for dealing with compiler replacement.

This provides a few key functions to deal with varying/measuring the compilers used inside the benchbuild study. From a high-level view, there are 2 interesting functions:

• lt_clang(cflags, ldflags, func)
• lt_clang_cxx(cflags, ldflags, func)

These generate a wrapped clang/clang++ in the current working directory and hide the given cflags/ldflags from the calling build system. Both will give you a working plumbum command and call a python script that redirects to the real clang/clang++ given the additional cflags&ldflags.

The wrapper-script generated for both functions can be found inside:

• wrap_cc()

The remaining methods:

• llvm()
• llvm_libs()
• clang()
• clang_cxx()

Are just convencience methods that can be used when interacting with the configured llvm/clang source directories.

benchbuild.utils.compiler.clang()

Get a usable clang plumbum command.

This searches for a usable clang in the llvm binary path (See llvm()) and returns a plumbum command to call it.

Returns:plumbum Command that executes clang++

benchbuild.utils.compiler.clang_cxx()

Get a usable clang++ plumbum command.

This searches for a usable clang++ in the llvm binary path (See llvm()) and returns a plumbum command to call it.

Returns:plumbum Command that executes clang++

benchbuild.utils.compiler.llvm()

Get the path where all llvm binaries can be found.

Environment variable:

BB_LLVM_DIR

Returns:LLVM binary path.

benchbuild.utils.compiler.llvm_libs()

Get the path where all llvm libraries can be found.

Environment variable:

BB_LLVM_DIR

Returns:LLVM library path.

benchbuild.utils.compiler.lt_clang(cflags, ldflags, func=None)

Return a clang that hides CFLAGS and LDFLAGS.

This will generate a wrapper script in the current directory and return a complete plumbum command to it.

Parameters:

• cflags – The CFLAGS we want to hide.
• ldflags – The LDFLAGS we want to hide.
• func (optional) – A function that will be pickled alongside the compiler. It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.

Returns (benchbuild.utils.cmd):

Path to the new clang command.

benchbuild.utils.compiler.lt_clang_cxx(cflags, ldflags, func=None)

Return a clang++ that hides CFLAGS and LDFLAGS.

This will generate a wrapper script in the current directory and return a complete plumbum command to it.

Parameters:

• cflags – The CFLAGS we want to hide.
• ldflags – The LDFLAGS we want to hide.
• func (optional) – A function that will be pickled alongside the compiler. It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.

Returns (benchbuild.utils.cmd):

Path to the new clang command.

benchbuild.utils.compiler.wrap_cc_in_uchroot(cflags, ldflags, func=None, cc_name='clang')

Generate a clang wrapper that may be called from within a uchroot.

This basically does the same as lt_clang/lt_clang_cxx. However, we do not create a valid plumbum command. The generated script will only work inside a uchroot environment that has is root at the current working directory, when calling this function.

Parameters:

• cflags – The CFLAGS we want to hide
• ldflags – The LDFLAGS we want to hide
• func (optional) – A function that will be pickled alongside the compiler. It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.
• uchroot_path – Prefix path of the compiler inside the uchroot.
• cc_name – Name of the generated script.

benchbuild.utils.compiler.wrap_cxx_in_uchroot(cflags, ldflags, func=None)

Delegate to wrap_cc_in_uchroot).

benchbuild.utils.container module

Container utilites.

class benchbuild.utils.container.Container

Bases: object

filename

local

Finds the current location of a container. Also unpacks the project if necessary.

Returns:The path, where the container lies in the end. Return type:target

remote

class benchbuild.utils.container.Gentoo

Bases: benchbuild.utils.container.Container

latest_src_uri(*args, **kwargs)

name = 'gentoo'

remote

Get a remote URL of the requested container.

class benchbuild.utils.container.Ubuntu

Bases: benchbuild.utils.container.Container

name = 'ubuntu'

remote

Get a remote URL of the requested container.

benchbuild.utils.container.cached(func)

benchbuild.utils.container.is_valid_container(container, path)

Checks if a container exists and is unpacked.

Parameters:path – The location where the container is expected. Returns:True if the container is valid, False if the container needs to unpacked or if the path does not exist yet.

benchbuild.utils.container.unpack_container(container, path)

Method that checks if a directory for the container exists, checks if erlent support is needed and then unpacks the container accordingly.

Parameters:path – The location where the container is, that needs to be unpacked.

benchbuild.utils.db module

Database support module for the benchbuild study.

benchbuild.utils.db.create_run(cmd, prj, exp, grp)

Create a new ‘run’ in the database.

This creates a new transaction in the database and creates a new run in this transaction. Afterwards we return both the transaction as well as the run itself. The user is responsible for committing it when the time comes.

Parameters:

• cmd – The command that has been executed.
• prj – The project this run belongs to.
• exp – The experiment this run belongs to.
• grp – The run_group (uuid) we blong to.

Returns:

The inserted tuple representing the run and the session opened with the new run. Don’t forget to commit it at some point.

benchbuild.utils.db.create_run_group(prj)

Create a new ‘run_group’ in the database.

This creates a new transaction in the database and creates a new run_group within this transaction. Afterwards we return both the transaction as well as the run_group itself. The user is responsible for committing it when the time comes.

Parameters:- The project for which we open the run_group. (prj) – Returns:A tuple (group, session) containing both the newly created run_group and the transaction object.

benchbuild.utils.db.persist_compilestats(run, session, stats)

Persist the run results in the database.

Parameters:

• run – The run we attach the compilestats to.
• session – The db transaction we belong to.
• stats – The stats we want to store in the database.

benchbuild.utils.db.persist_config(run, session, cfg)

Persist the configuration in as key-value pairs.

Parameters:

• run – The run we attach the config to.
• session – The db transaction we belong to.
• cfg – The configuration we want to persist.

benchbuild.utils.db.persist_experiment(experiment)

Persist this experiment in the benchbuild database.

Parameters:experiment – The experiment we want to persist.

benchbuild.utils.db.persist_likwid(run, session, measurements)

Persist all likwid results.

Parameters:

• run – The run we attach our measurements to.
• session – The db transaction we belong to.
• measurements – The likwid measurements we want to store.

benchbuild.utils.db.persist_perf(run, session, svg_path)

Persist the flamegraph in the database.

The flamegraph exists as a SVG image on disk until we persist it in the database.

Parameters:

• run – The run we attach these perf measurements to.
• session – The db transaction we belong to.
• svg_path – The path to the SVG file we want to store.

benchbuild.utils.db.persist_project(project)

Persist this project in the benchbuild database.

Parameters:project – The project we want to persist.

benchbuild.utils.db.persist_time(run, session, timings)

Persist the run results in the database.

Parameters:

• run – The run we attach this timing results to.
• session – The db transaction we belong to.
• timings – The timing measurements we want to store.

benchbuild.utils.downloader module

Downloading helper functions for benchbuild.

The helpers defined in this module provide access to some common Downloading methods for the source code of benchbuild projects. All downloads will be cached in BB_TMP_DIR and locked down with a hash that is generated after the first download. If the hash matches the file/folder found in BB_TMP_DIR, nothing will be downloaded at all.

Supported methods:

Copy, CopyNoFail, Wget, Git, Svn, Rsync

benchbuild.utils.downloader.Copy(From, To)

Small copy wrapper.

Parameters:

• From (str) – Path to the SOURCE.
• To (str) – Path to the TARGET.

benchbuild.utils.downloader.CopyNoFail(src, root=None)

Just copy fName into the current working directory, if it exists.

No action is executed, if fName does not exist. No Hash is checked.

Parameters:

• src – The filename we want to copy to ‘.’.
• root – The optional source dir we should pull fName from. Defaults to benchbuild.settings.CFG[“tmpdir”].

Returns:

True, if we copied something.

benchbuild.utils.downloader.Git(src_url, tgt_name, tgt_root=None)

Get a shallow clone of the given repo

Parameters:

• src_url (str) – Git URL of the SOURCE repo.
• tgt_name (str) – Name of the repo folder on disk.
• tgt_root (str) – TARGET folder for the git repo. Defaults to CFG["tmpdir"]

benchbuild.utils.downloader.Rsync(url, tgt_name, tgt_root=None)

RSync a folder.

Parameters:

• url (str) – The url of the SOURCE location.
• fname (str) – The name of the TARGET.
• to (str) – Path of the target location. Defaults to CFG["tmpdir"].

benchbuild.utils.downloader.Svn(url, fname, to=None)

Checkout the SVN repo.

Parameters:

• url (str) – The SVN SOURCE repo.
• fname (str) – The name of the repo on disk.
• to (str) – The name of the TARGET folder on disk. Defaults to CFG["tmpdir"]

benchbuild.utils.downloader.Wget(src_url, tgt_name, tgt_root=None)

Download url, if required.

Parameters:

• src_url (str) – Our SOURCE url.
• tgt_name (str) – The filename we want to have on disk.
• tgt_root (str) – The TARGET directory for the download. Defaults to CFG["tmpdir"].

benchbuild.utils.downloader.get_hash_of_dirs(directory)

Recursively hash the contents of the given directory.

Parameters:directory (str) – The root directory we want to hash. Returns:A hash of all the contents in the directory.

benchbuild.utils.downloader.source_required(src_file, src_root)

Check, if a download is required.

Parameters:

• src_file – The filename to check for.
• src_root – The path we find the file in.

Returns:

True, if we need to download something, False otherwise.

benchbuild.utils.downloader.update_hash(src, root)

Update the hash for the given file.

Parameters:

• src – The file name.
• root – The path of the given file.

benchbuild.utils.log module

benchbuild.utils.log.configure()

Load logging configuration from our own defaults.

benchbuild.utils.log.set_defaults()

Configure the loggers default settings.

benchbuild.utils.path module

Path utilities for benchbuild.

benchbuild.utils.path.determine_path()

Borrowed from wxglade.py

benchbuild.utils.path.list_to_path(pathlist)

Convert a list of path elements to a path string.

benchbuild.utils.path.mkdir_uchroot(dirpath, root='.')

Create a file inside a uchroot env.

You will want to use this when you need to create a file with apropriate rights inside a uchroot container with subuid/subgid handling enabled.

Parameters:

• dirpath – The dirpath that should be created. Absolute inside the uchroot container.
• root – The root PATH of the container filesystem as seen outside of the container.

benchbuild.utils.path.mkfile_uchroot(filepath, root='.')

Create a file inside a uchroot env.

You will want to use this when you need to create a file with apropriate rights inside a uchroot container with subuid/subgid handling enabled.

Parameters:

• filepath – The filepath that should be created. Absolute inside the uchroot container.
• root – The root PATH of the container filesystem as seen outside of the container.

benchbuild.utils.path.path_to_list(pathstr)

Conver a path string to a list of path elements.

benchbuild.utils.path.template_files(path, exts=[])

Return a list of filenames found at @path.

The list of filenames can be filtered by extensions.

Parameters:

• path – Existing filepath we want to list.
• exts – List of extensions to filter by.

Returns:

A list of filenames found in the path.

benchbuild.utils.path.template_str(template)

Read a template file from the resources and return it as str.

benchbuild.utils.run module

Experiment helpers.

exception benchbuild.utils.run.GuardedRunException(what, db_run, session)

Bases: Exception

BB Run exception.

Contains an exception that ocurred during execution of a benchbuild experiment.

benchbuild.utils.run.begin(command, pname, ename, group)

Begin a run in the database log.

Parameters:

• command – The command that will be executed.
• pname – The project name we belong to.
• ename – The experiment name we belong to.
• group – The run group we belong to.

Returns:

(run, session), where run is the generated run instance and session the associated transaction for later use.

benchbuild.utils.run.begin_run_group(project)

Begin a run_group in the database.

A run_group groups a set of runs for a given project. This models a series of runs that form a complete binary runtime test.

Parameters:project – The project we begin a new run_group for. Returns:(group, session) where group is the created group in the database and session is the database session this group lives in.

benchbuild.utils.run.end(db_run, session, stdout, stderr)

End a run in the database log (Successfully).

This will persist the log information in the database and commit the transaction.

Parameters:

• db_run – The run schema object we belong to
• session – The db transaction we belong to.
• stdout – The stdout we captured of the run.
• stderr – The stderr we capture of the run.

benchbuild.utils.run.end_run_group(group, session)

End the run_group successfully.

Parameters:

• group – The run_group we want to complete.
• session – The database transaction we will finish.

benchbuild.utils.run.fail(db_run, session, retcode, stdout, stderr)

End a run in the database log (Unsuccessfully).

This will persist the log information in the database and commit the transaction.

Parameters:

• db_run – The run schema object we belong to
• session – The db transaction we belong to.
• retcode – The return code we captured of the run.
• stdout – The stdout we captured of the run.
• stderr – The stderr we capture of the run.

benchbuild.utils.run.fail_run_group(group, session)

End the run_group unsuccessfully.

Parameters:

• group – The run_group we want to complete.
• session – The database transaction we will finish.

benchbuild.utils.run.fetch_time_output(marker, format_s, ins)

Fetch the output /usr/bin/time from a.

Parameters:

• marker – The marker that limits the time output
• format_s – The format string used to parse the timings
• ins – A list of lines we look for the output.

Returns:

A list of timing tuples

benchbuild.utils.run.guarded_exec(cmd, project, experiment, **kwargs)

Guard the execution of the given command.

Parameters:

• cmd – the command we guard.
• pname – the database we run under.
• ename – the database session this run belongs to.
• run_group – the run group this execution will belong to.

Raises:

RunException – If the cmd encounters an error we wrap the exception in a RunException and re-raise. This ends the run unsuccessfully.

benchbuild.utils.run.handle_stdin(cmd, kwargs)

Handle stdin for wrapped runtime executors.

This little helper checks the kwargs for a has_stdin key containing a boolean value. If necessary it will pipe in the stdin of this process into the plumbum command.

Parameters:

• cmd (benchbuild.utils.cmd) – Command to wrap a stdin handler around.
• kwargs – Dictionary containing the kwargs. We check for they key has_stdin

Returns:

A new plumbum command that deals with stdin redirection, if needed.

benchbuild.utils.run.in_builddir(sub='.')

Decorate a project phase with a local working directory change.

Parameters:sub – An optional subdirectory to change into.

benchbuild.utils.run.run(command, retcode=0)

Execute a plumbum command, depending on the user’s settings.

Parameters:command – The plumbumb command to execute.

benchbuild.utils.run.store_config(func)

Decorator for storing the configuration in the project’s builddir.

benchbuild.utils.run.uchroot(*args, **kwargs)

Return a customizable uchroot command.

Parameters:args – List of additional arguments for uchroot (typical: mounts) Returns:chroot_cmd

benchbuild.utils.run.uchroot_env(mounts)

Compute the environment of the change root for the user.

Parameters:mounts – The mountpoints of the current user. Returns:paths ld_libs

benchbuild.utils.run.uchroot_mounts(prefix, mounts)

Compute the mountpoints of the current user.

Parameters:

• prefix – Define where to job was running if it ran on a cluster.
• mounts – All mounts the user currently uses in his file system.

Returns:

mntpoints

benchbuild.utils.run.uchroot_no_args()

Return the uchroot command without any customizations.

benchbuild.utils.run.uchroot_no_llvm(*args, **kwargs)

Return a customizable uchroot command.

The command will be executed inside a uchroot environment.

Parameters:args – List of additional arguments for uchroot (typical: mounts) Returns:chroot_cmd

benchbuild.utils.run.uchroot_with_mounts(*args, **kwargs)

Return a uchroot command with all mounts enabled.

benchbuild.utils.run.unionfs(base_dir='./base', image_dir='./image', image_prefix=None, mountpoint='./union')

Decorator for the UnionFS feature.

This configures a unionfs for projects. The given base_dir and/or image_dir are layered as follows:

image_dir=RW:base_dir=RO

All writes go to the image_dir, while base_dir delivers the (read-only) versions of the rest of the filesystem.

The unified version will be provided in the project’s builddir. Unmouting is done as soon as the function completes.

Parameters:

• base_dir – The unpacked container of a project delievered by a method out of the container utils.
• image_dir – Virtual image of the actual file system represented by the build_dir of a project.
• image_prefix – Useful prefix if the projects run on a cluster, to identify where the job came from and where it runs.
• mountpoint – Location where the filesystems merge, currently per default as ‘./union’.

benchbuild.utils.run.unionfs_set_up(ro_base, rw_image, mountpoint)

Setup a unionfs via unionfs-fuse.

Parameters:

• ro_base – base_directory of the project
• rw_image – virtual image of actual file system
• mountpoint – location where ro_base and rw_image merge

benchbuild.utils.run.unionfs_tear_down(mountpoint, tries=3)

Tear down a unionfs mountpoint.

benchbuild.utils.run.with_env_recursive(cmd, **envvars)

Recursively updates the environment of cmd and all its subcommands.

Parameters:

• - A plumbum command-like object (cmd) –
• - The environment variables to update (**envvars) –

Returns:

The updated command.

benchbuild.utils.schema module

Database schema for benchbuild

The schema should initialize itself on an empty database. For now, we do not support automatic upgrades on schema changes. You might encounter some roadbumps when using an older version of benchbuild.

Furthermore, for now, we are restricted to postgresql databases, although we already support arbitrary connection strings via config.

If you want to use reports that use one of our SQL functions, you need to initialize the functions first using the following command:

> BB_DB_CREATE_FUNCTIONS=true benchbuild run -E empty -l

After that you (normally) do not need to do this agains, unless we supply a new version that you are interested in. As soon as we have alembic running, we can provide automatic up/downgrade paths for you.

benchbuild.utils.schema.CONNECTION_MANAGER = <benchbuild.utils.schema.SessionManager object>

Import this session manager to create new database sessions as needes.

class benchbuild.utils.schema.CompileStat(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store compilestats as given by LLVM’s ‘-stats’ commoand.

component

id

name

run_id

value

class benchbuild.utils.schema.Config(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store customized information about a run.

You can store arbitrary configuration information about a run here. Use it for extended filtering against the run table.

name

run_id

value

class benchbuild.utils.schema.Event(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store PAPI profiling based events.

duration

id

name

run_id

start

tid

type

class benchbuild.utils.schema.Experiment(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store metadata about experiments.

begin

description

end

id

name

class benchbuild.utils.schema.GlobalConfig(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store customized information about a run.

You can store arbitrary configuration information about a run here. Use it for extended filtering against the run table.

experiment_group

name

value

class benchbuild.utils.schema.Likwid(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store measurement results of likwid based experiments.

core

metric

region

run_id

value

class benchbuild.utils.schema.Metadata(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store metadata information for every run.

If you happen to have some free-form data that belongs to the database, this is the place for it.

name

run_id

value

class benchbuild.utils.schema.Metric(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store default metrics, simple name value store.

name

run_id

value

class benchbuild.utils.schema.PerfEvent(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store PAPI profiling based events.

duration

id

name

run_id

start

tid

type

class benchbuild.utils.schema.Project(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store project metadata.

description

domain

group_name

name

src_url

version

class benchbuild.utils.schema.RegressionTest(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store regression tests for all projects.

This relation is filled from the PolyJIT side, not from benchbuild-study. We collect all JIT SCoPs found by PolyJIT in this relation for regression-test generation.

module

name

project_name

run_id

class benchbuild.utils.schema.Run(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store a run for each executed test binary.

begin

command

end

experiment_group

experiment_name

id

project_name

run_group

status

class benchbuild.utils.schema.RunGroup(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store information about a run group.

begin

end

experiment

id

project

status

class benchbuild.utils.schema.RunLog(**kwargs)

Bases: sqlalchemy.ext.declarative.api.Base

Store log information for every run.

Properties like, start time, finish time, exit code, stderr, stdout are stored here.

begin

config

end

run_id

status

stderr

stdout

class benchbuild.utils.schema.SessionManager

Bases: object

get()

benchbuild.utils.slurm module

SLURM support for the benchbuild study.

This module can be used to generate bash scripts that can be executed by the SLURM controller either as batch or interactive script.

benchbuild.utils.slurm.dump_slurm_script(script_name, benchbuild, experiment, projects)

Dump a bash script that can be given to SLURM.

Parameters:

• script_name (str) – name of the bash script.
• commands (list(benchbuild.utils.cmd)) – List of plumbum commands to write to the bash script.
• **kwargs – Dictionary with all environment variable bindings we should map in the bash script.

benchbuild.utils.slurm.prepare_directories(dirs)

Make sure that the required directories exist.

Parameters:- the directories we want. (dirs) –

benchbuild.utils.slurm.prepare_slurm_script(experiment, projects)

Prepare a slurm script that executes the experiment for a given project.

Parameters:

• experiment – The experiment we want to execute
• projects – All projects we generate an array job for.

benchbuild.utils.user_interface module

User interface helpers for benchbuild.

benchbuild.utils.user_interface.ask(question, default_answer=False, default_answer_str='no')

benchbuild.utils.user_interface.query_yes_no(question, default='yes')

Ask a yes/no question via raw_input() and return their answer.

Parameters:

• question (str) – Question hat is presented to the user.
• default (str) – The presumed answer, if the user just hits <Enter>. It must be “yes” (the default), “no” or None (meaning an answer is required of the user).

Returns (boolean):

True, if ‘yes’, False otherwise.

benchbuild.utils.versions module

Gather version information for BB.

benchbuild.utils.versions.get_git_hash(from_url)

Get the git commit hash of HEAD from :from_url.

Parameters:from_url – The file system url of our git repository. Returns:git commit hash of HEAD, or empty string.

benchbuild.utils.versions.get_version_from_cache_dir(src_file)

Creates a version for a project out of the hash.

The hash is taken from the directory of the source file.

Parameters:src_file – The source file of the project using this function. Returns:Either returns the first 8 digits of the hash as string, the entire hash as a string if the hash consists out of less than 7 digits or None if the path is incorrect.

benchbuild.utils.wrapping module

benchbuild.utils.wrapping.strip_path_prefix(ipath, prefix)

Strip prefix from path.

Parameters:

• ipath – input path
• prefix – the prefix to remove, if it is found in :ipath:

Examples

>>> strip_path_prefix("/foo/bar", "/bar")
'/foo/bar'
>>> strip_path_prefix("/foo/bar", "/")
'foo/bar'
>>> strip_path_prefix("/foo/bar", "/foo")
'/bar'
>>> strip_path_prefix("/foo/bar", "None")
'/foo/bar'

benchbuild.utils.wrapping.wrap(name, runner, sprefix=None, **template_vars)

Wrap the binary :name: with the function :runner:.

This module generates a python tool that replaces :name: The function in runner only accepts the replaced binaries name as argument. We use the cloudpickle package to perform the serialization, make sure :runner: can be serialized with it and you’re fine.

Parameters:

• name – Binary we want to wrap
• runner – Function that should run instead of :name:

Returns:

A plumbum command, ready to launch.

benchbuild.utils.wrapping.wrap_cc(filepath, cflags, ldflags, compiler, extension, compiler_ext_name=None, **template_vars)

Substitute a compiler with a script that hides CFLAGS & LDFLAGS.

This will generate a wrapper script in the current directory and return a complete plumbum command to it.

Parameters:

• filepath (str) – Path to the wrapper script.
• cflags (list(str)) – The CFLAGS we want to hide.
• ldflags (list(str)) – The LDFLAGS we want to hide.
• compiler (benchbuild.utils.cmd) – Real compiler command we should call in the script.
• extension – A function that will be pickled alongside the compiler. It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.
• compiler_ext_name – The name that we should give to the generated dill blob for :func:

Returns (benchbuild.utils.cmd):

Command of the new compiler we can call.

benchbuild.utils.wrapping.wrap_dynamic(self, name, runner, sprefix=None, **template_vars)

Wrap the binary :name with the function :runner.

This module generates a python tool :name: that can replace a yet unspecified binary. It behaves similar to the :wrap: function. However, the first argument is the actual binary name.

Parameters:

• name – name of the python module
• runner – Function that should run the real binary
• base_class – The base_class of our project.
• base_module – The module of base_class.

Returns: plumbum command, readty to launch.

benchbuild.utils.wrapping.wrap_dynamic_in_uchroot(self, name, runner, sprefix=None)

benchbuild.utils.wrapping.wrap_in_uchroot(name, runner, sprefix=None)