temci.utils package

Submodules

temci.utils.click_helper module

This module simplifies the creation of click options from settings and type schemes.

class temci.utils.click_helper.CmdOption(option_name: str, settings_key: Optional[str] = None, type_scheme: Optional[Type] = None, short: Optional[str] = None, completion_hints: Optional[Dict[str, Any]] = None, is_flag: Optional[bool] = None)[source]

Bases: object

Represents a command line option.

Initializes a option either based on a setting (via settings key) or on a type scheme. If this is backed by a settings key, the setting is automatically set. If is_flag is None, it is set True if type_scheme is an instance of Bool() or BoolOrNone()

Parameters
  • option_name – name of the option

  • settings_key – settings key of the option

  • type_scheme – type scheme with default value

  • short – short version of the option (ignored if is_flag=True)

  • completion_hints – additional completion hints (dict with keys for each shell)

  • is_flag – is the option a “–ABC/–no-ABC” flag like option?

callback

Callback that sets the setting

completion_hints

Additional completion hints (dict with keys for each shell)

default

Default value of this option

description

Description of this option

classmethod from_non_plugin_settings(settings_domain: str, exclude: Optional[List[Str]] = None, name_prefix: Optional[str] = None) CmdOptionList[source]

Creates a list of CmdOption object from all sub settings (in the settings domain). It excludes all sub settings that are either in the exclude list or end with “_active” or “_misc” (used for plugin settings). Also every setting that is of type Dict is ignored.

Parameters
  • settings_domain – settings domain to look into (or “” for the root domain)

  • exclude – list of sub keys to exclude

Returns

list of CmdOptions

classmethod from_registry(registry: type, name_prefix: Optional[str] = None) CmdOptionList[source]

Creates a list of CmdOption objects from an registry. It creates an activation flag (–OPT/–no-OPT) for each registered plugin and creates for each plugin preference an option with name OPT_PREF. Deeper nesting is intentionally not supported.

Parameters
  • registry – used registry

  • name_prefix – prefix of each option name (usable to avoid ambiguity problems)

Returns

list of CmdOptions

has_completion_hints

Does this option has completion hints?

has_default

Does this option has a default value?

has_description

Does this option has a description?

has_short

Does this option has a short version?

is_flag

Is this option flag like?

option_name

Name of this option

settings_key

Settings key of this option

short

Short version of the option (ignored if is_flag=True)

type_scheme

Type scheme with default value

class temci.utils.click_helper.CmdOptionList(*options: Tuple[Union[CmdOption, CmdOptionList]])[source]

Bases: object

A simple list for CmdOptions that supports list flattening.

Create an instance.

Parameters

options – options that this list consists of

append(options: Union[CmdOption, CmdOptionList]) CmdOptionList[source]

Appends the passed CmdÖptionList or CmdOption and flattens the resulting list.

Parameters

options – CmdÖptionList or CmdOption

Returns

self

get_sphinx_doc() str[source]

Returns the documentation string for the enclosed options

options

Options that build up this list

set_short(option_name: str, new_short: str) CmdOptionList[source]

Sets the short option name of the included option with the passed name.

Parameters
  • option_name – passed option name

  • new_short – new short option name

Returns

self

Raises

IndexError if the option with the passed name doesn’t exist

temci.utils.click_helper.cmd_option(option: Union[CmdOption, CmdOptionList], name_prefix: Optional[str] = None, validate: Optional[bool] = None) Callable[[Callable], Callable][source]

Wrapper around click.option that works with CmdOption objects. If option is a list of CmdOptions then the type_scheme_option decorators are chained. Also supports nested lists in the same manner.

Parameters
  • option – CmdOption or (possibly nested) list of CmdOptions

  • name_prefix – prefix of all options

  • validate – validate setting or validate only for outer most if None

Returns

click.option(…) like decorator

temci.utils.click_helper.document_func(description: str, *options: Tuple[Union[CmdOptionList, CmdOption]], argument: Optional[str] = None, only_if_sphinx_doc: bool = True) Callable[[Callable], Callable][source]

Function decorator that appends an command documentation to the functions __doc__ attribute.

Parameters
  • description – description of the command

  • options – options to generate a documentation for

  • argument – optional argument description of the command

  • only_if_sphinx_doc – only generate the documentation if the file is executed for documentation generation

temci.utils.click_helper.type_scheme_option(option_name: str, type_scheme: Type, is_flag: bool = False, callback=typing.Callable[[click.core.Context, str, typing.Any], typing.Any], short: Optional[str] = None, with_default: bool = True, default=None, validate_settings: bool = False) Callable[[Callable], Callable][source]

Is essentially a wrapper around click.option that works with type schemes.

Parameters
  • option_name – name of the option

  • type_scheme – type scheme to use

  • is_flag – is this option a “–ABC/–no-ABC” like flag

  • callback – callback that is called with the parameter and the argument and has to returns its argument

  • short – short name of the option (ignored if flag=True)

  • with_default – set a default value for the option if possible?

  • default – default value (if with_default is true), default: default value of the type scheme

  • validate_settings – call Settings().validate() in the callback

temci.utils.click_helper.validate(type_scheme: Type) Callable[[Context, str, Any], Any][source]

Creates a valid click option validator function that can be passed to click via the callback parameter. The validator function expects the type of the value to be the raw type of the type scheme.

Parameters

type_scheme – type scheme the validator validates against

Returns

the validator function

temci.utils.config_utils module

Types shared between different file config definitions

temci.utils.library_init module

Initialize temci for use as a library

temci.utils.mail module

Utilities to send mails.

temci.utils.mail.hostname() str[source]

Returns the hostname of the current machine

temci.utils.mail.send_mail(recipient: str, subject: str, content: str, attached_files: Optional[List[str]] = None)[source]

Sends a mail to the recipient with the passed subject, content and attached files.

Parameters
  • recipient – recipient of the mail, i.e. a mail address

  • subject – subject of the mail

  • content – content of the mail

  • attached_files – optional list of names of files that are attached to the mail

temci.utils.number module

class temci.utils.number.FNumber(number: Union[int, float], rel_deviation: Optional[Union[int, float]] = None, abs_deviation: Optional[Union[int, float]] = None, is_percent: Optional[bool] = None, scientific_notation: Optional[bool] = None, parentheses_mode: Optional[Union[str, ParenthesesMode]] = None, parentheses: Optional[bool] = None)[source]

Bases: object

A formattable number wrapper.

Configuration format, is in the settings under report/number

# Don't omit the minimum number of decimal places if insignificant?
force_min_decimal_places:         Bool()
            default: true

# The maximum number of decimal places
max_decimal_places:         Int(constraint=<function>)
            default: 5

# The minimum number of shown decimal places if decimal places are shown
min_decimal_places:         Int(constraint=<function>)
            default: 3

# Omit insignificant decimal places
omit_insignificant_decimal_places:         Bool()

# Show parentheses around non significant digits? (If a std dev is given)
parentheses:         Bool()
            default: true

# Mode for showing the parentheses: either d (Digits are considered significant if they don't change
# if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they
# are bigger than $sigmas * std dev)
parentheses_mode:         ExactEither('d'|'o')
            default: o

# Show as percentages
percentages:         Bool()

# Use the exponential notation, i.e. '10e3' for 1000
scientific_notation:         Bool()
            default: true

# Use si prefixes instead of 'e…'
scientific_notation_si_prefixes:         Bool()
            default: true

# Number of standard deviation used for the digit significance evaluation
sigmas:         Int(constraint=<function>)
            default: 2
deviation

Relative deviation

format() str[source]
classmethod init_settings(new_settings: Dict[str, Union[int, bool]])[source]
settings = {'force_min_decimal_places': True, 'max_decimal_places': 5, 'min_decimal_places': 3, 'omit_insignificant_decimal_places': False, 'parentheses': True, 'parentheses_mode': 'o', 'percentages': False, 'scientific_notation': True, 'scientific_notation_si_prefixes': True, 'sigmas': 2}
settings_format = # Don't omit the minimum number of decimal places if insignificant? force_min_decimal_places:         Bool()             default: true  # The maximum number of decimal places max_decimal_places:         Int(constraint=<function>)             default: 5  # The minimum number of shown decimal places if decimal places are shown min_decimal_places:         Int(constraint=<function>)             default: 3  # Omit insignificant decimal places omit_insignificant_decimal_places:         Bool()  # Show parentheses around non significant digits? (If a std dev is given) parentheses:         Bool()             default: true  # Mode for showing the parentheses: either d (Digits are considered significant if they don't change # if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they # are bigger than $sigmas * std dev) parentheses_mode:         ExactEither('d'|'o')             default: o  # Show as percentages percentages:         Bool()  # Use the exponential notation, i.e. '10e3' for 1000 scientific_notation:         Bool()             default: true  # Use si prefixes instead of 'e…' scientific_notation_si_prefixes:         Bool()             default: true  # Number of standard deviation used for the digit significance evaluation sigmas:         Int(constraint=<function>)             default: 2
temci.utils.number.Number

Numeric type

alias of Union[int, float]

class temci.utils.number.ParenthesesMode(value)[source]

Bases: Enum

An enumeration.

DIGIT_CHANGE = 'd'
ORDER_OF_MAGNITUDE = 'o'
classmethod map(key: Union[str, ParenthesesMode]) ParenthesesMode[source]
temci.utils.number.fnumber(number: Union[int, float], rel_deviation: Optional[Union[int, float]] = None, abs_deviation: Optional[Union[int, float]] = None, is_percent: bool = False) str[source]
temci.utils.number.format_number(number: Union[int, float], deviation: float = 0.0, parentheses: bool = True, explicit_deviation: bool = False, is_deviation_absolute: bool = True, min_decimal_places: int = 3, max_decimal_places: Optional[int] = None, omit_insignificant_decimal_places: bool = True, scientific_notation: bool = False, scientific_notation_steps: int = 3, scientific_notation_decimal_places: Optional[int] = None, scientific_notation_si_prefixes: bool = True, force_min_decimal_places: bool = True, relative_to_deviation: bool = False, sigmas: int = 2, parentheses_mode: ParenthesesMode = ParenthesesMode.ORDER_OF_MAGNITUDE) str[source]

Format the passed number

>>> format_number(1.0, 0.5)
'1.(000)'
>>> format_number(1.56, 0.005)
'1.56(0)'
>>> format_number(1560, scientific_notation=True)
'1.560k'
>>> format_number(1560, scientific_notation_si_prefixes=False, scientific_notation=True)
'1.560e3'
>>> format_number(float("inf"))
'inf'
Parameters
  • number – formatted number

  • deviation – standard deviation associated with the number

  • parentheses – show parentheses around non significant digits?

  • explicit_deviation – show the absolute deviation, e.g. “100±456.4”

  • is_deviation_absolute – is the given deviation absolute?

  • min_decimal_places – the minimum number of shown decimal places if decimal places are shown

  • max_decimal_places – the maximum number of decimal places

  • omit_insignificant_decimal_places – omit insignificant decimal places

  • scientific_notation – use the exponential notation, i.e. “10e3” for 1000

  • scientific_notation_steps – steps in which the exponential part is incremented

  • scientific_notation_decimal_places – number of decimal places that are shown in the scientic notation

  • scientific_notation_si_prefixes – use si prefixes instead of “e…”

  • force_min_decimal_places – don’t omit the minimum number of decimal places if insignificant?

  • relative_to_deviation – format the number relative to its deviation, i.e. “10 sigma”

  • sigmas – number of standard deviations for significance

  • parentheses_mode – mode for selecting the significant digits

Returns

the number formatted as a string

temci.utils.number.format_number_sn(number: Union[int, float], scientific_notation_steps: int = 3, deviation: Optional[float] = None, decimal_places: Optional[int] = None, si_prefixes: bool = True, **kwargs)[source]

temci.utils.plugin module

Utilities for loading plugins. Plugins are python files (ending .py) that are loaded prior to building the cli. These files may e.g. add runners, plugins, …

Plugins are loaded from the application directory (~/.temci) and from the paths given in the environment variable TEMCI_PLUGIN_PATH which contains a colon separated list of paths.

temci.utils.plugin.load_plugins()[source]

Load the plugins from the plugin folders :return:

temci.utils.plugin.plugin_paths() List[str][source]

Returns the paths that plugins are located in (might return folders and files)

temci.utils.registry module

class temci.utils.registry.AbstractRegistry[source]

Bases: object

An abstract registry. To create an own registry set the settings_key_path (type str), the use_key (type str), the use_list (type bool) and the default attribute (type (use_list ? list of strings : str).

Important: Be sure to have a “register = {}” line in your extending class.

default = None

Name(s) of the class(es) used by default. Type depends on the use_list property.

classmethod get_class(name: str) type[source]
classmethod get_for_name(name: str, *args, **kwargs) Any[source]

Creates a plugin with the given name.

Parameters

name – name of the registered class

Returns

object of the registered class

Raises

ValueError if there isn’t such a class

classmethod get_tester() Tester[source]

Returns the tester that is configured in the settings

Returns

tester instance

classmethod get_used() Union[str, List[str]][source]

Get the list of name of the used plugins (use_list=True) or the names of the used plugin (use_list=False).

plugin_synonym = ('plugin', 'plugins')

Singular and plural version of the word that is used in the documentation for the registered entities

classmethod register(name: str, klass: type, misc_type: Type, activate_by_default: Optional[bool] = None, deprecated: bool = False)[source]

Registers a new class. The constructor of the class gets as first argument the misc settings.

Parameters
  • name – common name of the registered class

  • klass – actual class

  • misc_type – type scheme of the {name}_misc settings

  • misc_default – default value of the {name}_misc settings

  • deprecated – is the registered class deprecated and should not be used?

registry = {}

Registered classes (indexed by their name)

settings_key_path = ''

Used settings key path

use_key = None

Used key that sets which registered class is currently used

use_list = False

Allow more than one class to used at a specific moment in time

temci.utils.registry.register(registry: type, name: str, misc_type: Type, deprecated: bool = False)[source]

Class decorator that calls the register method for the decorated method.

Parameters
  • registry – the registry class to register the class in

  • name – common name of the registered class

  • misc_type – type scheme of the {name}_misc settings (each dict key must have a default value)

  • deprecated – is the registered class deprecated and should not be used?

temci.utils.settings module

class temci.utils.settings.Settings(*args, **kwargs)[source]

Bases: object

Manages the Settings. The settings keys and sub keys are combined by a slash, e.g. “report/in”.

The current settings are:

The whole configuration file has the following structure:

# Alias for settings
config:         Str()

# Logging level
log_level:         ExactEither('debug'|'info'|'warn'|'error'|'quiet')
            default: info

# Additional settings file
settings:         Str()

# Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command
# line.
sudo:         Bool()

# Used temporary directory
tmp_dir:         Str()
            default: /tmp/temci

build:     
    # Input file with the program blocks to build
    in:         Str()
                default: build.yaml

    # Resulting run config file
    out:         Str()
                default: run_config.yaml

    # Number of simultaneous builds for a specific program block, only makes sense when
    # build_config/number > 1, and if the build commands create a different binary every time they are
    # run
    threads:         Int(constraint=<function>)
                default: 1

# Environment variables for the benchmarked programs, includes the user used for generated files
env:     
    PATH:         Str()

    USER:         Str()

report:     
    # Exclude all data sets that contain only NaNs.
    exclude_invalid:         BoolOrNone()
                default: true

    # Properties that aren't shown in the report.
    excluded_properties:         ListOrTuple(Str())
                default: [__ov-time]

    # Files that contain the benchmarking results
    in:         Either(Str()|ListOrTuple(Str()))
                default: run_output.yaml

    # List of included run blocks (all: include all), identified by their description or tag
    # attribute, can be regular expressions
    included_blocks:         ListOrTuple(Str())
                default: [all]

    # Replace the property names in reports with longer more descriptive versions?
    long_properties:         BoolOrNone()

    # Possible reporter are 'console', 'html2', 'csv', 'codespeed', 'codespeed2' and 'velcom'
    reporter:         ExactEither('console'|'html2'|'csv'|'codespeed'|'codespeed2'|'velcom')
                default: console

    # Produce xkcd like plots (requires the humor sans font to be installed)
    xkcd_like_plots:         BoolOrNone()

    #   Reporter that outputs JSON as specified by
    #   the `codespeed runner spec <https://git.scc.kit.edu/IPDSnelting/codespeed-runner>`_.
    codespeed2_misc:         Dict({}, False, keys=Any, values=Any, default = {})

    #   Reporter that outputs JSON as expected by `codespeed <https://github.com/tobami/codespeed>`_.
    #   Branch name and commit ID are taken from the current directory.
    #   Use it like this:
    #   .. code:: sh
    #   temci report --reporter codespeed ... | curl --data-urlencode json@-
    # http://localhost:8000/result/add/json/
    codespeed_misc:     
        # Branch name reported to codespeed. Defaults to current branch or else 'master'.
        branch:         Str()

        # Commit ID reported to codespeed. Defaults to current commit.
        commit_id:         Str()

        # Environment name reported to codespeed. Defaults to current host name.
        environment:         Str()

        # Executable name reported to codespeed. Defaults to the project name.
        executable:         Str()

        # Project name reported to codespeed.
        project:         Str()

    #   Simple reporter that outputs just text.
    console_misc:     
        # Matches the baseline block
        baseline:         Str()

        # Position of the baseline comparison: each: after each block, after: after each cluster, both:
        # after each and after cluster, instead: instead of the non baselined
        baseline_position:         ExactEither('each'|'after'|'both'|'instead')
                    default: each

        # 'auto': report clusters (runs with the same description) and singles (clusters with a single
        # entry, combined) separately, 'single': report all clusters together as one, 'cluster': report
        # all clusters separately, 'both': append the output of 'cluster' to the output of 'single'
        mode:         ExactEither('both'|'cluster'|'single'|'auto')
                    default: auto

        # Output file name or `-` (stdout)
        out:         FileName(allow_std=True)
                    default: '-'

        # Report on the failing blocks
        report_errors:         Bool()
                    default: true

        # Print statistical tests for every property for every two programs
        with_tester_results:         Bool()
                    default: true

    #   Simple reporter that outputs just a configurable csv table with rows for each run block
    csv_misc:     
        # List of valid column specs, format is a comma separated list of 'PROPERTY\[mod\]' or 'ATTRIBUTE'
        # mod is one of: mean, stddev, property, min, max and stddev per mean, optionally a formatting
        # option can be given viaPROPERTY\[mod|OPT1OPT2…\], where the OPTs are one of the following: %
        # (format as percentage), p (wrap insignificant digits in parentheses (+- 2 std dev)), s (use
        # scientific notation, configured in report/number) and o (wrap digits in the order of magnitude
        # of 2 std devs in parentheses). PROPERTY can be either the description or the short version of
        # the property. Configure the number formatting further via the number settings in the settings
        # file
        columns:         ListOrTuple(Str())
                    default: [description]
            # with constrain: List of valid column specs, format is a comma separated list of 'PROPERTY\[mod\]' or 'ATTRIBUTE' mod is one of: mean, stddev, property, min, max and stddev per mean, optionally a formatting option can be given viaPROPERTY\[mod|OPT1OPT2…\], where the OPTs are one of the following: % (format as percentage), p (wrap insignificant digits in parentheses (+- 2 std dev)), s (use scientific notation, configured in report/number) and o (wrap digits in the order of magnitude of 2 std devs in parentheses). PROPERTY can be either the description or the short version of the property. Configure the number formatting further via the number settings in the settings file

        # Output file name or standard out (-)
        out:         FileName(allow_std=True)
                    default: '-'

    #   Reporter that produces a HTML based report with lot's of graphics.
    #   A rewrite of the original HTMLReporter
    html2_misc:     
        # Alpha value for confidence intervals
        alpha:         T(<class 'float'>)
                    default: 0.05

        # Height per run block for the big comparison box plots
        boxplot_height:         T(<class 'float'>)
                    default: 2.0

        # Width of all big plotted figures
        fig_width_big:         T(<class 'float'>)
                    default: 25.0

        # Width of all small plotted figures
        fig_width_small:         T(<class 'float'>)
                    default: 15.0

        # Format string used to format floats
        float_format:         Str()
                    default: '{:5.2e}'

        # Override the contents of the output directory if it already exists?
        force_override:         Bool()

        # Generate pdf versions of the plotted figures?
        gen_pdf:         Bool()

        # Generate simple latex versions of the plotted figures?
        gen_tex:         Bool()
                    default: true

        # Generate excel files for all tables
        gen_xls:         Bool()

        # Hide warnings and errors related to statistical properties
        hide_stat_warnings:         Bool()

        # Name of the HTML file
        html_filename:         Str()
                    default: report.html

        # Use local versions of all third party resources
        local:         Bool()

        # Show the mean related values in the big comparison table
        mean_in_comparison_tables:         Bool()
                    default: true

        # Show the mininmum related values in the big comparison table
        min_in_comparison_tables:         Bool()

        # Output directory
        out:         Str()
                    default: report

        # Format string used to format floats as percentages
        percent_format:         Str()
                    default: '{:5.2%}'

        # Show zoomed out (x min = 0) figures in the extended summaries?
        show_zoomed_out:         Bool()
                    default: true

    number:     
        # Don't omit the minimum number of decimal places if insignificant?
        force_min_decimal_places:         Bool()
                    default: true

        # The maximum number of decimal places
        max_decimal_places:         Int(constraint=<function>)
                    default: 5

        # The minimum number of shown decimal places if decimal places are shown
        min_decimal_places:         Int(constraint=<function>)
                    default: 3

        # Omit insignificant decimal places
        omit_insignificant_decimal_places:         Bool()

        # Show parentheses around non significant digits? (If a std dev is given)
        parentheses:         Bool()
                    default: true

        # Mode for showing the parentheses: either d (Digits are considered significant if they don't
        # change if the number itself changes += $sigmas * std dev) or o (digits are considered
        # significant if they are bigger than $sigmas * std dev)
        parentheses_mode:         ExactEither('d'|'o')
                    default: o

        # Show as percentages
        percentages:         Bool()

        # Use the exponential notation, i.e. '10e3' for 1000
        scientific_notation:         Bool()
                    default: true

        # Use si prefixes instead of 'e…'
        scientific_notation_si_prefixes:         Bool()
                    default: true

        # Number of standard deviation used for the digit significance evaluation
        sigmas:         Int(constraint=<function>)
                    default: 2

    #   Reporter that outputs JSON as specified by
    #   the `velcom runner spec <https://github.com/IPDSnelting/velcom/wiki/Benchmark-Repo-
    # Specification>`_.
    velcom_misc:         Dict({}, False, keys=Any, values=Any, default = {})

run:     
    # Abort after the first failing build
    abort_after_build_error:         Bool()
                default: true

    # Append to the output file instead of overwriting by adding new run data blocks
    append:         Bool()

    # Disable the hyper threaded cores. Good for cpu bound programs.
    disable_hyper_threading:         Bool()

    # Discard all run data for the failing program on error
    discard_all_data_for_block_on_error:         Bool()

    # First n runs that are discarded
    discarded_runs:         Int(constraint=<function>)
                default: 1

    # Possible run drivers are 'exec' and 'shell'
    driver:         ExactEither('exec'|'shell')
                default: exec

    # Input file with the program blocks to benchmark
    in:         Str()
                default: input.exec.yaml

    # List of included run blocks (all: include all), or their tag attribute or their number in the
    # file (starting with 0), can be regular expressions
    included_blocks:         ListOrTuple(Str())
                default: [all]

    # Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
    max_block_time:         ValidTimespan()
                default: '-1'

    # Maximum number of benchmarking runs
    max_runs:         Int(constraint=<function>)
                default: 100

    # Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span
    # expressions
    max_time:         ValidTimespan()
                default: '-1'

    # Minimum number of benchmarking runs
    min_runs:         Int(constraint=<function>)
                default: 20

    # Do not build if build configs are present, only works if the working directory of the blocks
    # does not change
    no_build:         Bool()

    # Only build
    only_build:         Bool()

    # Output file for the benchmarking results
    out:         Str()
                default: run_output.yaml

    # Record the caught errors in the run_output file
    record_errors_in_file:         Bool()
                default: true

    # Number of benchmarking runs that are done together
    run_block_size:         Int(constraint=<function>)
                default: 1

    # if != -1 sets max and min runs to its value
    runs:         Int(constraint=<function>)
                default: -1

    # If not empty, recipient of a mail after the benchmarking finished.
    send_mail:         Str()

    # Print console report if log_level=info
    show_report:         Bool()
                default: true

    # Randomize the order in which the program blocks are benchmarked.
    shuffle:         Bool()
                default: true

    # Store the result file after each set of blocks is benchmarked
    store_often:         Bool()

    # Show the report continuously
    watch:         Bool()

    # Update the screen nth run (less updates are better for benchmarks)
    watch_every:         Int(constraint=<function>)
                default: 1

    cpuset:     
        # Use cpuset functionality?
        active:         Bool()

        # Number of cpu cores for the base (remaining part of the) system
        base_core_number:         Int(range=range(0, 2))
                    default: 1

        # 0: benchmark sequential, > 0: benchmark parallel with n instances, -1: determine n automatically
        parallel:         Int(constraint=<function>)

        # Number of cpu cores per parallel running program.
        sub_core_number:         Int(range=range(0, 2))
                    default: 1

        # place temci in the same cpu set as the rest of the system?
        temci_in_base_set:         Bool()
                    default: true

    #   Implements a simple run driver that just executes one of the passed run_cmds
    #   in each benchmarking run.
    #   It measures the time  using the perf stat tool (runner=perf_stat).
    #   The constructor calls the ``setup`` method.
    exec_misc:     
        # Argument passed to all benchmarked commands by replacing $ARGUMENT with this value in the
        # command
        argument:         Str()

        # Parse the program output as a YAML dictionary of that gives for a specific property a
        # measurement. Not all runners support it.
        parse_output:         Bool()

        # Order in which the plugins are used, plugins that do not appear in this list are used before all
        # others
        plugin_order:         ListOrTuple(Str())
                    default: [drop_fs_caches, sync, sleep, preheat, flush_cpu_caches]

        # Enable other plugins by default: none =  (enable none by default); all = cpu_governor,disable_sw
        # ap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable
        # all, might freeze your system); usable =
        # cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like
        # 'all' but doesn't affect other processes)
        preset:         ExactEither('none'|'all'|'usable')
                    default: none

        # Pick a random command if more than one run command is passed.
        random_cmd:         Bool()
                    default: true

        # If not '' overrides the runner setting for each program block
        runner:         ExactEither(''|'perf_stat'|'rusage'|'spec'|'spec.py'|'time'|'output')

    exec_plugins:     
        # Enable:   Allows the setting of the scaling governor of all cpu cores, to ensure that all use
        # the same.
        cpu_governor_active:         BoolOrNone()

        # Enable:   Enable cpusets, simply sets run/cpuset/active to true
        cpuset_active:         BoolOrNone()

        # Enable:   Disable amd turbo boost
        disable_amd_boost_active:         BoolOrNone()

        # Enable:   Disable address space randomization
        disable_aslr_active:         BoolOrNone()

        # Enable:   Disable the L1 and L2 caches on x86 and x86-64 architectures.
        #   Uses a small custom kernel module (be sure to compile it via 'temci setup
        # --build_kernel_modules').
        #   :warning: slows program down significantly and has probably other weird consequences
        #   :warning: this is untested
        #   :warning: a linux-forum user declared: Disabling cpu caches gives you a pentium I like
        # processor!!!
        disable_cpu_caches_active:         BoolOrNone()

        # Enable:   Disable hyper-threading
        disable_ht_active:         BoolOrNone()

        # Enable:   Disable intel turbo mode
        disable_intel_turbo_active:         BoolOrNone()

        # Enable:   Disables swapping on the system before the benchmarking and enables it after.
        disable_swap_active:         BoolOrNone()

        # Enable:   Disable amd and intel turbo boost
        disable_turbo_boost_active:         BoolOrNone()

        # Enable:   Sets run/discarded_runs
        discarded_runs_active:         BoolOrNone()

        # Enable:   Drop page cache, directoy entries and inodes before every benchmarking run.
        drop_fs_caches_active:         BoolOrNone()

        # Enable:   Adds random environment variables.
        env_randomize_active:         BoolOrNone()

        # Possible run driver plugins are 'nice', 'env_randomize', 'preheat', 'other_nice', 'stop_start',
        # 'sync', 'sleep', 'drop_fs_caches', 'disable_swap', 'disable_cpu_caches', 'flush_cpu_caches',
        # 'cpu_governor', 'disable_aslr', 'disable_ht', 'disable_turbo_boost', 'disable_intel_turbo',
        # 'disable_amd_boost', 'cpuset' and 'discarded_runs'
        exec_active:         StrList(allowed=['nice', 'env_randomize', 'preheat', 'other_nice', 'stop_start', 'sync', 'sleep', 'drop_fs_caches', 'disable_swap', 'disable_cpu_caches', 'flush_cpu_caches', 'cpu_governor', 'disable_aslr', 'disable_ht', 'disable_turbo_boost', 'disable_intel_turbo', 'disable_amd_boost', 'cpuset', 'discarded_runs'])

        # Enable:   Flushes the CPU caches on a x86 CPU using a small kernel module,
        #   see `WBINVD <https://www.felixcloutier.com/x86/wbinvd>`_
        flush_cpu_caches_active:         BoolOrNone()

        # Enable:   Allows the setting of the nice and ionice values of the benchmarking process.
        nice_active:         BoolOrNone()

        # Enable:   Allows the setting of the nice value of most other processes (as far as possible).
        other_nice_active:         BoolOrNone()

        # Enable:   Preheats the system with a cpu bound task
        #   (calculating the inverse of a big random matrix with numpy).
        preheat_active:         BoolOrNone()

        # Enable:   Sleep a given amount of time before the benchmarking begins.
        #   See Gernot Heisers Systems Benchmarking Crimes:
        #   Make sure that the system is really quiescent when starting an experiment,
        #   leave enough time to ensure all previous data is flushed out.
        sleep_active:         BoolOrNone()

        # Enable:   Stop almost all other processes (as far as possible).
        stop_start_active:         BoolOrNone()

        # Enable:   Calls ``sync`` before each program execution.
        sync_active:         BoolOrNone()

        #   Allows the setting of the scaling governor of all cpu cores, to ensure that all use the same.
        cpu_governor_misc:     
            # New scaling governor for all cpus
            governor:         Str()
                        default: performance

        #   Enable cpusets, simply sets run/cpuset/active to true
        cpuset_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disable amd turbo boost
        disable_amd_boost_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disable address space randomization
        disable_aslr_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disable the L1 and L2 caches on x86 and x86-64 architectures.
        #   Uses a small custom kernel module (be sure to compile it via 'temci setup
        # --build_kernel_modules').
        #   :warning: slows program down significantly and has probably other weird consequences
        #   :warning: this is untested
        #   :warning: a linux-forum user declared: Disabling cpu caches gives you a pentium I like
        # processor!!!
        disable_cpu_caches_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disable hyper-threading
        disable_ht_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disable intel turbo mode
        disable_intel_turbo_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disables swapping on the system before the benchmarking and enables it after.
        disable_swap_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Disable amd and intel turbo boost
        disable_turbo_boost_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Sets run/discarded_runs
        discarded_runs_misc:     
            # Number of discarded runs
            runs:         Int(constraint=<function>)
                        default: 1

        #   Drop page cache, directoy entries and inodes before every benchmarking run.
        drop_fs_caches_misc:     
            # Free dentries and inodes
            free_dentries_inodes:         Bool()
                        default: true

            # Free the page cache
            free_pagecache:         Bool()
                        default: true

        #   Adds random environment variables.
        env_randomize_misc:     
            # Maximum length of each random key
            key_max:         Int(constraint=<function>)
                        default: 4096

            # Maximum number of added random environment variables
            max:         Int(constraint=<function>)
                        default: 4

            # Minimum number of added random environment variables
            min:         Int(constraint=<function>)
                        default: 4

            # Maximum length of each random value
            var_max:         Int(constraint=<function>)
                        default: 4096

        #   Flushes the CPU caches on a x86 CPU using a small kernel module,
        #   see `WBINVD <https://www.felixcloutier.com/x86/wbinvd>`_
        flush_cpu_caches_misc:         Dict({}, False, keys=Any, values=Any, default = {})

        #   Allows the setting of the nice and ionice values of the benchmarking process.
        nice_misc:     
            # Specify the name or number of the scheduling class to use;0 for none, 1 for realtime, 2 for
            # best-effort, 3 for idle.
            io_nice:         Int(range=range(0, 4))
                        default: 1

            # Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the
            # process).
            nice:         Int(range=range(-20, 20))
                        default: -15

        #   Allows the setting of the nice value of most other processes (as far as possible).
        other_nice_misc:     
            # Processes with lower nice values are ignored.
            min_nice:         Int(range=range(-15, 20))
                        default: -10

            # Niceness values for other processes.
            nice:         Int(range=range(-20, 20))
                        default: 19

        #   Preheats the system with a cpu bound task
        #   (calculating the inverse of a big random matrix with numpy).
        preheat_misc:     
            # Number of seconds to preheat the system with an cpu bound task
            time:         Int(constraint=<function>)
                        default: 10

            # When to preheat
            when:         ListOrTuple(ExactEither('before_each_run'|'at_setup'))
                        default: [before_each_run]

        #   Sleep a given amount of time before the benchmarking begins.
        #   See Gernot Heisers Systems Benchmarking Crimes:
        #   Make sure that the system is really quiescent when starting an experiment,
        #   leave enough time to ensure all previous data is flushed out.
        sleep_misc:     
            # Seconds to sleep
            seconds:         Int(constraint=<function>)
                        default: 10

        #   Stop almost all other processes (as far as possible).
        stop_start_misc:     
            # Each process which name (lower cased) starts with one of the prefixes is not ignored. Overrides
            # the decision based on the min_id.
            comm_prefixes:         ListOrTuple(Str())
                        default: [ssh, xorg, bluetoothd]

            # Each process which name (lower cased) starts with one of the prefixes is ignored. It overrides
            # the decisions based on comm_prefixes and min_id.
            comm_prefixes_ignored:         ListOrTuple(Str())
                        default: [dbus, kworker]

            # Just output the to be stopped processes but don't actually stop them?
            dry_run:         Bool()

            # Processes with lower id are ignored.
            min_id:         Int(constraint=<function>)
                        default: 1500

            # Processes with lower nice values are ignored.
            min_nice:         Int(range=range(-15, 20))
                        default: -10

            # Suffixes of processes names which are stopped.
            subtree_suffixes:         ListOrTuple(Str())
                        default: [dm, apache]

        #   Calls ``sync`` before each program execution.
        sync_misc:         Dict({}, False, keys=Any, values=Any, default = {})

    # Maximum runs per tag (block attribute 'tag'), min('max_runs', 'per_tag') is used
    max_runs_per_tag:         Dict(, keys=Str(), values=Int(constraint=<function>), default = {})

    # Minimum runs per tag (block attribute 'tag'), max('min_runs', 'per_tag') is used
    min_runs_per_tag:         Dict(, keys=Str(), values=Int(constraint=<function>), default = {})

    # Runs per tag (block attribute 'tag'), max('runs', 'per_tag') is used
    runs_per_tag:         Dict(, keys=Str(), values=Int(constraint=<function>), default = {})

    #   Implements a run driver that runs the benched command a single time with redirected in- and
    # output.
    #   It can be used to run own benchmarking commands inside a sane benchmarking environment
    #   The constructor calls the ``setup`` method.
    shell_misc:     
        # Order in which the plugins are used, plugins that do not appear in this list are used before all
        # others
        plugin_order:         ListOrTuple(Str())
                    default: [drop_fs_caches, sync, sleep, preheat, flush_cpu_caches]

        # Enable other plugins by default: none =  (enable none by default); all = cpu_governor,disable_sw
        # ap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable
        # all, might freeze your system); usable =
        # cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like
        # 'all' but doesn't affect other processes)
        preset:         ExactEither('none'|'all'|'usable')
                    default: none

stats:     
    # Properties to use for reporting and null hypothesis tests, can be regular expressions
    properties:         ListOrTuple(Str())
                default: [all]

    # Possible testers are 't', 'ks' and 'anderson'
    tester:         ExactEither('t'|'ks'|'anderson')
                default: t

    # Range of p values that allow no conclusion.
    uncertainty_range:         Tuple(T(<class 'float'>):<function>, T(<class 'float'>):<function>)
                default: [0.05, 0.15]

    #   Tester that uses the Anderson statistic on 2 samples.
    anderson_misc:         Dict({}, False, keys=Any, values=Any, default = {})

    #   Tester that uses the Kolmogorov-Smirnov statistic on 2 samples.
    ks_misc:         Dict({}, False, keys=Any, values=Any, default = {})

    #   Tester that uses the student's t test.
    t_misc:         Dict({}, False, keys=Any, values=Any, default = {})

Initializes a Settings singleton object and thereby loads the Settings files. It loads the settings files from the app folder (config.yaml) and the current working directory (temci.yaml) if they exist.

Raises

SettingsError if some of the settings aren’t in the format described via the type_scheme class property

apply_override_actions()[source]

Applies actions like overriding max_runs with runs

config_file_name = 'temci.yaml'

Default name of the configuration files

default(value: Optional[Any], key: str)[source]

Returns the passed value if isn’t None else the settings value under the passed key.

Parameters
  • value – passed value

  • key – passed settings key

get(key: Union[str, List[str]]) Any[source]

Get the setting with the given key.

Parameters

key – name of the setting

Returns

value of the setting

Raises

SettingsError if the setting doesn’t exist

get_type_scheme(key: Union[str, List[str]]) Type[source]

Returns the type scheme of the given key.

Parameters

key – given key

Returns

type scheme

Raises

SettingsError if the setting with the given key doesn’t exist

has_key(key: str) bool[source]

Does the passed key exist?

has_log_level(level: str) bool[source]

Is the current log level the passed level?

Parameters

level – passed level (in [“error”, “warn”, “info”, “debug”])

is_obsolete(key: Union[str, List[str]]) bool[source]

Is the setting with the passed key obsolete?

Parameters

key – key or key path

Returns

obsolete setting?

load_file(file: str)[source]

Loads the configuration from the configuration YAML file.

Parameters

file – path to the file

Raises

SettingsError if the settings file is incorrect or doesn’t exist

load_files()[source]

Loads the configuration files from the current and the config directory

load_from_config_dir()[source]

Load the config file from the application directory (e.g. in the users home folder) if it exists.

load_from_current_dir()[source]

Load the configuration from the `configuration file in the current working directory if it exists.

load_from_dict(config_dict: Dict[str, Any])[source]

Load the configuration from the passed dictionary.

Parameters

config_dict – passed configuration dictionary

load_from_dir(dir: str)[source]

Load the configuration from the configuration file inside the passed directory.

Parameters

dir – path of the directory

modify_setting(key: str, type_scheme: Type)[source]

Modifies the setting with the given key and adds it if it doesn’t exist.

Parameters
  • key – key of the setting

  • type_scheme – Type of the setting

  • default_value – default value of the setting

Raises

SettingsError if the settings domain (the key without the last element) doesn’t exist

Raises

TypeError if the default value doesn’t adhere the type scheme

modify_type_scheme(key: str, modificator: Callable[[Type], Type])[source]

Modifies the type scheme of the given key via a modificator function.

Parameters
  • key – given key

  • modificator – gets the type scheme and returns its modified version

Raises

SettingsError if the setting with the given key doesn’t exist

obsoleteness_reason(key: Union[str, List[str]]) Optional[Obsolete][source]

Returns the obsolete type object for obsolete settings

Parameters

key – key or path

Returns

object that contains information on the obsoleteness or None

prefs

The set sonfigurations

reset()[source]

Resets the current settings to the defaults.

set(key: str, value, validate: bool = True, setup: bool = True)[source]

Sets the setting key to the passed new value

Parameters
  • key – settings key

  • value – new value

  • validate – validate after the setting operation

  • setup – call the setup function

Raises

SettingsError if the setting isn’t valid

store_into_file(file_name: str, comment_out_defaults: bool = False)[source]

Stores the current settings into a yaml file with comments.

Parameters
  • file_name – name of the resulting file

  • comment_out_defaults – comment out the default values

type_scheme = # Alias for settings config:         Str()  # Logging level log_level:         ExactEither('debug'|'info'|'warn'|'error'|'quiet')             default: info  # Additional settings file settings:         Str()  # Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command # line. sudo:         Bool()  # Used temporary directory tmp_dir:         Str()             default: /tmp/temci  build:          # Input file with the program blocks to build     in:         Str()                 default: build.yaml          # Resulting run config file     out:         Str()                 default: run_config.yaml          # Number of simultaneous builds for a specific program block, only makes sense when     # build_config/number > 1, and if the build commands create a different binary every time they are     # run     threads:         Int(constraint=<function>)                 default: 1  # Environment variables for the benchmarked programs, includes the user used for generated files env:          PATH:         Str()          USER:         Str()  report:          # Exclude all data sets that contain only NaNs.     exclude_invalid:         BoolOrNone()                 default: true          # Properties that aren't shown in the report.     excluded_properties:         ListOrTuple(Str())                 default: [__ov-time]          # Files that contain the benchmarking results     in:         Either(Str()|ListOrTuple(Str()))                 default: run_output.yaml          # List of included run blocks (all: include all), identified by their description or tag     # attribute, can be regular expressions     included_blocks:         ListOrTuple(Str())                 default: [all]          # Replace the property names in reports with longer more descriptive versions?     long_properties:         BoolOrNone()          # Possible reporter are 'console', 'html2', 'csv', 'codespeed', 'codespeed2' and 'velcom'     reporter:         ExactEither('console'|'html2'|'csv'|'codespeed'|'codespeed2'|'velcom')                 default: console          # Produce xkcd like plots (requires the humor sans font to be installed)     xkcd_like_plots:         BoolOrNone()          #   Reporter that outputs JSON as specified by     #   the `codespeed runner spec <https://git.scc.kit.edu/IPDSnelting/codespeed-runner>`_.     codespeed2_misc:         Dict({}, False, keys=Any, values=Any, default = {})          #   Reporter that outputs JSON as expected by `codespeed <https://github.com/tobami/codespeed>`_.     #   Branch name and commit ID are taken from the current directory.     #   Use it like this:     #   .. code:: sh     #   temci report --reporter codespeed ... | curl --data-urlencode json@-     # http://localhost:8000/result/add/json/     codespeed_misc:              # Branch name reported to codespeed. Defaults to current branch or else 'master'.         branch:         Str()                  # Commit ID reported to codespeed. Defaults to current commit.         commit_id:         Str()                  # Environment name reported to codespeed. Defaults to current host name.         environment:         Str()                  # Executable name reported to codespeed. Defaults to the project name.         executable:         Str()                  # Project name reported to codespeed.         project:         Str()          #   Simple reporter that outputs just text.     console_misc:              # Matches the baseline block         baseline:         Str()                  # Position of the baseline comparison: each: after each block, after: after each cluster, both:         # after each and after cluster, instead: instead of the non baselined         baseline_position:         ExactEither('each'|'after'|'both'|'instead')                     default: each                  # 'auto': report clusters (runs with the same description) and singles (clusters with a single         # entry, combined) separately, 'single': report all clusters together as one, 'cluster': report         # all clusters separately, 'both': append the output of 'cluster' to the output of 'single'         mode:         ExactEither('both'|'cluster'|'single'|'auto')                     default: auto                  # Output file name or `-` (stdout)         out:         FileName(allow_std=True)                     default: '-'                  # Report on the failing blocks         report_errors:         Bool()                     default: true                  # Print statistical tests for every property for every two programs         with_tester_results:         Bool()                     default: true          #   Simple reporter that outputs just a configurable csv table with rows for each run block     csv_misc:              # List of valid column specs, format is a comma separated list of 'PROPERTY\[mod\]' or 'ATTRIBUTE'         # mod is one of: mean, stddev, property, min, max and stddev per mean, optionally a formatting         # option can be given viaPROPERTY\[mod|OPT1OPT2…\], where the OPTs are one of the following: %         # (format as percentage), p (wrap insignificant digits in parentheses (+- 2 std dev)), s (use         # scientific notation, configured in report/number) and o (wrap digits in the order of magnitude         # of 2 std devs in parentheses). PROPERTY can be either the description or the short version of         # the property. Configure the number formatting further via the number settings in the settings         # file         columns:         ListOrTuple(Str())                     default: [description]             # with constrain: List of valid column specs, format is a comma separated list of 'PROPERTY\[mod\]' or 'ATTRIBUTE' mod is one of: mean, stddev, property, min, max and stddev per mean, optionally a formatting option can be given viaPROPERTY\[mod|OPT1OPT2…\], where the OPTs are one of the following: % (format as percentage), p (wrap insignificant digits in parentheses (+- 2 std dev)), s (use scientific notation, configured in report/number) and o (wrap digits in the order of magnitude of 2 std devs in parentheses). PROPERTY can be either the description or the short version of the property. Configure the number formatting further via the number settings in the settings file                  # Output file name or standard out (-)         out:         FileName(allow_std=True)                     default: '-'          #   Reporter that produces a HTML based report with lot's of graphics.     #   A rewrite of the original HTMLReporter     html2_misc:              # Alpha value for confidence intervals         alpha:         T(<class 'float'>)                     default: 0.05                  # Height per run block for the big comparison box plots         boxplot_height:         T(<class 'float'>)                     default: 2.0                  # Width of all big plotted figures         fig_width_big:         T(<class 'float'>)                     default: 25.0                  # Width of all small plotted figures         fig_width_small:         T(<class 'float'>)                     default: 15.0                  # Format string used to format floats         float_format:         Str()                     default: '{:5.2e}'                  # Override the contents of the output directory if it already exists?         force_override:         Bool()                  # Generate pdf versions of the plotted figures?         gen_pdf:         Bool()                  # Generate simple latex versions of the plotted figures?         gen_tex:         Bool()                     default: true                  # Generate excel files for all tables         gen_xls:         Bool()                  # Hide warnings and errors related to statistical properties         hide_stat_warnings:         Bool()                  # Name of the HTML file         html_filename:         Str()                     default: report.html                  # Use local versions of all third party resources         local:         Bool()                  # Show the mean related values in the big comparison table         mean_in_comparison_tables:         Bool()                     default: true                  # Show the mininmum related values in the big comparison table         min_in_comparison_tables:         Bool()                  # Output directory         out:         Str()                     default: report                  # Format string used to format floats as percentages         percent_format:         Str()                     default: '{:5.2%}'                  # Show zoomed out (x min = 0) figures in the extended summaries?         show_zoomed_out:         Bool()                     default: true          number:              # Don't omit the minimum number of decimal places if insignificant?         force_min_decimal_places:         Bool()                     default: true                  # The maximum number of decimal places         max_decimal_places:         Int(constraint=<function>)                     default: 5                  # The minimum number of shown decimal places if decimal places are shown         min_decimal_places:         Int(constraint=<function>)                     default: 3                  # Omit insignificant decimal places         omit_insignificant_decimal_places:         Bool()                  # Show parentheses around non significant digits? (If a std dev is given)         parentheses:         Bool()                     default: true                  # Mode for showing the parentheses: either d (Digits are considered significant if they don't         # change if the number itself changes += $sigmas * std dev) or o (digits are considered         # significant if they are bigger than $sigmas * std dev)         parentheses_mode:         ExactEither('d'|'o')                     default: o                  # Show as percentages         percentages:         Bool()                  # Use the exponential notation, i.e. '10e3' for 1000         scientific_notation:         Bool()                     default: true                  # Use si prefixes instead of 'e…'         scientific_notation_si_prefixes:         Bool()                     default: true                  # Number of standard deviation used for the digit significance evaluation         sigmas:         Int(constraint=<function>)                     default: 2          #   Reporter that outputs JSON as specified by     #   the `velcom runner spec <https://github.com/IPDSnelting/velcom/wiki/Benchmark-Repo-     # Specification>`_.     velcom_misc:         Dict({}, False, keys=Any, values=Any, default = {})  run:          # Abort after the first failing build     abort_after_build_error:         Bool()                 default: true          # Append to the output file instead of overwriting by adding new run data blocks     append:         Bool()          # Disable the hyper threaded cores. Good for cpu bound programs.     disable_hyper_threading:         Bool()          # Discard all run data for the failing program on error     discard_all_data_for_block_on_error:         Bool()          # First n runs that are discarded     discarded_runs:         Int(constraint=<function>)                 default: 1          # Possible run drivers are 'exec' and 'shell'     driver:         ExactEither('exec'|'shell')                 default: exec          # Input file with the program blocks to benchmark     in:         Str()                 default: input.exec.yaml          # List of included run blocks (all: include all), or their tag attribute or their number in the     # file (starting with 0), can be regular expressions     included_blocks:         ListOrTuple(Str())                 default: [all]          # Maximum time one run block should take, -1 == no timeout, supports normal time span expressions     max_block_time:         ValidTimespan()                 default: '-1'          # Maximum number of benchmarking runs     max_runs:         Int(constraint=<function>)                 default: 100          # Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span     # expressions     max_time:         ValidTimespan()                 default: '-1'          # Minimum number of benchmarking runs     min_runs:         Int(constraint=<function>)                 default: 20          # Do not build if build configs are present, only works if the working directory of the blocks     # does not change     no_build:         Bool()          # Only build     only_build:         Bool()          # Output file for the benchmarking results     out:         Str()                 default: run_output.yaml          # Record the caught errors in the run_output file     record_errors_in_file:         Bool()                 default: true          # Number of benchmarking runs that are done together     run_block_size:         Int(constraint=<function>)                 default: 1          # if != -1 sets max and min runs to its value     runs:         Int(constraint=<function>)                 default: -1          # If not empty, recipient of a mail after the benchmarking finished.     send_mail:         Str()          # Print console report if log_level=info     show_report:         Bool()                 default: true          # Randomize the order in which the program blocks are benchmarked.     shuffle:         Bool()                 default: true          # Store the result file after each set of blocks is benchmarked     store_often:         Bool()          # Show the report continuously     watch:         Bool()          # Update the screen nth run (less updates are better for benchmarks)     watch_every:         Int(constraint=<function>)                 default: 1          cpuset:              # Use cpuset functionality?         active:         Bool()                  # Number of cpu cores for the base (remaining part of the) system         base_core_number:         Int(range=range(0, 2))                     default: 1                  # 0: benchmark sequential, > 0: benchmark parallel with n instances, -1: determine n automatically         parallel:         Int(constraint=<function>)                  # Number of cpu cores per parallel running program.         sub_core_number:         Int(range=range(0, 2))                     default: 1                  # place temci in the same cpu set as the rest of the system?         temci_in_base_set:         Bool()                     default: true          #   Implements a simple run driver that just executes one of the passed run_cmds     #   in each benchmarking run.     #   It measures the time  using the perf stat tool (runner=perf_stat).     #   The constructor calls the ``setup`` method.     exec_misc:              # Argument passed to all benchmarked commands by replacing $ARGUMENT with this value in the         # command         argument:         Str()                  # Parse the program output as a YAML dictionary of that gives for a specific property a         # measurement. Not all runners support it.         parse_output:         Bool()                  # Order in which the plugins are used, plugins that do not appear in this list are used before all         # others         plugin_order:         ListOrTuple(Str())                     default: [drop_fs_caches, sync, sleep, preheat, flush_cpu_caches]                  # Enable other plugins by default: none =  (enable none by default); all = cpu_governor,disable_sw         # ap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable         # all, might freeze your system); usable =         # cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like         # 'all' but doesn't affect other processes)         preset:         ExactEither('none'|'all'|'usable')                     default: none                  # Pick a random command if more than one run command is passed.         random_cmd:         Bool()                     default: true                  # If not '' overrides the runner setting for each program block         runner:         ExactEither(''|'perf_stat'|'rusage'|'spec'|'spec.py'|'time'|'output')          exec_plugins:              # Enable:   Allows the setting of the scaling governor of all cpu cores, to ensure that all use         # the same.         cpu_governor_active:         BoolOrNone()                  # Enable:   Enable cpusets, simply sets run/cpuset/active to true         cpuset_active:         BoolOrNone()                  # Enable:   Disable amd turbo boost         disable_amd_boost_active:         BoolOrNone()                  # Enable:   Disable address space randomization         disable_aslr_active:         BoolOrNone()                  # Enable:   Disable the L1 and L2 caches on x86 and x86-64 architectures.         #   Uses a small custom kernel module (be sure to compile it via 'temci setup         # --build_kernel_modules').         #   :warning: slows program down significantly and has probably other weird consequences         #   :warning: this is untested         #   :warning: a linux-forum user declared: Disabling cpu caches gives you a pentium I like         # processor!!!         disable_cpu_caches_active:         BoolOrNone()                  # Enable:   Disable hyper-threading         disable_ht_active:         BoolOrNone()                  # Enable:   Disable intel turbo mode         disable_intel_turbo_active:         BoolOrNone()                  # Enable:   Disables swapping on the system before the benchmarking and enables it after.         disable_swap_active:         BoolOrNone()                  # Enable:   Disable amd and intel turbo boost         disable_turbo_boost_active:         BoolOrNone()                  # Enable:   Sets run/discarded_runs         discarded_runs_active:         BoolOrNone()                  # Enable:   Drop page cache, directoy entries and inodes before every benchmarking run.         drop_fs_caches_active:         BoolOrNone()                  # Enable:   Adds random environment variables.         env_randomize_active:         BoolOrNone()                  # Possible run driver plugins are 'nice', 'env_randomize', 'preheat', 'other_nice', 'stop_start',         # 'sync', 'sleep', 'drop_fs_caches', 'disable_swap', 'disable_cpu_caches', 'flush_cpu_caches',         # 'cpu_governor', 'disable_aslr', 'disable_ht', 'disable_turbo_boost', 'disable_intel_turbo',         # 'disable_amd_boost', 'cpuset' and 'discarded_runs'         exec_active:         StrList(allowed=['nice', 'env_randomize', 'preheat', 'other_nice', 'stop_start', 'sync', 'sleep', 'drop_fs_caches', 'disable_swap', 'disable_cpu_caches', 'flush_cpu_caches', 'cpu_governor', 'disable_aslr', 'disable_ht', 'disable_turbo_boost', 'disable_intel_turbo', 'disable_amd_boost', 'cpuset', 'discarded_runs'])                  # Enable:   Flushes the CPU caches on a x86 CPU using a small kernel module,         #   see `WBINVD <https://www.felixcloutier.com/x86/wbinvd>`_         flush_cpu_caches_active:         BoolOrNone()                  # Enable:   Allows the setting of the nice and ionice values of the benchmarking process.         nice_active:         BoolOrNone()                  # Enable:   Allows the setting of the nice value of most other processes (as far as possible).         other_nice_active:         BoolOrNone()                  # Enable:   Preheats the system with a cpu bound task         #   (calculating the inverse of a big random matrix with numpy).         preheat_active:         BoolOrNone()                  # Enable:   Sleep a given amount of time before the benchmarking begins.         #   See Gernot Heisers Systems Benchmarking Crimes:         #   Make sure that the system is really quiescent when starting an experiment,         #   leave enough time to ensure all previous data is flushed out.         sleep_active:         BoolOrNone()                  # Enable:   Stop almost all other processes (as far as possible).         stop_start_active:         BoolOrNone()                  # Enable:   Calls ``sync`` before each program execution.         sync_active:         BoolOrNone()                  #   Allows the setting of the scaling governor of all cpu cores, to ensure that all use the same.         cpu_governor_misc:                  # New scaling governor for all cpus             governor:         Str()                         default: performance                  #   Enable cpusets, simply sets run/cpuset/active to true         cpuset_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disable amd turbo boost         disable_amd_boost_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disable address space randomization         disable_aslr_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disable the L1 and L2 caches on x86 and x86-64 architectures.         #   Uses a small custom kernel module (be sure to compile it via 'temci setup         # --build_kernel_modules').         #   :warning: slows program down significantly and has probably other weird consequences         #   :warning: this is untested         #   :warning: a linux-forum user declared: Disabling cpu caches gives you a pentium I like         # processor!!!         disable_cpu_caches_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disable hyper-threading         disable_ht_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disable intel turbo mode         disable_intel_turbo_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disables swapping on the system before the benchmarking and enables it after.         disable_swap_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Disable amd and intel turbo boost         disable_turbo_boost_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Sets run/discarded_runs         discarded_runs_misc:                  # Number of discarded runs             runs:         Int(constraint=<function>)                         default: 1                  #   Drop page cache, directoy entries and inodes before every benchmarking run.         drop_fs_caches_misc:                  # Free dentries and inodes             free_dentries_inodes:         Bool()                         default: true                          # Free the page cache             free_pagecache:         Bool()                         default: true                  #   Adds random environment variables.         env_randomize_misc:                  # Maximum length of each random key             key_max:         Int(constraint=<function>)                         default: 4096                          # Maximum number of added random environment variables             max:         Int(constraint=<function>)                         default: 4                          # Minimum number of added random environment variables             min:         Int(constraint=<function>)                         default: 4                          # Maximum length of each random value             var_max:         Int(constraint=<function>)                         default: 4096                  #   Flushes the CPU caches on a x86 CPU using a small kernel module,         #   see `WBINVD <https://www.felixcloutier.com/x86/wbinvd>`_         flush_cpu_caches_misc:         Dict({}, False, keys=Any, values=Any, default = {})                  #   Allows the setting of the nice and ionice values of the benchmarking process.         nice_misc:                  # Specify the name or number of the scheduling class to use;0 for none, 1 for realtime, 2 for             # best-effort, 3 for idle.             io_nice:         Int(range=range(0, 4))                         default: 1                          # Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the             # process).             nice:         Int(range=range(-20, 20))                         default: -15                  #   Allows the setting of the nice value of most other processes (as far as possible).         other_nice_misc:                  # Processes with lower nice values are ignored.             min_nice:         Int(range=range(-15, 20))                         default: -10                          # Niceness values for other processes.             nice:         Int(range=range(-20, 20))                         default: 19                  #   Preheats the system with a cpu bound task         #   (calculating the inverse of a big random matrix with numpy).         preheat_misc:                  # Number of seconds to preheat the system with an cpu bound task             time:         Int(constraint=<function>)                         default: 10                          # When to preheat             when:         ListOrTuple(ExactEither('before_each_run'|'at_setup'))                         default: [before_each_run]                  #   Sleep a given amount of time before the benchmarking begins.         #   See Gernot Heisers Systems Benchmarking Crimes:         #   Make sure that the system is really quiescent when starting an experiment,         #   leave enough time to ensure all previous data is flushed out.         sleep_misc:                  # Seconds to sleep             seconds:         Int(constraint=<function>)                         default: 10                  #   Stop almost all other processes (as far as possible).         stop_start_misc:                  # Each process which name (lower cased) starts with one of the prefixes is not ignored. Overrides             # the decision based on the min_id.             comm_prefixes:         ListOrTuple(Str())                         default: [ssh, xorg, bluetoothd]                          # Each process which name (lower cased) starts with one of the prefixes is ignored. It overrides             # the decisions based on comm_prefixes and min_id.             comm_prefixes_ignored:         ListOrTuple(Str())                         default: [dbus, kworker]                          # Just output the to be stopped processes but don't actually stop them?             dry_run:         Bool()                          # Processes with lower id are ignored.             min_id:         Int(constraint=<function>)                         default: 1500                          # Processes with lower nice values are ignored.             min_nice:         Int(range=range(-15, 20))                         default: -10                          # Suffixes of processes names which are stopped.             subtree_suffixes:         ListOrTuple(Str())                         default: [dm, apache]                  #   Calls ``sync`` before each program execution.         sync_misc:         Dict({}, False, keys=Any, values=Any, default = {})          # Maximum runs per tag (block attribute 'tag'), min('max_runs', 'per_tag') is used     max_runs_per_tag:         Dict(, keys=Str(), values=Int(constraint=<function>), default = {})          # Minimum runs per tag (block attribute 'tag'), max('min_runs', 'per_tag') is used     min_runs_per_tag:         Dict(, keys=Str(), values=Int(constraint=<function>), default = {})          # Runs per tag (block attribute 'tag'), max('runs', 'per_tag') is used     runs_per_tag:         Dict(, keys=Str(), values=Int(constraint=<function>), default = {})          #   Implements a run driver that runs the benched command a single time with redirected in- and     # output.     #   It can be used to run own benchmarking commands inside a sane benchmarking environment     #   The constructor calls the ``setup`` method.     shell_misc:              # Order in which the plugins are used, plugins that do not appear in this list are used before all         # others         plugin_order:         ListOrTuple(Str())                     default: [drop_fs_caches, sync, sleep, preheat, flush_cpu_caches]                  # Enable other plugins by default: none =  (enable none by default); all = cpu_governor,disable_sw         # ap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable         # all, might freeze your system); usable =         # cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like         # 'all' but doesn't affect other processes)         preset:         ExactEither('none'|'all'|'usable')                     default: none  stats:          # Properties to use for reporting and null hypothesis tests, can be regular expressions     properties:         ListOrTuple(Str())                 default: [all]          # Possible testers are 't', 'ks' and 'anderson'     tester:         ExactEither('t'|'ks'|'anderson')                 default: t          # Range of p values that allow no conclusion.     uncertainty_range:         Tuple(T(<class 'float'>):<function>, T(<class 'float'>):<function>)                 default: [0.05, 0.15]          #   Tester that uses the Anderson statistic on 2 samples.     anderson_misc:         Dict({}, False, keys=Any, values=Any, default = {})          #   Tester that uses the Kolmogorov-Smirnov statistic on 2 samples.     ks_misc:         Dict({}, False, keys=Any, values=Any, default = {})          #   Tester that uses the student's t test.     t_misc:         Dict({}, False, keys=Any, values=Any, default = {})

Type scheme of the settings

validate()[source]

Validate this settings object

Raises

SettingsError if the setting isn’t valid

validate_key_path(path: List[str]) bool[source]

Validates a path into in to the settings trees, :param path: list of sub keys :return: Is this key path valid?

exception temci.utils.settings.SettingsError[source]

Bases: ValueError

Error raised if something with the settings goes wrong

temci.utils.settings.ValidCPUCoreNumber() Int[source]

Creates a Type instance that matches all valid CPU core numbers.

temci.utils.sudo_utils module

temci.utils.sudo_utils.bench_as_different_user() bool[source]
temci.utils.sudo_utils.chown(path: Union[str, TextIOWrapper, BufferedRandom, BufferedRWPair, BufferedWriter, IOBase])[source]
temci.utils.sudo_utils.get_bench_uid_and_gid() Tuple[int, int][source]
temci.utils.sudo_utils.get_bench_user() str[source]
temci.utils.sudo_utils.get_env_setting() Dict[str, str][source]

temci.utils.typecheck module

Implements basic type checking for complex types.

Why? Because it’s nice to be able to type check complex structures that come directly from the user (e.g. from YAML config files).

The Type instance are usable with the standard isinstance function:

isinstance(4, Either(Float(), Int()))

Type instances also support the “&” (producres All(one, two)) and “|” (produces Either(one, two)) operators. The above sample code can therefore be written as:

isinstance(4, Float() | Int())

The native type wrappers also support custom constraints. With help of the fn module one can write:

t = Float(_ > 0) | Int(_ > 10)
isinstance(var, t)

“t” is a Type that matches only floats greater than 0 and ints greater than 10.

For more examples look into the test_typecheck.py file.

class temci.utils.typecheck.All(*types: Tuple[Type])[source]

Bases: Type

Checks for the value to be of all of several types.

Creates an All instance.

Parameters

types – list of types (or SpecialType subclasses)

Raises

ConstraintError if some of the contraints aren’t (typechecker) Types

types

Expected types

class temci.utils.typecheck.Any(completion_hints: Optional[Dict[str, Any]] = None)[source]

Bases: Type

Checks for the value to be of any type.

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

class temci.utils.typecheck.Bool[source]

Bases: Type, ParamType

Like Bool but with a third value none that declares that the value no boolean value. It has None as its default value (by default).

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

completion_hints

Completion hints for supported shells for this type instance

name: str = 'bool'

click.ParamType name, that makes this class usable as a click type

class temci.utils.typecheck.BoolOrNone[source]

Bases: Type, ParamType

Like Bool but with a third value none that declares that the value no boolean value. It has None as its default value (by default).

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

completion_hints

Completion hints for supported shells for this type instance

convert(value, param, ctx: Context) Optional[bool][source]

Convert method that makes this class usable as a click type.

default

The default value of this instance

name: str = 'bool_or_none'

click.ParamType name, that makes this class usable as a click type

class temci.utils.typecheck.CompletionHint(**hints)[source]

Bases: object

A completion hint annotation for a type. Usage example:

Int() // Completion(zsh="_files")
hints

Completion hints for every supported shell

class temci.utils.typecheck.Constraint(constraint: Callable[[Any], bool], constrained_type: Type = Any, description: Optional[str] = None)[source]

Bases: Type

Checks the passed value by an user defined constraint.

Creates an Constraint instance.

Parameters
  • constraint – function that returns True if the user defined constraint is satisfied

  • constrained_type – Type that the constraint is applied on

  • description – short description of the constraint (e.g. “>0”)

Raises

ConstraintError if constrained_type isn’t a (typechecker) Types

constrained_type

Type that the constraint is applied on

constraint

Function that returns True if the user defined constraint is satisfied

description

Short description of the constraint (e.g. “>0”)

string_representation(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None) Union[str, List[str]][source]

Produce a YAML string that contains the default value (if possible), the description of this type and more and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

class temci.utils.typecheck.Default(default)[source]

Bases: object

A default value annotation for a Type. Usage example:

Int() // Default(3)

Especially useful to declare the default value for a key of an dictionary. Allows to use Dict(…).get_default() -> dict.

default

Default value of the annotated type

class temci.utils.typecheck.Description(description: str)[source]

Bases: object

A description of a Type, that annotates it. Usage example:

Int() // Description("Description of Int()")
description

Description string

class temci.utils.typecheck.Dict(data: Optional[Dict[Any, Type]] = None, unknown_keys: bool = False, key_type: Type = Any, value_type: Type = Any)[source]

Bases: Type

Checks for the value to be a dictionary with expected keys and values satisfy given type constraints.

Creates a new instance.

Parameters
  • data – dictionary with the expected keys and the expected types of the associated values

  • unknown_keys – accept unknown keys in value

  • key_type – expected Type of all dictionary keys

  • value_type – expected Type of all dictionary values

Raises

ConstraintError if one of the given types isn’t a (typechecker) Types

get_default() dict[source]

Returns the default value of this type :raises: ValueError if the default value isn’t set

get_default_yaml(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None, comment_out_defaults: bool = False) Union[str, List[str]][source]

Produce a YAML like string that contains the default value and the description of this type and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

  • comment_out_defaults – comment out default values

get_description(key: str) str[source]

Returns the description for the passed key or None if there isn’t one.

Parameters

key – passed key

has_default() bool[source]

Does this type instance have an default value?

is_obsolete(key: str) bool[source]

Is the type that belongs to the key Obsolete?

Parameters

key – dict key

Returns

obsolete?

key_type

Expected Type of all dictionary keys

obsoleteness_reason(key: str) Optional[Obsolete][source]

Return the obsoleteness reason (the Obsolete type object) if the key is obsolete

Parameters

key – dict key

Returns

Obsolete object or none

string_representation(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None) Union[str, List[str]][source]

Produce a YAML string that contains the default value (if possible), the description of this type and more and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

unknown_keys

Fail if value contains unknown keys

value_type

Expected Type of all dictionary values

class temci.utils.typecheck.DirName(constraint: Optional[Callable[[Any], bool]] = None)[source]

Bases: Str

A valid directory name. If the directory doesn’t exist, at least the parent directory must exist.

Creates an instance.

Parameters

constraint – function that returns True if the user defined constraint is satisfied

completion_hints

Completion hints for supported shells for this type instance

constraint

Function that returns True if the user defined constraint is satisfied

temci.utils.typecheck.E(exp_value) Exact[source]

Alias for Exact.

class temci.utils.typecheck.Either(*types: tuple)[source]

Bases: Type

Checks for the value to be of one of several types.

Creates an Either instance.

Parameters

types – list of types (or SpecialType subclasses)

Raises

ConstraintError if some of the contraints aren’t (typechecker) Types

types

Possible types

class temci.utils.typecheck.Exact(exp_value)[source]

Bases: Type

Checks for value equivalence.

Creates an Exact object.

Parameters

exp_value – value to check for

exp_value

Expected value

class temci.utils.typecheck.ExactEither(*exp_values: tuple)[source]

Bases: Type

Checks for the value to be of one of several exact values.

Creates an ExactEither instance.

Parameters

exp_values – list of types (or SpecialType subclasses)

Raises

ConstraintError if some of the contraints aren’t (typechecker) Types

exp_values

Expected values

class temci.utils.typecheck.FileName(constraint: Optional[Callable[[Any], bool]] = None, allow_std: bool = False, allow_non_existent: bool = True)[source]

Bases: Str

A valid file name. If the file doesn’t exist, at least the parent directory must exist and the file must be creatable.

Creates an instance.

Parameters
  • constraint – function that returns True if the user defined constraint is satisfied

  • allow_std – allow ‘-’ as standard out or in

  • allow_non_existent – allow files that don’t exist

allow_non_existent

Allow files that don’t exist

allow_std

Allow ‘-’ as standard out or in

completion_hints

Completion hints for supported shells for this type instance

constraint

Function that returns True if the user defined constraint is satisfied

temci.utils.typecheck.FileNameOrStdOut() FileName[source]

A valid file name or “-” for standard out.

temci.utils.typecheck.Float(constraint: Optional[Callable[[Any], bool]] = None) Union[T, Constraint][source]

Alias for Constraint(constraint, T(float)) or T(float)

Parameters

constraint – function that returns True if the user defined constraint is satisfied

class temci.utils.typecheck.Info(value_name: Optional[str] = None, value=None, _app_str: Optional[str] = None)[source]

Bases: object

Information object that is used to produce meaningful type check error messages.

Creates a new info object.

Parameters
  • value_name – name of the value that is type checked

  • value – value that is type checked

add_to_name(app_str: str) Info[source]

Creates a new info object based on this one with the given appendix to its value representation. It’s used to give information about what part of the main value is currently examined.

Parameters

app_str – app string appended to the own app string to create the app string for the new info object

Returns

new info object

errormsg(constraint: Type, value, msg: Optional[str] = None) InfoMsg[source]

Creates an info message object with the passed expected type and the optional message.

Parameters
  • constraint – passed expected type

  • value – value that violates th constraint

  • msg – additional message, it should give more information about why the constraint isn’t met

errormsg_cond(cond: bool, constraint: Type, value, msg: Optional[str] = None) InfoMsg[source]

Creates an info message object with the passed expected type and the optional message.

Parameters
  • cond – if this is false InfoMsg(True) is returned.

  • constraint – passed expected type

  • value – value that violates th constraint

  • msg – additional message, it should give more information about why the constraint isn’t met

errormsg_key_non_existent(constraint: Type, key: str) InfoMsg[source]

Creates an info message object with the passed expected type that contains the message that currently examined part of the value is unexpected.

Parameters

constraint – passed expected type

errormsg_unexpected(key: str) InfoMsg[source]

Creates an info message object with the passed expected type that contains the message that currently examined part of the value has an unexpected key.

Parameters

key – the unexpected key

get_value() Any[source]

Get the main value of this object. :raises: ValueError if the main value isn’t set

has_value

Is the value property of this info object set to a meaningful value?

set_value(value)[source]

Set the main value of this object

value

Main value that is type checked

value_name

Name of the value that is typechecked

wrap(result: bool) InfoMsg[source]

Wrap the passed bool into a InfoMsg object.

class temci.utils.typecheck.Int(constraint: Optional[Callable[[Any], bool]] = None, range: Optional[range] = None, description: Optional[str] = None)[source]

Bases: Type

Checks for the value to be of type int and to adhere to some constraints.

Creates an instance.

Parameters
  • constraint – function that returns True if the user defined constraint is satisfied

  • range – range (or list) that the value has to be part of

  • description – description of the constraints

completion_hints

Completion hints for supported shells for this type instance

constraint

Function that returns True if the user defined constraint is satisfied

description

Description of the constraints

range

Range (or list) that the value has to be part of

class temci.utils.typecheck.List(elem_type: Type = Any)[source]

Bases: Type

Checks for the value to be a list with elements of a given type.

Creates a new instance.

Parameters

elem_type – type of the list elements

Raises

ConstraintError if elem_type isn’t a (typechecker) Types

elem_type

Expected type of the list elements

get_default() Any[source]

Returns the default value of this type :raises: ValueError if the default value isn’t set

get_default_yaml(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None, comment_out_defaults: bool = False) Union[str, List[str]][source]

Produce a YAML like string that contains the default value and the description of this type and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

  • comment_out_defaults – comment out default values

class temci.utils.typecheck.ListOrTuple(elem_type: Type = Any)[source]

Bases: Type

Checks for the value to be a list or tuple with elements of a given type.

Creates an instance.

Parameters

elem_type – type of the list or tuple elements

Raises

ConstraintError if elem_type isn’t a (typechecker) Types

elem_type

Expected type of the list or tuple elements

temci.utils.typecheck.NaturalNumber(constraint: Optional[Callable[[Any], bool]] = None) Int[source]

Matches all natural numbers (ints >= 0) that satisfy the optional user defined constrained.

Parameters

constraint – function that returns True if the user defined constraint is satisfied

class temci.utils.typecheck.NonErrorConstraint(constraint: Callable[[Any], Any], error_cls: type, constrained_type: Type = Any, description: Optional[str] = None)[source]

Bases: Type

Checks the passed value by an user defined constraint that fails if it raises an error.

Creates a new instance

Parameters
  • constraint – function that doesn’t raise an error if the user defined constraint is satisfied

  • error_cls – class of the errors the constraint method raises

  • constrained_type – Type that the constraint is applied on

  • description – short description of the constraint (e.g. “>0”)

Raises

ConstraintError if constrained_type isn’t a (typechecker) Types

constrained_type

Type that the constraint is applied on

constraint

Function that returns True if the user defined constraint is satisfied

description

Short description of the constraint (e.g. “>0”)

error_cls

Class of the errors the constraint method raises

class temci.utils.typecheck.NonExistent(completion_hints: Optional[Dict[str, Any]] = None)[source]

Bases: Type

Checks a key of a dictionary for existence if its associated value has this type.

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

class temci.utils.typecheck.Optional(other_type: Type)[source]

Bases: Either

Checks the value and checks that its either of native type None or of another Type constraint. Alias for Either(Exact(None), other_type)

Creates an Optional instance.

Parameters

other_type – type to make optional

Raises

ConstraintError if other_type isn’t a (typechecker) Types

temci.utils.typecheck.PositiveInt(constraint: Optional[Callable[[Any], bool]] = None) Int[source]

Matches all positive integers that satisfy the optional user defined constrained.

Parameters

constraint – function that returns True if the user defined constraint is satisfied

class temci.utils.typecheck.Str(constraint: Optional[Callable[[Any], bool]] = None)[source]

Bases: Type

Checks for the value to be a string an optionally meet some constraints.

Creates an instance.

Parameters

constraint – function that returns True if the user defined constraint is satisfied

constraint

Function that returns True if the user defined constraint is satisfied

class temci.utils.typecheck.StrList[source]

Bases: Type, ParamType

A comma separated string list which contains elements from a fixed set of allowed values.

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

allowed_values

Possible values that can appear in the string list, if None all values are allowed.

convert(value, param, ctx: Context) List[str][source]

Convert method that makes this class usable as a click type.

get_default_yaml(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None, comment_out_defaults: bool = False) str[source]

Produce a YAML like string that contains the default value and the description of this type and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

  • comment_out_defaults – comment out default values

name: str = 'comma_sep_str_list'

click.ParamType name, that makes this class usable as a click type

class temci.utils.typecheck.T(native_type: type)[source]

Bases: Type

Wrapper around a native type.

Creates an isntance.

Parameters

native_type – wrapped native type

native_type

Native type that is wrapped

class temci.utils.typecheck.Tuple(*elem_types: Tuple[Type])[source]

Bases: Type

Checks for the value to be a tuple (or a list) with elements of the given types.

Creates a new instance.

Parameters

elem_types – types of each tuple element

Raises

ConstraintError if elem_type isn’t a (typechecker) Types

elem_types

Expected type of each tuple element

class temci.utils.typecheck.Type(completion_hints: Optional[Dict[str, Any]] = None)[source]

Bases: object

A simple type checker type class.

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

check(value) bool[source]

Checks whether or not the passed value has the type specified by this instance.

Parameters
  • value – passed value

  • info – info object for creating error messages

completion_hints

Completion hints for supported shells for this type instance

default

Default value of this type instance

description

Description of this type instance

dont_typecheck_default() Type[source]

Disable type checking the default value. :return: self

get_default() Any[source]

Returns the default value of this type :raises: ValueError if the default value isn’t set

get_default_yaml(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None, comment_out_defaults: bool = False) Union[str, List[str]][source]

Produce a YAML like string that contains the default value and the description of this type and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

  • comment_out_defaults – comment out default values

has_default() bool[source]

Does this type instance have an default value?

string_representation(indents: int = 0, indentation: int = 4, str_list: bool = False, defaults=None) Union[str, List[str]][source]

Produce a YAML string that contains the default value (if possible), the description of this type and more and its possible sub types.

Parameters
  • indents – number of indents in front of each produced line

  • indentation – indentation width in number of white spaces

  • str_list – return a list of lines instead of a combined string?

  • defaults – default value that should be used instead of the default value of this instance

typecheck_default

Type check the default value

class temci.utils.typecheck.ValidTimeSpan[source]

Bases: Type, ParamType

A string that is parseable as timespan by pytimeparse. E.g. “32m” or “2h 32m”.

Creates an instance.

Parameters

completion_hints – completion hints for supported shells for this type instance

convert(value, param, ctx: Context) int[source]

Convert method that makes this class usable as a click type.

name: str = 'valid_timespan'

click.ParamType name, that makes this class usable as a click type

class temci.utils.typecheck.ValidYamlFileName(allow_non_existent: bool = False)[source]

Bases: Str

A valid file name that refers to a valid YAML file.

Create an instance.

Parameters

allow_non_existent – allow files that don’t exist

allow_non_existent

Allow files that don’t exist

completion_hints

Completion hints for supported shells for this type instance

temci.utils.typecheck.YAML_FILE_COMPLETION_HINT = "_files -g '*\\.yaml'"

YAML file name completion hint for ZSH

temci.utils.typecheck.typecheck(value, type: Union[Type, type], value_name: Optional[str] = None)[source]

Like verbose_isinstance but raises an error if the value hasn’t the expected type.

Parameters
  • value – passed value

  • type – expected type of the value

  • value_name – optional description of the value

Raises

TypeError

temci.utils.typecheck.typecheck_locals(locals: Optional[Dict[str, Any]] = None, **variables: Dict[str, Union[Type, type]])[source]

Like typecheck but checks several variables for their associated expected type. The advantage against typecheck is that it sets the value descriptions properly. Example usage:

def func(a: str, b: int):
    typecheck_locals(locals(), a=Str(), b=Int())
Parameters
  • locals – directory to get the variable values from

  • variables – variable names with their associated expected types

Raises

TypeError

temci.utils.typecheck.verbose_isinstance(value, type: Union[Type, type], value_name: Optional[str] = None) InfoMsg[source]

Verbose version of isinstance that returns a InfoMsg object.

Parameters
  • value – value to check

  • type – type or Type to check for

  • value_name – name of the passed value (improves the error message)

temci.utils.util module

Utility functions and classes that don’t depend on the rest of the temci code base.

class temci.utils.util.InsertionTimeOrderedDict[source]

Bases: object

A dictionary which’s elements are sorted by their insertion time.

classmethod from_list(items: Optional[list], key_func: Callable[[Any], Any]) InsertionTimeOrderedDict[source]

Creates an ordered dict out of a list of elements.

Parameters
  • items – list of elements

  • key_func – function that returns a key for each passed list element

Returns

created ordered dict with the elements in the same order as in the passed list

items() List[Tuple[Any, Any]][source]
keys() List[source]

Returns all keys of this dictionary. They are sorted by their insertion time.

values() List[source]

Rerturns all values of this dictionary. They are sorted by their insertion time.

class temci.utils.util.Singleton[source]

Bases: type

Singleton meta class. @see http://stackoverflow.com/a/6798042

temci.utils.util.allow_all_imports = True

Allow all imports (should the can_import method return true for every module)?

temci.utils.util.can_import(module: str) bool[source]

Can a module (like scipy or numpy) be imported without a severe and avoidable performance penalty? The rational behind this is that some parts of temci don’t need scipy or numpy.

Parameters

module – name of the module

temci.utils.util.document(**kwargs: Dict[str, str])[source]

Document

Parameters

kwargs – class attribute, documentation prefix

temci.utils.util.does_command_succeed(cmd: str) bool[source]

Does the passed command succeed (when executed by /bin/sh)?

temci.utils.util.does_program_exist(program: str) bool[source]

Does the passed program exist?

temci.utils.util.geom_std(values: List[float]) float[source]

Calculates the geometric standard deviation for the passed values. Source: https://en.wikipedia.org/wiki/Geometric_standard_deviation

temci.utils.util.get_cache_line_size(cache_level: Optional[int] = None) Optional[int][source]

Returns the cache line size of the cache on the given level. Level 0 and 1 are actually on the same level.

Parameters

cache_level – if None the highest level cache is used

Returns

cache line size or none if the cache on the given level doesn’t exist

temci.utils.util.get_distribution_name() str[source]

Returns the name of the current linux distribution (requires lsb_release to be installed)

temci.utils.util.get_distribution_release() str[source]

Returns the used release of the current linux distribution (requires lsb_release to be installed)

temci.utils.util.get_doc_for_type_scheme(type_scheme: Type) str[source]

Return a class documentation string for the given type scheme. Use the default_yaml method.

temci.utils.util.get_memory_page_size() int[source]

Returns the size of a main memory page

temci.utils.util.handler = <RainbowLoggingHandler <stderr> (NOTSET)>

Colored logging handler that is used for the root logger

temci.utils.util.has_pdflatex() bool[source]

Is pdflatex installed?

temci.utils.util.has_root_privileges() bool[source]

Has the current user root privileges?

temci.utils.util.in_standalone_mode = False

In rudimentary standalone mode (executed via run.py)

temci.utils.util.join_strs(strs: List[str], last_word: str = 'and') str[source]

Joins the passed strings together with “, ” except for the last to strings that separated by the passed word.

Parameters
  • strs – strings to join

  • last_word – passed word that is used between the two last strings

temci.utils.util.on_apple_os() bool[source]

Is the current operating system an apple OS X?

temci.utils.util.parse_timespan(time: str) float[source]

Parse a time span expression, see https://pypi.org/project/pytimeparse/

Supports -1 to express an infinite time span

Parameters

time – time span expression, mixture of different time units is possible

Returns

time span in seconds

class temci.utils.util.proc_wait_with_rusage[source]

Bases: object

Each Popen object gets a field rusage

temci.utils.util.recursive_exec_for_leafs(data: dict, func, _path_prep=[])[source]

Executes the function for every leaf key (a key without any sub keys) of the data dict tree.

Parameters
  • data – dict tree

  • func – function that gets passed the leaf key, the key path and the actual value

temci.utils.util.rusage_header() str[source]
temci.utils.util.sphinx_doc() bool[source]

Is the code only loaded to document it with sphinx?

temci.utils.util.warn_for_pdflatex_non_existence_once(_warned=[False])[source]

Log a warning if the pdflatex isn’t available, but only if this function is called the first time

temci.utils.vcs module

class temci.utils.vcs.FileDriver(dir: str = '.', branch: Optional[str] = None)[source]

Bases: VCSDriver

The default driver, that works with plain old files and directories without any vcs. Therefore its also usable with every directory. It has only one revision: Number -1 or ‘HEAD’, the current directory content.

This class is also a simple example implementation of a VCSDriver.

Initializes the VCS driver for a given base directory. It also sets the current branch if it’s defined in the Settings

Parameters
  • dir – base directory

  • branch – used branch

copy_revision(id_or_num: Union[int, str], sub_dir: str, dest_dirs: List[str])[source]

Copy the sub directory of the current vcs base directory into all of the destination directories.

Parameters
  • id_or_num – id or number of the revision (-1 and ‘HEAD’ represent the uncommitted changes)

  • sub_dir – sub directory of the current vcs base directory relative to it

  • dest_dirs – list of destination directories in which the content of the sub dir is placed or dest dir string

Raises

VCSError if something goes wrong while copying the directories

get_branch() Optional[str][source]

Gets the current branch.

Returns

current branch name

Raises

VCSError if something goes terribly wrong

get_info_for_revision(id_or_num: Union[int, str]) dict[source]

Get an info dict for the given commit (-1 and ‘HEAD’ represent the uncommitted changes). Structure of the info dict:

"commit_id"; …,
"commit_message": …,
"commit_number": …,
"is_uncommitted": True/False,
"is_from_other_branch": True/False,
"branch": … # branch name or empty string if this commit belongs to no branch
Parameters

id_or_num – id or number of the commit

Returns

info dict

Raises

VCSError if the number or id isn’t valid

has_uncommitted() bool[source]

Check for uncommitted changes in the repository.

classmethod is_suited_for_dir(dir: str = '.')[source]

Checks whether or not this vcs driver can work with the passed base directory.

Parameters

dir – passed base directory path

number_of_revisions() int[source]

Number of committed revisions in the current branch (if branches are supported). :return: number of revisions

set_branch(new_branch: str)[source]

Sets the current branch and throws an error if the branch doesn’t exist.

Parameters

new_branch – new branch to set

Raises

VCSError if new_branch doesn’t exist

validate_revision(id_or_num: Union[int, str]) bool[source]

Validate the existence of the referenced revision.

Parameters

id_or_num – id or number of the reverenced revision

Returns

does it exists?

class temci.utils.vcs.GitDriver(dir: str = '.', branch: Optional[str] = None)[source]

Bases: VCSDriver

The driver for git repositories.

Initializes the VCS driver for a given base directory. It also sets the current branch if it’s defined in the Settings

Parameters
  • dir – base directory

  • branch – used branch

copy_revision(id_or_num: Union[int, str], sub_dir: str, dest_dirs: List[str])[source]

Copy the sub directory of the current vcs base directory into all of the destination directories.

Parameters
  • id_or_num – id or number of the revision (-1 and ‘HEAD’ represent the uncommitted changes)

  • sub_dir – sub directory of the current vcs base directory relative to it

  • dest_dirs – list of destination directories in which the content of the sub dir is placed or dest dir string

Raises

VCSError if something goes wrong while copying the directories

get_branch() Optional[str][source]

Gets the current branch.

Returns

current branch name

Raises

VCSError if something goes terribly wrong

get_info_for_revision(id_or_num: Union[int, str]) dict[source]

Get an info dict for the given commit (-1 and ‘HEAD’ represent the uncommitted changes). Structure of the info dict:

"commit_id"; …,
"commit_message": …,
"commit_number": …,
"is_uncommitted": True/False,
"is_from_other_branch": True/False,
"branch": … # branch name or empty string if this commit belongs to no branch
Parameters

id_or_num – id or number of the commit

Returns

info dict

Raises

VCSError if the number or id isn’t valid

get_valid_branches() Optional[List[str]][source]

Gets the valid branches for the associated repository or None if the vcs doesn’t support branches.

has_uncommitted() bool[source]

Check for uncommitted changes in the repository.

classmethod is_suited_for_dir(dir: str = '.') bool[source]

Checks whether or not this vcs driver can work with the passed base directory.

Parameters

dir – passed base directory path

number_of_revisions() str[source]

Number of committed revisions in the current branch (if branches are supported). :return: number of revisions

set_branch(new_branch: str)[source]

Sets the current branch and throws an error if the branch doesn’t exist.

Parameters

new_branch – new branch to set

Raises

VCSError if new_branch doesn’t exist

validate_revision(id_or_num: Union[int, str]) bool[source]

Validate the existence of the referenced revision.

Parameters

id_or_num – id or number of the reverenced revision

Returns

does it exists?

exception temci.utils.vcs.NoSuchRevision[source]

Bases: VCSError

Thrown if a specific revision doesn’t exist

exception temci.utils.vcs.NoSuchVCSError[source]

Bases: VCSError

Thrown if there isn’t a vcs with the specific name

class temci.utils.vcs.VCSDriver(dir: str = '.', branch: Optional[str] = None)[source]

Bases: object

Abstract version control system driver class used to support different vcss.

Initializes the VCS driver for a given base directory. It also sets the current branch if it’s defined in the Settings

Parameters
  • dir – base directory

  • branch – used branch

branch

Used branch

copy_revision(id_or_num: Union[int, str], sub_dir: str, dest_dirs: List[str])[source]

Copy the sub directory of the current vcs base directory into all of the destination directories.

Parameters
  • id_or_num – id or number of the revision (-1 and ‘HEAD’ represent the uncommitted changes)

  • sub_dir – sub directory of the current vcs base directory relative to it

  • dest_dirs – list of destination directories in which the content of the sub dir is placed or dest dir string

Raises

VCSError if something goes wrong while copying the directories

dir

Base directory

get_branch() Optional[str][source]

Gets the current branch.

Returns

current branch name

Raises

VCSError if something goes terribly wrong

get_info_for_all_revisions(max: int = - 1) List[Dict[str, Any]][source]

Get an info dict for all revisions. A single info dict has the following structure:

"commit_id"; …,
"commit_message": …,
"commit_number": …,
"is_uncommitted": True/False,
"is_from_other_branch": True/False,
"branch": … # branch name or empty string if this commit belongs to no branch
Parameters

max – if max isn’t -1 it gives the maximum number of revision infos returned

Returns

list of info dicts

get_info_for_revision(id_or_num: Union[int, str]) dict[source]

Get an info dict for the given commit (-1 and ‘HEAD’ represent the uncommitted changes). Structure of the info dict:

"commit_id"; …,
"commit_message": …,
"commit_number": …,
"is_uncommitted": True/False,
"is_from_other_branch": True/False,
"branch": … # branch name or empty string if this commit belongs to no branch
Parameters

id_or_num – id or number of the commit

Returns

info dict

Raises

VCSError if the number or id isn’t valid

classmethod get_suited_vcs(mode='auto', dir='.', branch: Optional[str] = None) VCSDriver[source]

Chose the best suited vcs driver for the passed base directory and the passed mode. If mode is “auto” the best suited vcs driver is chosen. If mode is “git” or “file”, the GitDriver or the FileDriver is chosen. If the chosen driver isn’t applicable than a VCSError is raised.

Parameters
  • mode – passed mode

  • dir – base directory

  • branch – used branch

Returns

vcs driver for the base directory

Raises

VCSError if the selected driver isn’t applicable

get_valid_branches() Optional[List[str]][source]

Gets the valid branches for the associated repository or None if the vcs doesn’t support branches.

has_uncommitted() bool[source]

Check for uncommitted changes in the repository.

id_type = Either(Str()|Int())
classmethod is_suited_for_dir(dir: str = '.') bool[source]

Checks whether or not this vcs driver can work with the passed base directory.

Parameters

dir – passed base directory path

number_of_revisions() int[source]

Number of committed revisions in the current branch (if branches are supported). :return: number of revisions

set_branch(new_branch: str)[source]

Sets the current branch and throws an error if the branch doesn’t exist.

Parameters

new_branch – new branch to set

Raises

VCSError if new_branch doesn’t exist

validate_revision(id_or_num: Union[int, str]) bool[source]

Validate the existence of the referenced revision.

Parameters

id_or_num – id or number of the reverenced revision

Returns

does it exists?

exception temci.utils.vcs.VCSError[source]

Bases: OSError

Base error for the errors that occur during vcs handling

Module contents

Package with utility modules.