temci.scripts package¶
Submodules¶
temci.scripts.cli module¶
- class temci.scripts.cli.ErrorCode(value)[source]¶
Bases:
EnumAn enumeration.
- NO_ERROR = 0¶
- PROGRAM_ERROR = 1¶
- TEMCI_ERROR = 255¶
- temci.scripts.cli.benchmark_and_exit(runs: Optional[List[dict]] = None)[source]¶
Benchmark and exit with an exit code if an error occurred: 0 if everything went okay, 1 if at least one benchmarked program failed, 255 if temci itself failed
- temci.scripts.cli.cli_with_error_catching()[source]¶
Process the command line arguments and catch (some) errors.
- temci.scripts.cli.create_run_driver_function(driver: str, options: CmdOptionList)[source]¶
- temci.scripts.cli.temci__build(build_file: str, **kwargs)[source]¶
- Language
python
- Command
temci build- Description
Build programs and output modified run config file
- Argument
build configuration YAML file
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--in:- Description
Input file with the program blocks to build
- Argument type
Str()
- Default
'build.yaml'
--out:- Description
Resulting run config file
- Argument type
Str()
- Default
'run_config.yaml'
--threads:- Description
Number of simultaneous builds for a specific program block, only makes sense when build_config/number > 1, and if the build commands create a different binary every time they are run
- Argument type
Int(constraint=<function>)
- Default
1
- temci.scripts.cli.temci__clean(**kwargs)[source]¶
- Language
python
- Command
temci clean- Description
Clean up the temporary files
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
- temci.scripts.cli.temci__completion__bash()[source]¶
- Language
python
- Command
temci completion bash- Description
Creates a new tab completion file for zsh and returns its file name
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
- temci.scripts.cli.temci__completion__zsh()[source]¶
- Language
python
- Command
temci completion zsh- Description
Creates a new tab completion file for zsh and returns its file name
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
- temci.scripts.cli.temci__exec(run_file, **kwargs)¶
- Language
python
- Command
temci exec- Description
Implements a simple run driver that just executes one of the passed run_cmds
- Argument
configuration YAML file
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
--nice|--no-nice:- Description
Enable: Allows the setting of the nice and ionice values of the benchmarking process.
- Default
None
--nice_nice:- Description
Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process).
- Argument type
Int(range=range(-20, 20))
- Default
-15
--nice_io_nice:- Description
Specify the name or number of the scheduling class to use;0 for none, 1 for realtime, 2 for best-effort, 3 for idle.
- Argument type
Int(range=range(0, 4))
- Default
1
--env_randomize|--no-env_randomize:- Description
Enable: Adds random environment variables.
- Default
None
--env_randomize_min:- Description
Minimum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_max:- Description
Maximum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_var_max:- Description
Maximum length of each random value
- Argument type
Int(constraint=<function>)
- Default
4096
--env_randomize_key_max:- Description
Maximum length of each random key
- Argument type
Int(constraint=<function>)
- Default
4096
--preheat|--no-preheat:- Description
Enable: Preheats the system with a cpu bound task
- Default
None
--preheat_time:- Description
Number of seconds to preheat the system with an cpu bound task
- Argument type
Int(constraint=<function>)
- Default
10
--preheat_when:- Description
When to preheat
- Argument type
ListOrTuple(ExactEither(‘before_each_run’|’at_setup’))
- Default
['before_each_run']
--other_nice|--no-other_nice:- Description
Enable: Allows the setting of the nice value of most other processes (as far as possible).
- Default
None
--other_nice_nice:- Description
Niceness values for other processes.
- Argument type
Int(range=range(-20, 20))
- Default
19
--other_nice_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start|--no-stop_start:- Description
Enable: Stop almost all other processes (as far as possible).
- Default
None
--stop_start_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start_min_id:- Description
Processes with lower id are ignored.
- Argument type
Int(constraint=<function>)
- Default
1500
--stop_start_comm_prefixes:- Description
Each process which name (lower cased) starts with one of the prefixes is not ignored. Overrides the decision based on the min_id.
- Argument type
ListOrTuple(Str())
- Default
['ssh', 'xorg', 'bluetoothd']
--stop_start_comm_prefixes_ignored:- Description
Each process which name (lower cased) starts with one of the prefixes is ignored. It overrides the decisions based on comm_prefixes and min_id.
- Argument type
ListOrTuple(Str())
- Default
['dbus', 'kworker']
--stop_start_subtree_suffixes:- Description
Suffixes of processes names which are stopped.
- Argument type
ListOrTuple(Str())
- Default
['dm', 'apache']
--stop_start_dry_run|--no-stop_start_dry_run:- Description
Just output the to be stopped processes but don’t actually stop them?
- Default
False
--sync|--no-sync:- Description
Enable: Calls
syncbefore each program execution.- Default
None
--sleep|--no-sleep:- Description
Enable: Sleep a given amount of time before the benchmarking begins.
- Default
None
--sleep_seconds:- Description
Seconds to sleep
- Argument type
Int(constraint=<function>)
- Default
10
--drop_fs_caches|--no-drop_fs_caches:- Description
Enable: Drop page cache, directoy entries and inodes before every benchmarking run.
- Default
None
--drop_fs_caches_free_pagecache|--no-drop_fs_caches_free_pagecache:- Description
Free the page cache
- Default
True
--drop_fs_caches_free_dentries_inodes|--no-drop_fs_caches_free_dentries_inodes:- Description
Free dentries and inodes
- Default
True
--disable_swap|--no-disable_swap:- Description
Enable: Disables swapping on the system before the benchmarking and enables it after.
- Default
None
--disable_cpu_caches|--no-disable_cpu_caches:- Description
Enable: Disable the L1 and L2 caches on x86 and x86-64 architectures.
- Default
None
--flush_cpu_caches|--no-flush_cpu_caches:- Description
Enable: Flushes the CPU caches on a x86 CPU using a small kernel module,
- Default
None
--cpu_governor|--no-cpu_governor:- Description
Enable: Allows the setting of the scaling governor of all cpu cores, to ensure that all use the same.
- Default
None
--cpu_governor_governor:- Description
New scaling governor for all cpus
- Argument type
Str()
- Default
'performance'
--disable_aslr|--no-disable_aslr:- Description
Enable: Disable address space randomization
- Default
None
--disable_ht|--no-disable_ht:- Description
Enable: Disable hyper-threading
- Default
None
--disable_turbo_boost|--no-disable_turbo_boost:- Description
Enable: Disable amd and intel turbo boost
- Default
None
--disable_intel_turbo|--no-disable_intel_turbo:- Description
Enable: Disable intel turbo mode
- Default
None
--disable_amd_boost|--no-disable_amd_boost:- Description
Enable: Disable amd turbo boost
- Default
None
--cpuset|--no-cpuset:- Description
Enable: Enable cpusets, simply sets run/cpuset/active to true
- Default
None
--discarded_runs|--no-discarded_runs:- Description
Enable: Sets run/discarded_runs
- Default
None
--discarded_runs_runs:- Description
Number of discarded runs
- Argument type
Int(constraint=<function>)
- Default
1
--runner:- Description
If not ‘’ overrides the runner setting for each program block
- Argument type
ExactEither(‘’|’perf_stat’|’rusage’|’spec’|’spec.py’|’time’|’output’)
- Default
''
--random_cmd|--no-random_cmd:- Description
Pick a random command if more than one run command is passed.
- Default
True
--preset:- Description
Enable other plugins by default: none = (enable none by default); all = cpu_governor,disable_swap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable all, might freeze your system); usable = cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like ‘all’ but doesn’t affect other processes)
- Argument type
ExactEither(‘none’|’all’|’usable’)
- Default
'none'
--parse_output|--no-parse_output:- Description
Parse the program output as a YAML dictionary of that gives for a specific property a measurement. Not all runners support it.
- Default
False
--plugin_order:- Description
Order in which the plugins are used, plugins that do not appear in this list are used before all others
- Argument type
ListOrTuple(Str())
- Default
['drop_fs_caches', 'sync', 'sleep', 'preheat', 'flush_cpu_caches']
--argument:- Description
Argument passed to all benchmarked commands by replacing $ARGUMENT with this value in the command
- Argument type
Str()
- Default
''
--cpuset_active|--no-cpuset_active:- Description
Use cpuset functionality?
- Default
False
--cpuset_base_core_number:- Description
Number of cpu cores for the base (remaining part of the) system
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_parallel:- Description
0: benchmark sequential, > 0: benchmark parallel with n instances, -1: determine n automatically
- Argument type
Int(constraint=<function>)
- Default
0
--cpuset_sub_core_number:- Description
Number of cpu cores per parallel running program.
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_temci_in_base_set|--no-cpuset_temci_in_base_set:- Description
place temci in the same cpu set as the rest of the system?
- Default
True
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
- temci.scripts.cli.temci__format(number, abs_deviation, **kwargs)[source]¶
- Language
python
- Command
temci format- Description
Format a number
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--parentheses|--no-parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--scientific_notation|--no-scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--scientific_notation_si_prefixes|--no-scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--omit_insignificant_decimal_places|--no-omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--force_min_decimal_places|--no-force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--percentages|--no-percentages:- Description
Show as percentages
- Default
False
--sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
- temci.scripts.cli.temci__init__build_config(file, **kwargs)[source]¶
- Language
python
- Command
temci init build_config- Description
Creates a sample build config file build_config.yaml (or the file name passed) in the current directory
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
- temci.scripts.cli.temci__init__run_config(file, **kwargs)[source]¶
- Language
python
- Command
temci init run_config- Description
Creates a sample exec run config file run_config.yaml (or the file name passed) in the current directory
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
- temci.scripts.cli.temci__init__settings(file, **kwargs)[source]¶
- Language
python
- Command
temci init settings- Description
Create a new settings file temci.yaml (or the file name passed) in the current directory
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
- temci.scripts.cli.temci__report(report_file: List[str], **kwargs)[source]¶
- Language
python
- Command
temci report- Description
Generate a report from benchmarking result
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--in:- Description
Files that contain the benchmarking results
- Argument type
Either(Str()|ListOrTuple(Str()))
- Default
'run_output.yaml'
--excluded_properties:- Description
Properties that aren’t shown in the report.
- Argument type
ListOrTuple(Str())
- Default
['__ov-time']
--exclude_invalid|--no-exclude_invalid:- Description
Exclude all data sets that contain only NaNs.
- Default
True
--long_properties|--no-long_properties:- Description
Replace the property names in reports with longer more descriptive versions?
- Default
False
--xkcd_like_plots|--no-xkcd_like_plots:- Description
Produce xkcd like plots (requires the humor sans font to be installed)
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), identified by their description or tag attribute, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--reporter:- Description
Possible reporter are ‘console’, ‘html2’, ‘csv’, ‘codespeed’, ‘codespeed2’ and ‘velcom’
- Argument type
ExactEither(‘console’|’html2’|’csv’|’codespeed’|’codespeed2’|’velcom’)
- Default
'console'
--properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--console_out:- Description
Output file name or - (stdout)
- Argument type
FileName(allow_std=True)
- Default
'-'
--console_with_tester_results|--no-console_with_tester_results:- Description
Print statistical tests for every property for every two programs
- Default
True
--console_mode:- Description
‘auto’: report clusters (runs with the same description) and singles (clusters with a single entry, combined) separately, ‘single’: report all clusters together as one, ‘cluster’: report all clusters separately, ‘both’: append the output of ‘cluster’ to the output of ‘single’
- Argument type
ExactEither(‘both’|’cluster’|’single’|’auto’)
- Default
'auto'
--console_report_errors|--no-console_report_errors:- Description
Report on the failing blocks
- Default
True
--console_baseline:- Description
Matches the baseline block
- Argument type
Str()
- Default
''
--console_baseline_position:- Description
Position of the baseline comparison: each: after each block, after: after each cluster, both: after each and after cluster, instead: instead of the non baselined
- Argument type
ExactEither(‘each’|’after’|’both’|’instead’)
- Default
'each'
--html2_out:- Description
Output directory
- Argument type
Str()
- Default
'report'
--html2_html_filename:- Description
Name of the HTML file
- Argument type
Str()
- Default
'report.html'
--html2_fig_width_small:- Description
Width of all small plotted figures
- Argument type
T(<class ‘float’>)
- Default
15.0
--html2_fig_width_big:- Description
Width of all big plotted figures
- Argument type
T(<class ‘float’>)
- Default
25.0
--html2_boxplot_height:- Description
Height per run block for the big comparison box plots
- Argument type
T(<class ‘float’>)
- Default
2.0
--html2_alpha:- Description
Alpha value for confidence intervals
- Argument type
T(<class ‘float’>)
- Default
0.05
--html2_gen_tex|--no-html2_gen_tex:- Description
Generate simple latex versions of the plotted figures?
- Default
True
--html2_gen_pdf|--no-html2_gen_pdf:- Description
Generate pdf versions of the plotted figures?
- Default
False
--html2_gen_xls|--no-html2_gen_xls:- Description
Generate excel files for all tables
- Default
False
--html2_show_zoomed_out|--no-html2_show_zoomed_out:- Description
Show zoomed out (x min = 0) figures in the extended summaries?
- Default
True
--html2_percent_format:- Description
Format string used to format floats as percentages
- Argument type
Str()
- Default
'{:5.2%}'
--html2_float_format:- Description
Format string used to format floats
- Argument type
Str()
- Default
'{:5.2e}'
--html2_min_in_comparison_tables|--no-html2_min_in_comparison_tables:- Description
Show the mininmum related values in the big comparison table
- Default
False
--html2_mean_in_comparison_tables|--no-html2_mean_in_comparison_tables:- Description
Show the mean related values in the big comparison table
- Default
True
--html2_force_override|--no-html2_force_override:- Description
Override the contents of the output directory if it already exists?
- Default
False
--html2_hide_stat_warnings|--no-html2_hide_stat_warnings:- Description
Hide warnings and errors related to statistical properties
- Default
False
--html2_local|--no-html2_local:- Description
Use local versions of all third party resources
- Default
False
--csv_out:- Description
Output file name or standard out (-)
- Argument type
FileName(allow_std=True)
- Default
'-'
--csv_columns:- Description
List of valid column specs, format is a comma separated list of ‘PROPERTY[mod]’ or ‘ATTRIBUTE’ mod is one of: mean, stddev, property, min, max and stddev per mean, optionally a formatting option can be given viaPROPERTY[mod|OPT1OPT2…], where the OPTs are one of the following: % (format as percentage), p (wrap insignificant digits in parentheses (+- 2 std dev)), s (use scientific notation, configured in report/number) and o (wrap digits in the order of magnitude of 2 std devs in parentheses). PROPERTY can be either the description or the short version of the property. Configure the number formatting further via the number settings in the settings file
- Argument type
ListOrTuple(Str()):List of valid column specs, format is a comma separated list of ‘PROPERTY[mod]’ or ‘ATTRIBUTE’ mod is one of: mean, stddev, property, min, max and stddev per mean, optionally a formatting option can be given viaPROPERTY[mod|OPT1OPT2…], where the OPTs are one of the following: % (format as percentage), p (wrap insignificant digits in parentheses (+- 2 std dev)), s (use scientific notation, configured in report/number) and o (wrap digits in the order of magnitude of 2 std devs in parentheses). PROPERTY can be either the description or the short version of the property. Configure the number formatting further via the number settings in the settings file
- Default
['description']
--codespeed_project:- Description
Project name reported to codespeed.
- Argument type
Str()
- Default
''
--codespeed_executable:- Description
Executable name reported to codespeed. Defaults to the project name.
- Argument type
Str()
- Default
''
--codespeed_environment:- Description
Environment name reported to codespeed. Defaults to current host name.
- Argument type
Str()
- Default
''
--codespeed_branch:- Description
Branch name reported to codespeed. Defaults to current branch or else ‘master’.
- Argument type
Str()
- Default
''
--codespeed_commit_id:- Description
Commit ID reported to codespeed. Defaults to current commit.
- Argument type
Str()
- Default
''
- temci.scripts.cli.temci__setup(build_kernel_modules: bool)[source]¶
- Language
python
- Command
temci setup- Description
Compile all needed binaries in the temci scripts folder
Options:
--build_kernel_modules|--no-build_kernel_modules:- Description
Build the rusage helper program and the kernel modules used for disabling the CPU caches (with the –build_kernel_modules option)
- Default
False
- temci.scripts.cli.temci__shell(run_file, **kwargs)¶
- Language
python
- Command
temci shell- Description
Implements a run driver that runs the benched command a single time with redirected in- and output.
- Argument
configuration YAML file
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
--nice|--no-nice:- Description
Enable: Allows the setting of the nice and ionice values of the benchmarking process.
- Default
None
--nice_nice:- Description
Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process).
- Argument type
Int(range=range(-20, 20))
- Default
-15
--nice_io_nice:- Description
Specify the name or number of the scheduling class to use;0 for none, 1 for realtime, 2 for best-effort, 3 for idle.
- Argument type
Int(range=range(0, 4))
- Default
1
--env_randomize|--no-env_randomize:- Description
Enable: Adds random environment variables.
- Default
None
--env_randomize_min:- Description
Minimum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_max:- Description
Maximum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_var_max:- Description
Maximum length of each random value
- Argument type
Int(constraint=<function>)
- Default
4096
--env_randomize_key_max:- Description
Maximum length of each random key
- Argument type
Int(constraint=<function>)
- Default
4096
--preheat|--no-preheat:- Description
Enable: Preheats the system with a cpu bound task
- Default
None
--preheat_time:- Description
Number of seconds to preheat the system with an cpu bound task
- Argument type
Int(constraint=<function>)
- Default
10
--preheat_when:- Description
When to preheat
- Argument type
ListOrTuple(ExactEither(‘before_each_run’|’at_setup’))
- Default
['before_each_run']
--other_nice|--no-other_nice:- Description
Enable: Allows the setting of the nice value of most other processes (as far as possible).
- Default
None
--other_nice_nice:- Description
Niceness values for other processes.
- Argument type
Int(range=range(-20, 20))
- Default
19
--other_nice_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start|--no-stop_start:- Description
Enable: Stop almost all other processes (as far as possible).
- Default
None
--stop_start_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start_min_id:- Description
Processes with lower id are ignored.
- Argument type
Int(constraint=<function>)
- Default
1500
--stop_start_comm_prefixes:- Description
Each process which name (lower cased) starts with one of the prefixes is not ignored. Overrides the decision based on the min_id.
- Argument type
ListOrTuple(Str())
- Default
['ssh', 'xorg', 'bluetoothd']
--stop_start_comm_prefixes_ignored:- Description
Each process which name (lower cased) starts with one of the prefixes is ignored. It overrides the decisions based on comm_prefixes and min_id.
- Argument type
ListOrTuple(Str())
- Default
['dbus', 'kworker']
--stop_start_subtree_suffixes:- Description
Suffixes of processes names which are stopped.
- Argument type
ListOrTuple(Str())
- Default
['dm', 'apache']
--stop_start_dry_run|--no-stop_start_dry_run:- Description
Just output the to be stopped processes but don’t actually stop them?
- Default
False
--sync|--no-sync:- Description
Enable: Calls
syncbefore each program execution.- Default
None
--sleep|--no-sleep:- Description
Enable: Sleep a given amount of time before the benchmarking begins.
- Default
None
--sleep_seconds:- Description
Seconds to sleep
- Argument type
Int(constraint=<function>)
- Default
10
--drop_fs_caches|--no-drop_fs_caches:- Description
Enable: Drop page cache, directoy entries and inodes before every benchmarking run.
- Default
None
--drop_fs_caches_free_pagecache|--no-drop_fs_caches_free_pagecache:- Description
Free the page cache
- Default
True
--drop_fs_caches_free_dentries_inodes|--no-drop_fs_caches_free_dentries_inodes:- Description
Free dentries and inodes
- Default
True
--disable_swap|--no-disable_swap:- Description
Enable: Disables swapping on the system before the benchmarking and enables it after.
- Default
None
--disable_cpu_caches|--no-disable_cpu_caches:- Description
Enable: Disable the L1 and L2 caches on x86 and x86-64 architectures.
- Default
None
--flush_cpu_caches|--no-flush_cpu_caches:- Description
Enable: Flushes the CPU caches on a x86 CPU using a small kernel module,
- Default
None
--cpu_governor|--no-cpu_governor:- Description
Enable: Allows the setting of the scaling governor of all cpu cores, to ensure that all use the same.
- Default
None
--cpu_governor_governor:- Description
New scaling governor for all cpus
- Argument type
Str()
- Default
'performance'
--disable_aslr|--no-disable_aslr:- Description
Enable: Disable address space randomization
- Default
None
--disable_ht|--no-disable_ht:- Description
Enable: Disable hyper-threading
- Default
None
--disable_turbo_boost|--no-disable_turbo_boost:- Description
Enable: Disable amd and intel turbo boost
- Default
None
--disable_intel_turbo|--no-disable_intel_turbo:- Description
Enable: Disable intel turbo mode
- Default
None
--disable_amd_boost|--no-disable_amd_boost:- Description
Enable: Disable amd turbo boost
- Default
None
--cpuset|--no-cpuset:- Description
Enable: Enable cpusets, simply sets run/cpuset/active to true
- Default
None
--discarded_runs|--no-discarded_runs:- Description
Enable: Sets run/discarded_runs
- Default
None
--discarded_runs_runs:- Description
Number of discarded runs
- Argument type
Int(constraint=<function>)
- Default
1
--preset:- Description
Enable other plugins by default: none = (enable none by default); all = cpu_governor,disable_swap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable all, might freeze your system); usable = cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like ‘all’ but doesn’t affect other processes)
- Argument type
ExactEither(‘none’|’all’|’usable’)
- Default
'none'
--plugin_order:- Description
Order in which the plugins are used, plugins that do not appear in this list are used before all others
- Argument type
ListOrTuple(Str())
- Default
['drop_fs_caches', 'sync', 'sleep', 'preheat', 'flush_cpu_caches']
--cpuset_active|--no-cpuset_active:- Description
Use cpuset functionality?
- Default
False
--cpuset_base_core_number:- Description
Number of cpu cores for the base (remaining part of the) system
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_parallel:- Description
0: benchmark sequential, > 0: benchmark parallel with n instances, -1: determine n automatically
- Argument type
Int(constraint=<function>)
- Default
0
--cpuset_sub_core_number:- Description
Number of cpu cores per parallel running program.
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_temci_in_base_set|--no-cpuset_temci_in_base_set:- Description
place temci in the same cpu set as the rest of the system?
- Default
True
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
- temci.scripts.cli.temci__short()[source]¶
Utility commands to ease working directly on the command line
- temci.scripts.cli.temci__short__exec(commands: list, with_description: Optional[list] = None, without_description: Optional[list] = None, **kwargs)[source]¶
- Language
python
- Command
temci short exec- Description
Execute commands directly with the exec run driver
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--with_description|-d:- Description
DESCRIPTION COMMAND: Benchmark the command and set its description attribute.Appends ‘$ARGUMENT’ to the command if the string isn’t present. Use the ‘–argument’ option to set the value that this string is replaced with.
- Argument type
ListOrTuple(Tuple(Str(), Str()))
- Default
None
--without_description|-wd:- Description
COMMAND: Benchmark the command and use itself as its description. Appends ‘$ARGUMENT’ to the command if the string isn’t present. Use the ‘–argument’ option to set the value that this string is replaced with.
- Argument type
ListOrTuple(Str())
- Default
None
--nice|--no-nice:- Description
Enable: Allows the setting of the nice and ionice values of the benchmarking process.
- Default
None
--nice_nice:- Description
Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process).
- Argument type
Int(range=range(-20, 20))
- Default
-15
--nice_io_nice:- Description
Specify the name or number of the scheduling class to use;0 for none, 1 for realtime, 2 for best-effort, 3 for idle.
- Argument type
Int(range=range(0, 4))
- Default
1
--env_randomize|--no-env_randomize:- Description
Enable: Adds random environment variables.
- Default
None
--env_randomize_min:- Description
Minimum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_max:- Description
Maximum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_var_max:- Description
Maximum length of each random value
- Argument type
Int(constraint=<function>)
- Default
4096
--env_randomize_key_max:- Description
Maximum length of each random key
- Argument type
Int(constraint=<function>)
- Default
4096
--preheat|--no-preheat:- Description
Enable: Preheats the system with a cpu bound task
- Default
None
--preheat_time:- Description
Number of seconds to preheat the system with an cpu bound task
- Argument type
Int(constraint=<function>)
- Default
10
--preheat_when:- Description
When to preheat
- Argument type
ListOrTuple(ExactEither(‘before_each_run’|’at_setup’))
- Default
['before_each_run']
--other_nice|--no-other_nice:- Description
Enable: Allows the setting of the nice value of most other processes (as far as possible).
- Default
None
--other_nice_nice:- Description
Niceness values for other processes.
- Argument type
Int(range=range(-20, 20))
- Default
19
--other_nice_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start|--no-stop_start:- Description
Enable: Stop almost all other processes (as far as possible).
- Default
None
--stop_start_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start_min_id:- Description
Processes with lower id are ignored.
- Argument type
Int(constraint=<function>)
- Default
1500
--stop_start_comm_prefixes:- Description
Each process which name (lower cased) starts with one of the prefixes is not ignored. Overrides the decision based on the min_id.
- Argument type
ListOrTuple(Str())
- Default
['ssh', 'xorg', 'bluetoothd']
--stop_start_comm_prefixes_ignored:- Description
Each process which name (lower cased) starts with one of the prefixes is ignored. It overrides the decisions based on comm_prefixes and min_id.
- Argument type
ListOrTuple(Str())
- Default
['dbus', 'kworker']
--stop_start_subtree_suffixes:- Description
Suffixes of processes names which are stopped.
- Argument type
ListOrTuple(Str())
- Default
['dm', 'apache']
--stop_start_dry_run|--no-stop_start_dry_run:- Description
Just output the to be stopped processes but don’t actually stop them?
- Default
False
--sync|--no-sync:- Description
Enable: Calls
syncbefore each program execution.- Default
None
--sleep|--no-sleep:- Description
Enable: Sleep a given amount of time before the benchmarking begins.
- Default
None
--sleep_seconds:- Description
Seconds to sleep
- Argument type
Int(constraint=<function>)
- Default
10
--drop_fs_caches|--no-drop_fs_caches:- Description
Enable: Drop page cache, directoy entries and inodes before every benchmarking run.
- Default
None
--drop_fs_caches_free_pagecache|--no-drop_fs_caches_free_pagecache:- Description
Free the page cache
- Default
True
--drop_fs_caches_free_dentries_inodes|--no-drop_fs_caches_free_dentries_inodes:- Description
Free dentries and inodes
- Default
True
--disable_swap|--no-disable_swap:- Description
Enable: Disables swapping on the system before the benchmarking and enables it after.
- Default
None
--disable_cpu_caches|--no-disable_cpu_caches:- Description
Enable: Disable the L1 and L2 caches on x86 and x86-64 architectures.
- Default
None
--flush_cpu_caches|--no-flush_cpu_caches:- Description
Enable: Flushes the CPU caches on a x86 CPU using a small kernel module,
- Default
None
--cpu_governor|--no-cpu_governor:- Description
Enable: Allows the setting of the scaling governor of all cpu cores, to ensure that all use the same.
- Default
None
--cpu_governor_governor:- Description
New scaling governor for all cpus
- Argument type
Str()
- Default
'performance'
--disable_aslr|--no-disable_aslr:- Description
Enable: Disable address space randomization
- Default
None
--disable_ht|--no-disable_ht:- Description
Enable: Disable hyper-threading
- Default
None
--disable_turbo_boost|--no-disable_turbo_boost:- Description
Enable: Disable amd and intel turbo boost
- Default
None
--disable_intel_turbo|--no-disable_intel_turbo:- Description
Enable: Disable intel turbo mode
- Default
None
--disable_amd_boost|--no-disable_amd_boost:- Description
Enable: Disable amd turbo boost
- Default
None
--cpuset|--no-cpuset:- Description
Enable: Enable cpusets, simply sets run/cpuset/active to true
- Default
None
--discarded_runs|--no-discarded_runs:- Description
Enable: Sets run/discarded_runs
- Default
None
--discarded_runs_runs:- Description
Number of discarded runs
- Argument type
Int(constraint=<function>)
- Default
1
--runner:- Description
If not ‘’ overrides the runner setting for each program block
- Argument type
ExactEither(‘’|’perf_stat’|’rusage’|’spec’|’spec.py’|’time’|’output’)
- Default
''
--random_cmd|--no-random_cmd:- Description
Pick a random command if more than one run command is passed.
- Default
True
--preset:- Description
Enable other plugins by default: none = (enable none by default); all = cpu_governor,disable_swap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable all, might freeze your system); usable = cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like ‘all’ but doesn’t affect other processes)
- Argument type
ExactEither(‘none’|’all’|’usable’)
- Default
'none'
--parse_output|--no-parse_output:- Description
Parse the program output as a YAML dictionary of that gives for a specific property a measurement. Not all runners support it.
- Default
False
--plugin_order:- Description
Order in which the plugins are used, plugins that do not appear in this list are used before all others
- Argument type
ListOrTuple(Str())
- Default
['drop_fs_caches', 'sync', 'sleep', 'preheat', 'flush_cpu_caches']
--argument:- Description
Argument passed to all benchmarked commands by replacing $ARGUMENT with this value in the command
- Argument type
Str()
- Default
''
--cpuset_active|--no-cpuset_active:- Description
Use cpuset functionality?
- Default
False
--cpuset_base_core_number:- Description
Number of cpu cores for the base (remaining part of the) system
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_parallel:- Description
0: benchmark sequential, > 0: benchmark parallel with n instances, -1: determine n automatically
- Argument type
Int(constraint=<function>)
- Default
0
--cpuset_sub_core_number:- Description
Number of cpu cores per parallel running program.
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_temci_in_base_set|--no-cpuset_temci_in_base_set:- Description
place temci in the same cpu set as the rest of the system?
- Default
True
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
- temci.scripts.cli.temci__short__shell(command: str, **kwargs)[source]¶
- Language
python
- Command
temci short shell- Description
Execute a command in a shell with benchmarking setup
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
--nice|--no-nice:- Description
Enable: Allows the setting of the nice and ionice values of the benchmarking process.
- Default
None
--nice_nice:- Description
Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process).
- Argument type
Int(range=range(-20, 20))
- Default
-15
--nice_io_nice:- Description
Specify the name or number of the scheduling class to use;0 for none, 1 for realtime, 2 for best-effort, 3 for idle.
- Argument type
Int(range=range(0, 4))
- Default
1
--env_randomize|--no-env_randomize:- Description
Enable: Adds random environment variables.
- Default
None
--env_randomize_min:- Description
Minimum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_max:- Description
Maximum number of added random environment variables
- Argument type
Int(constraint=<function>)
- Default
4
--env_randomize_var_max:- Description
Maximum length of each random value
- Argument type
Int(constraint=<function>)
- Default
4096
--env_randomize_key_max:- Description
Maximum length of each random key
- Argument type
Int(constraint=<function>)
- Default
4096
--preheat|--no-preheat:- Description
Enable: Preheats the system with a cpu bound task
- Default
None
--preheat_time:- Description
Number of seconds to preheat the system with an cpu bound task
- Argument type
Int(constraint=<function>)
- Default
10
--preheat_when:- Description
When to preheat
- Argument type
ListOrTuple(ExactEither(‘before_each_run’|’at_setup’))
- Default
['before_each_run']
--other_nice|--no-other_nice:- Description
Enable: Allows the setting of the nice value of most other processes (as far as possible).
- Default
None
--other_nice_nice:- Description
Niceness values for other processes.
- Argument type
Int(range=range(-20, 20))
- Default
19
--other_nice_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start|--no-stop_start:- Description
Enable: Stop almost all other processes (as far as possible).
- Default
None
--stop_start_min_nice:- Description
Processes with lower nice values are ignored.
- Argument type
Int(range=range(-15, 20))
- Default
-10
--stop_start_min_id:- Description
Processes with lower id are ignored.
- Argument type
Int(constraint=<function>)
- Default
1500
--stop_start_comm_prefixes:- Description
Each process which name (lower cased) starts with one of the prefixes is not ignored. Overrides the decision based on the min_id.
- Argument type
ListOrTuple(Str())
- Default
['ssh', 'xorg', 'bluetoothd']
--stop_start_comm_prefixes_ignored:- Description
Each process which name (lower cased) starts with one of the prefixes is ignored. It overrides the decisions based on comm_prefixes and min_id.
- Argument type
ListOrTuple(Str())
- Default
['dbus', 'kworker']
--stop_start_subtree_suffixes:- Description
Suffixes of processes names which are stopped.
- Argument type
ListOrTuple(Str())
- Default
['dm', 'apache']
--stop_start_dry_run|--no-stop_start_dry_run:- Description
Just output the to be stopped processes but don’t actually stop them?
- Default
False
--sync|--no-sync:- Description
Enable: Calls
syncbefore each program execution.- Default
None
--sleep|--no-sleep:- Description
Enable: Sleep a given amount of time before the benchmarking begins.
- Default
None
--sleep_seconds:- Description
Seconds to sleep
- Argument type
Int(constraint=<function>)
- Default
10
--drop_fs_caches|--no-drop_fs_caches:- Description
Enable: Drop page cache, directoy entries and inodes before every benchmarking run.
- Default
None
--drop_fs_caches_free_pagecache|--no-drop_fs_caches_free_pagecache:- Description
Free the page cache
- Default
True
--drop_fs_caches_free_dentries_inodes|--no-drop_fs_caches_free_dentries_inodes:- Description
Free dentries and inodes
- Default
True
--disable_swap|--no-disable_swap:- Description
Enable: Disables swapping on the system before the benchmarking and enables it after.
- Default
None
--disable_cpu_caches|--no-disable_cpu_caches:- Description
Enable: Disable the L1 and L2 caches on x86 and x86-64 architectures.
- Default
None
--flush_cpu_caches|--no-flush_cpu_caches:- Description
Enable: Flushes the CPU caches on a x86 CPU using a small kernel module,
- Default
None
--cpu_governor|--no-cpu_governor:- Description
Enable: Allows the setting of the scaling governor of all cpu cores, to ensure that all use the same.
- Default
None
--cpu_governor_governor:- Description
New scaling governor for all cpus
- Argument type
Str()
- Default
'performance'
--disable_aslr|--no-disable_aslr:- Description
Enable: Disable address space randomization
- Default
None
--disable_ht|--no-disable_ht:- Description
Enable: Disable hyper-threading
- Default
None
--disable_turbo_boost|--no-disable_turbo_boost:- Description
Enable: Disable amd and intel turbo boost
- Default
None
--disable_intel_turbo|--no-disable_intel_turbo:- Description
Enable: Disable intel turbo mode
- Default
None
--disable_amd_boost|--no-disable_amd_boost:- Description
Enable: Disable amd turbo boost
- Default
None
--cpuset|--no-cpuset:- Description
Enable: Enable cpusets, simply sets run/cpuset/active to true
- Default
None
--discarded_runs|--no-discarded_runs:- Description
Enable: Sets run/discarded_runs
- Default
None
--discarded_runs_runs:- Description
Number of discarded runs
- Argument type
Int(constraint=<function>)
- Default
1
--runner:- Description
If not ‘’ overrides the runner setting for each program block
- Argument type
ExactEither(‘’|’perf_stat’|’rusage’|’spec’|’spec.py’|’time’|’output’)
- Default
''
--random_cmd|--no-random_cmd:- Description
Pick a random command if more than one run command is passed.
- Default
True
--preset:- Description
Enable other plugins by default: none = (enable none by default); all = cpu_governor,disable_swap,sync,stop_start,other_nice,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (enable all, might freeze your system); usable = cpu_governor,disable_swap,sync,nice,disable_aslr,disable_ht,cpuset,disable_turbo_boost (like ‘all’ but doesn’t affect other processes)
- Argument type
ExactEither(‘none’|’all’|’usable’)
- Default
'none'
--parse_output|--no-parse_output:- Description
Parse the program output as a YAML dictionary of that gives for a specific property a measurement. Not all runners support it.
- Default
False
--plugin_order:- Description
Order in which the plugins are used, plugins that do not appear in this list are used before all others
- Argument type
ListOrTuple(Str())
- Default
['drop_fs_caches', 'sync', 'sleep', 'preheat', 'flush_cpu_caches']
--argument:- Description
Argument passed to all benchmarked commands by replacing $ARGUMENT with this value in the command
- Argument type
Str()
- Default
''
--cpuset_active|--no-cpuset_active:- Description
Use cpuset functionality?
- Default
False
--cpuset_base_core_number:- Description
Number of cpu cores for the base (remaining part of the) system
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_parallel:- Description
0: benchmark sequential, > 0: benchmark parallel with n instances, -1: determine n automatically
- Argument type
Int(constraint=<function>)
- Default
0
--cpuset_sub_core_number:- Description
Number of cpu cores per parallel running program.
- Argument type
Int(range=range(0, 2))
- Default
1
--cpuset_temci_in_base_set|--no-cpuset_temci_in_base_set:- Description
place temci in the same cpu set as the rest of the system?
- Default
True
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
--discarded_runs:- Description
First n runs that are discarded
- Argument type
Int(constraint=<function>)
- Default
1
--min_runs:- Description
Minimum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
20
--max_runs:- Description
Maximum number of benchmarking runs
- Argument type
Int(constraint=<function>)
- Default
100
--runs:- Description
if != -1 sets max and min runs to its value
- Argument type
Int(constraint=<function>)
- Default
-1
--max_time:- Description
Maximum time the whole benchmarking should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--max_block_time:- Description
Maximum time one run block should take, -1 == no timeout, supports normal time span expressions
- Argument type
ValidTimespan()
- Default
'-1'
--run_block_size:- Description
Number of benchmarking runs that are done together
- Argument type
Int(constraint=<function>)
- Default
1
--in:- Description
Input file with the program blocks to benchmark
- Argument type
Str()
- Default
'input.exec.yaml'
--out:- Description
Output file for the benchmarking results
- Argument type
Str()
- Default
'run_output.yaml'
--store_often|--no-store_often:- Description
Store the result file after each set of blocks is benchmarked
- Default
False
--included_blocks:- Description
List of included run blocks (all: include all), or their tag attribute or their number in the file (starting with 0), can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--disable_hyper_threading|--no-disable_hyper_threading:- Description
Disable the hyper threaded cores. Good for cpu bound programs.
- Default
False
--show_report|--no-show_report:- Description
Print console report if log_level=info
- Default
True
--append|--no-append:- Description
Append to the output file instead of overwriting by adding new run data blocks
- Default
False
--shuffle|--no-shuffle:- Description
Randomize the order in which the program blocks are benchmarked.
- Default
True
--send_mail:- Description
If not empty, recipient of a mail after the benchmarking finished.
- Argument type
Str()
- Default
''
--discard_all_data_for_block_on_error|--no-discard_all_data_for_block_on_error:- Description
Discard all run data for the failing program on error
- Default
False
--record_errors_in_file|--no-record_errors_in_file:- Description
Record the caught errors in the run_output file
- Default
True
--no_build|--no-no_build:- Description
Do not build if build configs are present, only works if the working directory of the blocks does not change
- Default
False
--only_build|--no-only_build:- Description
Only build
- Default
False
--abort_after_build_error|--no-abort_after_build_error:- Description
Abort after the first failing build
- Default
True
--watch|--no-watch:- Description
Show the report continuously
- Default
False
--watch_every:- Description
Update the screen nth run (less updates are better for benchmarks)
- Argument type
Int(constraint=<function>)
- Default
1
--driver:- Description
Possible run drivers are ‘exec’ and ‘shell’
- Argument type
ExactEither(‘exec’|’shell’)
- Default
'exec'
--stats_properties:- Description
Properties to use for reporting and null hypothesis tests, can be regular expressions
- Argument type
ListOrTuple(Str())
- Default
['all']
--stats_uncertainty_range:- Description
Range of p values that allow no conclusion.
- Argument type
Tuple(T(<class ‘float’>):<function>, T(<class ‘float’>):<function>)
- Default
[0.05, 0.15]
--stats_tester:- Description
Possible testers are ‘t’, ‘ks’ and ‘anderson’
- Argument type
ExactEither(‘t’|’ks’|’anderson’)
- Default
't'
--report_number_parentheses|--no-report_number_parentheses:- Description
Show parentheses around non significant digits? (If a std dev is given)
- Default
True
--report_number_min_decimal_places:- Description
The minimum number of shown decimal places if decimal places are shown
- Argument type
Int(constraint=<function>)
- Default
3
--report_number_max_decimal_places:- Description
The maximum number of decimal places
- Argument type
Int(constraint=<function>)
- Default
5
--report_number_scientific_notation|--no-report_number_scientific_notation:- Description
Use the exponential notation, i.e. ‘10e3’ for 1000
- Default
True
--report_number_scientific_notation_si_prefixes|--no-report_number_scientific_notation_si_prefixes:- Description
Use si prefixes instead of ‘e…’
- Default
True
--report_number_omit_insignificant_decimal_places|--no-report_number_omit_insignificant_decimal_places:- Description
Omit insignificant decimal places
- Default
False
--report_number_force_min_decimal_places|--no-report_number_force_min_decimal_places:- Description
Don’t omit the minimum number of decimal places if insignificant?
- Default
True
--report_number_percentages|--no-report_number_percentages:- Description
Show as percentages
- Default
False
--report_number_sigmas:- Description
Number of standard deviation used for the digit significance evaluation
- Argument type
Int(constraint=<function>)
- Default
2
--report_number_parentheses_mode:- Description
Mode for showing the parentheses: either d (Digits are considered significant if they don’t change if the number itself changes += $sigmas * std dev) or o (digits are considered significant if they are bigger than $sigmas * std dev)
- Argument type
ExactEither(‘d’|’o’)
- Default
'o'
- temci.scripts.cli.temci__version(**kwargs)[source]¶
- Language
python
- Command
temci version- Description
Print the current version (0.8.5)
Options:
--settings:- Description
Additional settings file
- Argument type
Str()
- Default
''
--config:- Description
Alias for settings
- Argument type
Str()
- Default
''
--tmp_dir:- Description
Used temporary directory
- Argument type
Str()
- Default
'/tmp/temci'
--log_level:- Description
Logging level
- Argument type
ExactEither(‘debug’|’info’|’warn’|’error’|’quiet’)
- Default
'info'
--sudo|--no-sudo:- Description
Acquire sudo privileges and run benchmark programs with non-sudo user. Only supported on the command line.
- Default
False
temci.scripts.temci_completion module¶
Just a more performant version of temci completion that rebuilds the completion files only if the temci version changed. The advantage over using temci completion directly is, that it’s normally significantly faster.
Usage:
``` This returns the location of the completion file.
- temci.scripts.temci_completion.cli()[source]¶
Process the command line arguments and call
temci completionif needed.
- temci.scripts.temci_completion.completion_dir() str[source]¶
Get the name of the completion directory
- temci.scripts.temci_completion.completion_file_name(shell: str) str[source]¶
Get the completion file name for the passed shell and the current temci version
temci.scripts.version module¶
Contains the current version of temci.
- temci.scripts.version.version = '0.8.5'¶
The current version of temci
Module contents¶
This directory contains the command line interface and tab completion code and also the several wrapper scripts and the projects C++ code in sub directories.
