This lesson is in the early stages of development (Alpha version)

Developing Benchmarks

Benchmarks are scripts that run detector simulations and analyze the resulting data to extract quantities related to the detector performance.

Prerequisites

The following tutorial assumes basic knowledge of shell. It may be practical to use eic-shell for benchmark development, as it is the environment that will closely match the one during benchmark execution. It also assumes that you are a member of the EIC organization on GitHub and belong to the “epic-devs” team. You also need to have your local ssh key added to GitHub so that you can push.

This tutorial follows other tutorials in the EIC series:

Further information is included in other tutorials:

Schedule

Setup Download files required for the lesson
00:00 1. Exercise 1: Analysis Scripts and Snakemake How does one set up data analysis workflows?
00:30 2. Exercise 2: Setting up your first benchmark with pipelines How do we create a new pipeline with GitLab CI?
01:00 3. Exercise 3: Filling out your benchmark How do we fill in each stage of the benchmark pipeline?
01:30 4. Exercise 4: Adding a Status Flag How can your benchmark indicate that there were detrimental changes to software or detector design?
01:50 5. Exercise 5: Making Useful Figures How does one make useful benchmark figures?
02:10 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.