Introduction

Overview

Teaching: 5 min
Exercises: 0 min

Questions

• What is the EIC environment?

Objectives

• Understand what we mean with the EIC environment, or eic-shell.

• Appreciate the benefits of containers as the development environment.

In this episode we will cover two aspects of the EIC Software Environment:

• the actual commands you can run to access the software components we rely on for simulations and analysis,
• the services and repositories you will want to contribute to in order to work collaboratively.

Much of the work we do at the EIC relies on both of these collaborative environments.

Key Points

• The EIC environment eic-shell is a singularity/docker container with a curated selection of software components.

The EIC Software Environment

Overview

Teaching: 5 min
Exercises: 5 min

Questions

• What is the EIC environment?

Objectives

• Understand what we mean with the EIC environment, or eic-shell.

• Appreciate the benefits of containers as development environment.

Why a EIC standard environment?

• EIC software is modular, so not just a matter of a single make install command but close to 20 necessary dependencies and 10 EIC-specific packages.
• Laboratory operating systems are stable and (therefore) slow to upgrade to new versions, even though we need those versions in the modern software we write:
  • gcc-11 or gcc-12 for C++17 ranges, C++20 for concepts and filtered ranges, and C++23 for zipped ranges
  • underlying dependencies like ACTS, DD4hep, podio, etc, each have many options, some of which are important to set right (or you end up with a DD4hep that doesn’t do some of the things we need it to do)
• The standard environment gives all users access to exactly the same environment that is used for benchmarks, productions, and continuous integration on the github and gitlab servers.
• The standard environments are versioned and can be retrieved (or rebuilt) at a later time in case you wish to revisit an old result. Specifying the specific version of the standard environment you used for a particular plot will definitively determine what versions of e.g. geant4 and ROOT you used.
• If you encounter a bug that you can reproduce in a standard environment, it makes it much much easier for software developers to fix. Making it easier for developers to fix you bug will typically result in your bug getting fixed quicker. If developers cannot reproduce the bug, it often becomes a fishing expedition that few developers have appetite for.

Are you required to use the EIC standard environment?

• No. You can use any environment you want. We are merely trying to make this easy for all users to get started, in particular if you are not yet familiar with compiling source code from scratch.
• Most of the people in the software groups use a mix of the standard environment and other installations (for example, we may have ROOT installed separately).
• We may ask you to reproduce issues that you observe in the standard environment before we can act on them.
• If there is a specific workflow or use case that you cannot achieve in the standard environment, let us know and we’ll work with you to address that.

How to get eic-shell, the EIC standard environment, and what it is?

• The EIC standard environment is accessed with the eic-shell command. That will start an eic-shell session in the standard environment.
• To install the eic-shell command in the directory ~/eic/ (which you may need to create), you can run the following command:

  cd ~/eic
  curl --location https://get.epic-eic.org | bash
  

• You can also save the file at https://get.epic-eic.org as install.sh and run this script by hand:

  cd ~/eic
  wget --output-document install.sh https://get.epic-eic.org
  bash install.sh
  

• The install script will search for several components that are required (see the prerequisites):
  • which operating system you are running (linux and mac are supported),
  • whether you have singularity/apptainer or docker installed (see prerequisites),
  • whether you have access to the Cern VM file system at /cvmfs (typically available on clusters, and can be easily installed on individual linux systems, see https://cernvm.cern.ch/fs/).
• If you are on a system that requires downloading the full container (up to 10 GB), this may take a while and no progress bar is shown.
• The install script will create a new file ./eic-shell which will start the eic-shell environment, i.e. the EIC standard environment.

Note: An alternative location for downloading this script is at https://eicweb.phy.anl.gov/containers/eic_container/-/raw/master/install.sh, in case you experience difficulties accessing the custom domain.

Note: On some high performance computing servers, there are security restrictions that require the use of additional flags curl --insecure and wget --no-check-certificate.

Exercise 1:

• Install the eic-shell executable in a suitable location on the system you will be using mainly for EIC work.
• Look at the output and compare with what is on the instructor’s screen. If you do not have a /cvmfs directory on your system, this may take a while as the environment has to be downloaded. Keep this in mind for later, as you will have to perform updates periodically.
• Advanced users: Take a look at the installation options when you run bash install.sh --help. The include information on how to tweak your setup for specific use scenarios.

How to start eic-shell on common systems?

You can start eic-shell from inside the directory where it is installed with

cd ~/eic
./eic-shell

or you can start it from any directory with ~/eic/eic-shell (if you used the same directory structure).

• When you have successfully started eic-shell you should see a jug_xl> prompt, indicating that you are now inside the EIC standard environment.
• Pro-tip: You can also add the directory where eic-shell is installed to your search path, so you can start eic-shell from anywhere. Or you can install eic-shell in a directory that is already in your search path.

Exercise:

• Run the eic-shell script from the current directory. You should get the jug_xl> prompt. Exit the environment again.
• Take a look at the running options with eic-shell --help. If you do not have a /cvmfs directory on your system, you will need to run eic-shell --upgrade periodically to ensure that you are staying up to date.
• Advanced users: Set up your environment to be able to run eic-shell from anywhere.

What is where in the eic-shell environment?

When you start eic-shell, you enter into a container: a self-contained operating system that integrates with the core of the operating system on the host system. When you look at the top-level directories, you’ll see that they are similar to the directories on the host system. Don’t let that fool you: you are looking at content that is only inside the container.

There are a few directories that are shared between the container and the host operating system: the current directory you are working on, your home directories, etc. Essentially, the typical directories where you would store your own work (as opposed to system directories).

All programs that are installed in the container, are accessible under the /usr/local directory tree. This is a standard location for programs that are not installed by the operating system’s package manager, which is how we install the dependencies for the EIC software stack. It also means that these programs are automatically found by the eic-shell without needing to modify the search path.

The jumble of files in /usr/local are in fact merely a view into software installed in a more organized fashion in /opt/software, potentially for different compilers, operating systems, etc. The full definition of all software installed in /opt/software is in /opt/spack-environment/dev/spack.yaml, the ultimate definition of the EIC standard environment.

Exercise:

• Using cd and ls, check that you can navigate to various directories in the container, and that you can access the directories for your own work.
• Verify that you can run some of the software dependencies that are installed in eic-shell. For example, run the command ddsim --help to get some informaton on one way to run Geant4 simulations on DD4hep-based geometries. Use the command which ddsim to see where ddsim is located.
• Advanced users: Based on the location of ddsim found in the previous step, verify that this is a symbolic link to a different location under /opt/software. Verify that the version in the path of that location agrees with the version specified in the spack.yaml file.

Notes on starting graphical programs

Graphical programs work just as well inside the eic-shell environment as outside, with the exception of docker on Mac where some additional work is needed. On a Mac it is therefore often easier to open ROOT files outside the container.

Key Points

• The EIC environment eic-shell is a singularity/docker container with a curated selection of software components.

Using GitHub with EIC Software

Overview

Teaching: 10 min
Exercises: 5 min

Questions

• How do we use GitHub within the EIC community?

Objectives

• Several repositories contain the key geometry, simulation, reconstruction, and analysis software.

• Working with repositories requires membership of the EIC organization on GitHub.

• Issue reports and pull requests are ways to collaborate virtually on collaboration software tools.

Where are repositories located?

The main repositories are located on two locations:

• the GitHub service contains most user-facing repositories: https://github.com/eic.
• the eicweb GitLab server contains mainly continuous integration infrastructure: https://eicweb.phy.anl.gov/EIC.

We will use GitHub as the main code repository tool. The top of the github.com/eic page has several ‘pinned’ repositories that are most important. You can also see where most activity is happening based on the ordering of the repositories in the list.

You can ‘watch’ repositories to be notified of activity. This is helpful if you wish to remain up-to-date on the activity. You can ‘star’ repositories as another way to find them easily.

Exercise 1:

• Verify that are a member of the EIC organization on GitHub: Do you see the members-only page with the grid of software meetings? If not, send you GitHub user name to the conveners.
• Check which teams you are in: Enter the search terms “members:me” into the search box in the Teams tab to see only the teams you are a member of. If you are not in ePIC Devs, request to be added at https://github.com/orgs/eic/teams/epic-devs.
• Choose one repository to subscribe to, at some level of activity.

What is the proper way to work with these repositories?

Duplication of work is avoided when everyone knows what is going on.

1. A typical workflow starts from the creation of an issue with a feature request or a bug report. This happens in the browser on the github repository site. When creating the issue, feel free to tag people you want to notify.
2. Create a branch that includes the issue number in the name. This can be done from the browser or command line as git checkout -b issue123
3. Edit and work on your branch. When you are ready, you can commit your changes to the branch.
4. If you are ready to merge your branch into the main repository, you can generate a pull request from the browser. You can assign a reviewer at this stage and add informationin the Write box that explains the changes made. If your work is not yet completed, you can utilize the Draft PR.
5. Once you’ve generated a pull request, continuous integration tests are run to ensure compatibility. The reviewer will approve the changes, merging your code into the main branch.
6. After the pull request is approved and the code is merged, you can delete your branch using the browser or via the command line.

A good example of the above workflow is shown in detail herefor the EICrecon repository.

Exercise 2:

• Think of one issue that is related to the analysis or detector you are mainly working on.
• Think of a good title that summarizes what the issue is.
• Open an issue in the relevant repository and fill out the provided template.
• Tag one person (who is not the instructor) whose input you would like on this issue.
• Advanced users: Create a branch from the issue using the ‘Development’ section in the right side bar of the issue.

How can you use eic-shell with the repositories that you are working on?

What are the checks that are run on pull requests?

Key Points

• We use several tools on GitHub to ensure we keep the overview of who does what work.