Easybuild
Installing software from source is a non-trivial problem. That’s why we are a huge fan of package managers at the RSEpedia software showcase, and why this week we are featuring easybuild. Easybuild is especially pertinent to the research community because it’s intended to install and manage scientific software on High Performance Computing (HPC) resources. Keep reading the following sections to learn more!
If you are already an easybuilder, we encourage you to contribute to the research software encyclopedia and annotate the respository:
otherwise, keep reading!
- What is Easybuild
- How do I cite it?
- How do I contribute to the software survey
- Where can I learn more?
What is Easybuild?
Let’s say you’re an HPC administrator. Or maybe you’re a researcher that has found himself unexpectedly as manager of a few servers. You likely have an entire suite of scientific software packages that you want to provide to your user base, whether that be a small group like a lab or an entire academic center. You likely have different versions too, and subtle differences in dependencies. What to do? While you might decide that environment modules are the way to go, or perhaps a previous week’s package manager, spack, easybuild offers a set of features that might whet your appetite:
- fully automated, reproducible software builds with dependency resolution
- custom install procedures (instead of traditional configure, make, make install)
- co-existence of different versions via dedicated installation prefixes and module files
- build recipes are simple and human readable (this is a huge deal!)
- sharing of recipes within the HPC community
- logs are kept as a record of the build process
- install in parallel, if desired
- a thriving, growing community.
The last point about community is a big deal! It means that if you use easybuild and need help, there are many others that are able and willing to help you. Here is a group picture from the 5th EasyBuild User Meeting, held in Jan 2020. Yes, right before the coronapocalypse!
In terms of software supported, check out the list of supported software - there are over 2000! But don’t take our word for it! The easybuild documentation has a What is easybuild? page that can tell you more.
How do I cite it?
While there isn’t a recommended paper to cite in the repository, you can likely reference this ACM publication:
@inproceedings{10.1109/HUST.2014.8,
author = {Geimer, Markus and Hoste, Kenneth and McLay, Robert},
title = {Modern Scientific Software Management Using EasyBuild and Lmod},
year = {2014},
isbn = {9781467367554},
publisher = {IEEE Press},
url = {https://doi.org/10.1109/HUST.2014.8},
doi = {10.1109/HUST.2014.8},
abstract = {HPC user support teams invest a lot of time and effort in installing scientific software for their users. A well-established practice is providing environment modules to make it easy for users to set up their working environment. Several problems remain, however: user support teams lack appropriate tools to manage a scientific software stack easily and consistently, and users still struggle to set up their working environment correctly. In this paper, we present a modern approach to installing (scientific) software that provides a solution to these common issues. We show how EasyBuild, a software build and installation framework, can be used to automatically install software and generate environment modules. By using a hierarchical module naming scheme to offer environment modules to users in a more structured way, and providing Lmod, a modern tool for working with environment modules, we help typical users avoid common mistakes while giving power users the flexibility they demand.},
booktitle = {Proceedings of the First International Workshop on HPC User Support Tools},
pages = {41–51},
numpages = {11},
location = {New Orleans, Louisiana},
series = {HUST '14}
}
There is an earlier paper from 2012 as well.
How do I get started?
- The EasyBuild tutorial is a great place to start. It was intended to be in-person at ISC’20, but COVID-19 mucked up that plan. The easybuilders community persisted anyway! It was held 100% virtually, and hugely successful, with close to 100 participants. The exercises are quick and useful, as there is a container image provided to get you up and running! Not only are there working cases to learn from, but also debugging challenges.
- Easybuild Tech Talks might you learn watching something over dinner? EasyBuild has started a small series of tech talks that currently include the topics of Arm and Open MPI, the first of which is still coming up on September 30th!
- Kenneth Hoste on RSE Stories: if you want to snuggle into a warm chair and listen to learn, you might be interested in listening to Kenneth talk about easybuild on the Research Software Engineer Stories Podcast.
- The EESSI Project: pronounced “easy,” this project grown out of the easybuild community aims to build a ready-to-go stack of properly optimized software installations for HPC systems, cloud, and workstations. It’s heavily inspired by the Compute Canada software stack. See maintainer Kenneth Hoste’s introduction to the project video, and if you’re super eager, jump right into the pilot.
- How to Make Package Mangers Cry if you want an introduction to some of the problems of package management, and a good laugh.
- Kenneth Hoste on Twitter and Easybuild on Twitter if you want a quick ping for social media.
You can also post questions on the GitHub issues board.
How do I contribute to the software survey?
or read more about annotation here. You can clone the software repository to do bulk annotation, or annotation any repository in the software database, We want annotation to be fun, straight-forward, and easy, so we will be showcasing one repository to annotate per week. If you’d like to request annotation of a particular repository (or addition to the software database) please don’t hesitate to open an issue or even a pull request.
Where can I learn more?
You might find these other resources useful:
- The Research Software Database on GitHub
- RSEpedia Documentation
- Google Docs Manuscript you are invited to contribute to.
- Annotation Documentation for RSEpedia
- Annotation Tutorial in RSEng docs
For any resource, you are encouraged to give feedback and contribute!
Recent Posts
- Posted on 21 Mar 2021
- Posted on 07 Mar 2021
- Posted on 21 Feb 2021
- Posted on 21 Feb 2021