Simplifying scientific Python package installation and environment management
Amiya works to simplify scientific Python package installation by streamlining environment management, dependency tracking, and runtime customizations through easy-to-use tools. With the growing popularity of Python, installation and management of python packages on HPC clusters is emerging as a critical problem for researchers and is complicated by the need for providing consistency across traditional batch workloads and interactive notebooks. Amiya will collaborate with various HPC centers to document and present their best practices for managing Python applications and implement these practices in the development of the python-env-mod tool to simplify these processes. Python-env-mod helps users manage their Python environments more efficiently and load runtime configurations through the familiar abstraction of environment modules. HPC centers can further customize the module file templates to incorporate additional software dependencies and provide descriptive help messages. This work will significantly improve scientific productivity, reduce user errors, and enable sharing of Python package installations among users.
Amiya is a Senior Computational Scientist at Purdue Research Computing where he collaborates with faculty and researchers from various scientific domains to optimize their computational and data analysis workflows. Being an avid advocate for software reliability and security, Amiya has developed several algorithms and tools for software testing both during his graduate studies at Purdue ECE and then at Research Computing. He co-invented the “Testpilot” regression testing framework at Purdue (HUST17) and also developed the “conda-env-mod” tool for easy deployment of scientific Python applications (HUST20). Amiya currently leads the software build automation project for Purdue’s community clusters. Amiya’s contributions towards the Community Cluster program were recognized by the Bravo Award (2020) given to Purdue employees for outstanding achievement. Amiya also serves as a fellow of Trusted CI (2021) where he promotes best practices for secure computing.
Selected Resources
Simplifying scientific Python package installation and environment management – challenges and solutions conda-env-mod: A tool for automating HPC Python package installation
