Details

Parent Category: blog

Published on Sunday, 10 September 2023 00:43

By: Swathi Vallabhajosyula

Being a 5th-year Ph.D. student, most of my clock hours are spent during the literature review, curating, creating, and analyzing datasets, understanding fundamentals of different machine learning models, and constructing custom artificially intelligent workflow frameworks for addressing various research problems. Relatively less time is spent converting the research application code into reproducible and readable software components that could be integrated into inferencing workflows for end users or collaborators. 

My primary research focuses on creating an AI-driven HARP (HPC Application Resource Predictor) framework for estimating high-performance computing (HPC) resource requirements in scientific workflows. Scientists utilize HPC cyberinfrastructures like the Ohio Supercomputer Center (OSC) and the Texas Advanced Computing Center (TACC) for computationally intensive tasks (Figure 1 provides an overview of the HARP framework). Each workflow demands specific resources, necessitating a deep understanding of optimal resource allocation. Given the competitive nature of resource allocation and the high cost of these resources, predicting resource needs is essential. To address this, we have developed a framework that emulates executions with sample configurations as history and trains regression models to estimate execution times for targeted applications.

Through the SGX3 internship, I had an opportunity to focus on the software engineering aspect of development to apply the FAIR (Findable, Accessible, Interoperable, and Reusable) principles to my research software, HARP. While trying to integrate this profiler with TAPIS, I identified several generalizability issues with my existing framework. This drove me to learn newer technologies to build cyberinfrastructure agnostic software components for generalizability and centralization of predictive modeling for reusability. I explored distributing the components of my framework as Images that could be executed within containers that store the history of executions as machine-readable metadata on a central server using MongoDB and Flask API calls.

The PEARC (Practice & Experience in Advanced Research Computing) conference series catalyzes conversations about the research computing community's complexities, prospects, and resolutions. This community-led initiative proactively involves a broad spectrum of participants, ranging from local to international, encompassing academia, government, and industry, and includes cyberinfrastructure and research computing collaborators. The PEARC23 conference at the Oregon Convention Center in Portland, Oregon, centered around "Computing for the Common Good." Portland, OR, is a vibrant city known for its lush greenery, thriving arts scene, and eco-friendly initiatives, nestled alongside the Willamette River. It uniquely blends outdoor activities, culinary delights, and a welcoming, laid-back atmosphere.

During PEARC23, I was privileged to represent OSU as a graduate student and TACC as an SGX3 intern. I had a chance to present my ongoing work and discuss plans for integration with TAPIS, aimed at increasing accessibility through a paper presentation and a poster session. This engagement resulted in valuable feedback from the conference attendees, and was awarded the Phil Andrews Award for PEARC23 Best Paper Overall from among all tracks/categories! Furthermore, I actively participated in various activities that enabled me to connect with fellow students, researchers, and industry experts. These activities included attending paper presentations, tutorials, workshops, and Birds of a Feather session. I drew inspiration from keynote speeches from Dr. Panda from OSU, who spoke about the ICICLE Institute, and Dr. Pierce from TACC, who discussed Decision Pathways.

In addition to rekindling my passion for Software Engineering, my time in the SGX3 fellowship has allowed me to rediscover my enthusiasm for outdoor pursuits. Surrounded by a wonderful group of peers, I had the privilege of experiencing diverse culinary delights in Austin and engaging in activities such as hiking, ziplining, kayaking, and bowling.