Details

Published on Thursday, 01 December 2022 00:49

Popularizing OneDataShare and enabling scientists to move High Amounts of Data over Wide Area Networks

By: Jacob Goldverg

The Gateways 2022 is a unique conference that attracts a broad audience from backgrounds ranging in but not limited to: NASA's satellite development, a simplified version of bash that is executed through the python runtime, and educational programs for high school students to uncover Pulsars. The scientific breadth that Gateways 2022 had was amazing, as it really opened me up to just how many people in varying fields are dealing with data that is just too big.

Upon arriving and to start the conference off was a great run down of the motive, and initiative that the Gateways community is focusing on. The venue selected was beautiful, the four olive trees at the venue made is very scene-esque and on-top of this it was hosted in downtown San Diego all the sun was fantastic.
The first keynote speaker who lead a very mind-blowing talk, Matthew Greenhouse. The amount of innovation that NASA performed to launch the James Webb Space Telescope was astounding, and the photos this telescope was able to take were truly marvelous. From engineering a completely new design(the individual gold like lenses), to a new transport container to ship the telescope to various testing facilities really gave comic book feeling in the amount of mind breaking ideas NASA produced from 1996-2022 (the launching).

After shaking off the Star Trek feeling I had a very fun time talking about my poster on OneDataShare. I was amazed to find how many people in varying backgrounds that have Terabytes of data with no other way to move it than Fedex. Being able to speak with scientists, biologists, climatologists, network administrators, really gave the diverse meaning of users who are limited by their networks in moving big data. Which proved to be very enlightening talking with scientists in varying backgrounds was highly enlightening on my own work.

The most common problems I heard at the conference that are relatable to OneDataShare are:

• Due to all the cloud storage providers, working with their API's to access you data is a long and tedious process. Which really boils down to vendor locking on users data.
• Having an internet connection that is unstable leading to poor performance in standard utilities.
• Users are also unable to determine the optimal parameters in which to send their data, due to the restraints from standard utilities and the complexity of modeling threading to throughput utilization.
• ODS is able to optimize throughput by using ML/AI to monitor the conditions of the data transfer and then tune the transfer in real time.

Being able to talk to the people at TACC, and various other institutions also gave me new insight on how ODS can be used not just by people but by HPC sites as well. I could summarize my poster presentation as being very engaging and stimulating.

After the poster presentation there were many talks that covered a very diverse set of problems. As a PhD student in Computer Science it is really hard to understand the problems pertaining in other fields as PhD students are entirely saturated with their own work, which is exactly why I found SGCI to be very inspirational. Seeing the work done in space, education, HPC left a positive impression of the future and the kinds of solutions being created today.

What really stuck out to me was how HPC is now getting put into undergraduate education. As someone who did their undergraduate degree in computer science and mathematics at the University at Buffalo I never got take any classes that covered distributed work flow, Tapis, Globus, Slurm and really had to self teach a lot of these critical tools. Education in HPC will help other bodies of research in how to use these sites efficiently is essential and the focus that SGCI has for it is wonderful.

If I were to summarize what I enjoyed about Gateways 2022:

• The conference really covers a wide range of bodies which leads to very educational conversations.
• The large focus on education and enabling students and scientists to gain access and use HPC resources to solve their own problems.
• Very well organized, and engaging. With fantastic keynote speakers.
• The food! Dinner, breakfast, all the wonderful finger food during the conference.
•  The poster session was a really easy way to end the first day of the conference.

What is exciting is to see the continued growth of the science gateways community and the impact it continues to have on the community around it.

I would really like to express my gratitude to the conference and the organizers for the support they have shown me in attending Gateways 2022.

Gateways 2022 took place from October 18-20, 2022 in San Diego, CA.
 

Join us for the next Gateways conference!
Stay informed by subscribing to the Science Gateways Community newsletter.