Geoweaver: From Incubator Project to a Broader Audience Scope
By Claire Stirm
No longer focusing on agriculture alone, Dr. Ziheng Sun has expanded Geoweaver in support of many Earth sciences, and he was recognized by NASA for his initiative. Geoweaver is a science gateway designed to manage artificial-intelligence/machine-learning (AI/ML) research in Earth science. Researchers can use Geoweaver to compose and execute full-stack, deep-learning workflows. Geoweaver takes advantage of hybrid resources including online, spatial-data facilities; high-performance computing; and open-source, deep-learning libraries. It is a useful alternative to SSH client (e.g., Putty), FTP client, and scientific workflow software.
Initiated as a 2018 Earth Science Information Partners (ESIP) Lab project, Ziheng received funding to work towards turning large-scale, distributed, deep networks into manageable, modernized workflows. Ziheng, Research Assistant Professor at George Mason University, attended the ESIP- and SGCI-facilitated Gateway Focus Week (formerly Bootcamp) in November 2018. Ziheng found the sustainability workshop to be very interactive and inspiring and, afterward, he started to shape an idea for Geoweaver to move beyond its initial phase of just supporting agriculture-mapping workflows. Following the workshop, Ziheng met with Juliana Casavan through a short-term sustainability engagement provided by SGCI. “Geoweaver shifted its focus from agriculture mapping to a broader scope,” says Ziheng. He goes on to say, “In order to broaden my audience community, Juliana taught me how to find out who my target audience is within the Earth science community and who is interested in my software products.”
Following his engagement with Juliana, Ziheng met with the SGCI usability team as they reviewed Geoweaver. “They were really amazing,” says Ziheng. “They gave me a lot of useful advice on the user-interface design. I have adopted their suggestions and now the interface for Geoweaver is a lot different from two years ago.” With the theme of keeping Geoweaver’s users at the forefront of Ziheng’s work, he went on to say that now “the users are more comfortable using Geoweaver.”
SGCI also provided technical support. Geoweaver was designed to store a detailed history of every process that a user runs. These processes can be independent, using separate execution resources, and can be linked together to form workflows. Users can run processes in parallel or sequentially across separate resources, but all managed in one place. Jupyter servers can also be added as host resources. Ziheng worked with Andrew Magill, an SGCI technology consultant with the Extended Developer Support team to implement a unique capability. Andrew and Ziheng enabled Geoweaver to intercept websocket traffic between a user and the user’s Jupyter Server. This allows versions of notebooks to be saved as the user edits their notebook. The user can also choose to return to previous states of the notebook. Ziheng commented that in this six-month engagement, “We created a bridge between Geoweaver and Jupyter Notebooks. Geoweaver can capture all the history of what a user did in a Jupyter Notebook, and now it is a selling point.” Andrew demonstrated this unique capability at Gateways 2020. You can check out the recording of this demo at Geoweaver as a Gateway to Manage Distributed Geoscientific Workflows.
Within the last two years, Ziheng has pivoted Geoweaver from a small, agriculture-focused “Lab” project to a gateway reaching new horizons across the Earth science community. Ziheng was one of the recipients of the 2019 NASA ACCESS award. The NASA ACCESS announcement specifically sought technology developments for one or more combinations of ML, science in the cloud, and support of high-value, open-source, Earth-science tools and libraries. This three-year award will fund the continued development of Geoweaver. Ziheng was able to demo the new Jupyter Notebook feature on Geoweaver at a recent NASA ACCESS meeting. He received good feedback from the attendees and views the NASA award as proof of how Geoweaver is designed to support the Earth science community beyond agriculture.
Sun, Ziheng, Liping Di, Annie Burgess, Jason A. Tullis, and Andrew B. Magill. "Geoweaver: Advanced Cyberinfrastructure for Managing Hybrid Geoscientific AI Workflows." ISPRS International Journal of Geo-Information 9, no. 2 (2020): 119.