Webinar: Hosting Gateways on Bridges, a Converged HPC, AI, and Big Data Platform
August 8, 2018
Hosting Gateways on Bridges, a Converged HPC, AI, and Big Data Platform
Presented by Sergiu Sanielevici, Director, User Support for Scientific Applications, Pittsburgh Supercomputing Center
Bridges converges high-performance computing (HPC), artificial intelligence (AI), and Big Data and offers a familiar and exceptionally flexible user environment, applicable to whatever data analytics or simulation exceed groups’ local capabilities. Supported by NSF award 1445606, Bridges is designed to enable communities and applications that traditionally have not used high performance computing (HPC). Science gateways provide a particularly effective means for bringing the power of HPC and high performance data analytics (HPDA) to domain experts and students having little or no exposure to HPC, HPDA, or even Linux. To drive science gateways, Bridges provides dedicated hardware for running persistent databases and web servers, together with virtual machines (VMs) and containers for provisioning specific, secure software environments. In this overview, we will describe the design and features of Bridges that are conducive to deploying science gateways. We will also highlight The Causal Web as an example that leverages key elements of HPDA (specifically, causal analysis using Bridges’ large-memory nodes) and Big Data.
Questions about or seeking help with Bridges? Email firstname.lastname@example.org.
Resources mentioned during the webinar:
- Bridges User Guide
- XSEDE Resource Allocation information
- Paper about a science gateway using Bridges:
Webinar: BOINC - Volunteer Computing for Science Gateways
October 11, 2017
BOINC: Volunteer Computing for Science Gateways
Presented by David Anderson, Research Scientist, Space Sciences Laboratory, at the University of California, Berkeley and Steven Clark, Purdue and nanoHUB
BOINC (Berkeley Open Infrastructure for Network Computing) is a distributed computing infrastructure based on a centralized server that coordinates volunteer computer resources. The volunteered resources can come from a variety of types of systems including home computers, institutional servers, and smartphones. BOINC has been used as the underlying foundation for a number of distributed computing projects.
Now, a collaboration between UC Berkeley and Purdue is adding volunteer computing to the nanoHUB nanoscience gateway. Owners of personal computers — Windows, Mac, Linux — will be able to support nanoHUB by transparently running compute-intensive nanoHub applications in the background on these computers. The goal is to greatly increase the computing throughput available to nanoHUB (perhaps tens of thousands of CPUs) at a lower cost than that of commercial clouds and dedicated hardware. This will support new paradigms, such as uncertainty quantification and anticipated computing, that can add significant scientific utility to nanoHUB.
We will describe the technical aspects of this work — moving jobs between batch systems, and using virtualization to run Linux jobs on consumer devices — as well as our plans for recruiting volunteers.
Our work will become part of the HUBzero software, allowing any Hub to add its own volunteer computing capability. More generally, the technology we're developing will simplify the task of adding volunteer computing to any science gateway.
BOINC Slides (David Anderson's part)
nanoHUB Slides (Steven Clark's part)
Webinar: Building a Modern Research Data Portal with Globus - Introduction to the Globus Platform
March 8, 2017
Building a Modern Research Data Portal with Globus - Introduction to the Globus Platform
Presented by Steve Tuecke and Greg Nawrocki, University of Chicago - Globus.org
Abstract: Science DMZ (a portion of the network optimized for high-performance scientific applications) architectures provide frictionless end-to-end network paths; and Globus APIs allow programmers to create powerful research data portals that leverage these paths for data distribution, staging, synchronization, and other useful purposes. In this tutorial, we use real-world examples to show how these new technologies can be applied to realize immediately useful capabilities.
Attendees will develop an understanding of key identity management concepts as they are applied to data management across the research lifecycle, and will be exposed to tools and techniques for implementing these concepts in their own systems.
We will explain how the Globus APIs provide intuitive access to authentication, authorization, sharing, transfer, and synchronization services. Companion iPython/Jupyter notebooks will provide application skeletons that workshop participants can adapt to realize their own research data portals, science gateways, and other web applications that support research data workflows.
Download a PDF of the slides
Answers to questions asked during the webinar
The slides have many links to various online resources. If you don't see what you are looking for, feel free to contact Greg email@example.com directly.
Q: The globus sample portal is written in which language?
Q: For share endpoint, one can't see another share endpoint right?
A: Someone can see the data in an endpoint only if it's been explicitly shared with them. The endpoints themselves are all publicly visible.
Q: Does the Transfer/Share API include download from a share/endpoint to local machine that is not an endpoint?
A: All transfers are to and from endpoints. Globus Connect Personal is a very easy way to set up an endpoint on a local machine: https://www.globus.org/globus-connect-personal
Q: What if someone doesn't want to set up a personal endpoint? We just have resistance from people who don't want to setup a personal endpoint for infrequent downloads.
A: Native “in browser” http transfers are on the roadmap. Transfers themselves are easy; getting them to work within the constraints of our security model requires care. We should have some more concrete timelines for delivery at GlobusWorld in April.
Q: When the user "logins" to the gateway, the gateway redirects to Globus and the user signs in, then Globus redirects back to gateway, is https required for this process or is http ok?
A: Https is required, standard OAuth.