Jetstream is the first production science and engineering cloud. It currently provides hosting for several gateways and has the capacity for many more. The flexibility of the cloud environment combined with the mission from the NSF to make the environment available free of charge to US-based scientific researchers means that Jetstream offers considerable resources to the gateway community.
By providing a freely available resource as an option to SGCI and their clients, costs that might have been used to purchase commercial capacity might be otherwise used for development. Additionally, using Jetstream would provide another resource for the gateway community making it more robust.
XSEDE ECSS Symposium 12/19/17: Overview of Deployment Options
The next XSEDE ECSS Symposium will take place December 19, 2017, at 10 am Pacific/1 pm ET.
Andrea Zonca (SDSC) will be the first speaker. Andrea will give an overview of deployment options for Jupyter Notebooks at scale on XSEDE resources. They are all based on deploying Jupyterhub on Jetstream, then either spawn Notebooks on a traditional HPC system or setup a distributed scalable system on Jetstream instances either via Docker Swarm or Kubernetes.
Mahidhar Tatineni (SDSC) will give an overview of the deployment and performance of Remote Direct Memory Access (RDMA) Hadoop, Spark, and HBase middleware on the XSEDE Comet HPC resource. These packages have been developed by Dr. D.K. Panda’s Network-Based Computing (NBC) Laboratory at the Ohio State University. The talk will cover details of the integration with the HPC scheduling framework, the design and components of the packages, and the performance benefits of the design. Applications tested include the Kira toolkit (astronomy image processing), latent Dirichlet allocation (LDA) for topic modeling, and BigDL (distributed deep learning library).
Full abstracts and connection details are available at on XSEDE's website.