Webinar: Data and Software Carpentry: Using Training to Build a Worldwide Research Community
May 10, 2017
Data and Software Carpentry: Using Training to Build a Worldwide Research Community
Presented by Tracy Teal, co-founder and the Executive Director of Data Carpentry, and Adjunct Assistant Professor with BEACON, Michigan State University
Although petabytes of data are now available, most scientific disciplines are failing to translate this sea of data into scientific advances. The missing step between data collection and research progress is a lack of training for scientists in crucial skills for effectively and reproducibly managing and analyzing large amounts of data. Already faced with a deluge of data, researchers themselves are demanding this training. Short, intensive, hands-on Software and Data Carpentry workshops give researchers the opportunity to engage in deliberate practice as they learn these skills. This model has been shown to be effective, with the vast majority (more than 90%), of learners saying that participating in the workshop was worth their time and led to improvements in their data management and data analysis skills. Data Carpentry events have trained over 20,000 learners since 2014 on 6 continents with over 800 volunteer instructors. The strategies of growing this community could be applied toward growing communities of gateway users, particularly by offering training and demonstrating the value of the skills and tools that will enhance their work.
View the slides (Slideshare)
Questions asked during the webinar (and some answers)
If you have further questions, you are welcome to contact Tracy at tkteal AT datacarpentry DOT org.
Q: What is the relationship between SGCI and her organization? There seems to be some overlap...in training, for example?
A: Currently there is no formal relationship between SGCI and Data Carpentry, but we definitely want to look into that option further!
Q: I wonder how additional topics and instructors get added to the set of offerings.
Q: When discussing "active learning", she used an acronym - IBU? IVU? What's that?
A: I, We, You [First the instructor shows it, then we do it together, and then you do it yourself.]
Q: Do you request attendees install software before a workshop (or during)? In the Python ecosystem, do you recommend a particular distribution?
Example of a lesson: http://swcarpentry.github.io/lesson-example/
Q: So what about Jupyter? Do you use it?
Q: Where does the instructor training take place? and is there also a cost for this?
Q: Who pays for the volunteer instructor's travel?
Q: What is the relationship between Data Carpentry and Software Carpentry.
Webinar: Gateway Showcase featuring Ensayo Project's SimEOC and Spatial Portal for Analysis of Climatic Effects on Species (SPACES)
April 12, 2017
Gateway Showcase featuring
Ensayo Project's SimEOC: A Web-Based Virtual Emergency Operations Center Simulator for Training and Research
Presented by Greg Madey, University of Notre Dame
Abstract: Training is an integral part of disaster preparedness. Practice in dealing with crises improves one’s ability to manage emergency situations. As an emergency escalates, more and more agencies get involved. These agencies require training to learn how to manage the crisis and to work together across jurisdictional boundaries. Consequently, training requires participation from many individuals, consumes a great deal of resources in vendor cost for support and staff time, and cannot be conducted often. Moreover, in the current crisis management environment, most training is conducted through discussion-based tabletop and paper-based scenario performance exercises. SimEOC was developed under the NSF-funded Ensayo Project. It is a web-based training simulator and research tool. SimEOC is built using MongoDB, Express.js, Angular and Node.js (the MEAN stack). A design overview and demonstration will be provided.
Download a PDF of the Ensayo Project's slides
Questions asked about Ensayo Project (answered in the video)
If you want to try out the gateway or have further questions, email gmadey AT nd DOT edu
- Can anyone get an account for the SimEOC and do the exercises?
- Please comment on the development process....
- Who would configure these exercises? How do you add new facilities related to emergency management?
- When did collaboration with CRC begin? how was the broader development team scoped and organized?
- Has this system been used in a real training situation?
- Did CRC have enough visibility at ND that you knew to reach out? Otherwise, how did you learn of them?
Spatial Portal for Analysis of Climatic Effects on Species (SPACES)
Presented by Dilkushi de Alwis Pitts, University of Cambridge
Abstract: To deal with escalating environmental shifts caused by climate change and other factors, ecologists are increasingly called upon to make risk assessment decisions about affected natural resources. As a result, there is a rapidly growing need for niche modeling of species projections to guide management decisions and activities related to intervention.
A number of software applications exist for carrying out fundamental niche modeling, but they present several problems for users, including distinct approaches to algorithms, data, and outputs, among others. The openModeller software was created to address these concerns by providing transparent, open-source tools under a common architecture.
SPACES builds on openModeller to manage for biologists the complication of running niche models, including data formatting and the complexities of the modeled systems. SPACES has endeavored to resolve the issues mentioned above by obtaining, handling, and storing the large quantities of data that niche models require, processing the data in a user-controlled way, and presenting the results in convenient formats. Through SPACES, extensive, quality spatial data are made available alongside species data and a variety of niche models that can be executed, analyzed, and compared—all through a common Web browser interface, designed to support a virtual scientific community and share the results of research.
Questions asked about SPACES Project (answered in the video)
If you want to try out the gateway or have further questions, email kad49 AT cam DOT ac DOT uk
- Does SPACES use HPC to run jobs? If yes, which one(s)?
- Are all the output spatial raster data? Otherwise, how to do map algebra on vector data? Where does map algebra happen, in user's browser or a backend server? If latter, how to manage the computation?
- What does it cost to add a new model algorithm? How much automation has been done to speed it up?
- How do other researchers put their algorithms into SPACES portal?
Webinar: Building a Modern Research Data Portal with Globus - Introduction to the Globus Platform
March 8, 2017
Building a Modern Research Data Portal with Globus - Introduction to the Globus Platform
Presented by Steve Tuecke and Greg Nawrocki, University of Chicago - Globus.org
Abstract: Science DMZ (a portion of the network optimized for high-performance scientific applications) architectures provide frictionless end-to-end network paths; and Globus APIs allow programmers to create powerful research data portals that leverage these paths for data distribution, staging, synchronization, and other useful purposes. In this tutorial, we use real-world examples to show how these new technologies can be applied to realize immediately useful capabilities.
Attendees will develop an understanding of key identity management concepts as they are applied to data management across the research lifecycle, and will be exposed to tools and techniques for implementing these concepts in their own systems.
We will explain how the Globus APIs provide intuitive access to authentication, authorization, sharing, transfer, and synchronization services. Companion iPython/Jupyter notebooks will provide application skeletons that workshop participants can adapt to realize their own research data portals, science gateways, and other web applications that support research data workflows.
Download a PDF of the slides
Answers to questions asked during the webinar
The slides have many links to various online resources. If you don't see what you are looking for, feel free to contact Greg email@example.com directly.
Q: The globus sample portal is written in which language?
Q: For share endpoint, one can't see another share endpoint right?
A: Someone can see the data in an endpoint only if it's been explicitly shared with them. The endpoints themselves are all publicly visible.
Q: Does the Transfer/Share API include download from a share/endpoint to local machine that is not an endpoint?
A: All transfers are to and from endpoints. Globus Connect Personal is a very easy way to set up an endpoint on a local machine: https://www.globus.org/globus-connect-personal
Q: What if someone doesn't want to set up a personal endpoint? We just have resistance from people who don't want to setup a personal endpoint for infrequent downloads.
A: Native “in browser” http transfers are on the roadmap. Transfers themselves are easy; getting them to work within the constraints of our security model requires care. We should have some more concrete timelines for delivery at GlobusWorld in April.
Q: When the user "logins" to the gateway, the gateway redirects to Globus and the user signs in, then Globus redirects back to gateway, is https required for this process or is http ok?
A: Https is required, standard OAuth.
Webinar: Creating a developer pipeline by teaching gateway technologies
February 8, 2017
Creating a developer pipeline by teaching gateway technologies
Presented by Marlon Pierce and Suresh Marru, Indiana University
Abstract: Do you have trouble recruiting and retaining good gateway developers? Common challenges include the allure of the cutting-edge commercial sector, academic pay scales, and the specialized knowledge necessary for development. The Indiana University Science Gateways Research Center is attempting to turn this challenge into an opportunity by teaching Web-based cyberinfrastructure systems using concepts and technologies that will benefit students in their non-academic careers. Such distributed systems require knowledge of both classic and cutting-edge topics, such as microservices and “DevOps” practices. Meanwhile, the instructors can identify and connect with promising students while keeping themselves aware of important trends and technologies. This webinar will provide an overview of the material and concepts that we cover in the class, describe the student project assignments and class organization, and present outcomes and student feedback that we have gotten over the last two semesters. We will also discuss possibilities of making the course material and instruction available to other interested universities. Course information and material is available from http://courses.airavata.org/.
View the slides (Slideshare)
Answers to questions asked during the webinar
Q: Are services mostly on the gateway side or mostly at the clusters?
A: Services are mostly middleware.
Q: Were all/most students CS students? Were they undergraduate or graduate students?
A: Yes, all of them ended up being CS students. We had Data Science students enrolled but we probably scared them away with programming-heavy projects. So far, it has been restricted to graduate students.
Q: Were distributed data services part of the class?
A: Distributed data services are part of the advanced class we are teaching in Spring. We are trying to make this is a rhythm; in Fall, the foundation gateway architecture class, and in Spring, we recruit selected students from the Fall class to do advanced topics like you mention.
Q: Were there exams? Or was it all homework/project-based?
A: Everything is project and class participation based. Here's an example of our grading in Fall: http://courses.airavata.org/fall2016/index.html#grading
Q: Do you ever invite "real" scientists to discuss their use of gateways?
A: Yes, we do guest lectures when we have to travel. We also tried to bring in guest lectures from industry, which also worked out well.
Webinar: How you and your gateway can benefit from the services of the Science Gateways Community Institute (SGCI)
January 11, 2017
How you and your gateway can benefit from the services of the Science Gateways Community Institute (SGCI)
with Nancy Wilkins-Diehr, Michael Zentner, Marlon Pierce, Maytal Dahan, Katherine Lawrence, and Linda Hayden, all of the Science Gateways Community Institute
Abstract: Are you developing gateways and related software that serve your science discipline? A domain-based scientist or engineer who needs a gateway? A campus-based IT administrator or staff member lacking manpower or expertise to provide specialized services?
The Science Gateways Community Institute (SGCI) was funded by the National Science Foundation on August 1, 2016 to provide and leverage resources, expertise, community support, and education to create and sustain science and engineering gateways. Many of our services are now up and running and ready to support your gateway project. This webinar will describe the types of FREE services you can request and provide examples of clients that we are already helping. Among the many services that might interest you, we offer consultants with specialized expertise, hands-on gateway development support, ways of publicizing your gateway/software, learning opportunities, and programs for young professionals and students.
Answers to questions asked during the webinar
Q: Tell me about groups on Purdue's campus (such as HUBzero) that develop gateways for the university.
A: HUBzero is just one instance of a successful campus gateway development group. SGCI offers consulting about how to create your own campus group. Sandra Gesing of Notre Dame leads this initiative. For more information or to request a presentation to your campus, email firstname.lastname@example.org.
Q: How many people per gateway project can attend an Incubator Bootcamp?
A: You can send two people: a PI and a technical/development person.
Q: Will SGCI provide gateway hosting in addition to the container sandbox for developing gateways?
A: SGCI will offer temporary hosting directly; this isn’t meant to be permanent hosting but more of an area to play with. Excellent permanent options are available through providers such as XSEDE Quarry (and perhaps Jetstream) and Amazon Web Services.
Q: How can I find out more about Workforce Development's Summer Programs?
A: A description of the three 2017 programs and a link to the application (due March 17) is available online. Information about the programs in general is in the Student-Focused Opportunities section of this website.
Q: How can I join the SGCI Young Professionals Network?
A: Soon there will be a form on the Young Professionals page.
Q: Will you be offering immersive learning experiences through SGCI, such as in-residence programming collaborations?
A: We don't currently have that planned, but that's a great idea to consider in the future.
Q: How are SGCI services distinct from XSEDE's ECSS program?
A: In short, XSEDE's programs are limited to connecting to XSEDE resources whereas the SGCI can help you create a gateway that connects to any type of resource. In addition, our services are more diverse, offering specialty services such as usability evaluation, graphic design, technology consulting, and more. SGCI will be a Level 2 XSEDE service provider.