Talk at eResearch New Zealand Conference, June 2011 (given remotely from Italy, unfortunately!) Abstract: Whitehead observed that "civilization advances by extending the number of important operations which we can perform without thinking of them." I propose that cloud computing can allow us to accelerate dramatically the pace of discovery by removing a range of mundane but timeconsuming research data management tasks from our consciousness. I describe the Globus Online system that we are developing to explore these possibilities, and propose milestones for evaluating progress towards smarter science.

1. Accelerating data-intensive scienceby outsourcing the mundane Ian Foster

2. Alfred North Whitehead (1911) Civilization advances by extending the number of important operations which we can perform without thinking about them

3. J.C.R. Licklider reflects on thinking (1960) About 85 per cent of my “thinking” time was spent getting into a position to think, to make a decision, to learn something I needed to know

4. For example … (Licklider again) At one point, it was necessary to compare six experimental determinations of a function relating speech-intelligibilityto speech-to-noise ratio. No two experimenters had used the same definition or measure of speech-to-noise ratio. Several hours of calculating were required to get the data into comparable form. When they were in comparable form, it took only a few seconds to determine what I needed to know.

5. Research hasn’t changed much in 300 years Analyzedata Collectdata Publish results Identify patterns Design experiment Pose question Test hypotheses Hypothesize explanation

6. Discovery 1960: Data collection dominates Janet Rowley: chromosome translocationsand cancer

7. 800,000,000,000 bases/day 30,000,000,000,000 bases/year Discovery 2010: Data overflows

8. 42%!! Meanwhile, we drown in administrivia The Federal Demonstration Partnership’s faculty burden survey

9. You can run a company from a coffee shop

10. Salesforce.com, Google, Animoto, …, …, caBIG, TeraGrid gateways Software Platform Infrastructure Varieties of “* as a Service” (*aaS)

11. Salesforce.com, Google, Animoto, …, …, caBIG, TeraGrid gateways Software Platform Amazon, GoGrid,Microsoft, Flexiscale, … Infrastructure Varieties of * as a service (*aaS)

12. Salesforce.com, Google, Animoto, …, …, caBIG, TeraGrid gateways Software Google, Microsoft, Amazon, … Platform Amazon, GoGrid,Microsoft, Flexiscale, … Infrastructure Varieties of * as a service (*aaS)

13. Perform important tasks without thinking Web presence Email (hosted Exchange) Calendar Telephony (hosted VOIP) Human resources and payroll Accounting Customer relationship mgmt Data analytics Content distribution IaaS

14. Perform important tasks without thinking Web presence Email (hosted Exchange) Calendar Telephony (hosted VOIP) Human resources and payroll Accounting Customer relationship mgmt Data analytics Content distribution SaaS IaaS

15. What about small and medium labs?

16. Research IT is a growing burden Big projects can build sophisticated solutions to IT problems Small labs and collaborations have problems with both They need solutions, not toolkits—ideally outsourced solutions

17. Medium science: Dark Energy Survey Blanco 4m on Cerro Tololo Image credit: Roger Smith/NOAO/AURA/NSF Every night, they receive 100,000 files in Illinois They transmit these files to Texas for analysis (35 msec latency) Then move the results back to Illinois This whole process must run reliably & routinely

18. Open transfer sockets vs. time [Image: Don Petravick, NCSA]

19. A new approach to research IT Goal: Accelerate discovery and innovation worldwide by providing research IT as a service Leverage software-as-a-service (SaaS) to provide millions of researchers with unprecedented access to powerful research tools, and enable a massive shortening of cycle times intime-consuming research processes

21. Publish papers

22. Find, configure, install relevant software

23. Find, access, analyze relevant data

24. Order supplies

25. Write proposals

26. Write reports

28. Publish papers

29. Find, configure, install relevant software

30. Find, access, analyze relevant data

31. Order supplies

32. Write proposals

33. Write reports

35. Grid (aka federation) as a service Globus Toolkit Globus Online Build the Grid Components for building custom grid solutions globustoolkit.org Use the Grid Cloud-hostedfile transfer service globusonline.org

36. Globus Online’s Web 2.0 architecture Command line interface lsalcf#dtn:/ scpalcf#dtn:/myfile br />nersc#dtn:/myfile HTTP REST interface POST https://transfer.api.globusonline.org/ v0.10/transfer <transfer-doc> Web interface Fire-and-forget data movement Many files and lots of data Credential management Performance optimization Expert operations and monitoring GridFTP servers FTP servers High-performance data transfer nodes Globus Connect on local computers

37. Globus Connect to/from your laptop 25

38. Almost always faster than other methods 0.001 0.01 0.1 1 10 100 1000 Megabyte/file Argonne  NERSC

39. Monitoring provides deep visibility

41. Globus Online runs on the cloud

42. Data movers scale well on Amazon

43. 11 x 125 files 200 MB each 11 users 12 sites SaaS facilitates troubleshooting

44. Moving 586 Terabytes in two weeks

45. NSF XSEDE architecture incorporatesGlobus Toolkit and Globus Online XSEDE 33

46. Next steps: Outsource additional activities Analyzedata Collectdata Publish results Identify patterns Design experiment Pose question Test hypotheses Hypothesize explanation

47. A use case for the next steps Medical image data is acquired at multiple sites Uploaded to a commercial cloud Quality control algorithms applied Anonymization procedures applied Metadata extracted and stored Access granted to clinical trial team Interactive access and analysis More metadata generated and stored Access granted to subset of data for education

48. Required building blocks Group management for data sharing Scheduled September, 2011, for BIRN biomedical Metadata management Create, update, query a hosted metadata catalog Data publication workflows Data movement, naming, metadata operations, etc. Cloud storage access And HTTP, WebDAV, SRM, iRODS, … Computation on shared data E.g., via Galaxy workflow system

49. www.globusoline.org 37

50. Summary To accelerate discovery, automate the mundane Data-intensive computing is particularly full of mundane tasks Outsourcing complexity to SaaS providers is a promising route to automation Globus Online is an early experiment in SaaS for science

51. For more information Foster, I. Globus Online: Accelerating and democratizing science through cloud-based services. IEEE Internet Computing(May/June):70-73, 2011. Allen, B., Bresnahan, J., Childers, L., Foster, I., Kandaswamy, G., Kettimuthu, R., Kordas, J., Link, M., Martin, S., Pickett, K. and Tuecke, S. Globus Online: Radical Simplification of Data Movement via SaaS. Preprint CI-PP-05-0611, Computation Institute, 2011.

52. Thank you! foster@anl.gov foster@uchicago.edu