Connections between big data and open data. Includes a case study of Data.gov and the ways that companies, charities, and others are using open data to improve the lives of people around the planet.

1. The Evolution of Open Data
@JeanneHolm Jeanne Holm
Evangelist, Data.gov
11 June 2014

2. TheTimesTheyAre A’Changing
2
People
People  Knowledge  Problems
June 11, 2014

3. Data, Data Everywhere
 Smart phones
 Smart cars
 Smart people
 Sensors
 RFID
 Cameras…everywhere
June 11, 2014 3
Americanis.net

4. Move from Structured to Unstructured
 Heterogeneous sources of data
 Structured (tables, transactions) = schema
 Semi-structured (human-readable, XML, JSON)
 Unstructured (images, audio, videos) = no relationship
June 11, 2014 4

5. Recent Data Growth Web 2.0
 Social media
 Facebook
 Twitter
 Skype
 The Internet ofThings
 Many sources
 Varied formats
 Relatively timely
 Web content
 Many authors
 Unstructured
 Highly variable trust
and provenance
 Gaming
 Highly specific
 Huge transactional data
 Real-time, high
bandwidth usage
June 11, 2014 5

6. Recent Data Growth Web 2.0
 Open data
 Government and industry
 Structured and unstructured
 Accessible
 Private data
 Apps
 Health data
 Credit card and financial data
 The Web
 Browsers
 Search engines
 Web site metrics
June 11, 2014 6

7. Creating Order from the Chaos
 Open vs. closed
 Multiple formats
 Unstructured
 Trusted vs. unvalidated
June 11, 2014 7

8. Releasing and using
open data is about
empowering people to
make better decisions
June 11, 2014 8

9. Presidential Executive Order
June 11, 2014 9

10. Project Open Data: Open Source Policy
 Open source
government policy,
technical guidance,
and software
 Citizen contributions
to policy, code, and
content
 http://project-open-
data.github.io/
June 11, 2014 10

11. 105,000+ collections
349 citizen apps
500,000 data
resources
175 agencies
450 APIs
June 11, 2014 11

12. Creating the Open Data Community
Open Data
is an
Ecosystem
June 11, 2014 12

13. Open Exchanges with Citizens
Questions and answers at
the new Open Data Stack
Exchange
http://opendata.stackexchange.com/
Data jams and data
paloozas at theWhite
House
June 11, 2014 13

14. Connecting Over Social Media
June 11, 2014 14

15. Developer Feedback
June 11, 2014 15
https://github.com/GSA/data.gov

16. Open Exchange with Developers
 Created a new Open Data Stack Exchange to field
questions to the global community:
http://opendata.stackexchange.com/
June 11, 2014 16

17. Citizen Participation: Redesigning Data.gov
 In looking at the redesign, conducted multiple places for citizens to
say what they wanted
 Formal usability testing (3 rounds)
 Blogs
 Next.Data.gov
 Quora
 Twitter @usdatagov
 Open Data Stack Exchange
 Multiple social media platforms
 All the comments in one place  Github
 Issues tracked at
https://github.com/GSA/data.gov/issues?labels=&milestone=&pag
e=1&state=open
June 11, 2014 17

18. UsabilityTesting
 Created and vetted usability test that focused on what actions
people completed on the site and expectations they had for what
they would find
 Face-to-face testing inWashington D.C.
 Virtual testing via Skype and phone
 Online testing using Loop 11
June 11, 2014 18
 Reached out to key users
 Data journalists
 Researchers
 Developers
 Entrepreneurs
 Data scientists
 Businesses
 Students and teachers
 Advocacy groups
 People who had complained
about Data.gov

19. Evaluate the Feedback
 All issues were copied, connected, or added to Github from any
public communication channel
 Issues were assigned to a person and a build
 Discussion was encouraged on each and people were invited to
the conversation
June 11, 2014 19

20. Designs Exposed Openly
June 11, 2014 20

21. U.S. Open Data for Cities, Counties, and
States
21June 11, 2014

22. Linked Data and the SemanticWeb
June 11, 2014 22
Join theW3C
eGovernment Interest
Group
www.w3.org/egov

23. Open Communities
Community
Developers ✓
Safety ✓
Energy ✓
Health ✓
Law ✓
Education ✓
Ocean ✓
Manufacturing ✓
Business ✓
Ethics ✓
States ✓
Counties ✓
Cities ✓
Agriculture ✓
+ many more…
June 11, 2014 23

24. Agriculture Drives Innovation and Saves Lives
Farmers’ Markets
iCow
June 11, 2014 24

25. Open Government Platform (OGPL)
Email, Github, Facebook, and Twitter for discussion
https://github.com/opengovtplatform
http://www.opengovplatform.org
June 11, 2014 25

26. A Global
Movement Has
Begun to Provide
Transparency and
Democratization
of Data
June 11, 2014 26

27. Is open data useful?
June 11, 2014 27

28. Creating a multi-
billion dollar
industry
June 11, 2014 28

29. NOAA weather data + Air Force GPS services =
$100B industry
June 11, 2014 29

30. Climate Corporation
June 11, 2014 30

31. Helping others
through a disaster
June 11, 2014 31

32. Hurricane Sandy
700,000 downloads of a
transportation app using
data from Data.gov
Early warnings save lives
June 11, 2014 32

33. 33
Saving Lives
June 11, 2014

34. 34June 11, 2014

35. It’s Not Rocket
Science…or Maybe It Is
June 11, 2014 35

36. International Space Apps
 Open annual event, around the world and in space
 95 physical locations in 46 countries participated;, 8,195
participants, 671 projects
 Where on Earth
 Exomars Rover is My Robot
 Asteroid Prospector
 SpaceWearables
 Growing Food for A MartianTable
June 11, 2014 36

37. June 11, 2014 37

38. Predicting the
Future
June 11, 2014 38

39. June 11, 2014 39

40. June 11, 2014 40
Empowering Others to
Use Open Data

41. Organizing and Understanding the Data
 Web searching, mining, and crawling
 Algorithms
 Visualizations
 Text mining
 Clustering
 Semantic analysis
 Linked data
 Machine learning
June 11, 2014 41

42. New Role: Data Scientist
 Combines technical and business skills
 Looks at complex data problems with subject matter
expertise
 Applies technologies to mine, analyze, and visualize the
data
 Understands statistics and math, coding and algorithms
 Can explain the significance of the data to others
 Leader of the data scientists: The Chief Data Officer
June 11, 2014 42

43. June 11, 2014 43

44. Open Data Matters
 Connect citizens to open data to transform their world and
empower them through education
 Connect developers to open data to create new ways of
using the data to inform others
 Connect businesses to open data to provide new services
and products for everyone to use
 Connect data scientists to open data to analyze the past and
predict the future
 Encourage governments to release more open data
June 11, 2014 44

45. Helping to improve the
lives of people in our
community
June 11, 2014 45