Build AI maturity and transformation playbook
•
2 j'aime•1,355 vues
The document discusses several important considerations for companies looking to implement artificial intelligence, including developing an AI transformation playbook, assessing an organization's AI maturity, anticipating costs and timing, deciding whether to build or buy AI solutions, and addressing important legal and ethical issues around explainability, privacy, fairness, and safety. The document provides guidance on how companies can effectively lead their organization into the AI era by establishing the right strategies, processes, and safeguards.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Build AI maturity and transformation playbook
Similaire à Build AI maturity and transformation playbook (20)
Dernier
Dernier (20)
Build AI maturity and transformation playbook
- 1. Build AI in your company management, culture, legal, ethics, governance Orange Belt - Session 3 1
- 2. What we have seen so far 1. What is AI 2. What can AI do and what it can’t do 3. How to select a project 4. What are the steps necessary for a first successful ML project 2
- 3. AI is not traditional software A totally different lifecycle 3
- 4. 4
- 5. 5
- 6. We saw this last week 01 03 02 06 04 05 Monitoring & Updates Have the right talents & solutions Maintenance Select the right question Choose the performance metric Decide the level of explainability Identify Use the right architecture Have the talents in place Deploy Find the right data Structure annotate data Clean Data Data Decide on an acceptable error Test on the right scope Evaluate Select the right algorithm Tune the model Model 6
- 7. In practice, everything is more iterative 7
- 8. Plan for today 0 (Base on the case seen in the assignment) What does it feel like to work on a complex ML project ? 1. AI Transformation Playbook 2. AI Maturity of your company 3. Perspectives on costs & roadmap 4. Build vs Buy 5. Legal & ethics considerations 6. Human and AI interactions 8
- 9. 1. AI Transformation Playbook How to lead your company into the AI era 9 Source : https://landing.ai/ai-transformation-playbook/
- 10. AI Transformation Playbook 1. Execute pilot projects to gain momentum 2. Build an in-house AI team 3. Provide broad AI training 4. Develop an AI strategy 5. Develop internal and external communications 10
- 11. 1. Execute pilot project to gain momentum • More important for the initial project to succeed rather than be the most valuable • Show traction within 6-12 months • Can be in-house or outsourced 11
- 12. 2. Build an in-house AI team Centralized AI Platform 12
- 13. 3. Provide broad AI training Role What they should learn Nb hours of training Executives and senior business leaders - What AI can do for the company ? - AI Strategy - Resource allocation >= 4 hours Leaders of divisions working on AI projects - Set project direction (both technical and business diligence) - Resource allocation - Monitor progress >= 12 hours AI engineer trainees - Build and ship AI software - Gather data - Execute on specific AI projects >= 100 hours 13
- 14. How to document yourselves AI for Everyone, by Andrew Ng Deep Learning Specialization 3blue1brown Siraj OpenAI ImportAI https://towardsdatascience.c om/ 14
- 15. 4. Develop an AI strategy • Build several difficult AI assets that are broadly aligned with a coherent strategy • Leverage AI to create an advantage specific to your industry sector. • Design strategy aligned with the “Virtuous Cycle of AI” AI plays a role here 15
- 16. 4. Develop an AI strategy • Consider creating a data strategy • Strategic data acquisition • Unified data warehouse • Create network effects and platform advantages • In industries with “winner take all” dynamics, AI can be an accelerator • What about more traditional strategy framework ? • AI can allow a low cost strategy • AI can allow a high value product strategy 16
- 17. 5. Develop internal and external communications • Investor relations • Government relations • Consumer / user education • Talen / recruitment • Internal communications 17
- 18. AI pitfalls to avoid Don’t : Do : - Expect AI to solve everything - Be realistic about what AI can and cannot do, given limitations of technology, data and engineering resources - Hire 2-3 ML engineers and count solely on them to come up with use cases - Pair engineers with business talent and work across cross-functional team to find valuable projects - Expect the AI project to work the first time - Plan for AI development to be an iterative process, with multipe attemps needed to succeed. - Expect traditional planning processes to apply without changes - Work with AI team to establish timeline estimates, milestones, KPIs, etc. - Think you need superstar AI engineers before you can do anything - Keep building the team but get going with the team you already have 18
- 19. Some initial steps you can take • Start learning (with this course) • Start brainstorming projets • Hire a few ML/DS people to help • Hire or appoint an AI leader (VP AI, CAIO, …) • Discuss with CEO/Board possibilities of AI Transformation • Will your company be much more valuable and/or more effective if it were good at AI ? 19
- 20. Exercise What is the first step in the AI Transformation Playbook for helping your company become good at AI? 20
- 21. Exercise Of the following options, which is the most important trait of your first pilot project? A) Succeed and show traction within 6-10 months B) Drive extremely high value for the business C) Be executed by an in-house team D) None of the above 21
- 22. Exercise Say you are building the DropBox OCR system, and want to accumulate data for your product through having many users. Which of these represents the “Virtuous circle of AI” for this product? 22
- 23. Exercise Why is developing an AI strategy NOT the first step in the AI Transformation Playbook? 23
- 24. 2. AI Maturity Assess the readiness of your organisation 24
- 25. 25
- 26. A startup mindset is always nice 26
- 27. 3-steps roadmap to maturity Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY DATA PEOPLE LEGALÐICS PRODUCT 27
- 28. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget DATA No infrastructure Data Silos Descriptive analytics PEOPLE No data scientists No education LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 28
- 29. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics PEOPLE No data scientists No education LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 29
- 30. 30
- 31. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding value PEOPLE No data scientists No education LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 31
- 32. 32
- 33. Data requirements according to business needs 33
- 34. Define data acquisition strategy It’s not a one shot 34
- 35. Data annotation pipeline To go at scale, you need guidelines, internal and external annotators. Even pre-annotation with machine learning that can be validated DOG CAT 35
- 36. Standardise quality check Find the relevant metrics Exemple Sound: signal to noise ratio / cross-talk / silence detection 36
- 37. Govern data Different access restrictions, stay compliant 37
- 38. 38
- 39. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 39
- 40. Example roles • Software Engineer • Build user interface, web & mobile applications, back end operations, … • Machine Learning Engineer • Input (A) to Output (B) • Machine Learning Researcher • Extend state-of-the-art in ML 40
- 41. Example roles • Data Scientist • Examine data and provide insights • Make presentation and communicate to team / executives • Data Engineer • Organize Data • Make sure data is saved in an easily accessible, documented and cost effective way • More and more required as the amount of data managed by companies increases • AI Product Manager • Help decide what to build; what’s feasible and valuable 41
- 42. Getting started with a small team • 1 Software Engineer, or • 1 Machine Learning Engineer / Data Scientist, or • Nobody by yourself 42
- 44. Structure a team - decentralised 44
- 45. Structure a team centralised (SWAT) 45
- 46. Structure a team add use case specialists 46
- 47. Exercise Suppose you are building a trigger word detection system, and want to hire someone to build a system to map from Inputs A (audio clip) to Outputs B (whether the trigger word was said), using existing AI technology. Out of the list below, which of the following hires would be most suitable for writing this software? 47
- 48. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development 48
- 49. Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development Use of APIs Discrete proof of concepts Pilots AI is core product & core competency 49
- 50. Exercise : create your own roadmap Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development Use of APIs Discrete proof of concepts Pilots AI is core product & core competency 50
- 51. Exercise Try to identify the critical 3 next steps for your organization, using the matrix 51
- 52. 3. AI Management Anticipate costs and timing 52
- 53. prices & timing are not at all set in stone.. 53
- 54. but we can still assess feasibility 54
- 55. but we can still assess feasibility 55
- 56. but we can still assess feasibility 56
- 57. but we can still assess feasibility 57
- 58. Machine Learning is not regular Software 58
- 59. Bring back your expected performance requirements 59
- 60. 60
- 61. 61
- 62. To sum up 1. Anticipate the cost drivers (data, accuracy, problem difficulty) 2. There is a big upfront cost to anticipate as opposed to regular projects 3. The cost scales nonlinearily with the accuracy requirements 4. The time it takes to get that accuracy is nonlinear as well 62
- 63. Exercise What is the biggest cost of your current idea? What do you think will be the bottleneck? What type of strategy could you devise to anticipate that? 63
- 64. 4. Build vs Buy The eternal dilemma 64
- 65. 65
- 66. With a consultant you don’t know, always look to start with a small proof of concept deliverable to prove to yourself that this consultant knows their stuff. Work with the consultant to come up with a project that is a low hanging fruit. Something that they can deliver on quickly without much development effort (e.g. based on existing code they already have, and data you have already collected). If this first step goes well, then you can confidently move to a bigger project scope. 66
- 67. 67
- 68. Access to resources (cloud, ML libraries, production systems) Access to talent Proven success stories to get you up and running in no time So why would you do everything yourselves ? Don’t try to run faster than the train Collaborate with Startups! 68
- 69. 5. Legal & Ethics What should you be careful about 69
- 71. What does a Convolution sees ? VGG16, convolutional layer 1-1, a few of the 64 filters 71
- 72. What does a Convolution sees ? Variation of kernel size Source : 72
- 73. Adversarial Examples The left image is predicted with 99.9% confidence as a magpie. 73
- 74. Adversarial Examples Machine Learning classifiers today are easily fooled ! 74
- 75. Adversarial Examples Machine Learning classifiers today are easily fooled ! 75
- 77. The problem with explainability https://distill.pub/2019/activation-atlas/ 77
- 79. • Maximise customer satisfaction, • Think about human decision making expectations • Complex or direct simple explanations? • Stay pragmatic - “reasonably necessary” Reasonable explanation 79
- 80. Alternative accountability Flagging - Stress test - Auditing 81
- 81. Privacy 82
- 82. 1. AI systems must be transparent. 2. An AI must have a “deeply rooted” right to the information it is collecting. 3. Consumers must be able to opt out of the system. 4. The data collected and the purpose of the AI must be limited by design. 5. Data must be deleted upon consumer request. Bernhard DebatinProfessor and Director of the Institute for Applied and Professional Ethics Guidelines for ethical privacy 83
- 83. 1. Clear consent requests to customer (and easy out) 2. Breach notifications within 72h 3. Right to access all personal data upon request 4. Right to be forgotten in practice 84
- 84. Fairness & Bias What is fair anyways? 85
- 85. Risks & Safety How to protect from potential dangers 86
- 86. 87
- 87. 88
- 88. 89
- 89. 90
- 90. ‘build a world-class face recognition AI model’ Specific, focus & measurable 91
- 91. ‘build a face recognition service that can detect male/female genders, with pre- defind specific age groups, and these specific subset of races, and ethnicities in the requirements document (which is grounded in a standard taxonomy from a neutral organization such as the United Nations Race and Ethnicity taxonomy)) with at least 90% accuracy on ‘these’ given specific test datasets’ where ‘these’ test datasets were carefully crafted by the offering management team to have an even distribution of all the genders, age groups, specific races, and ethnicities for which the model is supposed do well’ Specific, focus & measurable 92
- 92. 93
- 93. 94
- 94. 95
- 95. 96
- 96. 97
- 97. Foster trust By Element AI 98
- 98. To summarise 1. Specify a robust goal 2. Make sure you respect privacy 3. Make it explainable 4. Make it secure 5. Make it transparent 99
- 99. Exercise Can you identify the most sensitive data in your organizations? Do you already have a strategy to be compliant? Do you need a big explainability or would you rather get better performance? 100
- 100. 6. Towards better HCI Human-Computer interactions have to be reconsidered 101
- 101. Human-in-the-loop Design for better collaboration 102
- 102. The problem with hype… 103
- 103. 104
- 104. 105
- 105. 106
- 106. 107
- 107. Human vs Machine • Machines never forget • Crunch numbers and scan fast • Never bored or impatient or tired • People have better emotional nuances + humanity • People have better common sense • People can tackle new tasks easily 108
- 108. 3 types of collaboration 109
- 109. AI recommends multiple options 110
- 110. AI makes the decision 111
- 111. AI coaches 112
- 112. Keep human psychology in mind Lots of false positive = irritating Introducing without consultation = resentment Most often good decision = overtrust (automation bias) … 113
- 113. Exercise Try to imagine the end product of your project, leveraging both AI and humans using the type of collaborations we just talked about What would the user be the most sensitive about ? 114
- 114. Conclusion Create your AI implementation strategy 115
- 115. Quiz 116