Pollyanna Jones of NHS England & David Terrar of Agile Elephant, introduced by John Glover of Kahootz, use the NHS England futureNHS platform as a case study story for implementing an effective collaboration solution across silos, teams and organisational boundaries. The story shows: * How the Department of Health and their Arm’s-Length Bodies are using a shared service arrangement to improve team working and stakeholder engagement across the UK health sector * The potential, drivers and enablers that are necessary for success and the impending blockers and pitfalls with advice as how to overcome them. * Where to start, how to educate your staff, and an understanding as to how to select and drive benefit from collaboration tools across the value chain * How to tap into the collective knowledge and expertise of your stakeholders to foster a sense of shared purpose and community involvement * Building a solid business case. Where the value and ROI of collaboration tools could lie as your organisation looks to improve team working with external parties and across organisational boundaries.

1. #futureNHS
The Business Case for Deploying Online
Collaboration Across Organisational Boundaries
Government Computing 2018 | 19 April
Pollyanna Jones @PollyannaJones | David Terrar @DT | John Glover @JDGlover

2. #futureNHS
Agenda
• Agenda and Introductions – John Glover
• Delivering across the collaboration value chain – Pollyanna Jones
– Why do we need enterprise collaboration in the NHS?
– futureNHS – collaborating across national, regional and local footprints
• Evaluating the futureNHS platform – David Terrar
– Key findings
– Engagement and roll out
– How collaboration systems create value
– Quantifying the costs
– Base Case, Upside benefits and ROI
– Executive Summary from the Agile Elephant report
– What next?
– Where are we now?
• Questions - Pollyanna Jones, David Terrar, John Glover

3. 3
The economic case for intelligent knowledge management
in the NHS?
01
02
03
04
`
05
• 7 out 10 internet users are using social
media (Social Media Around the World 2012 – In Sites)
• 96% of execs cite lack of collaboration or ineffective communications
for work-place failures (SalesforceRypple)
• Failing to implement social technology makes high-skiIl employees and
management 20-25% less productive (McKinsey Global Institute)
• In some cases, there is up to 50% duplication of effort in documents
produced by the NHS (observational estimate)
EMAIL
• The average office worker checks
their mail 36 times an hour
(Atlassian)
• The typical Corporate user spends
over 2 hours a day reading and
responding to email (McKinsey – The
Social Economy)
• Almost 20% of the EU workforce spend more than 10 working hours
per week out of the office (Manpower)
• By 2015 the mobile work force will reach 1.3 billion globally. That’s
37.2% of the total work force (IDC)
• A mobile or dispersed workforce as seen in STPs/ACSs will need to
collaborate more effectively
• On average, people send and receive 15 emails with attachments a day
(SkyDrive)
• 59% of middle managers miss valuable information every day, simply because
they can ‘t find it or never see it (Accenture)
• Sharing documents by email prevents team-members from simultaneously
making edits and leads to wasted time and effort
• Almost half of organisations see
knowledge sharing as the top
purpose for social collaboration
(Ventana Research)
• Employees spend an average of 9-
hours a week searching for
information (Atlassian)
• For STPs/ACSs to work at scale
effectively, knowledge sharing must
be enabled and happen in near real
time (P3 team)
MISSED OPPORTUNITIES
COLLABORATION
MOBILE WORKFORCE
KNOWLEDGE SHARING
DOCUMENT SHARING
DEVELOPING A LEARNING SYSTEM
THROUGH INTELLIGENT KNOWLEDGE
MANAGEMENT
THE 5 YEAR FORWARD: KEY CHALLENGES
• The challenges faced by STPs and ICSs will be overcome by those that have a strong sense of
working collaboratively and sharing knowledge in a real-time, agile fashion. It is no longer
unacceptable to think of knowledge management as a storage repository of loosely
connected files.
• Our srategy is initially focussed on all those individuals and teams within NHS England who
are involved in supporting the transformation agenda along the whole supply chain as
shown on the next slide. The audience will include those involved in the national population
health dashboard, data design services, etc. ranging from customers to those working within
these teams. We need one version of the truth in our day to day working across the system
and this starts with effective KM.
Fostering a strong sense of community with purpose
Tapping into collective knowledge and expertise
Keeping deliverables on track
Implementing the FYFW will require dynamic engagement of staff at all levels at
the centre, region and locally. Within each of these ecosystems, senior and
frontline staff will need to be connected with the goals and objectives of their
organisation(s) and be clear on how they fit in or whether their work is making a
difference
Two old-fashioned techniques of information overload via email and top-down
knowledge mobilisation MUST NOT be implemented. To drive sustainable
change, solutions to implementing integrated care will need to be developed and
implemented locally and disseminated regionally and shared nationally.
Communities of practice need to created to solve some of the wicked problems
of transformation in the NHS
Given the exponential growth in programme activities in STPs coupled with the
variants of each within each geographical footprint, a new approach to ensuring
project milestones are met will be required. The onus must be on all those
delivering change to be part of the process and ensure accountability. A top-
down accountability mechanism will not work

4. #futureNHS
4
futureNHS: The start of the collaboration journey
WHAT?
• As a collaboration platform for the new care models programme, Better Care Exchange, STPs and ACSs, etc., to allow teams and work streams to share information
• As a way of managing the administration for the vanguards including funding submissions and evaluation
• As a space for retaining and sharing best practice from the vanguards and new care models programme. A number of products are being developed for the platform to be launched in the autumn
including the population health readiness tool
• As a collaboration space for individual vanguards and STPs to use internally
• Improving internal team working including: acting as a shared drive in the cloud for cross-organisational teams; providing teams with stronger search powers into their files; improving internal team
communications; and, for management of PMO
• Hub for communities of practices to communicate and share important information across broad geographies.
• Knowledge management and curation of information and evidence - users are creating, sharing and adding to each other’s knowledge on the platform
• Improving business intelligence by providing information on useful topics of information for people
• NHS England was one of the first of the Arms’ Length Bodies (ALBs) that wanted to develop a new way of sharing learning and developing policy
collaboratively and in one place with stakeholders
• The New Care Models (NCM) programme was chosen to be the flagship programme to test enterprise collaboration using a single platform that
brings NHS, local government and third sector organisations together to help deliver integrated care
• This test has proved successful and Sustainability and Transformation Partnerships (STPs) and Accountable Care Systems (ACSs) are now using this
platform as the standard approach for sharing learning
233 workspaces
1.6 million
page views
15k+users 1 platform 16 months running
WHO?
The platform is being used in a number of different ways to support knowledge sharing:
Its purpose is to support users to access and exchange information around transformation across health and social care
The aim is to deliver a platform that provides enterprise level collaboration and one that goes beyond a learning management system
WHY?
HOW?

5. #futureNHS
5
Collaborating across the national, regional and local footprints
Knowledge sharing
through communities
of practice and WIKIs
Increased efficiencies
in communication
Collaboration between
health and social care,
NHS and non-NHS partners
Ensuring delivery
through seamless PMO
Single version of the truth being
cascaded as information
National
Regional
Local

6. #futureNHS
A little about Agile Elephant?
• Business transformation + Digital enablement
• Decades of business experience and over 10 years
working with digital media and social
collaboration in practice
• Digital Diagnostic methodology and 7E model for
continuous improvement of collaboration
solutions
• 9 years of collaboration solution research
• Enterprise Digital Summit – London June 6 &
November 21, 2018 is our 5th year)

7. #futureNHS
What were we asked to do?
• A checkpoint review of the platform provided by Kahootz
• Review the set-up and support from the internal team
• Recommendations for improving effectiveness and roll-out
• Create a plan with specific actions for the next 12 months
• Create a business case for expansion
• Return on investment and potential upside

8. #futureNHS
Key findings
• Remarkably fast organic roll out
• Impressive engagement (surveys and interviews were easy)
• Excellent (and vital) support from the central team
• Easier than the other options with reach across multiple IT environments
• Kahootz is a good fit, balances ease of use with functionality, covers 80% of
Lecko feature map
• Proven “new power model” project approach
• Empowerment across teams is driving new ways of collaborative working
• Supporting new care models, improving patient care

9. #futureNHS
The Lecko analysis looks at 100 criteria, split into 10 sections, and evaluates each package
in detail according to these criteria. The survey is done annually.
Process Collaboration
Ergonomics & Customisation
Conversation
Connecting People
Productivity
Content Management
Communicate & Co-Ordinate
Knowledge Management
External Access
Plus: Cross Functional integration criteria
Detailed tool analysis approach
= 80% coverage

10. #futureNHS
What people said:
“Version control is a nightmare. Writing documents with 25 people trying to rewrite it is not easy. Kahootz
makes it way better than before. It’s so much easier to see who has worked on what.”
“You can make better progress and better decisions as you are all signed up to one version of the truth”
“It’s facilitated joint working; it speeds stuff up. You can get a quicker dialogue and pace between
specialists that have different skillsets”.
“Promoting the sharing of ideas and content is crucial. The NHS has a large amount of content which is
‘forgotten’ about in 12 months and it is not uncommon for the ‘wheel to be reinvented’’.
“It does help that we can connect with so many different areas, Kahootz does break down silos”
“The Population Health Analytics Network is an exemplary space. It is designed effectively, is easy to
engage with and has a vibrant community”

11. #futureNHS
Engagement - why so important?
Research shows that teams with high employee engagement rates are 21% more productive than
those with low engagement (Source: Gallup)
Collaboration platforms can help:
• Drive engagement by connecting people across silos
• Drive engagement by enabling new ways of working to evolve (eg: transparency of working)
To release the full potential of the platform also means that people need to engage with it:
• Focusing on methods to increase usage and engagement of the platform itself
• Providing an environment where users trust:
- the platform
- those supporting and educating them
- the people and processes to develop new ways of working

12. #futureNHS
Above average system engagement
Major factors from interviews and surveys were:
System is easy to learn, use and be useful
• System is seen as very easy to learn, intuitive to use – very little “friction”
• But also has enough good functionality to make it useful for an increasing number of
use cases
Connects people across different IT silos
• Only system that can cross into many environments
• Allows multiple environments to be joined
Effective recruitment, training and support
• Central team have been effective for a number of reasons
• System itself is very stable, needs very little user support

13. #futureNHS
How rollout was done – central team
Task Area
Methodology • Central support team are undertaking the role of Change Agents for driving new ways of
working via this system
• Empowering the frontline users by enabling them to take ownership of the platform
• Fostering a strong sense of community with purpose.
Driving
Engagement
• Use the collaboration platform as a fundamental part of business process. They had to
reintegrate business processes to make that happen.
o Export from Sharepoint for example
Continuing
Education
• Follow up publicity campaign to show wider functionality explaining:
o Surveys / Database / How it could be used like NHS Digital’s Delen site
• Producing an operational manual, including a section on how they actually use the
collaboration platform
Continuing
Support
• Must be able to have the capacity to support the community to skill up as it gets more
experienced. To solve this they may get others involved, about 12-20 managers who are
doing innovative things
• Participating in NHS England initiative to share experiences – they have learned a lot from
meeting with others

14. #futureNHS
How rollout was done – early supporters & champions
Task Area
Setting up
the system
• Train 12 people to be the initial workgroup moderators.
o Chose 3 business areas and led by example:
o Took 2-3 people from each area.
Workgroup
Design
• It is critical to design a workgroup structure first. They find that some people don’t listen
to this advice, don’t think through the structure first, and subsequently end up having to
make changes which can upset existing users.
• Keep it simple and only have what’s needed, particularly when starting.
• Don’t overload content for the sake of it
• Curate content in a logical way to result in a sensible architecture of the workgroup.
Managing
Groups
• Insist on having 2 internal managers - though they do allow external managers. Maximum
4-5 managers per group - prefer no more.
• Sheer variety - can push in all sorts of ways - groups can spring up like mushrooms, can
lose control easily. They are starting to map their universe.
Increasing
Engagement
• Comments within the collaboration platform not widely used:
o Functionality isn’t used a lot under docs
o Forums drive engagement - less formal engagement drives usage

15. #futureNHS
Questionnaires and Interviews –
what worked, what didn’t, what’s next
Task Area
Setting up the
system
• Resource intensive when set up, need to start with people who want to do it
• Setting up the environment in advance – “Rules of Engagement” incl. Privacy rules
• Need a mix of systems for different communication roles.
Workgroup
Design
• Spend time getting design right up-front, make changes early.
• “Keep it Simple” structure - organised files, documents etc. into areas
• Make content useful for people’s work and don’t overfill
Managing Groups • Need to be constantly active with users - A “Mother chicken” and a “terrier”
• Continual Housekeeping is required to “tidy up” the site
• Drive Enthusiasm - If users get what they want quickly they love it.
Increasing
Engagement
• Content drives engagement - Set up, then follow up. Monitor what documents
used/downloaded closely
• Needs to be done pro-actively, Aim is to make everyone look to the platform as a trusted
repository
• Drive behaviours, make it useful – refuse to use email
• Group conversations were really useful.

16. #futureNHS
Questionnaires and Interviews –
what worked, what didn’t, what’s next (part 2)
Task Area
Continue
Training
• To train others, use pre-produced materials.
Continue
Support
• Set up a support group - try and recruit keen people initially, gets higher initial
success as they are self-motivated
• One or two people as super users - they may take a lead on that.
• Kahootz hold events with other organisations, both in NHS and outside – very
useful.
• Easy to learn and use, most found it intuitive

17. #futureNHS
Emerging process/organisational barriers
Structuring for further rollout
All the evidence is that the system does increase workflow fluidity and saves time, but if the
desire is to drive adoption faster there needs to be two new structures put into place :
• Structuring the training/ongoing education output so it is user self-service, i.e. more
structured material to reduce the time spent by central resources.
• Some formal “outer layer” of workgroup management to cope with the rising number of
workgroups and users, and to spread best practice of workgroup structure, perform
housekeeping etc.
Build a solid community of skilled workgroup leaders
In order to scale the system further, the central team will need a self-motivating community of
workgroup leaders. Building a community of leaders early on in the process is crucial as this
drives enthusiasm, provides important mutual support to help accelerate driving KH adoption.

18. #futureNHS
Engagement: 4 C’s – our methodology
Many of the interview questions we asked dealt with drivers and paths to increasing collaboration and
engagement. Increasing user engagement on a collaboration platform is typically dependent on:
• Driving engagement between platform users (Cooperation)
• Creating clear and compelling content (Communication)
• Ensuring that the platform itself is easy to engage with/
use/navigate (Coordination)
For the purpose of this analysis all sides of the engagement equation are covered as the interplay between
all 3 is crucial for the creation of a truly engaging platform (Collaboration).

19. #futureNHS
Engagement: 4 C’s – what we saw working well
The interview questions for Engagement and Collaboration were split up as follows:
Cooperation Communication Coordination
• User engagement
• Team Engagement
• Mutual assistance between
people
• Personal development
• Building Trust
• New Ways of Working
• Content
• A Wikipedia System
• Informal engagement
• Forums and Open discussions
• Support networks
• Management structure
• User Training
• System setup and
“housekeeping”
• The System’s User Experience
and Interface
• Using Analytics to increase
Engagement

20. #futureNHS
How Collaboration systems create value – generic benefits
Base Case – estimating generic benefits across all types of enterprises with high level quantification
Modifications for calculating
potential value
• Validation – test the
assumptions using
questionnaire & survey
• Engagement – 100% benefit is
only available to those that are
100% engaged with system
(daily)
• Experience – Users start at 0%
effective and go up a linear
learning curve to become 100%
effective on system over 6 days
usage

21. #futureNHS
End to end analysis – adding value
“Upside” Cases – qualifying end to end areas where potential benefits can be found in any specific enterprise
• User/Customer
Intelligence
• Scheduling • Delivery of
service/goods
• Derive Insights • Spares & supplies • Improve
collaboration
• Co Create
Products &
Services
• Complex Task
Management
• Travel • Communication/
Interaction
• Stakeholder Care • Information Flow
• Ecosystem
monitoring
• Forecasting/
Monitoring
• Reputation
Management
• Knowledge
management
• Distribute
processes
• “Voice of the
Stakeholder”
• Matching talent
to tasks
• Recruitment &
Retention
Business
Support
End User/
Customer /
Patient Support
“Sales &
Marketing”
LogisticsOperations
Service / Product
Development
End to end analysis – adding value
“Upside” Cases – qualifying end to end areas where potential benefits can be found in any specific enterprise
• User/Customer
Intelligence
• Scheduling • Delivery of
service/goods
• Derive Insights • Spares & supplies • Improve
collaboration
• Co Create
Products &
Services
• Complex Task
Management
• Travel • Communication/
Interaction
• Stakeholder Care • Information Flow
• Ecosystem
monitoring
• Forecasting/
Monitoring
• Reputation
Management
• Knowledge
management
• Distribute
processes
• “Voice of the
Stakeholder”
• Matching talent
to tasks
• Recruitment &
Retention
Business
Support
End User/
Customer /
Patient Support
“Sales &
Marketing”
LogisticsOperations
Service / Product
Development

22. #futureNHS
Calculating cost factors
There are 3 main cost areas – IT system costs, central system support, workgroup management and user training
Cost Group Factors Assumptions
System Costs • Sessions per month
• Pageviews/User/Month
• Storage Requirements/User (GB)
• Average no of Workgroups / Use
• Kahootz monthly price per active user per month will cover all variations
Central System
Team
• Internal socialisation
• Recruitment of Users
• Initial Training
• Ongoing support
• User Retraining/Upskilling
• Central System team can carry on with current average growth of registered
users at current capacity
• Over time (Year 2+) can divest Ongoing Support to Workgroup Managers
• Cost of staff + % cost of others drawn into project
Workgroup
Managers
• Workgroup management
• (Increasing) Ongoing support
• (Increasing) Train the trainers
• Approximate times from interviews with NHS England workgroup mangers
plus DoH and NHS Digital experience
• Average cost of typical system user (which is higher than average staff)
Users • Cost of training • 0.5 days lost for initial training per registered user
• Additional 5.5 days lost per active user @ 50% as effectiveness increases
• Average cost of typical system user

23. #futureNHS
Quantifying people costs
The model uses the Full Time Equivalent (FTE) cost approach – benefits may not be able to save on headcount,
but can save time, and this can then be used to do extra work or reduce new recruits
Full Time Equivalent (FTE)
The FTE value is assumed to have the equivalent total cost of employment set against it (annual salary or pay for 7.5
hours a day for 240 days + Benefits + NI) to put a conservative value to savings.
Support and Clinical staff
The NHS model differentiates between Support staff and Clinical staff, it is assumed that:
• Support staff have an average FTE cost of £40k pa, Clinical Staff £60k – higher than averages, assumes high
weighting of higher bands
• Clinical staff have a higher % of time spent on role specific tasks
• The other 3 areas are split pro rata across the remaining 33% of task work time
• Productivity Improvement is assumed to be the same for each task.

24. #futureNHS
ROI Base Case and Upside potential benefits
Total benefits £m
100 %
Base Case
Upside
“Double
Counting”
Additional
Base Case
Potential
Upside Cases
(not quantified)
Case A
Case B
Case C
£ Total
Total
Benefits
Notes
• “Additional Base Case Potential” is the
Maximum potential of the Base Case
as learning curve increases
• Keep it as a part of Upside Case
• The real Upside Case will be
composed of a number of more
detailed Use Cases, for example in the
areas tested in the Audit (Case A, B, C
in diagram)
• Example – Reduce report replication
case

25. #futureNHS
Impact of year-on-year growth
Difference between 1 year and subsequent ROI is significant as initial costs drop away
100 %
Year 1
Base Case
Benefits
Major
Cost
Annual
operating
costs
Year 1
Benefits
ROI
Year 1
training cost
Year 2
£ Total
Minor
Cost
Year 1
£ Total
100 %
Major
Cost
Year 2
Base Case
Benefits
Annual
operating
costs
Year 2
Benefits
ROI
Year 2
training cost

26. #futureNHS
Executive Summary from our report
• futureNHS collaboration platform:
– in use for 9 months, over 5000 registered & 3,200 active users, over 100 active groups, compound
growth rate of 38% per month
• Good engagement, high user satisfaction, adds value beyond document creation
• The easiest platform for collaboration across departmental boundaries and siloes
• Specific recommendations include keeping the current resources and adding more
• Assuming Year 1 growth to 12,000 registered users, approximate benefits are:
– ROI of 52% in the first year, about 700% in Year 2 and beyond if no extra growth
– First year net operating benefits of about £3.5m, rising to £8.9m in Year 2 and beyond if no extra
– Assuming Year 2 growth to a total of 24,000 users, ROI is about 185% and operating benefits about £13m
– High ROI in Year 2 as Year 1 staff are trained so little additional training cost but benefits still accrue
• Additional upside benefits - these are qualified but not quantified
• Strong evidence base for the need for collaboration in health and social care
• With a few improvements, we strongly recommend the team fully endorse and ramp up this work

27. #futureNHS
• Continue with the Central Support Team to drive a wider roll-out and new ways of
working
• More formal workgroup management, with a community space to swap use cases,
ideas and help
• Continual training and building up of additional training assets
• Visibility of more senior level sponsorship and those senior people being actively
involved on the platform
• Create a Wikipedia style knowledge resource and repository of curated content that
is easy to search
• Development budget for improved monitoring and analytics to support workgroup
managers
What next?

28. #futureNHS
What next? Wikipedia
Main views about a Wikipedia approach can be summarised as:
• Make it easier for users to see what is available on the platform – everyone is in favour ,
but issues are:
• Some content is in private groups.
• The search functionality would need to be better than at the time.
• Make it easier for users to see what is available elsewhere on other NHS systems. Again,
most people are in favour and see huge benefits, but there are concerns:
• Search, as above.
• How to link the content technically
• Create more content from various sources – but there are 2 main concerns:
• Who would do the work, people are very busy with their own day to day jobs?
• To do this it would be essential to be able to search for content more easily.
• Drive a knowledge curation phase – but who would have the time.
Privacy Issues
Layers of privacy
came up
frequently– to be
really useful,
people should be
able to see
everything – but
given there are 3rd
parties on the
system, how would
one ensure that
content is not seen
by people who
should not see it.

29. #futureNHS
Where are we now?
• As of 28th March 2018 - 15k users, 175 groups
• Success and user engagement from proactive “new power
model” approach
• Platform licences were supplied team and organisation wide
• futureNHS has spread to other NHSE programmes
• This platform could be a key transformation asset
• Users have been actively participating in the development of
the platform and its functions

30. #futureNHS
Questions?

31. #futureNHS
Dates for your diary:
Kahootz User Conference | techUK, London | 22 June
Making Sense of Blockchain for Business Leaders | techUK, London | 23 May
Pollyanna Jones
Associate Director for Collaboration, New models of care
NHS England
https://www.linkedin.com/in/pollyanna-jones-27a83713/
David Terrar
Founder & CXO
Agile Elephant
https://www.linkedin.com/in/davidterrar
John Glover
Sales & Marketing Director
Kahootz
https://www.linkedin.com/in/jglover1/