1. Impact/legacy measurement and evaluation in
mega events projects with focus on intangible
assets
Mauricio Nunes Rodrigues
Marcos Cavalcanti (COPPE/UFRJ)
Ahmed Bounfour (UPSUD)
The 8th International Doctoral Consortium on Intellectual Capital Management
27 may 2015
2. Content
1. Introduction
2. Research Question
3. Methodological approach
4. Model development
5. Model validation study (FIFA 2014 World Cup
Tourism impacts)
6. Next steps
3. 1. Why hosting mega event projects?
• Increasing number of stakeholders interested in hosting
international mega events, despite criticism regarding:
• costly and complex bidding and implementation projects
• actual scene of poor performance in terms of public support
and economic / environmental outcomes
• Why these stakeholders place value on hosting such events (e.g.
Olympic Games, Sports World Championships, Festivals, Cultural
and Political summits)?
4. 1. Potential economic benefits
Directs
• Capital flow to host city/country
• Infrastructure construction or
upgrade
• Lower transportation costs due
improved networks
• Increase in tourists spending
Indirects
• Advertising effect of the host
city/country as a potential
tourist or business destination
• more local business
opportunities
• Improved local sense of
community and civic pride
• Improved perceived abroad
image of the host city/country
• Citizen entertainment and
welfare
• Human resources skills
development
• Motivation to a more active life
(Clark, 2008; Kasimati, 2003;
Preuss, 2007; Preuss, 2010;
Zimbalist, 2010)
5. 1. Potential downsides / risks
• Don`t be able to deliver all positive impacts (planned or unplanned)
• Socially unjust displacement and re-distributions
• Poor urban land use
• Underused facilities after the event
• High public debts
• Excessive costs
• Athens 2004 more than US$ 10 bi
• Beijing 2008 more than US$ 40 bi
• London 2012 about US$ 15 bi
(Cashman, 2010; Haußermann & Simons, 2000; Flyvbjerg et al. 2003; Zimbalist, 2010)
6. 1. Costs x Investment - Are it worth it?
• Key role on public policies and investment (tourism and business
destination attractiveness, business growth, urban regeneration,
and infrastructure, image, environmental and local population
quality of life improvements)
(Clark, 2008; OECD, 2010; Orueta & Fainstein, 2008; Preuss, 2007; Zimbalist, 2010)
• When hosted well, the mega event project can play a significant
role in city/region local development, growth and competitiveness
(OECD, 2010)
• The benefits do not occur by accident or without an effective action
Need strategic vision and a proper impact planning and
management
(Clark, 2008; IOC, 2009b; OECD, 2010)
7. • Impact strategic planning and forecasting traditionally done by
a best practice / benchmarking approach with basis on tangible
socio-environment-economic indicators
(Preuss, 2007)
• Do not provide relevant information for effective decision-
making, neither for the strategic management of the future
positive impacts, legacies and benefits
• The value of nations, regions, organizations is directly related to
their intangible capitals and depends on systems to visualize,
cultivate and capitalize on value-creation interactions
(Edvinsson, 2003; Edvinsson & Bounfour, 2004)
1. Impacts planning and management
8. 2. Research question
How could we measure and evaluate the impacts
generated for and by mega event projects, taking into
account the intangible assets, with a focus on future
value creation (legacies)?
9. 3. Methodological approach
• Qualitative and strategic design with basis on the
intangibles approach
Hypothesis Evidence
1. The mega event project’s decision-makers
and managers should base their decisions
not only on tangible socio-environment-
economic analysis
The traditional approaches to mega projects
performance measurement and evaluation
are insufficient to support the strategic
maximization of the potential benefits
(Flyvbjerg et al., 2003)
2. The measurement and evaluation of the
intangible aspects can help to improve the
significance of the positive impacts expected
by some stakeholders
The intangible assets have become strategic
factors for value creation, and are considered
central factors to economies’ growth and
competitiveness (OECD, 2008)
3. It is possible to develop a reliable
operational method to measure and evaluate
the mega event projects intangible aspects
????
10. 3. Objective
To develop a diagnostic model for measurement and
evaluation of the mega event projects impacts and
legacies, taking into account the intangible assets
12. 4. Model development
• Focus
• to identify the relevant intangible assets, the
interdependencies between then, the success factors
generated, and the variables that should be collected
• Strategy = research case study
• To guarantee the external validity and follow the strategic
approach
• To prevent the lack of a real-life event context and a well
defined objective to the impact analysis
• Widely used in organizational and managerial studies when the
investigators are interested in understand complex social
phenomena (Kohlbacher, 2006; Yin, 2003)
13. 4. Model development
• Case analysis
• Assessment and evaluation of the impacts on the intangible
assets and outcomes of the FIFA 2014 World Cup interventions
on the Rio de Janeiro city tourism industry
• Mixed approach
• Based on the Intangible Capital Rating - CRIE/BNDES
(Deutscher, 2007, 2008 and Cavalcanti, 2007) and the
Intellectual Capital Dynamic Value - IC-dVAL (Bounfour, 2000,
2003)
CRIE/BNDES IC-dVAL
Identification of the assets which the
organizations should have to
implement their future vision
Integration of a link between the
financial value of assets and the
internal performance
14. 4. The MEI2 Model
The operational version of the Mega Event Intangibles Impacts
Model (MEI2 Model) consist of:
• 5 Dimensions
• 15 Assets (resources, competencies and processes)
• 42 Indicators
16. 5. The MEI2 Model Validation
• Data collection
• Interviews with FIFA 2014 World Cup stakeholders directly and indirectly
impacted by the interventions on the Rio de Janeiro city tourism industry
• The sample was divided in two groups
• Focus
• Stakeholders perceptions of the performance and impacts of the
interventions on the intangible assets and outcomes listed as main sources
of competitive advantage for its strategic objectives
Internal stakeholders External stakeholders
Decision-makers and local project
management staff from
government bodies
Tourism industry business
associations and unions
Other sectors business
associations and agencies
Megaproject / event specialists
17. • Data collection in 3 steps
• Step 1 – Impact analysis
• Step 2 – Foundations of the stakeholders perceptions
• Step 3 – Identification of the ideal competitive positioning
5. The MEI2 Model Validation
Improved
No
impact
Worsened
Completely Quite Somewhat Slightly Neutral Slightly Somewhat Quite Completely
2,0 1,5 1,0 0,5 0 -0,5 -1,0 -1,5 -2,0
19. 5. Preliminary results
Intangible capital
dimensions
Internal
stakeholders
mean (±sd)
External
stakeholders
mean (±sd)
Human 35,0% (0,07) 21,7% (0,08)
Strategic 15,0% (0,00) 26,7% (0,13)
Structural 22,5% (0,11) 16,7% (0,06)
Ecosystem 12,5% (0,14) 18,3% (0,05)
Relationship 15,0% (0,00) 16,7% (0,04)
No significant difference between groups (Student’s t-Test)
Degree of importance (the relative value) for each dimension of intangible capital
21. 5. The MEI2 Model Validation
• Preliminary conclusions
• The concentration of importance in human capital by internal
stakeholders (35% of the overall weight) was reflected in the division
of financial resources to professional and languages skills training
(about 74% of specific budget for tourism intervention)
• The strategic positioning captured by the model in both groups (but
more concentrated in internal stakeholders) leads us to infer that the
strategic positioning of tourism interventions was more focused on
project execution internal questions than to the lasting potential
benefits of the project for the tourism
• The Overall performance rating (11,5%) situated in the first
improvement quartile indicated a fair overall positive impact of the
FIFA 2014 World Cup interventions on the Rio de Janeiro city Tourism
industry
• There is a significant different performance perception between the
two groups of stakeholders (38,5% - internal X 2,5% - external)
derived by a pessimist perspective from the external stakeholders
22. 6. Next steps
• Increase the sample
• to expand the field of analysis. E.g., increasing the number of
subgroups regarding the external stakeholders (tourism industry, non-
tourism industry and megaproject specialists)
• To reduce the risk of the Type I error (false positive) in comparisons
between groups
• Collect data about the intervention expenditures for each dimension of
intangible capital
• to calculate the dynamic value of the Intangible capital
• Define a set of external indexes and socioeconomic indicators
• to compare and benchmarking the expected benefits (tourism and
business destination attractiveness, business growth, urban
regeneration, and infrastructure, image, environmental and local
population quality of life improvements) with our results
23. Thanks for your attention!
Questions and comments?
Mauricio Rodrigues
mauricio@pep.ufrj.br
Editor's Notes
In last years, the application process to host mega event projects are becoming increasingly popular.
Despite of the costly bidding and implementing phases and the actual scene of poor performance in terms of public support, economic and environmental outcomes.
Why some stakeholders place value on hosting such events?
2024 = Paris, Rome, Hamburg and Boston - Probably Budapest
The literature indicates potential economic benefits from hosting a mega event, such as…
However, mega event projects also present high risk of potential downsides, such as:
Don`t be able to deliver all positive impacts
Socially unjust displacement
Poor urban land use
Underused facilities after the event
High public debts, and
Excessive costs
But, despite the potential downsides and high costs, some of the expenditures described, results in an improved infrastructure that can generate significant tangible and intangible benefits to the host country, such as tourism and business destination attractiveness, business growth, urban regeneration, and improvements on infrastructure, image, environment and local population quality of life
So, when hosted well, mega event projects can play a significant role in local development, growth and competitiveness.
However, the benefits do not occur by accident or without an effective action. They need a strategic vision and a proper impact planning and management.
To do that…The mega event project’s organizers support their planning and strategic forecasting in a best practice / benchmarking approach / looking for something which have already proven to be successful at the past
But, these traditional approaches to evaluate and forecast the mega events benefits do not provide relevant information for effective decision-making, neither for the strategic management of future positive impacts.
As, nowadays, the value of nations, regions and organizations is directly related to their intangible capitals and depends on systems to visualize, cultivate and capitalize on value-creation interactions,
We arrived to our research question:
How could we measure and evaluate the impacts generated for and by mega event projects, taking into account the intangible assets and resources, with a focus on future value creation (legacies)?
To answer the research question, we decided to apply a qualitative and strategic design with basis on the intangibles approach supported on the following tree hypotheses:
For the first two hypothesis, we have already found evidence in the literature review. And for the third one...
We decided to develop a diagnostic model for measurement and evaluation of the mega event projects impacts and legacies, taking into account the intangible assets
To achieve the objective, we’ve decided on conduct the research in tree main phases: Concept definition, model development and model validation.
The concept definition phase is composed by a critical literature review to:
Identify the mega projects main characteristics, issues and benefits
Explore the strengths and weaknesses of the established frameworks for measure the mega event projects impacts and legacies
Reflect about the paradigm changes on the stakeholders’ value and performance perception
Identify the fundamentals of the intangibles
During the development phase, our concern was to identify what would be the managerially relevant intangible assets, the interdependencies between then, the success factors generated for and by the mega event projects, and the variables that should be collected and taking into account
To try guarantee the external validity, the strategic approach of the study and to prevent the lack of a real-life event context and a well defined objective to the impact analysis we took the decision of running the model development phase in a research case study
Such strategy is widely used in organizational and managerial studies and across the social sciences when the investigators are interested in understand complex social phenomena (Kohlbacher, 2006; Yin, 2003).
So, we are conducting a case study on the measurement of the impacts on the intangible assets and outcomes of the FIFA 2014 World Cup interventions on the Rio de Janeiro city Tourism industry.
In a mixed approach on intangibles
Based on the Intangible Capital Rating model, proposed by Deutscher and Cavalcanti and the Intellectual Capital Dynamic Value, proposed by Bounfour
Among the different options of models available on the literature,
we chose the Intangible Capital Rating model due its focus on the identification of the assets which an organizations should have to implement their future vision.
On the other hand, the Intellectual Capital Dynamic Value was chose because of the link done between the financial value of assets (USING THE PROXY VALUE = THE EXPENDITURES) and the internal performance
Then, the concept of the operational version of the Mega Event Intangibles Impacts Model consist of 5 dimensions of the intangible Capitals, 15 Assets (including resources, competencies and processes) and 42 Indicators
Internally to the mega event project we have = The strategic, the human and the structural capitals.
On the borderline = The relationship capital
And externally = The Ecosystem in which the mega event project operates
Each capital’s dimension incorporates a group of assets (resources, competencies and processes) we think relevant for value creation.
For each asset was proposed at least one indicator to characterize it
The data collection effort has consisted of interviews with FIFA 2014 World Cup stakeholders directly and indirectly impacted by the interventions on the Rio de Janeiro city tourism industry
The interviews have been conducted within two groups of analysis: the internal stakeholders i.e. Decision-makers and local project management staff from government bodies and the external stakeholders, impacted directly in tourism industry business and indirectly in other sectors business associations and agencies
The focus of such interviews is gathering information about the stakeholders perceptions of the performance (efficiency and effectiveness) and impacts of the interventions on the intangible assets and outcomes listed as main sources of competitive advantage for the achievement of its strategic objectives
The data collection have been taken in 3 steps:
1st step - The evaluation of the impacts on the listed assets has been carried out from a matrix in which the effect was distributed into quartiles to both the positive and negative side.
Starting from zero (no impact) until the maximum theoretical potential of improvement (+2) or worsening (-2), each interviewed has a 5-point scale to evaluate his/her performance perceptions in a positive or negative perspective to assign the notes for each of the 42 indicators.
2nd step - Besides asking for the evaluation of the effects of the FIFA 2014 World Cup interventions on each asset, the interviewer must understand the foundations of the perceptions. The indicators should not necessarily be quantitative, but should represent an observable phenomena. So, for each indicator was formulated one question with such regard
And finally, as 3nr step, In order to identify the ideal competitive positioning according the view of each stakeholder we asked him/her to indicate the degree of relative importance of each dimension of intangible capital, asset and indicator from the assignment of weights
This way, the MEI2 model returns 3 different answers:
a) The Degree of importance (the relative value) for each dimension of intangible capitals by weight assignment
The Performance Ratings of the FIFA 2014 World Cup interventions on the Rio de Janeiro city Tourism industry, in a overall fashion and in respect with each listed capital and asset.
The Dynamic Value of the Intangible capital when we link the performance ratings with the financial value of the assets by using the interventions expenditures as a proxy.
A quarter (25%) of the overall model weight was attributed to human capital, followed by the strategic capital (23,8%)