This paper outlines a study conducted in Mombasa, Kenya where real-time consumer data collection techniques (also known as known as big data, real-time data, crowdsourced data or open source data) were used to prove or disprove hypothesis about macroeconomic trends. It concludes that there are many reasons to feel confident that these techniques may serve as sufficient alternatives for economic forecasts in countries where traditional means of microeconomic data collection are sparse due to poor infrastructure and other circumstance. Further research is needed to verify the repeatability of these findings and the methods soundness statistically.
A summary presentation can be found here - http://www.slideshare.net/jongos1/predicting-macrodeck
The Triple Threat | Article on Global Resession | Harsh Kumar
Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection [Paper]
1. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
Jon Gosier
D8A Group, LLC
Conducted on Behalf of Market Atlas, LLC
Telephone: (+1) 520-301-7906; jon@d8a.com
Abstract: The ultimate goal of this project is to see if there are strong
correlations that can be found between real-time consumer spending patterns and
macro-economic trends and market fluctuations in African countries. Such
methodologies, if proven to be reliable and consistent, would offer a new way
investment decisions can be made as it relates to Africa and other emerging market
countries which suffer from poor private sector visibility and financial infrastructure.
This paper outlines a study conducted in Mombasa, Kenya where real-time
consumer data collection techniques (also known as known as big data, real-time data,
crowdsourced data or open source data) were used to prove or disprove hypothesis
about macroeconomic trends. It concludes that there are many reasons to feel
confident that these techniques may serve as sufficient alternatives for economic
forecasts in countries where traditional means of microeconomic data collection are
sparse due to poor infrastructure and other circumstance. Further research is needed to
verify the repeatability of these findings and the methods soundness statistically.
Acknowledgements: The research contained in this report was conducted on
behalf of Market Atlas via a generous grant offered by the John S. and James L.
Knight Foundation. It was a pleasure to conduct this study and I look forward to the
many new experiments it leads to. The author would also like to thank Justin Mahwikizi
and Akin Sawyerr of Market Atlas for their support.
2. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
2
Introduction
The ultimate goal of this project is to see if there are strong correlations that can be
found between real-time consumer market spending patterns and macro-economic
trends and market fluctuations in African countries.
As part of these experiments we’ve coined a new term to refer to the economic
indicator that this type of micro-economic data represents. This term, Real-Time
Consumer Spending (RTCS), will be used throughout this document.
If successful, our methodology will indicate a new way investment decisions can be
made as it relates to Africa and other emerging market countries which suffer from poor
private sector visibility and financial infrastructure.
It is our hope that by making Africa more attractive to private equity investors, more
trade will occur and more jobs and wealth will created on the continent as a result.
3. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
3
Methodology
Real-Time Consumer Spending (RTCS)
is the term the author uses for
collecting data from populations by
attempting to use a statistically
relevant sample of data providers to
represent larger population trends.
While the goal is to eventually do this
through SMS, due to privacy concerns,
we hired individuals who visited
various vendors and shop owners and
simply surveyed them face-to-face.
Data was verified by viewing any
records the shop owners may have
kept.
Since the price of goods can be
subject to bias, the author decided to
focus on volume of units sold per
vendor and how that changed from
month to month. It’s the author’s
assumption that these monthly
changes strongly correlate with consumer demand and therefore can serve as the
microeconomic indicator for consumer demand. If we can collect enough RTCS data
from enough statistically representative sources, we can test for strong correlations with
macroeconomic indicators like Gross Domestic Product, Purchasing Power Parity,
Inflation, Foreign Direct Investment, Debt and others. Then we may be able to test for
causation which ultimately allows for creating predictive models.
This first half of the experiment is limited to collecting RTCS data and using it to prove
or disprove hypotheses about macroeconomic trends.
4. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
4
Profile of Data Sources
Cell Phone Vendors
Shop Location
Mombasa Town CBD
Who are the Customers?
People from the rural areas buy these phones in dozen(s) and sell them in retail in the
rural area. People living in Mombasa town and the surrounding area.
Way of carrying out the business?
The owner of the shop has specific customers those who buy in dozens.
The shop owners has a small tent outside the shop which plays music and advertise the
phones. The owners says phones are sold with a discounted prices when they are
advertising the phone.
5. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
5
SIM Card Resellers
Shop Location
Mombasa Town CBD
Who are the Customers?
Most customers are people who offer Mobile Money Transfer (Mpesa). Other customers
are Individuals who want their SIM cards replaced.
Way of carrying out the business?
The owner of this shop does not have much advertisement. Only a label on the door
that reads “wholesale and retail SIM cards”. This shop also sells Airtime, retail phones,
and other phone accessories.
6. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
6
Fruit Vendor
Shop Location
Mombasa. Market Place Kongowea
Who are the Customers?
Most customers buy fruit in bulk and then resell the fruit via their kiosks. Other
customers are hotels and restaurants.
Way of carrying out the business?
The owner has an open space where he conducts his business.
7. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
7
Meat Vendor
Shop Location
Mombasa Town
Who are the Customers?
Customers are individuals who buy meat for their families or themselves. Others
customers are hotels and restaurants.
Way of carrying out the business?
The owner has a rental house where he conducts his business.
8. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
8
Grains/Rice Vendor
Shop Location
Mombasa Town
Who are the Customers?
Most customers are people who buy grains (rice) in bulk and resell them in small
quantities in shops. Other customers are Hotels and Restaurants
Way of carrying out the business?
The owner has a rental house where he conducts his business.
9. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
9
General Store
Source of information
The owner of the business
Shop Location
Mombasa. Tudor Estate
Who are the Customers?
Most customers are individuals who live very close the shop.
Other customers are students who from the schools around the store.
Way of carrying out the business?
The owner ahas a rental house where he conducts his business
10. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
10
Clothes Vendors
Shop Location
Mombasa
Who are the Customers?
Customers are shop owners and entrepreneurs who buy several pieces of clothes for
resell. Other customers are individuals who buy for their own necessity.
Way of carrying out the business?
The owner has a rental house where he conducts his business
11. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
11
Kenya Macroeconomic Data
These are some macro economic trends and indicators.
Annual 1
2005 2007 2009 2011 2013 Trend
GDP ($ billions) $18.7 $31.9 $37.0 $41.9 $55.2 ↑
GDP (2-year growth) 25.5% 70.59% 15.99% 13.24% 31.74% ↑
GDP (growth rate) 5.91% 6.99% 2.74% 4.42% 4.69% ↓
GDP (per capita) $523.61 $721.46 $771.29 $816.44 $994.31 ↑
Real Interest Rate 7.6% 5.0% 2.8% 3.8% 10.9% ↑
Consumer Price
Index
72.57 80.24 102.09 121.17 140.11 ↑
Inflation (consumer
prices annual %)
10.3% 9.8% 9.2% 14.0% 5.7% ↓
Monthly 2
NOV DEC Trend
Inflation Rate 6.43% 6.09% ↓
Food Inflation 8.16% 7.54% ↓
Consumer Price
Index
151.92 pts 151.85 pts ↓
CPI (% change) -0.21% -0.05% ↓
1
http://data.worldbank.org
2
http://www.tradingeconomics.com
3
“The
Shadow
Economy
and
Work
in
the
Shadow:
What
Do
We
(Not)
Know?”
Friedrich
Schneider,
Forschungsinstitut
zur
Zukunft
der
Arbeit
Institute
for
the
Study
of
Labor,
March
2012.
2
http://www.tradingeconomics.com
12. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
12
Kenya Forecast Data
Forecasts of Kenya’s various economic indicators. The forecast data isn’t used in this
experiment but is included because historic and current observations should have some
bearing on future trends and projections.
Will RTCS data also offer new ways of projecting macroeconomic data?
Annual Forecast Data
Indicator 2020 2030 Trend
GDP (billions) $55.71 $57.46 ↑
GDP (growth rate) 3.09% 3.09% -
GDP (per capita) $625.00 $625.00 -
Inflation 7.33% 7.30% ↓
Interest Rate 8.9% 10.85% ↑
Stock Market 4997 5755 ↑
13. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
13
Collected Microeconomic Data
The RTCS microeconomic data collected was from small scale vendors around
Mombasa, Kenya.
Data collection isn’t wide spread enough or on a long enough time scale to make any
definitive conclusions but the emerging trends would be interesting if they continue to
hold.
Monthly Consumer Purchases (Mombasa)
Vendor Type NOV-14 DEC-14 Trend Change % Over
Inflation
Clothes 200 211 ↑ 5.5% ↓
Everything 8.16 7.54 ↓ -7.59% ↓
Grains/Rice 900 910 ↑ 1.11% ↓
Meat 450 390 ↓ -13.33% ↓
Fruit 19000 21774 ↑ 14.6% ↑
Cell Phones 44 27 ↓ -38.64 ↓
SIM Cards 300 100 ↓ -66.66% ↓
All 2986.02 3345.65 ↓ -15.00% ↓
14. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
14
Lessons Learned
We've learned a lot that has changed the general methodology of this project to be
more practical and scalable.
• Small scale shop owners in principle do not want to share daily volume of sale
data with any third-party, regardless of the intent. This is because the majority
of them are not paying taxes which is illegal. There was always some level of
suspicion of why anyone would ask for this type of information. Their fear was that
it would be reported to the government and they would be penalized for their
disclosure. Even if it was deliberately reported, it might accidentally leak. Even
after assuring them that the data was anonymized and that the project was not for
the government, not enough would participate.
• This ‘shadow’ economic activity accounts for 40.2% of all economic activity
according to a 2012 report.3
This shadow economy (those who do not pay taxes,
the black market, and cash-based ecosystems) may account for more economic
activity than researchers have yet to truly understand.
• Real-Time Consumer Spending data collection is a cost-efficient means of
sampling this shadow economy.
• By looking at the cost to the vendor, we loose some insight to consumer behavior
because costs to consumer is not always determined by macroeconomic trends.
However, if we look at volume of goods sold and changes in volume, we may
have found a way to avoid this bias.
3
“The
Shadow
Economy
and
Work
in
the
Shadow:
What
Do
We
(Not)
Know?”
Friedrich
Schneider,
Forschungsinstitut
zur
Zukunft
der
Arbeit
Institute
for
the
Study
of
Labor,
March
2012.
http://ftp.iza.org/dp6423.pdf
15. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
15
Conclusions
We've come to a number of conclusions through the course of this experiment that may
tell us much about the link between RTCS and other economic indicators.
Typically, as interest rates are lowered, people have more money to spend and the
economy grows but inflation increases. However, as inflation rises, currency buys a
smaller percentage of a good or service. This should mean if consumer earning doesn’t
outpace inflation, poverty and other social inequalities will increase.
The observations from the charts above tell us…
Trends
GDP ($ billions) ↑
GDP (growth rate) ↓
GDP (per capita) ↑
Real Interest Rate ↑
Consumer Price
Index
↑
Inflation (consumer
prices annual %)
↓
Our observations around consumer spending are as such….
Vendor Type NOV-14 DEC-14 Trend Change % Inflation
Average 2986.02 3345.65 ↓ -15.00% ↓
One would think that because interest rates are up, consumer spending should be
down. Our observations do match this.
However, four weeks of data is hardly a statistical trend. It will take several months of
further data collection to prove any correlations of significant statistical significance.
16. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
16
Is Fruit An Anomaly?
While the Collected Data is not yet enough to be statistically relevant, there are other
indicators to look to in the mean time, for instance the Kenya Consumer Price Index:
The Consumer Price Index (CPI) measures changes in the price consumers pay for a
basket of goods and services. As inflation rises, the CPI should rise because CPI is often
use to calculate inflation. Interestingly enough, CPI was rising for the entirety of 2014
until August when CPI began to hit some volatility.
This parallels the annual trend in inflation.
When we look back to the Monthly Consumer Purchases (Mombasa) chart we can see
a few areas that may prove to be resilient to Inflation and CPI fluctuations. These are
17. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
17
Clothing, Grains/Rice and Fruit. Fruit especially seems to be resilient with fruit vendors
moving volume that has grown month on month at a percentage that is higher than that
of the change in inflation.
Some might conclude that this is because these are ‘essentials’ that people will buy
regardless of economic trends. One might also conclude that the data collected from
Mombasa is too geography specific, unless the data from other regions of the country
match the trends, there’s really nothing learned. While there is no conclusive evidence
that this is the case yet, we will continue to observe the fluctuations for further
correlations. We will also work to raise capital to extend this research to multiple
locations across Kenya.
18. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
18
Can Microeconomic Data Predict the Market?
Another interesting observation is the apparent link between these various trends and
the performance of the Kenyan Stock Market.
Kenyan Stock Market Performance (2005 to 2013)
Kenyan Stock Market Performance (2013 to 2014)
Looking at the trailing months of Kenyan Stock Market Performance (2013 to 2014),
there does seem to be a loose correlation between consumers spending, other
economic indicators, and the performance of the Nairobi Securities Exchange 20 Share
Index NSE20! This was our original assumption for this grant and should provide
enough encouragement that this research should continue.
19. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
19
Is There A Link Between Inflation and The Market?
Looking at historic performance of the of Kenyan Stock Market Performance (2005 to
2014) on the right and the historic rates of inflation on the left, there seem to be some
parallels in how the market has performed and inflation. Yet as of 2012 these links seem
to have diverged. Was there a correlation to begin with? If so, what changed?
20. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
20
Questions to Investigate
These conclusions lead to new questions for investigation and testing:
• Why does fruit appear to be so resilient in the face of external economic factors
that would suggest otherwise?
• What is the link between the demand for fruit and consumer demand for
clothing and grains/rice?
• Is the apparent correlation between the Consumer Price Index and Kenyan Stock
Market real or is it coincidence?
• Are these findings location specific? Do the trends hold in the northern and
southern parts of the country? What about Urban versus rural?
• Is there a link between the Consumer Price Index and Stock Market
Performance.
• Is there a causal link between Real-Time Consumer Spending (RTPS), Consumer
Price Index and GDP Per Capita? How can this be tested on an ongoing basis?
• Is sampling Real-Time Consumer Spending the best method for measuring the
shadow economy and consumer spending generally?
• In addition to RTCS, there seems to be other indicators that would allow one to
predict macroeconomic trends from microeconomic data. However, which of
these are most statistically sound? What new tests can we perform to repeat and
scale this experiment?
21. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
21
Updated Activities
The activities for our pilot have changed based on the aforementioned findings…
1- We are now sending agents out as a data collectors. They are no longer asking
questions about volume of sale, but rather, price of purchase. This indicates
whether or not the vendors are paying more or less for the items they sell on a
monthly basis (which might indicate inflation).
2- The software developed for collecting the data has also changed. The data is
now collected by mobile survey and added to a simple database that can pushed
data out via API.
3- The graph search features of the core Market Atlas product have been
completed.
22. Predicting Macroeconomic Trends Through Real-Time Mobile Data Collection
22
Next Steps
1- Now that data is successfully being collected from one area (Mombasa, Kenya)
we hope to replicate this model in other countries to try to get a diverse enough
dataset to draw statistically sound conclusions.
2- I've found numerous parallels between this data collection methodology and
needs of public health organizations, banks, education and other industries which
could point to a big opportunity for life after this pilot concludes. Perhaps there
are private sector market opportunities to explore?