Making Sense of Complicated Systems: Computer Simulation and Public Health

Making Sense of Complicated Systems:
Computer Simulation and Public
Health

Edmund Chattoe-Brown (ecb18@le.ac.uk)
Department of Sociology, University of Leicester

http://www.simian.ac.uk

Thanks

• This research (just about) funded by the Economic
and Social Research Council as part of the
National Centre for Research Methods
(http://www.ncrm.ac.uk).
• The usual disclaimer applies regarding Nigel
Gilbert (co-PI SIMIAN, Sociology, Surrey).

2 http://www.simian.ac.uk

How do we know what is going on?
• Evidence based policy?
• Expertise: But how does one self check?
• Sticking a finger in the air.
• “The ideas of economists and political philosophers, both when
they are right and when they are wrong, are more powerful than
is commonly understood. Indeed the world is ruled by little else.
Practical men, who believe themselves to be quite exempt from
any intellectual influence, are usually the slaves of some defunct
economist. Madmen in authority, who hear voices in the air, are
distilling their frenzy from some academic scribbler of a few
years back.” (J. M. Keynes, The General Theory of Employment,
Interest and Money, 1936)


We all know …
• … social systems are complicated.
• … organisations are even more so.
• … things don’t always turn out how we expect.
• … attempts to make things better regularly have the
opposite effect.
• Can we do anything about this? If so, what?
• For people who have to intervene, this is pretty
important.
• Famously: “When the only tool you have is a hammer,
every problem begins to resemble a nail.” (Abraham
Maslow)


Case study 1: HIV transmission
• How do we find out what is going on at
present? (With a view to trying to do something
about it.)
• A simple “agent-based” model.
• What this model does (and equally importantly
doesn’t do).
• How should this kind of modelling actually be
done?
• Two main approaches at present: Quantitative
and qualitative. (Randomised control trials?
Intervention experiments?)

The quantitative approach
• Woodford, Michael R. et al. (2012) ‘Correlates of HIV Testing
Uptake among Kothi-Identified Men who have Sex with Men in
Public Sex Environments in Chennai, India’, AIDS Behav, 16, pp.
53-62.
• “At the bivariate level, married men, those with low HIV
transmission knowledge, those who engaged in unprotected anal
sex and unprotected receptive anal sex were at lower odds of
reporting testing uptake. In multivariate analysis, married men
and those with low levels of HIV transmission knowledge were at
decreased odds of being tested, as were kothis who experienced
forced sex.” (p. 53)
• “Culturally competent programs engaging married kothis are
needed. Interventions to facilitate HIV prevention education and
systemic interventions to combat sexual violence may facilitate
HIV testing uptake among kothis.” (p. 53)


Challenges
• Association may not be causation: What actually explains the
patterns we observe?
• Where is agency?
• The more you try to “chop the world up” into attributes (age,
gender, marital status, ethnicity) the more data of particular kinds
you need.
• It seems that people “don’t want to know” if a) they are likely to
give it to someone else, b) they had “no choice” but to incur the
risk.
• What is “culturally competent” is therefore the $64,000 question.
(Let us fight the fights that must be fought.)
• Here (in a nutshell) the problem is that we have the big pattern but
aren’t sure what happened during individual interaction and
decision processes to give it. How many theories could produce
what we observe?

The qualitative approach
• Li, Haochu et al. (2010) ‘Sociocultural Facilitators and Barriers to
Condom Use During Anal Sex among Men who have Sex with
Men in Guangzhou, China: An Ethnographic Study’, AIDS Care:
Psychological and Socio-medical Aspects of AIDS/HIV, 22(12),
pp. 1481-1486.
• “Min: Not all of clients used condoms . . . If they feel zai [“money
boys”] are very liang [handsome], sometimes clients will ask how
long the zai have worked. If the zai have not been working for too
long, clients will not use condoms.” (p. 1484)
• “Yu: We did not use a condom because he came from a rural
area. My evaluation was that rural people have less sex … I didn’t
ask him to use a condom.” (p. 1484)
• “Gu: I went to his home. It was a big, double style apartment. We
didn’t use condoms because I felt that he would not be an
unsanitary person, and his body condition was healthy.” (p. 1484)

Challenges
• Here we have lots of micro detail and ideas for “process theory”.
(Risk assessments of potential partners and impact on health
education: “You can’t tell by looking”, “It’s their friends you have to
worry about not them.”)
• However, we have no way of telling what it all adds up to “in large”.
• Should we put our money into condoms or testing? Why?
• Might something that “ought” to help, through interactions in the
system, actually make things worse? (Example: Strict needle
exchange policy stops better organised IDU from redistributing
informally to worse organised IDU. Famously: “The assumptions
you don’t realise you are making are the ones that will do you in.”)
• We just aren’t very good in seeing the implications of complicated
systems by casual reasoning.


What is Agent-based modelling?
• Represent social theories as computer
programmes rather than narratives or equation
systems.
• Try to represent social behaviour “directly”
rather than through theory constructs or
parameters. (Different from older simulation
approaches.)
• Like statistics (and ethnography) you can get a
good idea of it without doing it yourself.
• Tries to combine the rigour of mathematics with
the richness of narratives.

A picture is worth …

NetLogo can
be
downloaded
free for
PC/Mac and
UNIX and
includes this
model


Assumptions
• Only dealing with sexual transmission.
• Distributed chance to form sexual relationship.
• Distributed chance to use condoms.
• Distributed chance for (assumed monogamous)
sexual relationship to break up.
• Distributed chance to test.
• All those who test positive always use condoms
thereafter.
• As you would expect with HIV: No symptoms
initially, won’t show on tests very early.
• All “chances” are normal distributions around the
stated mean.

Very important …
• I don’t believe this model (and can probably see at
least as many things wrong with it as you can) but if
you are showing someone statistics, you don’t start
with with an “all guns blazing” model.
• I’m showing you this to trace out the novelty of the
approach and justify my claims about the method not
as a piece of social science.
• I am not an expert on public health (though I do
know a fair bit about simulation).


What happens?

The job of the
computer,
following the
programme, is for
each agent to
“behave”
according to rules
and probabilities
and the outcome
to be unfolded
over time.


And then?

Recognise the S-
shaped epidemic
curve? Even with
(almost) no
condoms and no
testing, the
distributions mean
that some people
never have sex or
are in very long
relationships.
(This run is about
40 years.)


Is real life an outlier?
• The first novel thing we can do with a simulation is to run it
repeatedly with nothing changed except “the randomness”.
• Using settings of 5 for a-c-t, 52 weeks for a-c and “0” condom
use and testing (this is a kind of baseline of “uncontrolled
spread”), the % infected for 10 runs are: 40.6, 44, 55.7, 44.3,
42.0, 43.4, 47.1, 45.1, 48.0, 46.0,
• This is mean of 45.6% but plus 10 and minus 5!
• Unless you think there is no randomness in reality, don’t be too
impressed with any numerical values you happen to get.
• How big does an intervention effect have to be before you can
pick it out of the “inbuilt” variability?

Nonlinearity I
% Infection After 10 Years Varying Rates of
Condom Usage Should we
conclude
60
condoms
50 don’t
work, the
40
effect isn’t
30 linear or
% Infected
20 that we
might get
10
a proper
0 trend in
0 2 4 6 8 10 12 averages?
Rate of Condom Usage


Nonlinearity II

% Infection After 10 Years Varying Length of Fairly
Relationship
clearly not a
120 straight line.
Incremental
100
change over
80 the wrong
60 ranges may
show little
% 40
Infected
effect.
20

0
0 50 100 150 200 250
Average Length of Relationship (Weeks)


Nonlinearity III
% Infection After 10 Years Varying Again fairly
Chance To Test clearly not a
straight line.
60
Is the
50 effectiveness
40
of testing a
“cheat” on
30
the
20 assumption
% Infected
10
that HIV+
always use
0
0 0.5 1 1.5 2 2.5
condoms?
-10
Chance to Test


Nonlinearity IV
• Baseline case (10 runs) gives average 45.6% infected after 10
years.
• Adding condom use “5” alone (5 runs) reduces this to 40.7%: a
5% “gain”.
• Adding testing “0.25” alone (5 runs) reduces this to 14.5%: a 31%
“gain”.
• Doing both gives 16.6% (a 29% “gain”): Adding two apparently
“good” things appears to makes things (admittedly marginally)
worse.
• Uh-oh? (Or would we rather hope this is all just random noise?)


Things to notice
• Even an agent-based model as simple as this doesn’t produce “straight
line” causation common in social statistics.
• Assumptions of the model can be mapped onto real data (changes in
behaviour with HIV+ result, distribution of condom usages, relationship
lengths and so on.)
• The model is mainly based on process and doesn’t contain theoretical
constructs or “fiddle factors” (though there may be things we don’t yet
know).
• Macro effects (simulated population infection levels over time) can be
mapped against real data.
• If they “match” then maybe these micro assumptions could actually have
“generated” what we see. (Combining qualitative and quantitative
approaches for a kind of falsification. Not just model fitting.)


Thinking about complexity
• Because of the mode of transmission, it doesn’t take many failures to use a
condom within a stable relationship to create another infected partner for
future pairings.
• How dangerous this is to the wider population depends on the length of
relationships.
• We tend to think of condom use in terms of a “lottery” with casual partners
who may or may not be infected (even a little more will help) but the
addition of relationships means that, unless condom use is almost total,
HIV status of partners will tend to coincide. Constant condom use also
goes against the tendency for stable relationships to move to “safer” and
“less fiddly” means of contraception like the pill.
• This is why condom use after testing is so effective because it is assumed
to be motivated differently. (Rather than a “sex lottery” logic it reflects
permanently changed sexual status.)

Backing up a bit
• Very important: These are not policy conclusions because the model is “made up”.
• However, you can see how it would go for a proper model and what is distinctive about
this approach.
• Start with the literature and find out what is best known.
• Build a “version 0” model and explore it.
• Find the mistakes and “puzzles”.
• Talk to experts to try and fill in the data and process gaps even anecdotally.
• Identify what is still not known and collect more data.
• Compare aggregate behaviour with simulation outputs.
• Repeat?
• If this is done properly, the problem is not complicated models (if you can describe it I
can build it) but getting the data (both for calibration and validation). The complexity of
models tends to be limited by the realisation of how much we don’t know. (Needle stock
example.)

Another example: Networks

Simple SIR
model but on a
network with
plausible
“cliques and
bridges”
structure.


Mid run

Island
population
protected by
immunity of
bridging actor.
(Only works
for a static
network of
course.)


End of run
Infection “burns
out” as
network is
partitioned by
resistant agents.
New infections
“from outside”
would cause
new flare ups.


Another example: BBVsim
• Collaboration between modeller and domain experts to build a
teaching tool to show drug educator that interventions are
complex. (Not going to run you through another whole model.)
• First lesson: Establish clearly what counts as plausible model
output.
• Second lesson: Modeller can ask questions experts can’t
answer. (ABM as a “new place to stand”). Example of held
stocks of needles and “sharing etiquette”.
• Third lesson: “Official” targets sometimes make no sense at all.
50% of users with all the needles they need and 100% with half
of what they need constitute the same coverage but could have
very different effects in a complex system.

Impact and serendipity
• Idea originally from organised crime: Traditional (“egocentric”)
social network studies don’t work well with secretive groups. (But
there is extra data like phone taps and observation.)
• “Adding up” lots of subjective networks (even if pretty inaccurate)
can be better than an objective networks with non response.
• Used a simulation to show that, under the right circumstances,
qualitative research can be “better” than quantitative! Simulation
was needed to model how information about networks diffused in
the network: How do you know how well Bob knows Mary?
• NHS Wales are now thinking about how to use simulation and
ideas like this to innovate in contact tracing.
• Interesting ethical issue arises. http://www.simian.ac.uk
28

Conclusions for discussion
• This approach is significantly different from both qualitative and
quantitative research. In particular it can cope with the likely
nonlinearity of social systems.
• It gives a distinctive way of building theories, of testing them, of
focusing data collection and of thinking about systems (including
computational experiments).
• It is relevant to “complicated” public health issues (which is
rather likely to be all of them I suspect).
• The “entry costs” are now rather low (and programmers are quite
cheap as long as you know what you want.)
• There is “plenty of room” for original research.

Randomised control trials
• Most interventions aren’t like a pill that can be active or chalk.
• If there is randomness in the system you may appear to get
exactly the opposite effect from what is really there comparing
single runs!


“Real” (published) examples
• Chattoe-Brown, E. (2009) ‘The Social Transmission of Choice: A Simulation with
Applications to Hegemonic Discourse’, Mind and Society, 8(2), December, pp. 193-207.
• Chattoe, E. (2006) ‘Using Simulation to Develop and Test Functionalist Explanations: A
Case Study of Dynamic Church Membership’, British Journal of Sociology, 57(3),
September, pp. 379-397.
• Chattoe, Edmund and Hamill, Heather (2005) 'It's Not Who You Know - It's What You
Know About People You Don't Know That Counts: Extending the Analysis of Crime
Groups as Social Networks', British Journal of Criminology, 45(6), November, pp.
860-876.
• Chattoe, Edmund and Gilbert, Nigel (1997) 'A Simulation of Adaptation Mechanisms in
Budgetary Decision Making', in Conte, Rosaria, Hegselmann, Rainer and Terna, Pietro
(eds.) Simulating Social Phenomena, Lecture Notes in Economics and Mathematical
Systems 456 (Berlin: Springer-Verlag), pp. 401-418.
• Blood Borne Virus Simulator <http://www.bbvsim.org.uk/>. [For training.]
• Chattoe, Edmund, Hickman, Matthew and Vickerman, Peter (2005) Drug Futures
2025? Modelling Drug Use (London: DTI Foresight/Office of Science and Technology),
<http://www.bis.gov.uk/assets/foresight/docs/brain-science/dti-modelling.pdf>.
[Replication.]

Now what?
• NetLogo (software mainly used, free, works on
Mac/PC/Unix, with a nice library of examples):
<http://ccl.northwestern.edu/netlogo/>.
• Simulation for the Social Scientist, 2nd edition, 2005,
Gilbert/Troitzsch. [Don’t get first edition, not in NL!]
• Agent-Based Models, 2007, Gilbert.
• Journal of Artificial Societies and Social Simulation
(JASSS): <http://jasss.soc.surrey.ac.uk/JASSS.html>. [Free
online, interdisciplinary and peer reviewed.]
• simsoc (email discussion group for the social simulation
community): <https://www.jiscmail.ac.uk/cgi-bin/webadmin?
A0=SIMSOC>.

Making Sense of Complicated Systems: Computer Simulation and Public Health

Recommandé

Recommandé

Contenu connexe

Similaire à Making Sense of Complicated Systems: Computer Simulation and Public Health

Similaire à Making Sense of Complicated Systems: Computer Simulation and Public Health (20)

Plus de Edmund Chattoe-Brown

Plus de Edmund Chattoe-Brown (20)

Dernier

Dernier (20)

Making Sense of Complicated Systems: Computer Simulation and Public Health