Student Profile Sample - We help schools to connect the data they have, with ...
Ā
Bibliometrics and scientometrics
1. Bibliometrics and
scientometrics
Part I: the classical performance indicators
Pablo Achard - NCCR Affective Sciences
Tuesday, February 24, 2009
2. 1.What are we talking about?
2.Basic indicators
3. Performance indicators at the
micro level
4.Performance indicators at the
meso and macro level
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
3. 1.What are we talking about?
2.Basic indicators
3. Performance indicators at the
micro level
4.Performance indicators at the
meso and macro level
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
4. Deļ¬nitions
āBibliometricsā is introduced by Pritchard in 1969:
āthe application of mathematical and statistical
methods to books and other media of
communicationā
āScientometricsā is the science of measuring and
analyzing science. As such, it includes the bibliometrics of
scientiļ¬c books and articles; but it also takes into
account funding, demography, geography, etc.
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
5. What is it used for
1. Science and Technology Studies (epistemology, science
sociology, science history,ā¦) and the scientiļ¬c study of
idea spread
2. Biblioeconomics
3. Science management
4. Rankings (used by journalists, students, scientists,ā¦)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
6. What is it used for
P. Achard āBibliometrics and scientometricsā
Lindberg, PhD Thesis
Tuesday, February 24, 2009
7. Why is it becoming so
important
1. Availability of large databases
2. Increased use of management tools in the research
administration
3. Globalization of the education market
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
8. What is this talk focusing on?
The managerial use of performance indicators at the
micro level (individuals, groups), and at the meso and
macro levels (from departments to continents)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
9. What is this talk focusing on?
The managerial use of performance indicators at the
micro level (individuals, groups), and at the meso and
macro levels (from departments to continents)
Therefore we will not deal with important topics like:
- The study of idea spread
- Demographics, education, collaborations
- Links between Research and Development, public
P. Achard āBibliometrics and scientometricsā
understanding of science
Tuesday, February 24, 2009
11. 1.What are we talking about?
2.Basic indicators
3. Performance indicators at the
micro level
4.Performance indicators at the
meso and macro level
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
12. Direct input indicators
Number of scientists
ā¢
R&D budgets
ā¢
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
13. Direct output indicators
Number of publications / books / abstracts
ā¢
Number of patents
ā¢
Number of PhDs
ā¢
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
14. Inļ¬uence indicators
Measure the āpassiveā effects
Number of citations
ā¢
Number of downloads
ā¢
Pagerank of a website
ā¢
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
15. Number of citations ā quality
The number of citations measures the reception
āif a paper receives 5 or 10 citations a year throughout
several years after its publication, it is very likely that its
content will become integrated into the body of knowledge
of the respective subject ļ¬eld; if, on the other hand, no
reference is made at all to the paper during 5 to 10 years
after publication, it is likely that the results involved do not
contribute essentially to the contemporary scientiļ¬c
paradigm system of the subject ļ¬eld in questionā
P. Achard āBibliometrics and scientometricsā
(Braun et al. 1985)
Tuesday, February 24, 2009
16. Number of citations ā quality
Book evaluation vs citations:
āThe J-shaped distribution of
citednessā (Nicolaisen 2002)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
17. Number of citations: issues
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
18. Number of citations: issues
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
19. Normalized indicators
Number of citations / number of expected citations for
similar publications
The āCrown indicatorā of Leiden University: Normalized
by publication type, year and ļ¬eld
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
20. The myth of delayed
recognition
Papers highly cited only after a period of 5 years = 60
P. Achard āBibliometrics and scientometricsā
out of 450ā000 published in 1980 or 0.013%
Tuesday, February 24, 2009
22. IF
Statistics matter: the mean
value is a very rough
description of a distribution
P. Achard āBibliometrics and scientometricsā
(W. GlƤnzel, 2003)
Tuesday, February 24, 2009
24. Methodological pitfalls
Field delineation
Aggregation level
Time frame
Counting scheme
Data quality
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
25. 1.What are we talking about?
2.Basic indicators
3. Performance indicators at the
micro level
4.Performance indicators at the
meso and macro level
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
26. Warning 1: Mating...
You wouldnāt mary someone based on his/her picture
on a mating website
But if you look for partners on a mating website, you
should care about the picture you upload
(using Photoshop is a matter of personal ethics)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
27. Warning 1: Mating...
You wouldnāt mary someone based on his/her picture
on a mating website
But if you look for partners on a mating website, you
should care about the picture you upload
(using Photoshop is a matter of personal ethics)
Apply the same rules with evaluating someoneās
P. Achard āBibliometrics and scientometricsā
scientiļ¬c value and bibliometrics
Tuesday, February 24, 2009
28. Warning 2: Distortion...
1. Funding agencies and science policy makers want to
get the best science for their money. But there are
more scientists than one can know and more ļ¬elds
than one can understand.
2. They fund / hire preferentially projects / scientists
with the best score on a metric that correlates with the
āqualityā they are looking for.
3. Scientists adapt their publication behavior to
P. Achard āBibliometrics and scientometricsā
increase their score on this metric.
Tuesday, February 24, 2009
29. Individual performance
indicators
Number of publications / books / abstracts
Number of patents
Problem: no indication of the inļ¬uence of the work
Distortions: multiplication of small papers (Least
P. Achard āBibliometrics and scientometricsā
Publishable Unit); āhonoraryā authors
Tuesday, February 24, 2009
30. Individual performance
indicators
Number of citations received
Problems: one article can make it all (highly skewed); too
long to build
Distortions: self-citations; citation exchange
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
31. Individual performance
indicators
H-index:
quot;A scientist has
index h if h of his or
her Np papers have
at least h citations
each and the other
(Np - h) papers have
fewer than h
citations eachquot;
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
32. Individual performance
indicators
H-index:
Problems: very strong
quot;A scientist has
correlation with the
index h if h of his or
number of publications;
her Np papers have
perfect to compare
at least h citations
scientists... at the end of
each and the other
their career!
(Np - h) papers have
fewer than h
Distortions: same as
citations eachquot;
P. Achard āBibliometrics and scientometricsā
publication numbers
Tuesday, February 24, 2009
33. Individual performance
indicators
Many variants of the h-index
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
34. Individual performance
indicators
Number of publications x IF of the journal in which they
where published
Problems: highly noisy; very ļ¬eld-dependent; IF doesnāt
determine future citation
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
35. The ļ¬eld dependency of IF
P. Achard āBibliometrics and scientometricsā
(Leydesdorff, 2008)
Tuesday, February 24, 2009
36. IF-citation correlation?
Number of citations vs
IF of the journal in
which articles were
published for each and
every article of 4
researchers
P. Achard āBibliometrics and scientometricsā
(Seglen, 1997)
Tuesday, February 24, 2009
37. Individual performance
indicators
Number of publications x IF of the journal in which they
where published
Problems: highly noisy; very ļ¬eld-dependent; IF doesnāt
determine future citation
Distortions: editors āhot topicsā; worse at the
P. Achard āBibliometrics and scientometricsā
macroscopic level
Tuesday, February 24, 2009
38. How to increase the IF of your
university?
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
39. How to increase the IF of your
university?
Just cut the departments with low citation rates!
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
40. How to increase the IF of your
university?
Just cut the departments with low citation rates!
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
41. How to increase the IF of your
university?
Just cut the departments with low citation rates!
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
42. How to increase the IF of your
university?
Just cut the departments with low citation rates!
Do you think Iām kidding?
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
43. How to increase the IF of your
university?
Just cut the departments with low citation rates!
Do you think Iām kidding?
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
44. How to increase the IF of your
university?
Just cut the departments with low citation rates!
Do you think Iām kidding?
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
45. Individual performance
indicators
P. Achard āBibliometrics and scientometricsā
www.phdcomics.com
Tuesday, February 24, 2009
46. 1.What are we talking about?
2.Basic indicators
3. Performance indicators at the
micro level
4.Performance indicators at the
meso and macro level
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
47. University rankings
Shanghai = alumni awards score + staff awards score +
highly cited score + Nature and Science articles +
articles + a mixture of all this/number of faculty
Times Higher Education = peer-review score +
employer-review score + staff/student score + citation/
staff score + international student score
UNIGE ranking 2007 = 105 ranking 2008 = 68
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
48. University rankings
Shanghai = alumni awards score + staff awards score +
highly cited score + Nature and Science articles +
articles + a mixture of all this/number of faculty
Times Higher Education = peer-review score +
employer-review score + staff/student score + citation/
staff score + international student score
UNIGE ranking 2007 = 105 ranking 2008 = 68
P. Achard āBibliometrics and scientometricsā
Letās have a look at more serious indicators!
Tuesday, February 24, 2009
52. Publications vs GDP
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
53. Citations vs GDP (normalized)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
54. Field dependency (again)
Clinical medicine (MED)
Biomedical research (BRE)
Biology (BIO)
Chemistry (CHE)
Physics (PHY)
Mathematics (MAT)
Engineering (ENG)
Earth and space sciences (ESS)
P. Achard āBibliometrics and scientometricsā
(GlƤnzel, 2003)
Tuesday, February 24, 2009
55. Citations vs GDP (normalized)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009
57. āThe use of a single index crashes the multidimensional
space of bibliometrics into one single dimension.ā
(Wolfgang GlƤnzel)
āNot everything that can be counted counts,
and not everything that counts can be counted.ā
(Albert Einstein)
P. Achard āBibliometrics and scientometricsā
Tuesday, February 24, 2009