Statistical methods for questionnaire development: Questionnaire reliability analysis, Factor Analysis, and Principal Component Analysis

1. Statistical Methods in
Questionnaire Development
Ahmed Said Negida
School of Medicine, Zagazig University, Egypt

2. About the lecturer
AHMED NEGIDA
• Final Year Medical Student (Dec 2018)
• Early career researcher
• Founder of MRGE (national research team)
• Published ˃ 40 research papers
• Co-author of 2 international book chapters
• Trained 750 researchers in Egypt since 2014

3. Learning objectives
• Reliability vs. Validity vs. Accuracy
• Reliability
• Validity

4. Reliability
Cronbach’s Alpha
The most widely used measure for reliability

5. Cronbach’s Alpha
How it is calculated?
• n = number of questions
• Vi = variance of scores on each question
• Vtest = total variance of overall scores on the entire test





 



Vtest
Vi
n
n
1
1


6. Cronbach’s Alpha on SPSS

7. Cronbach’s Alpha on SPSS

8. Cronbach’s Alpha on SPSS

9. Cronbach’s Alpha on SPSS

10. Cronbach’s Alpha on SPSS

11. Cronbach’s Alpha on SPSS

12. Cronbach’s Alpha on SPSS

13. Cronbach’s Alpha on SPSS

14. Cronbach’s Alpha on SPSS

17. Writing Results of Reliability Analysis

18. Writing Results of Reliability Analysis

19. Principal Component Analysis
Principal components analysis (PCA, for short) is a variable-reduction
technique that shares many similarities to exploratory factor analysis. Its
aim is to reduce a larger set of variables into a smaller set of 'articifial'
variables, called 'principal components', which account for most of the
variance in the original variables.

20. Uses of the Principal Component Analysis
(a) you have measured many variables (e.g., 7-8 variables, represented as 7-8 questions/statements in a
questionnaire) and you believe that some of the variables are measuring the same underlying construct
(e.g., depression). If these variables are highly correlated, you might want to include only those variables
in your measurement scale (e.g., your questionnaire) that you feel most closely represent the construct,
removing the others
(b) you want to create a new measurement scale (e.g., a questionnaire), but are unsure whether all the
variables you have included measure the construct you are interested in (e.g., depression). Therefore,
you test whether the construct you are measuring 'loads' onto all (or just some) of your variables. This
helps you understand whether some of the variables you have chosen are not sufficiently representative
of the construct you are interested in, and should be removed from your new measurement scale
(c) you want to test whether an existing measurement scale (e.g., a questionnaire) can be shortened to
include fewer items (e.g., questions/statements), perhaps because such items may be superfluous (i.e.,
more than one item may be measuring the same construct) and/or there may be the desire to create a
measurement scale that is more likely to be completed (i.e., response rates tend to be higher in shorter
questionnaires).

21. Requirements of the PCA
1. Sample Adequacy
2. Type of Data (Continuous)
3. No outliers
4. All in the same direction
5. Linear correlation between the items

22. Requirements of the PCA
Are items correlated?

23. Requirements of the PCA
Is the sample size adequate?
Kaiser-Meyer-Olkin
Measure of Sampling Adequacy

24. Requirements of the PCA
Is the sample size adequate?
Kaiser-Meyer-Olkin Measure of Sampling Adequacy
• KMO values between 0.8 and 1 indicate the sampling is adequate.
• KMO values less than 0.6 indicate the sampling is not adequate and that
remedial action should be taken. Some authors put this value at 0.5, so use
your own judgment for values between 0.5 and 0.6.
• KMO Values close to zero means a large problem for factor analysis.

25. Kaiser-Meyer-Olkin (KMO) Test

26. Kaiser-Meyer-Olkin (KMO) Test

27. Kaiser-Meyer-Olkin (KMO) Test
• KMO values between 0.8 and 1 indicate the sampling is adequate.
• KMO values less than 0.6 indicate the sampling is not adequate and
that remedial action should be taken. Some authors put this value at
0.5, so use your own judgment for values between 0.5 and 0.6.
• KMO Values close to zero means a large problem for factor analysis.
1. Bartlett's Test of Sphericity P<0.001 (Interpretation: the correlation
matrix is not an identity matrix; the correlation matrix shows unique
correlations)

28. Bartlett's Test of Sphericity
Bartlett's Test of Sphericity P<0.001
Interpretation: the correlation matrix is not an identity matrix;
the correlation matrix shows unique correlations

29. Factor Extraction
The most common method
eigenvalue >1 approach

30. Requirements of the PCA
Is the sample size adequate?
Kaiser-Meyer-Olkin Measure of Sampling Adequacy
• KMO values between 0.8 and 1 indicate the sampling is adequate.
• KMO values less than 0.6 indicate the sampling is not adequate and that
remedial action should be taken. Some authors put this value at 0.5, so use
your own judgment for values between 0.5 and 0.6.
• KMO Values close to zero means a large problem for factor analysis.

31. Factor Analysis

32. Total variance explained
Total variance explained this indicates how much of the variability in
the data has been modelled by the extracted factors.

33. Total variance explained

34. SCREE PLOT

35. Component Matrix

36. Component Matrix