European Journal of Adapted Physical Activity 13, 2 (2020) | DOI: 10.5507/euj.2019.014

Cognitive interviews with children as a research tool for instrument validation in adapted physical activity

Nancy L. I. Spencer1, Marcel Bouffard1, E. Jane Watkinson1
1 Faculty of Kinesiology, Sport, and Recreation, University of Alberta, Canada

The development and use of instruments to assess individuals with impairments in various domains is common practice in the field of Adapted Physical Activity. Test developers and users often question the validity of their instruments and use different conceptual and/or empirical strategies for validation purposes. One validation strategy, still rarely used in the sport sciences, is cognitive interviews with participants. This study is an attempt to show the utility of cognitive interviews for instrument development with children with relevance to the field of adapted physical activity.  Specifically, we investigated the question-and-answer processes of children with impairments when responding to the Athletic Competence Domain Subscale from Harter’s (1985) Self-Perception Profile for Children. Eight children with different diagnoses (ages 8-13 years) took part in cognitive interviews. The study revealed sources of validity and invalidity in the instrument. Differences and concerns in the children’s comprehension of questions and interpretation of words, leading to potentially limited and varied sources of information for response production, emerged. While judgment generally appeared to be unproblematic, topic sensitivity and limited response options were a common constraint. The data obtained from this study could be used to further refine the instrument.

Keywords: perceived competence; questionnaire research; validity

Received: January 8, 2019; Accepted: October 11, 2019; Published: April 30, 2020  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Spencer, N.L.I., Bouffard, M., & Jane Watkinson, E. (2020). Cognitive interviews with children as a research tool for instrument validation in adapted physical activity. European Journal of Adapted Physical Activity, 13(1), 2. doi: 10.5507/euj.2019.014
Share...
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287-311. doi:10.1093/poq/nfm006 Go to original source...
    2. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge, MA: Cambridge University Press. Go to original source...
    3. Bouffard, M., & Spencer-Cavaliere, N. (2016). Interdisciplinarity in Adapted Physical Activity. Quest, 68, 4-14. doi:10.1080/00336297.2015.1117002 Go to original source...
    4. Branch, J. L. (2006). Using think alouds, think afters, and think togethers to research adolescents' inquiry experiences. Alberta Journal of Educational Research, 52, p. 148-159. https://journalhosting.ucalgary.ca/index.php/ajer/article/view/55153 Go to original source...
    5. Bredhal, A.-M. (2008). Ethical aspects in research in adapted physical activity. Sport, Ethics, and Philosophy, 2, 257-270. doi:10.1080/17511320802223881 Go to original source...
    6. Brian, A., De Meester, A., Klavina, A., Irwin, J. M., Taunton, S., Pennell, A., & Lieberman, L. (2019). Exploring children/adolescents with visual impairments' physical literacy: A preliminary investigation of autonomous motivation. Journal of Teaching in Physical Education, 38, 155-161. doi: 10.1123/jtpe.2018-0194 Go to original source...
    7. Brian, A., Haegele, J.A., & Bostick, L. (2016). Perceived motor competence of children with visual impairments: A preliminary investigation. British Journal of Visual Impairment, 34, 151-155. doi:10.1177/ 0264619616628575 Go to original source...
    8. Brian, A., Taunton, S., Haibach-Beach, P., & Lieberman, L.J. (2018). Influence of a sports camp on perceived physical competence of children who are visually impaired. Journal of Visual Impairment & Blindness, 112, 505-509. doi: 10.1177/0145482X1811200508 Go to original source...
    9. Causgrove Dunn, J., & Dunn, J. G. H. (2006). Perceptions of the motivational climate, perceived competence, and participation behaviors of children with movement difficulties in physical education. Adapted Physical Activity Quarterly, 23, 293-309. doi: 10.1123/apaq.23.3.293 Go to original source...
    10. Castillo-Díaz, M., & Padilla, J.-L. (2013). How cognitive interviewing can provide validity evidence of the response processes to scale items. Social Indicators Research, 114, 963-975. doi: 10.1007/s11205-012-0184-8 Go to original source...
    11. Collins, D. (2015). Cognitive interviewing: Origin, purpose and limitations. In, Collins, D. (Ed.), Cognitive interviewing practice (2nd ed., pp. 3-27). Thousand Oaks, CA: Sage. Go to original source...
    12. Danermark, B., Ekström, M., Jakobsen, L., & Karlsson, J.C. (2002). Explaining society: Critical realism in the social sciences. New York: Routledge.
    13. Dietrich, H., & Ehrlenspiel, F. (2010). Cognitive interviewing: A qualitative tool for improving questionnaires in sport science. Measurement in Physical Education and Exercise Science, 14, 51-60. doi:10.1080/10913670903455025 Go to original source...
    14. Dockett, S., Einarsdottir, J., & Perry, B. (2009). Researching with children: Ethical tensions. Early Childhood Research, 7, 283-298. doi: 10.1177/1476718X09336971 Go to original source...
    15. Ericsson, K. A. & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press. Go to original source...
    16. Groves, R. M., Fowler, F. J., Jr., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed.). New Jersey: John Wiley & Sons Inc.
    17. Harter, S. (1978). Effectance motivation reconsidered: Toward a developmental model. Human Development, 21, 34-64. doi:10.1159/000271574 Go to original source...
    18. Harter, S. (1981a). A model of mastery motivation in children: Individual differences and developmental change. In W. A. Collins (Ed.), Minnesota Symposium on Child Psychology, (Vol. 14, pp. 215-255). Hillsdale, NJ: Erlbaum.
    19. Harter, S. (1981b). A new self-report scale of intrinsic versus extrinsic orientation in the classroom: Motivational and informational components. Developmental Psychology, 17, 300-312. doi:10.1037/0012-1649.17.3.300 Go to original source...
    20. Harter, S. (1982). The perceived competence scale for children. Child Development, 53, 87-97. doi: 10.2307/1129640 Go to original source...
    21. Harter, S. (1985). Manual for the Self-Perception Profile for Children. University of Denver. Go to original source...
    22. Harter, S. (1990). Issues in the assessment of the self-concept of children and adolescents. In A. M. La Greca (Ed.), Through the eyes of the child: Obtaining self-reports from children (pp. 292-325). Boston: Allyn and Bacon:
    23. Harter, S. (1999). The construction of the self: A developmental perspective. New York, NY: The Guilford Press.
    24. Horvat, M., Block, M. E., & Kelly, L. E. (2007). Developmental and adapted physical activity assessment. Champaign, IL: Human Kinetics.
    25. Knafl, K., Deatrick, J., Gallo, A., Holcombe, G., Bakitas, M., Dixon, J., & Grey, M. (2007). The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health, 30, 224-234. doi: 10.1002/nur.20195 Go to original source... Go to PubMed...
    26. Leary, J. M., Ice, C., & Cottrell, L. (2012). Adaptation and cognitive testing of physical activity measures for use with young, school-aged children and their parents. Quality of Life Research, 21, 1815-1828. doi: 10.1007/s11136-011-0095-1 Go to original source... Go to PubMed...
    27. Messick, S. (1989). Validity. In R. L. Linn, Educational measurement (3rd ed.). NewYork: Macmillan Publishing Company.
    28. Neilson, H. K., Ullman, R., Robson, P. J., Friedenreich, C. M., & Csizmadi, I. (2013). Cognitive testing of the STAR-Q: Insights in activity and sedentary time reporting. Journal of Physical Activity and Health, 10, 379-389. doi: 10.1123/jpah.10.3.379 Go to original source... Go to PubMed...
    29. Oberg, D., & Ellis, J. (2006). Exploring the new paradigm for researching with children and youth. Alberta Journal of Educational Research, 52(3), 107-110. https://journalhosting.ucalgary.ca/index.php/ajer/article/view/55150 Go to original source...
    30. Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications.
    31. Renick, M. J., & Harter, S. (1989). Impact of social comparisons on the developing self perceptions of learning disabled students. Journal of Educational Psychology, 81, 631-638. doi:10.1037/0022-0663.81.4.631 Go to original source...
    32. Schaeffer, N. C., & Presser, S. (2003). The science of asking questions. Annual Review of Sociology, 29, 65-88. doi: 10.1146/annurev.soc.29.110702.110112 Go to original source...
    33. Schuengel, C., Voorman, J., Stolk, J., Dallmeijer, A., Vermeer, A., & Becher, J. (2006). Self-worth, perceived competence, and behaviour problems in children with cerebral palsy. Disability and Rehabilitation, 28, 1251-1258. doi:10.1080/09638280600554652 Go to original source... Go to PubMed...
    34. Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93-105. doi:10.1037/0003-066X.54.2.93 Go to original source...
    35. Shapiro, D. R., Moffett, A., Lieberman, L., & Dummer, G. M. (2005). Perceived competence of children with visual impairments. Journal of Visual Impairment & Blindness, 99, 15-25. doi: 10.1177/0145482X0509900103 Go to original source...
    36. Shapiro, D. R., & Ulrich, D. A. (2002). Expectancies, values, and perceptions of physical competence of children with and without learning disabilities. Adapted Physical Activity Quarterly, 19, 318-333. doi: 10.1123/apaq.19.3.318 Go to original source... Go to PubMed...
    37. Silon, E. L., & Harter, S. (1985). Assessment of perceived competence, motivational orientation, and anxiety in segregated and mainstreamed educable mentally retarded children. Journal of Educational Psychology, 77, 217-230. doi:10.1037/0022-0663.77.2.217 Go to original source...
    38. Spencer-Cavaliere, N., Thai, J., & Kingsley, B. (2017). A part of and apart from sport: Practitioner's experiences coaching in segregated youth sport. Social Inclusion, 5, 120-129. doi:10.17645/si.v5i2.889 Go to original source...
    39. Spencer-Cavaliere, N., & Watkinson, E. J. (2010). Inclusion understood from the perspectives of children with disability. Adapted Physical Activity Quarterly, 27, 275-293. doi: 10.1123/apaq.27.4.275 Go to original source... Go to PubMed...
    40. Tourangeau, R. (1984). Cognitive science and survey methods. In T. Jabine, M. Straf, J. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey design: Building a bridge between disciplines (pp. 73-100). Washington, DC: National Academy Press.
    41. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge, UK: Cambridge University Press. Go to original source...
    42. Tsalavoutas, I., & Reid, G. (2006). Competence satisfaction: Risk taking and achievement. Adapted Physical Activity Quarterly, 23, 410-423. doi: 10.1123/apaq.23.4.410 Go to original source...
    43. Ullrich-French, S., Cox, A., Cole, A., Roades Cooper, B., & Gotch, C. (2017). Initial validity evidence for the state mindfulness scale for physical activity with youth. Measurement in Physical Education and Exercise Science, 21, 177-189. doi: 10.1080/1091367X.2017.1321543 Go to original source...
    44. Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage Publications. Go to original source...
    45. Willis, G. B. (2015). Analysis of the cognitive interview in questionnaire design. New York, NY: Oxford University Press.
    46. Woolley, M. E., Bowen, G. L., & Bowen, N. K. (2004). Cognitive pretesting and the developmental validity of child self-report instruments: Theory and applications. Research on Social Work Practice, 14, 191-200. doi:10.1177/1049731503257882 Go to original source... Go to PubMed...
    47. Wynnyk, K., & Spencer-Cavaliere, N. (2013). Children's social relationships and motivation in sledge hockey. Adapted Physical Activity Quarterly, 30, 299-316. doi: 10.1123/apaq.30.4.299 Go to original source... Go to PubMed...
    48. Yun, J, & Ulrich, D. A. (1997). Perceived and actual physical competence in children with mild mental retardation. Adapted Physical Activity Quarterly, 14, 285-297. doi: 10.1123/apaq.14.4.285 Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content