Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory
Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, unt...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The Evaluation Center at Western Michigan University
2020-04-01
|
| Series: | Journal of MultiDisciplinary Evaluation |
| Subjects: | |
| Online Access: | https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/591 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850216848723804160 |
|---|---|
| author | Jason S. McIntosh Jane Buckley Thomas Archibald |
| author_facet | Jason S. McIntosh Jane Buckley Thomas Archibald |
| author_sort | Jason S. McIntosh |
| collection | DOAJ |
| description |
Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, until recently little empirical inquiry on the dimensions of evaluative thinking has been conducted.
Purpose: To address that lack, the purpose of the study presented in this paper is to refine the construct of evaluative thinking by exploring its underlying dimensions and to ascertain the internal consistency of an instrument developed to measure evaluative thinking, the Evaluative Thinking Inventory (ETI).
Setting: The ETI was developed as part of an ECB initiative focused on non-formal science, engineering, technology, and math (STEM) education in the United States, and was tested as part of a study focused on evaluating gifted education programs, also in the United States.
Intervention: Not applicable.
Research design: Survey research and exploratory factor analysis (EFA).
Data collection & analysis: The ETI was administered to participants in a study measuring the effectiveness of a tool used to conduct internal evaluations of gifted education programs. SPSS was used to conduct an EFA on 96 completed ETIs. Cronbach’s alpha was used to estimate the internal consistency of the instrument.
Findings: The analysis of the ETI revealed a two-factor model of evaluative thinking (i.e., believe in and practice evaluation and pose thoughtful questions and seek alternatives). This study also provided internal consistency evidence for the ETI showing alpha reliabilities for the two factors ranging from 0.80 to 0.82. The ETI has potentially wide applicability in research and practice in ECB and in the field of evaluation more generally.
|
| format | Article |
| id | doaj-art-0d5041aed0364b81a1a59eefbc057b68 |
| institution | OA Journals |
| issn | 1556-8180 |
| language | English |
| publishDate | 2020-04-01 |
| publisher | The Evaluation Center at Western Michigan University |
| record_format | Article |
| series | Journal of MultiDisciplinary Evaluation |
| spelling | doaj-art-0d5041aed0364b81a1a59eefbc057b682025-08-20T02:08:12ZengThe Evaluation Center at Western Michigan UniversityJournal of MultiDisciplinary Evaluation1556-81802020-04-01163410.56645/jmde.v16i34.591Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking InventoryJason S. McIntosh0Jane Buckley1https://orcid.org/0000-0003-4206-2341Thomas Archibald2https://orcid.org/0000-0002-3567-9143Washington Elementary School DistrictJCB ConsultingVirginia Tech Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, until recently little empirical inquiry on the dimensions of evaluative thinking has been conducted. Purpose: To address that lack, the purpose of the study presented in this paper is to refine the construct of evaluative thinking by exploring its underlying dimensions and to ascertain the internal consistency of an instrument developed to measure evaluative thinking, the Evaluative Thinking Inventory (ETI). Setting: The ETI was developed as part of an ECB initiative focused on non-formal science, engineering, technology, and math (STEM) education in the United States, and was tested as part of a study focused on evaluating gifted education programs, also in the United States. Intervention: Not applicable. Research design: Survey research and exploratory factor analysis (EFA). Data collection & analysis: The ETI was administered to participants in a study measuring the effectiveness of a tool used to conduct internal evaluations of gifted education programs. SPSS was used to conduct an EFA on 96 completed ETIs. Cronbach’s alpha was used to estimate the internal consistency of the instrument. Findings: The analysis of the ETI revealed a two-factor model of evaluative thinking (i.e., believe in and practice evaluation and pose thoughtful questions and seek alternatives). This study also provided internal consistency evidence for the ETI showing alpha reliabilities for the two factors ranging from 0.80 to 0.82. The ETI has potentially wide applicability in research and practice in ECB and in the field of evaluation more generally. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/591evaluative thinkingevaluation capacity buildingresearch on evaluationexploratory factor analysis |
| spellingShingle | Jason S. McIntosh Jane Buckley Thomas Archibald Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory Journal of MultiDisciplinary Evaluation evaluative thinking evaluation capacity building research on evaluation exploratory factor analysis |
| title | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_full | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_fullStr | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_full_unstemmed | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_short | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_sort | refining and measuring the construct of evaluative thinking an exploratory factor analysis of the evaluative thinking inventory |
| topic | evaluative thinking evaluation capacity building research on evaluation exploratory factor analysis |
| url | https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/591 |
| work_keys_str_mv | AT jasonsmcintosh refiningandmeasuringtheconstructofevaluativethinkinganexploratoryfactoranalysisoftheevaluativethinkinginventory AT janebuckley refiningandmeasuringtheconstructofevaluativethinkinganexploratoryfactoranalysisoftheevaluativethinkinginventory AT thomasarchibald refiningandmeasuringtheconstructofevaluativethinkinganexploratoryfactoranalysisoftheevaluativethinkinginventory |