Reliability of subjective evaluation in assessing elite table tennis players’ performance
PurposeThis study aims to assess the reliability of subjective evaluations conducted under two information conditions and to explore the influence of observer expertise on the consistency of performance assessments of elite table tennis players.MethodsObservers of varying skill levels were invited t...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-06-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1432711/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849710519312711680 |
|---|---|
| author | Lulu Gan Jing Chen Luning Wang Yunfei Lu Jie Ren |
| author_facet | Lulu Gan Jing Chen Luning Wang Yunfei Lu Jie Ren |
| author_sort | Lulu Gan |
| collection | DOAJ |
| description | PurposeThis study aims to assess the reliability of subjective evaluations conducted under two information conditions and to explore the influence of observer expertise on the consistency of performance assessments of elite table tennis players.MethodsObservers of varying skill levels were invited to provide subjective evaluations of the elite table tennis players’ performance by observing specific rally strokes during the match. A Video Masking Paradigm approach was implemented to conceal motion information during critical moments of scoring and losing. The weighted Kappa coefficient (k) was employed to evaluate the inter-observer consistency between two observers. The Kendall’s coefficient of concordance (w) is a measure of inter-rater agreement, specifically used for ordinal scales (e.g., Likert five-point scale) when multiple raters are involved.ResultsIntra-observer reliability was good (r = 0.61–0.86), whereas inter-observer consistency between the two observers was low (k = 0.01–0.39). Among the observation indicators, the advanced group showed the lowest consistency in evaluating tactical behavior (without results, w = 0.44; with results, w = 0.76). Experiment 2: The consistency of the observers in the without results condition (expert group w = 0.75 vs. advanced group w = 0.57 vs. novice group w = 0.66) is lower than in the with results (expert group w = 0.84 vs. advanced group w = 0.78 vs. novice group w = 0.76). Across all three observation indicators, namely stroke quality, tactical intention, and competitive posture, the expert group demonstrated the highest level of consistency, followed by the advanced group, while the novice group exhibited the lowest level of agreement.ConclusionObservers with table tennis skill levels demonstrate high intra-observer test–retest reliability in subjective evaluations, but the inter-observer consistency is lower. Different information conditions (with or without results) are key variables affecting the consistency of subjective evaluations. When kinematic information is occluded (without results), the consistency of subjective evaluations decreases. The selection of observation indicators also impacts the consistency of subjective evaluations. Additionally, observers’ consistency in subjective evaluations is influenced by their level of experience and skill: the higher the observer’s level and experience, the greater the consistency of their subjective evaluations. |
| format | Article |
| id | doaj-art-3c32c7ab9df84679a52cd1999174b46e |
| institution | DOAJ |
| issn | 1664-1078 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj-art-3c32c7ab9df84679a52cd1999174b46e2025-08-20T03:14:53ZengFrontiers Media S.A.Frontiers in Psychology1664-10782025-06-011610.3389/fpsyg.2025.14327111432711Reliability of subjective evaluation in assessing elite table tennis players’ performanceLulu Gan0Jing Chen1Luning Wang2Yunfei Lu3Jie Ren4China Table Tennis College, Shanghai University of Sport, Shanghai, ChinaDivision of Health Professions, Texas A&M University–Texarkana, Texarkana, TX, United StatesSchool of Psychology, Shanghai University of Sport, Shanghai, ChinaChina Table Tennis College, Shanghai University of Sport, Shanghai, ChinaChina Table Tennis College, Shanghai University of Sport, Shanghai, ChinaPurposeThis study aims to assess the reliability of subjective evaluations conducted under two information conditions and to explore the influence of observer expertise on the consistency of performance assessments of elite table tennis players.MethodsObservers of varying skill levels were invited to provide subjective evaluations of the elite table tennis players’ performance by observing specific rally strokes during the match. A Video Masking Paradigm approach was implemented to conceal motion information during critical moments of scoring and losing. The weighted Kappa coefficient (k) was employed to evaluate the inter-observer consistency between two observers. The Kendall’s coefficient of concordance (w) is a measure of inter-rater agreement, specifically used for ordinal scales (e.g., Likert five-point scale) when multiple raters are involved.ResultsIntra-observer reliability was good (r = 0.61–0.86), whereas inter-observer consistency between the two observers was low (k = 0.01–0.39). Among the observation indicators, the advanced group showed the lowest consistency in evaluating tactical behavior (without results, w = 0.44; with results, w = 0.76). Experiment 2: The consistency of the observers in the without results condition (expert group w = 0.75 vs. advanced group w = 0.57 vs. novice group w = 0.66) is lower than in the with results (expert group w = 0.84 vs. advanced group w = 0.78 vs. novice group w = 0.76). Across all three observation indicators, namely stroke quality, tactical intention, and competitive posture, the expert group demonstrated the highest level of consistency, followed by the advanced group, while the novice group exhibited the lowest level of agreement.ConclusionObservers with table tennis skill levels demonstrate high intra-observer test–retest reliability in subjective evaluations, but the inter-observer consistency is lower. Different information conditions (with or without results) are key variables affecting the consistency of subjective evaluations. When kinematic information is occluded (without results), the consistency of subjective evaluations decreases. The selection of observation indicators also impacts the consistency of subjective evaluations. Additionally, observers’ consistency in subjective evaluations is influenced by their level of experience and skill: the higher the observer’s level and experience, the greater the consistency of their subjective evaluations.https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1432711/fullsports performancetable tenniselite athletereliabilitysubjective evaluation |
| spellingShingle | Lulu Gan Jing Chen Luning Wang Yunfei Lu Jie Ren Reliability of subjective evaluation in assessing elite table tennis players’ performance Frontiers in Psychology sports performance table tennis elite athlete reliability subjective evaluation |
| title | Reliability of subjective evaluation in assessing elite table tennis players’ performance |
| title_full | Reliability of subjective evaluation in assessing elite table tennis players’ performance |
| title_fullStr | Reliability of subjective evaluation in assessing elite table tennis players’ performance |
| title_full_unstemmed | Reliability of subjective evaluation in assessing elite table tennis players’ performance |
| title_short | Reliability of subjective evaluation in assessing elite table tennis players’ performance |
| title_sort | reliability of subjective evaluation in assessing elite table tennis players performance |
| topic | sports performance table tennis elite athlete reliability subjective evaluation |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1432711/full |
| work_keys_str_mv | AT lulugan reliabilityofsubjectiveevaluationinassessingelitetabletennisplayersperformance AT jingchen reliabilityofsubjectiveevaluationinassessingelitetabletennisplayersperformance AT luningwang reliabilityofsubjectiveevaluationinassessingelitetabletennisplayersperformance AT yunfeilu reliabilityofsubjectiveevaluationinassessingelitetabletennisplayersperformance AT jieren reliabilityofsubjectiveevaluationinassessingelitetabletennisplayersperformance |