Exploratory study of individual assessment practices: interrater reliability and judgments of assessor effectiveness

Article Abstract:

Three psychologists who regularly conduct individual assessments were asked to assess 3 individuals posing as job candidates for the same position. The materials from these 9 assessments - test scores, biographical information, and audiotapes of interviews - served as protocols for 50 industrial/organizational psychologists who rated the candidates and assessors. Comparisons of the approaches and conclusions of the assessors indicated variability in job/organizational information obtained, test instruments used, personal history information gathered, interview and the report generated, and conclusions regarding the candidates. On average, only one third of the raters agreed with the conclusions of the assessor whose protocol they were reviewing. Significant differences were found in the raters' evaluations of 2 of the assessors, depending on which candidate the assessor had evaluated. The study's design limits generalizability; however, the low interrater agreement is disturbing. (Reprinted by permission of the publisher.)

Employment, Personality assessment

---

Linearity of ability-performance relationships: a reconfirmation

Article Abstract:

Previous research on the nature of ability-performance relationships has found the relationship to be linear. Recent Monte Carlo work suggests that the test of the significance of the difference between the Pearson correlation and the correlation ratio (eta), used in previous research, has low power to detect nonlinear relationships. Using 174 studies of the relationship between 9 scales of the General Aptitude Test Battery and job performance, with a mean sample size of 210, and using a power polynomial approach, which has been shown to have higher power than the r vs eta test, the issue of linearity was reexamined. Similar to previous findings, nonlinear relationships were not found at levels substantially greater than would be expected by chance. (Reprinted by permission of the publisher.)

Performance, Measurement, Ability, Aptitude

---

Direct, indirect, and controlled observation and rating accuracy

Article Abstract:

Videotaping of assessment center exercises has become an increasingly common practice, yet little is known about the impact of video technology on rating accuracy. This study compared ratings of a group discussion made after live observation (direct), after viewing a video (indirect), or after viewing a video with opportunities to pause and rewind (controlled). Results indicated some differences in observational accuracy but not in rating accuracy. Implications for the use of video technology in assessment centers are discussed. (Reprinted by permission of the publisher.)

Video tape recorders and recording, Videocassette recorders, Videotape recording, Management development programs

Similar abstracts:

• Abstracts: A laboratory experiment testing the antecedents of leader cognitions. Cognitive-affective stress propensity: a field study
• Abstracts: Relations of emotional intelligence, practical intelligence, and trait affectivity with interview outcomes: it's not all just 'G.'
• Abstracts: Meta-analysis of assessment center validity. Number of assessment center dimensions as a determinant of assessor accuracy
• Abstracts: Distributional ratings, judgment decomposition, and their impact on interrater agreement and rating accuracy. Effects of subsequent performance on evaluations of previous performance
• Abstracts: The social psychology of eyewitness accuracy: misleading questions and communicator expertise. Eyewitness accuracy and confidence: within- versus between-subjects correlations

This website is not affiliated with document authors or copyright owners. This page is provided for informational purposes only. Unintentional errors are possible.