Reams, R., Chamrad, D. & Robinson, N.
Gifted Child Quarterly
National Association for Gifted Children
Vol. 34, No. 3, pp. 108-110
Summer 1990
This article examines the pros and cons of timed tests in intelligence testing. Many times, taking speed into account is an unfair practice and does not realistically portray intelligence levels in all young people. Some profoundly gifted young people are very deliberate and must think things through before they act, thus making them slower, but not less intelligent than someone who does not react in this manner. The article also suggests that if bonus points are given for speed, the person taking the test should be made aware of this fact. Authored by Redmond Reams, Diana Chamrad and Nancy Robinson.
Abstract
To investigate speed as a factor in WISC-R performance, scores of 66 high-scoring (IQ 120 +) and 36 average-scoring (IQ< 120) children were examined in a clinic for gifted children. Although significant mean group differences were found on all subtests, they were only marginal for Coding, which depends strongly on speed. On the three subtests using speed bonus points, the high-scoring group gave more correct answers, but on only one, Block Design, was there a trend toward more bonus points for speed. The results are interpreted as casting serious doubt on the utility of speed bonuses in tests of general intelligence with gifted children.
Putting the Research to Use
Although brighter children tend generally to learn more quickly and solve problems more rapidly than their age peers of average intelligence, intelligence tests which put a premium on very fast response may produce misleading results. Gifted children who are planful and/or cautious, or so perfectionistic that they think twice before responding, are likely to be penalized by intelligence test such as the WISC-R and the WPPSI-R, which award bonus points for very rapid responses to some subtests. Current norms do not permit separate scoring for speed and accuracy. A test such as the Stanford-Binet, Fourth Edition, which is almost entirely untimed, may be a better choice for gifted young people.
It has long been established that brighter individuals tend to solve problems and learn new information more quickly than those who are less bright. It is tempting to assume that speed should therefore play a central role in intelligence tests.
In constructing the prototype of his intelligence tests, David Wechsler (1944) took as his guide a comprehensive view of intelligence as a part of the total personality. The Performance Scales in particular were designed to include variables which Wechsler considered nonintellective, including speed of response. On the Wechsler Intelligence Scale for Children -- Revised (WISC-R) (Wechsler, 1974), for example, bonus points are awarded for three of the five subtests on the Performance Scale (Picture Arrangement, Block Design, and Object Assembly), and speed constitutes the major scoring criterion for a fourth Performance subtest, Coding. Similarly, for the new Wechsler Preschool and Primary Scale of Intelligence -- Revised (WPPSI-R) (Wechsler, 1989) for younger children, bonus points are awarded for rapid performance on two subtests. Object Assembly and Block Design, with speed the major variable in an optional subtest, Animal Pegs (formerly Animal House). On both WISC-R and WPPSI-R, additional performance subtests and a verbal sub-test, Arithmetic, also have time limits, as do those with bonus points, but the limits are sufficiently generous that correctness rather than speed usually determines the score.
Wechsler viewed mental speed, then, as a noncognitive aspect of intelligence. Similarly, Kagan (1965) viewed response speed as a matter of cognitive style, rather than cognitive ability. More recent research has, however, focused upon speed of processing as an index of cognitive ability. After a long period of disfavor, research into reaction time has re-emerged as a legitimate endeavor. Most investigators have looked at the relationship between IQ and reaction times on tasks such as choosing which key to touch in response to visual cues in the form of patterns of lights or shapes (e.g., Detterman, 1987; Jensen, 1982; Matthews & Dom, 1989; Vernon, 1987). Other investigators have looked at the relationship between IQ and inspection time, typically the minimum time needed to apprehend information in tasks such as identifying the longer of two lines (Kranzler & Jensen, 1989). Indeed, relationships have been discovered even in infancy between speed of habituation to a visual stimulus and intellectual performance in middle childhood (Fagan & McGrath, 1981). Most research has focused upon apparently simple tasks, but measurement of cognitive processing speed has also played a role in research which attempts to unravel the components of more complex problem-solving. Sternberg (1977), for example, demonstrated that brighter subjects differ from average subjects with respect to the amount of time allocated to different processes or cognitive components of a task: more time planning, less time executing, and less total time overall.
Despite the new information about speed of performance in the laboratory, Sternberg (1982) has argued that emphasizing speed of performance on intelligence tasks is likely to penalize individuals who approach tasks "intelligently" or strategically. In particular, gifted children who exhibit a thoughtful and high-level problem solving approach may not earn extra points for speed. Marr and Sternberg (1987) suggest that, while the capacity for rapid cognitive processing is, in most contexts, adaptive and "intelligent," there may be individual differences in preference for mental speed, and complex relationships with higher-level processes which are described as metacomponential. One source of individual difference is described by Siegler (1989), who found that in solving simple arithmetic tasks, children he described as "perfectionists" avoided the most efficient strategy, simple retrieval, unless they were very sure of their answers.
Direct translation from laboratory tasks to the kinds of problem-solving required on intelligence tests may be misleading. Most laboratory tasks present numerous trials with the same paradigm, so that it is both possible and efficient for the subject to devise a useful strategy. Intelligence tests, on the other hand, tend to present a variety of problems even within the same subtest, making strategy development difficult. Even when the task consists of repetitive responses (for example. Animal Pegs on the WPPSI-R or Coding on the WISC-R), the child is typically not given information which would permit an "intelligent" allocation of time between initial study and execution of the task.
The present study examines these issues by comparing high-scoring and average-scoring children on measures of speed on subtests of the WISC-R. Our hypothesis was that brighter children would show greater superiority of performance over average children on those intellectual measures on which power, rather than speed, was of a primary importance.
Subjects and Method
Subjects were 102 children brought to a university clinic which is designed to assist families of children ages 3 to 15 who have or are suspected to have advanced intellectual abilities. Typical referral issues include parents' wish to understand more precisely the nature of their child's abilities, help in making informed decisions about educational placement, discussion of personal/social issues, and/or a need to solve specific behavioral problems. The families are predominantly Caucasian and upper-middle class although a wide range of backgrounds is represented.
All children included in this study sample were administered the 10 ordinary subscales of the WISC-R using the usual sub-test ordering and prescribed time limits. Of the 102 children, those earning a Full Scale IQ of 120 or above (range 120-154) were categorized as the High group, while those earning an IQ below 120 (range 81-119) constituted the Average group. Verbal IQs showed occasional overlap (High range 107-149, Average range 81-130), as did Performance IQs (High range 109-151, Average range 85-126). Other sample characteristics resulting from this division are described in Table 1, as are the test scores of the groups. In the High group were 28 females and 38 males, ages 72 to 157 months. In the Average group were 16 females and 20 males, ages 68 to 177 months.
|
Table
1
Group
Means and Standard Deviations |
|
|
Average
Sample
N=36 |
High
Sample
N=66 |
|
|
|
Variable |
Mean |
S.D. |
Mean |
S.D. |
t |
p* |
|
Age
(months) |
108.1 |
27.7 |
102.7 |
22.1 |
1.09 |
NS |
|
Verbal
IQ |
112.0 |
8.2 |
128.8 |
10.1 |
8.58 |
<.0001 |
|
Performance
IQ |
109.7 |
9.1 |
125.8 |
9.4 |
8.42 |
<.0001 |
|
Full
Scale IQ |
112.3 |
7.5 |
131.2 |
8.3 |
8.83 |
<.0001 |
|
Information |
12.0 |
2.0 |
14.2 |
2.6 |
4.46 |
.0001 |
|
Similarities |
13.0 |
2.4 |
15.6 |
2.4 |
5.38 |
<.0001 |
|
Arithmetic |
10.6 |
2.7 |
13.5 |
2.5 |
5.78 |
<.0001 |
|
Vocabulary |
12.4 |
2.2 |
15.3 |
2.1 |
6.58 |
<.0001 |
|
Comprehension |
12.1 |
2.1 |
14.5 |
1.9 |
5.78 |
<.0001 |
|
Picture
Completion |
11.7 |
2.6 |
13.8 |
1.9 |
4.69 |
<.0001 |
|
Picture
Arrangement |
11.6 |
2.1 |
13.7 |
2.5 |
4.44 |
.0001 |
|
Block
Design |
11.4 |
2.6 |
14.8 |
2.8 |
6.27 |
<.0001 |
|
Object
Assembly |
11.4 |
2.3 |
13.6 |
2.8 |
4.14 |
.0002 |
|
Coding |
10.7 |
2.6 |
12.3 |
2.8 |
2.84 |
.0055 |
|
*One-tailed
probabilities |
Results
The rank order of the 10 mean subscale standard scores was virtually identical for both groups (rho = .81), except for Block Design, which was third highest for the High group and eighth for the Average Group. The scaled score difference of 1.59 favoring the High group on the timed Coding subtest was the smallest of the 10 discrepancies; the average difference between the groups on the other nine subtests was 2.53 (range = 2.12 to 3.42). Furthermore, correlations between Coding subtest scores and Performance and Full Scale IQs, corrected to eliminate the Coding subtest. were all near zero (.00 to .11).
As shown in Table 2, the groups were compared on the number of items solved correctly within the time limits and number of earned speed bonus points on Picture Arrangement, Block Design, and Object Assembly. Raw scores are reported because no norms are provided for the two factors separately. Despite a six-month mean age advantage for the Average group, the High group earned more points for correct answers on all three subtests. Only on Block Design, which requires subjects to copy a series of models, did they earn more bonus points for response speed.
|
Table
2
Performance
on WISC-R Timed Subtests (excluding Coding) |
|
|
Average
Sample
N=36 |
High
Sample
N=66 |
|
|
Variable |
Mean |
S.D. |
Mean |
S.D. |
t |
p* |
|
Picture
Arrangement |
|
|
|
|
|
|
|
Speed
Points |
4.8 |
4.2 |
4.9 |
3.4 |
.22 |
NS |
|
Correct
Points |
19.9 |
4.8 |
24.0 |
5.4 |
4.23 |
.0002 |
|
Block
Design |
|
|
|
|
|
|
|
Speed
Points |
5.0 |
4.6 |
7.5 |
5.6 |
2.32 |
.02 |
|
Correct
Points |
20.3 |
8.4 |
26.9 |
8.6 |
3.66 |
.0007 |
|
Object
Assembly |
|
|
|
|
|
|
|
Speed
Points |
2.1 |
2.5 |
3.0 |
2.8 |
1.56 |
NS |
|
Correct
Points |
17.8 |
3.5 |
19.9 |
2.6 |
3.56 |
.0009 |
|
*One
tailed test |
Discussion
The performance of the groups on Picture Arrangement and Object Assembly suggests that on these subtests, which demand insightful construction of the correct answers, higher capacity need not always be reflected in faster performance. On these subtests each item presents a unique problem which depresses cross-problem strategy formation and offers little payoff for such strategizing. On the other hand, the brighter children did apparently tend to devise more time-efficient strategies for copying models with blocks and copying codes than did children of average ability, but the differences were not as impressive as were differences reflecting accuracy of response.
With respect to the Coding task, the mean ages of the groups may have minimized group differences. Although standard scores reflect a comparison of children within an age group rather than across ages, differences in handwriting speed may mask cognitive process on this task (Lindley, Smith, & Thomas, 1988). Visual motor factors in normally developing children do not appear to play an important role in the other performance subtests.
Inclusion of bonus points for speed would seem to demand "honesty in advertising" if children are expected to respond as quickly as feasible. On the new WPPSI-R, for example, all the performance tasks as well as one verbal task have time limits, and within those limits there is a premium for speed on three (Block Design, Object Assembly, and Animal Pegs). Although children are told on these three tasks, "Let's see how fast you can do it," this instruction is not highlighted. Furthermore, the examiner is cautioned to be "unobtrusive" in timing so that the child is not distracted. It would take a very perceptive child to understand that speed itself, as opposed to completion within the time limits, will be rewarded.
We interpret these results as casting serious doubt on the utility of including speed bonuses in tests of general intelligence. Indeed, tests of higher-order capabilities which permit children to use their preferred problem-solving styles, be those rapid or cautious, would seem to make more sense. If speed bonuses are to be included, then we suggest that it would be wise to devise a scoring system which permits the examiner to look separately at this facet in evaluating a child's performance.
References
Detterman, D.K. (1987). What does reaction time tell us about intelligence? In P.A. Vernon (Ed.), Speed of information-processing and intelligence. Norwood. NJ: Ablex.
Fagan, J.F., & McGrath, S.K. (1981). Infant recognition memory and later intelligence. Intelligence, 5, 121-130.
Jensen, A.R. (1982). Reaction time and psychometric g. In H.J. Eysenck (Ed.), A model for intelligence. Berlin: Springer-Verlag.
Kagan. J. (1965). Impulsive and reflective children: Significance of conceptual tempo. In J.D. Krumboltz (Ed.), Learning and the educational process. Chicago: Rand McNally.
Kranzler, J.H., & Jensen A.R. (1989). Inspection time and intelligence: A meta-analysis. Intelligence, 13, 329-347.
Lindley, R.H., Smith, W.R., & Thomas, T.J. (1988). The relationship between speed of information processing as measured by timed paper-and-pencil tests and psychometric intelligence. Intelligence, 12, 17-25.
Marr, D.B., & Sternberg, R.J. (1987). The role of mental speed in intelligence: A triarchic perspective. In P.A. Vernon (Ed.), Speed of information-processing and intelligence (pp. 271-294). Norwood, NJ: Ablex.
Matthews, G., & Dorn, L. (1989). IQ and choice reaction time: An information processing analysis, Intelligence, 13, 299-318.
Siegler, R.S. (1989). Strategy diversity and cognitive assessment. Educational Researcher, 18 (9), 15-20.
Sternberg, R.J. (1977). Intelligence, information processing, and analogical reasoning. Hillsdale, NJ: Eribaum.
Sternberg, R.J. (1982). Lies we live by: Misapplications of tests in identifying the gifted. Gifted Child Quarterly, 26, 157-161.
Vernon, P.A. (1987). New developments in reaction time research. In P.A. Vernon (Ed.), Speed of information-processing and intelligence (pp. 1-20). Norwood, NJ: Ablex.
Wechsler, D. (1944). The measurement of adult intelligence (3rd ed.) Baltimore: Williams & Wilkins.
Wechsler, D. (1974). Wechsler Intelligence Scale for Children-Revised. New York: Psychological Corporation.
Wechsler, D. (1989). Wechsler Preschool and Primary Scale of Intelligence-Revised. San Antonio: Psychological Corporation Harcourt Brace Jovanovich.
Permission Statement
|