My life as a psychometrician, the ability algorithm, and some *R* procs to do the work

The number one job of the psychometrician, in the world of large-scale, state-wide assessments, is to produce the appropriate raw-to-scale tables on the day promised. When they are wrong or late, lawsuits ensue. When they are right and on time, everyone is happy. If we did nothing else, most wouldn’t notice; few would complain.

Once the setup is done, computers can produce the tables in a blink of an eye. It is so easy it is often better, especially in the universe beyond fixed forms, to provide the algorithm to produce scale scores on demand and not bother with lookup tables at all. Give it the item difficulties, feed in the raw score, and the scale score pops out. Novices must take care that management never finds out how easy this step really is.

With modern technology, the ability algorithm can be written in almost any computer language (there is probably an app for your phone) but some are easier than others. My native language is *Fortran, *so I am most at home with* C, C ^{++}, R, *or related dialects. I am currently using

*R*most of the time. For me with dichotomous items, this does it:

But first, I need to copy a few other things into the *R* window, like a procedure named *Ability*.

*(A simple cut and paste from this post into R probably won’t work but the code did work when I copied it from the Editor. The website seems to use a font for which not all math symbols are recognized by R. In particular, slash (/), minus (-), single and double quotes (‘ “), and ellipses (…) needed to fixed. I’ve done it with a “replace all” in a text editor before moving it into R by copying the offending symbol from the text into the “replace” box of the “replace all” command and typing the same symbol into the “with” box. Or leave a comment and I’ll email you a simple text version)*

Ability <- function (d, M=rep(1,length(d)), first=1, last=(length(d)-1), A = 500, B = 91, …) { b <- NULL; s <- NULL

b[first] <- first / (length(d) – first)

D <- exp(d)

for (r in first:last) {

b[r] <- Ablest(r, D, M, b[r], …)

s[r] <- SEM (exp(b[r]), D, M)

b[r+1] <- exp(b[r] + s[r]^2)

}

return (data.frame(raw=(first:last), logit=b[first:last], sem.logit=s[first:last],

GRit=round((A+B*b[first:last]),0), sem.GRit=round(B*s[first:last],1)))

} ##############################################################

This procedure is just a front for functions named *Ablest *and* SEM* that actually do the work so you will need to copy them as well:

# r is raw score; D is vector of exponential difficulties; M is vector of m[i]; stop is the convergence

repeat {

adjust <- (r – SumP(B,D,M)) / SumPQ (B,D,M)

B <- exp(log(B) + adjust)

if (abs(adjust) < stop) return (log(B))

}

} ##########################################################ok

SEM <- function (b, d, m=(rep(1,length(d)))) return (1 / sqrt(SumPQ(b,d,m)))

##############################################################

And *Ablest* needs some even more basic utilities copied into the window:

if (is.null(m)) return (sum (P(b,d))) # dichotomous case; sum() is a built-in function

k <- 1

Sp <- 0

for (i in 1:length(m)) {

Sp <- Sp + EV (b, d[k:(k+m[i]-1)])

k <- k + m[k]

}

return (Sp)

} ##################################################################ok

EV <- function (b, d) { # %*% is the inner product, produces a scalar

return (seq(1:length(d)) %*% P.Rasch(b, d, m=length(d)))

} ##################################################################ok

SumPQ <- function (B, D, m=NULL, P=function (B,D) {B/(B+D)}, PQ=function (p) {p-p^2}) {

if (is.null(m)) return (sum(PQ(P(B,D)))) # dichotomous case;

k <- 1

Spq <- 0

for (i in 1:length(m)) {

Spq = Spq + VAR (B,D[k:(k+m[i]-1)])

k <- k + m[k]

}

return (Spq)

} ##################################################################ok

VAR <- function (b,d) { # this is just the polytomous version of (p – p^2)

return (P.Rasch(b, d, m=length(d)) %*% ((1:length(d))^2) – EV(b,d)^2)

} ##################################################################ok

P.Rasch <- function (b, d, m=NULL, P=function (B,D) (B / (B+D)) ) {

if (is.null(m)) return (P(b,d)) # simple logistic

return (P.poly (P(b,d),m)) # polytomous

} ##################################################################ok

P.poly <- function (p, m) { # p is a simple vector of category probabilities

k <- 1

for (i in 1:length(m)) {

p[k:(k+m[i]-1)] = P.star (p[k:(k+m[i]-1)], m[i])

k <- k + m[i]

}

return (p)

} ##################################################################ok

P.star <- function (pstar, m=length(pstar)) {

#

# Converts p* to p; assumes a vector of probabilities

# computed naively as B/(B+D). This routine takes account

# of the Guttmann response patterns allowed with PRM.

#

q <- 1-pstar # all wrong, 000…

p <- prod(q) for (j in 1:m) {

q[j] <- pstar[j] # one more right, eg., 100…, or 110…, …

p[j+1] <- prod(q)

}

return (p[-1]/sum(p)) # Don’t return p for category 0

} ##################################################################ok

summary.ability <- function (score, dec=5) {

print (round(score,dec))

plot(score[,4],score[,1],xlab=”GRit”,ylab=”Raw Score”,type=’l’,col=’red’)

points(score[,4]+score[,5],score[,1],type=’l’,lty=3)

points(score[,4]-score[,5],score[,1],type=’l’,lty=3)

} ##################################################################

This is very similar to the earlier version of *Ablest* but has been generalized to handle polytomous items, which is where the vector *M* of maximum scores or number of thresholds comes in.

To use more bells and whistles, the call statement can be things like:

Ability (d, M, first, last, A, B, stop) # All the parameters it has Ability (d, M) # first, last, A, B, & stop have defaults Ability (d,,,,,, 0.0001) #*stop*is positional so the commas are needed Ability (d, ,10, 20,,, 0.001) # default for M assumes dichotomous items Ability (d, M,,, 100, 10) # defaults for A & B are 500 and 91

To really try it out, we can define a vector of item difficulties with, say, 25 uniformly spaced dichotomous items and two polytomous items, one with three thresholds and one with five. The vector *m* defines the matching vector of maximum scores.

Or give it your own vectors of logit difficulties and maximum scores.

For those who speak *R*, the code is fairly intuitive, perhaps not optimal, and could be translated almost line by line into *Fortran*, although some lines would become several. Most of the routines can be called directly if you’re so inclined and get the arguments right. Most importantly, *Ability* expects logit difficulties and returns logit abilities. Almost everything expects and uses exponentials. Almost all error messages are unintelligible and either because *d *and *m* don’t match or something is an exponential when it should be a logit or vice versa.

I haven’t mentioned what to do about zero and perfect scores today because, first, I’m annoyed that they are still necessary, second, these routines don’t do them, and, third, I talked about the problem a few posts ago. But, if you must, you could use *b[0] = b[1] – SEM[1]^2 *and* b[M] = b[M-1] + SEM[M-1]^2*, where M is the maximum possible score, not necessarily the number of items. Or you could appear even more scientific and use something like *b[0] = Ablest(0.3, D, m)* and *b[M] = Ablest(M-0.3, D, m)*. Here *D* is the vector of difficulties in the exponential form and *m* is the vector of maximum scores for the items (and *M *is the sum of the* m*‘s.) The length of *D* is the total number of thresholds (aka, *M*) and the length of *m* is the number of items (sometimes called *L.*) *Ablest* doesn’t care that the score isn’t an integer but *Ability* would care. The value *0.3 *was a somewhat arbitrary choice; you may prefer* 0.25 *or* 0.33* instead.

To call this the “setup” is a little misleading; we normally aren’t allowed to just make up the item difficulties this way. There are a few other preliminaries that the psychometrician might weigh in on or at least show up at meetings; for example, test design, item writing, field testing, field test analysis, item reviews, item calibration, linking, equating, standards setting, form development, item validation, and key verification. There is also the small matter of presenting the items to the students. Once those are out-of-the-way, the psychometrician’s job of producing the raw score to scale score lookup table is simple.

Once I deal with a few more preliminaries , I’ll go ahead and go back to the good stuff like diagnosing item and person anomalies.