var.roc(pROC) | Spotfire S+ Documentation |
These functions compute the variance of the AUC of a ROC curve.
var.roc(x, ...) ## S3 method for class 'auc': var.roc(auc, ...) ## S3 method for class 'roc': var.roc(roc, method=c("delong", "bootstrap", "obuchowski"), boot.n = 2000, boot.stratified = TRUE, reuse.auc=TRUE, progress = getOption("pROCProgress")$name, ...) ## S3 method for class 'smooth.roc': var.roc(smooth.roc, ...)
x, roc, smooth.roc, auc |
a “roc” object from the
roc function, a “smooth.roc” object from the
smooth.roc function or an “auc” object from
the auc function.
|
method |
the method to use, either “delong”, “bootstrap” or “obuchowski”. The first letter is sufficient. If omitted, the appropriate method is selected as explained in details. |
reuse.auc |
if TRUE (default) and the “roc” objects
contain an “auc” field, re-use these specifications for the
test. See details.
|
boot.n |
for method="bootstrap" only: the number of
bootstrap replicates or permutations. Default: 2000.
|
boot.stratified |
for method="bootstrap" only:
should the bootstrap be stratified (same number
of cases/controls in each replicate than in the original sample) or
not. Default: TRUE.
|
progress |
the name of progress bar to display. Typically
“none”, “win”, “tk” or “text” (see the
name argument to create_progress_bar for
more information), but a list as returned by create_progress_bar
is also accepted. See also the “Progress bars” section of
this package's documentation.
|
... |
further arguments passed to or from other methods,
especially arguments for var.roc when calling var ,
var.roc.auc and var.roc.smooth.roc . Arguments for
auc (if reuse.auc=FALSE ) and
txtProgressBar (only char and style ) if
applicable.
|
The var.roc
function computes the variance of the AUC of a ROC
curve. It is typically called with the roc object of
interest. Three methods are available: “delong”,
“bootstrap” and “obuchowski” (see “Computational
details” section below).
The default is to use “delong” method except for with
partial AUC and smoothed curves where “bootstrap” is employed.
Using “delong” for partial AUC and smoothed ROCs is not
supported (a warning is produced and “bootstrap” is employed
instead). “obuchowski” can be used on partial AUCs only if they
are defined as a region of specificity (AUC generated with
partial.auc.region="specificity"
). Requesting the variance to
be computed with method="obuchowski"
on an AUC with
partial.auc.region="specificity"
produces a warning and
“bootstrap” is employed instead.
For smoothed ROC curves, smoothing is performed again at each
bootstrap replicate with the parameters originally provided.
If a density smoothing was performed with user-provided
density.cases
or density.controls
the bootstrap cannot
be performed and an error is issued.
var.default
forces the usage of the
splus::var function in the splus package, so
that other code relying on var
should continue to function
normally.
The numeric value of the variance.
var.roc
needs a specification of the AUC to compute
the variance of the AUC of the ROC curve.
The specification is defined by:
reuse.auc
is set to TRUE
(default)
auc
with ...
(arguments partial.auc
, partial.auc.correct
and
partial.auc.focus
). In this case, you must ensure either that
the roc
object do not contain an auc
field (if
you called roc
with auc=FALSE
), or set
reuse.auc=FALSE
.
If reuse.auc=FALSE
the auc
function will always
be called with ...
to determine the specification, even if
the “roc” objects do contain an auc
field.
As well if the “roc” objects do not contain an auc
field, the auc
function will always be called with
...
to determine the specification.
Warning: if the roc object passed to roc.test contains an auc
field and reuse.auc=TRUE
, auc is not called and
arguments such as partial.auc
are silently ignored.
With method="bootstrap"
, the processing is done as follow:
boot.n
bootstrap replicates are drawn from the
data. If boot.stratified
is TRUE, each replicate contains
exactly the same number of controls and cases than the original
sample, otherwise if FALSE the numbers can vary.
With method="delong"
, the processing is done as described in
Hanley and Hajian-Tilaki (1997).
With method="obuchowski"
, the processing is done as described
in Obuchowski and McClish (1997), Table 1 and Equation 4, p. 1530–1531. The
computation of g for partial area under the ROC curve is
modified as:
expr1 * (2 * pi * expr2) ^ (-1) * (-expr4) - A * B * expr1 * (2 * pi * expr2^3) ^ (-1/2) * expr3
.
The “obuchowski” method makes the assumption that the data is binormal.
If the data shows a deviation from this assumption, it might help to
normalize the data first (that is, before calling roc
),
for example with quantile normalization:
norm.x <- qnorm(rank(x)/(length(x)+1)) var(roc(response, norm.x, ...), ...)
“delong” and “bootstrap” methods make no such assumption.
If method="delong"
and the AUC specification specifies a
partial AUC, the warning “Using DeLong for partial AUC is
not supported. Using bootstrap test instead.” is issued. The
method
argument is ignored and “bootstrap” is used instead.
If method="delong"
or method="obuchowski"
and the ROC
curve is smoothed, the warning “Using DeLong/Obuchowski for
smoothed ROCs is not supported. Using bootstrap test instead.” is
issued. The method
argument is ignored and “bootstrap”
is used instead.
When method="obuchowski"
is requested on a partial AUC with
method="obuchowski"
produces the warning “Using
Obuchowski for smoothed ROCs is not supported. Using bootstrap
instead.” The method
argument is ignored and “bootstrap”
is used instead.
If boot.stratified=FALSE
and the sample has a large imbalance between
cases and controls, it could happen that one or more of the replicates
contains no case or control observation, or that there are not enough
points for smoothing, producing a NA
area.
The warning “NA value(s) produced during bootstrap were ignored.”
will be issued and the observation will be ignored. If you have a large
imbalance in your sample, it could be safer to keep
boot.stratified=TRUE
.
If density.cases
and density.controls
were provided
for smoothing, the error “Cannot compute the covariance on ROC
curves smoothed with density.controls and density.cases.” is
issued.
Elisabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson (1988) ``Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach''. Biometrics 44, 837–845.
James A. Hanley and Karim O. Hajian-Tilaki (1997) ``Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: An update''. Academic Radiology 4, 49–58. DOI: href{http://dx.doi.org/10.1016/S1076-6332(97)80161-4}{10.1016/S1076-6332(97)80161-4}.
Nancy A. Obuchowski, Donna K. McClish (1997). ``Sample size determination for diagnostic accurary studies involving binormal ROC curve indices''. Statistics in Medicine, 16(13), 1529–1542. DOI: href{http://dx.doi.org/10.1002/(SICI)1097-0258(19970715)16:13<1529::AID-SIM565>3.0.CO;2-H}{(SICI)1097-0258(19970715)16:13<1529::AID-SIM565>3.0.CO;2-H}.
data(aSAH) ## Basic example roc1 <- roc(aSAH$outcome, aSAH$s100b) roc2 <- roc(aSAH$outcome, aSAH$wfns) var.roc(roc1) var.roc(roc2) # We could also write it in one line: var.roc(roc(aSAH$outcome, aSAH$s100b)) ### Not run: # The latter used Delong. To use bootstrap: var.roc(roc1, method="bootstrap") # Decrease boot.n for a faster execution var.roc(roc1,method="bootstrap", boot.n=1000) # ## End(Not run) # To use obuchowski: var.roc(roc1, method="obuchowski") ### Not run: # Variance of smoothed ROCs: # Smoothing is re-done at each iteration, and execution is slow var.roc(smooth(roc1)) # ## End(Not run) # or from an AUC (no smoothing) var.roc(auc(roc1)) ## Test data from Hanley and Hajian-Tilaki, 1997 disease.present <- c("Yes", "No", "Yes", "No", "No", "Yes", "Yes", "No", "No", "Yes", "No", "No", "Yes", "No", "No") field.strength.1 <- c(1, 2, 5, 1, 1, 1, 2, 1, 2, 2, 1, 1, 5, 1, 1) field.strength.2 <- c(1, 1, 5, 1, 1, 1, 4, 1, 2, 2, 1, 1, 5, 1, 1) roc3 <- roc(disease.present, field.strength.1) roc4 <- roc(disease.present, field.strength.2) # Assess the variance: var.roc(roc3) var.roc(roc4) ### Not run: # With bootstrap: var.roc(roc3, method="bootstrap") var.roc(roc4, method="bootstrap") # ## End(Not run)