pROC-package(pROC)Spotfire S+ Documentation

pROC

Description

Tools for visualizing, smoothing and comparing receiver operating characteristic (ROC curves). (Partial) area under the curve (AUC) can be compared with statistical tests based on U-statistics or bootstrap. Confidence intervals can be computed for (p)AUC or ROC curves. Sample size / power computation for one or two ROC curves are available.

Details

The basic unit of the pROC package is the roc function. It will build a ROC curve, smooth it if requested (if smooth=TRUE), compute the AUC (if auc=TRUE), the confidence interval (CI) if requested (if ci=TRUE) and plot the curve if requested (if plot=TRUE).

The roc function will call smooth.roc, auc, ci and plot as necessary. See these individual functions for the arguments that can be passed to them through roc. These function can be called separately.

Two paired (that is roc objects with the same response) or unpaired (with different response) ROC curves can be compared with the roc.test function. Sample size and power computations can be performed with the power.roc.test function.

Citation

If you use pROC in published research, please cite the following paper:

Xavier Robin, Natacha Turck, Alexandre Hainard, Natalia Tiberti, Frédérique Lisacek, Jean-Charles Sanchez and Markus Müller (2011). ``pROC: an open-source package for R and S+ to analyze and compare ROC curves''. BMC Bioinformatics, 12, p. 77. DOI: 10.1186/1471-2105-12-77

Type citation("pROC") for a BibTeX entry.

The authors would be glad to hear how pROC is employed. You are kindly encouraged to notify Xavier Robin <Xavier.Robin@unige.ch> about any work you publish.

Abbreviations

The following abbreviations are employed extensively in this package:

Functions

roc Build a ROC curve
are.paired Dertermine if two ROC curves are paired
auc Compute the area under the ROC curve
ci Compute confidence intervals of a ROC curve
ci.auc Compute the CI of the AUC
ci.se Compute the CI of sensitivities at given specificities
ci.sp Compute the CI of specificities at given sensitivities
ci.thresholds Compute the CI of specificity and sensitivity of thresholds
coords Coordinates of a ROC curve
cov Covariance between two AUCs
has.partial.auc Determine if the ROC curve have a partial AUC
lines.roc Add a ROC line to a ROC plot
plot.ci Plot CIs
plot Plot a ROC curve
power.roc.test Sample size and power computation
print Print a ROC curve object
roc.test Compare the AUC of two ROC curves
smooth Smooth a ROC curve
var Variance of the AUC

Dataset

This package comes with a dataset of 141 patients with aneurysmal subarachnoid hemorrhage: aSAH.

Installing and using

To install this package, make sure you are connected to the internet and issue the following command in the R prompt:

    install.pkgutils()
    library(pkgutils)
    install.packages("pROC")

To load the package in S+:

    library(pROC)

Bootstrap

All the bootstrap operations for significance testing and confidence interval computation are performed with non-parametric stratified or non-stratified resampling (according to the stratified argument) and with the percentile method, as described in Carpenter and Bithell (2000) sections 2.1 and 3.3.

Stratification of bootstrap can be controlled with boot.stratified. In stratified bootstrap (the default), each replicate contains the same number of cases and controls than the original sample. Stratification is especially useful if one group has only little observations, or if groups are not balanced.

The number of bootstrap replicates is controlled by boot.n. Higher numbers will give a more precise estimate of the significance tests and confidence intervals but take more time to compute. 2000 is recommanded by Carpenter and Bithell for confidence intervals. In our experience this is sufficient for a good estimation of the first significant digit only, so we recommend the use of 10000 bootstrap replicates to obtain a good estimate of the second significant digit whenever possible.

Author(s)

Xavier Robin, Natacha Turck, Jean-Charles Sanchez and Markus Müller

Maintainer: Xavier Robin <Xavier.Robin@unige.ch>

References

Tom Fawcett (2006) ``An introduction to ROC analysis''. Pattern Recognition Letters 27, 861–874. DOI: 10.1016/j.patrec.2005.10.010

Xavier Robin, Natacha Turck, Alexandre Hainard, et al. (2011) ``pROC: an open-source package for R and S+ to analyze and compare ROC curves''. BMC Bioinformatics, 7, 77. DOI: 10.1186/1471-2105-12-77

See Also

CRAN packages ROCR, verification or Bioconductor's roc.

Examples

data(aSAH)

# Build a ROC object and compute the AUC
roc(aSAH$outcome, aSAH$s100b)
roc(outcome ~ s100b, aSAH)

# Smooth ROC curve
roc(outcome ~ s100b, aSAH, smooth=TRUE)

# more options, CI and plotting
roc1 <- roc(aSAH$outcome,
            aSAH$s100b, percent=TRUE,
            # arguments for auc
            partial.auc=c(100, 90), partial.auc.correct=TRUE,
            partial.auc.focus="sens",
            # arguments for ci
            ci=TRUE, boot.n=100, ci.alpha=0.9, stratified=FALSE,
            # arguments for plot
            plot=TRUE, auc.polygon=TRUE, max.auc.polygon=TRUE, grid=TRUE,
            print.auc=TRUE, show.thres=TRUE)

# Add to an existing plot. Beware of 'percent' specification!
roc2 <- roc(aSAH$outcome, aSAH$wfns,
            plot=TRUE, add=TRUE, percent=roc1$percent)

## Confidence intervals ##

# CI of the AUC
ci(roc2)

## Not run: 
# CI of the curve
sens.ci <- ci.se(roc1, specificities=seq(0, 100, 5))
plot(sens.ci, type="shape", col="lightblue")
plot(sens.ci, type="bars")
## End(Not run)

# need to re-add roc2 over the shape
plot(roc2, add=TRUE)

## Not run: 
# CI of thresholds
plot(ci.thresholds(roc2))
## End(Not run)

## Comparisons ##

# Test on the whole AUC
roc.test(roc1, roc2, reuse.auc=FALSE)

## Not run: 
# Test on a portion of the whole AUC
roc.test(roc1, roc2, reuse.auc=FALSE, partial.auc=c(100, 90),
         partial.auc.focus="se", partial.auc.correct=TRUE)

# With modified bootstrap parameters
roc.test(roc1, roc2, reuse.auc=FALSE, partial.auc=c(100, 90),
         partial.auc.correct=TRUE, boot.n=1000, boot.stratified=FALSE)
## End(Not run)

[Package pROC version 1.4.9 Index]