-
Notifications
You must be signed in to change notification settings - Fork 755
C interface
This is an opinionated re-write of section 5 ("System and foreign language interfaces") of Writing R extensions, focussing on best practices and modern tools. This means it does not cover:
- the
.Cinterface - the old api defined in
Rdefines.h - esoteric language features that are rarely used
It focusses mainly on section 5.9, "Handling R objects in C", considerably expanding with many more examples. The main point of the guide is to help you read and understand R's C source code. It will also help you write your own C functions, but for anything more than the simplest function, we recommend using C++ and Rcpp.
All examples in this chapter use the inline package - this makes it extremely easy to get up and running with C code. Make sure you have it installed and loaded with the following code:
install.packages("inline")
library(inline)
You'll also (obviously) need a working C compiler. The final section of this chapter shows you how to turn C functions you've created with inline into C code for a package
Even if you've never used C before, you should be able to read C code because the basic structure is similar to R. If you want to learn it more formally The C programming language by Kernigan and Ritchie is a classic.
Important differences from R include:
- variables can only store specific types of object, and must be declared before use
- objects are modified in place, unless you specifically copy the object
- indices start at 0, not 1
- you must use a semi-colon at end of each line
- you must have an explicit return statement
- assignment is done with
=, not<-
Generally, calling C functions from R involves two parts: a C function and an R function that uses .Call. The simple function below adds two numbers together and illustrates some of the important features of coding in C (creating new R vectors, coercing input arguments to the appropriate type and dealing with garbage collection).
# In C ----------------------------------------
#include <R.h>
#include <Rinternals.h>
SEXP add(SEXP a, SEXP b) {
SEXP result;
PROTECT(result = allocVector(REALSXP, 1));
REAL(result)[0] = asReal(a) + asReal(b);
UNPROTECT(1);
return(result);
}
# In R ----------------------------------------
add <- function(a, b) {
.Call("add", a, b)
}
In this chapter we'll produce these two pieces in one step by using the inline package. This allows us to write:
add <- cfunction(signature(a = "integer", b = "integer"), "
SEXP result;
PROTECT(result = allocVector(REALSXP, 1));
REAL(result)[0] = asReal(a) + asReal(b);
UNPROTECT(1);
return(result);
")
add(1, 5)
The C functions and macros that R provides for us to modify R data structures are all defined in the header file Rinternals.h. It's easiest to find and display this file from within R:
rinternals <- file.path(R.home(), "include", "Rinternals.h")
file.show(rinternals)
Before we begin writing and reading C code, we need to know a little about the basic data structures.
At the C-level, all R objects are stored in a common datatype, the SEXP. (Technically, this is a pointer to a structure with typedef SEXPREC). A SEXP is a variant type, with subtypes for all R's data structures. The most important types are:
-
REALSXP: numeric vectors -
INTSXP: integer vectors -
LGLSXP: logical vectors -
STRSXP: character vectors -
VECSXP: lists -
CLOSXP: functions (closures) -
ENVSXP: environments
Beware: At the C level, R's lists are VECSXPs not LISTSXPs. This is because early implementations of R used LISP-like linked lists (now known as "pairlists") before moving to the S-like generic vectors that we now know as lists.
There are also SEXPs for less common object types:
-
CPLXSXP: complex vectors -
LISTSXP: "pair" lists, in R used only for function arguments -
DOTSXP: '...' -
SYMSXP: names/symbols -
NILSXP:NULL
And SEXPs for internal objects, objects that are usually only created and used by C functions, not R functions:
-
LANGSXP: language constructs -
CHARSXP: "scalar" strings (see below) -
PROMSXP: promises, lazily evaluated function arguments -
EXPRSXP: expressions
There's no built-in R function to easily access these names, but we can write one: (This is adapted from code in R's inspect.c)
sexp_type <- cfunction(c(x = "ANY"), '
switch (TYPEOF(x)) {
case NILSXP: return mkString("NILSXP");
case SYMSXP: return mkString("SYMSXP");
case LISTSXP: return mkString("LISTSXP");
case CLOSXP: return mkString("CLOSXP");
case ENVSXP: return mkString("ENVSXP");
case PROMSXP: return mkString("PROMSXP");
case LANGSXP: return mkString("LANGSXP");
case SPECIALSXP: return mkString("SPECIALSXP");
case BUILTINSXP: return mkString("BUILTINSXP");
case CHARSXP: return mkString("CHARSXP");
case LGLSXP: return mkString("LGLSXP");
case INTSXP: return mkString("INTSXP");
case REALSXP: return mkString("REALSXP");
case CPLXSXP: return mkString("CPLXSXP");
case STRSXP: return mkString("STRSXP");
case DOTSXP: return mkString("DOTSXP");
case ANYSXP: return mkString("ANYSXP");
case VECSXP: return mkString("VECSXP");
case EXPRSXP: return mkString("EXPRSXP");
case BCODESXP: return mkString("BCODESXP");
case EXTPTRSXP: return mkString("EXTPTRSXP");
case WEAKREFSXP: return mkString("WEAKREFSXP");
case S4SXP: return mkString("S4SXP");
case RAWSXP: return mkString("RAWSXP");
default: return mkString("<unknown>");
}')
sexp_type(10)
sexp_type(10L)
sexp_type("a")
sexp_type(T)
sexp_type(list(a = 1))
sexp_type(pairlist(a = 1))
R character vectors are stored as STRSXPs, a vector type where every element is a CHARSXP. CHARSXPs are read-only objects and must never be modified. In particular, the C-style string contained in a CHARSXP should be treated as read-only; it's hard to do otherwise because the CHAR accessor function returns a const char*.
Strings have this more complicated design because individual CHARSXP's (elements of a character vector) can be shared between multiple strings. This is an optimisation to reduce memory usage, and can result in unexpected behaviour:
x <- "banana"
y <- rep(x, 1e6)
object.size(x)
# 64 bytes
object.size(y) / 1e6
# 4.000056 bytes
(This is also the reason why factors are no more memory efficient than strings. factor(y) is actually slightly bigger than y)
At the heart of every C function will be a set of conversions between R data structures and C data structures. Inputs and output will always be R data structures (SEXPs) and you will need to convert them to C data structures in order to do any work. An additional complication is the garbage collector: if you don't claim every R object you create, the garbage collector will think they are unused and delete them.
The simplest way to create an new R-level object is allocVector, which takes two arguments, the type of SEXP (or SEXPTYPE) to create, and the length of the vector. The following code code creates a three element list containing a logical vector, a numeric vector and an integer vector:
dummy <- cfunction(body = '
SEXP vec, real, lgl, ints;
PROTECT(real = allocVector(REALSXP, 2));
PROTECT(lgl = allocVector(LGLSXP, 10));
PROTECT(ints = allocVector(INTSXP, 10));
PROTECT(vec = allocVector(VECSXP, 3));
SET_VECTOR_ELT(vec, 0, real);
SET_VECTOR_ELT(vec, 1, lgl);
SET_VECTOR_ELT(vec, 2, ints);
UNPROTECT(4);
return(vec);
')
dummy()
You might wonder what all the PROTECT calls do. They tell R that we're currently using each object, and not to delete it if the garbage collector is activated. (We don't need to protect objects that R already knows about, like function arguments).
You also need to make sure that every protected object is unprotected. UNPROTECT takes a single integer argument, n, and unprotects the last n objects that were protected. If your calls don't match, R will warn about a "stack imbalance in .Call".
Other specialised forms of PROTECT and UNPROTECT are needed in some circumstances: UNPROTECT_PTR(s) unprotects the
object pointed to by the SEXP s, PROTECT_WITH_INDEX saves an index of the protection location that can be used to replace the protected value using REPROTECT. Consult R externals for more details.
If you run dummy() a few times, you'll notice the output is basically random. This is because allocVector assigns memory to each output, but it doesn't clean it out first. For real functions, you'll want to loop through each element in the vector and zero it out. The most efficient way to do that is to use memset:
zeroes <- cfunction(c(n_ = "integer"), '
int n = asInteger(n_);
SEXP out;
PROTECT(out = allocVector(INTSXP, n));
memset(INTEGER(out), 0, n * sizeof(int));
UNPROTECT(1);
return out;
')
zeroes(10);
There are a few shortcuts for allocating matrices and 3d arrays:
allocMatrix(SEXPTYPE mode, int nrow, int ncol)
alloc3DArray(SEXPTYPE mode, int nrow, int ncol, int nface)
Beware allocList - it creates a pairlist, not a regular list.
The mkNamed function simplifies the creation of named vectors. The following code is equivalent to list(a = NULL, b = NULL, c = NULL):
const char *names[] = {"a", "b", "c", ""};
mkNamed(VECSXP, names);
There is a helper function for each atomic vector (apart from character, see following) that allows you to index into a SEXP and access the C-level data structure that lives at its heart.
The following example shows how to use the helper function REAL to inspect and modify a numeric vector:
add_one <- cfunction(c(x = "numeric"), "
SEXP out;
int n = length(x);
PROTECT(out = allocVector(REALSXP, n));
for (int i = 0; i < n; i++) {
REAL(out)[i] = REAL(x)[i] + 1;
}
UNPROTECT(1);
return out;
")
add_one(as.numeric(1:10))
There are similar helpers for logical, LOGICAL(x), integer, INTEGER(x), complex COMPLEX(x) and raw vectors RAW(x).
If you're working with long vectors, there's a performance advantage to using the helper function once and saving the result in a pointer:
add_two <- cfunction(c(x = "numeric"), "
SEXP out;
int n = length(x);
double *px, *pout;
PROTECT(out = allocVector(REALSXP, n));
px = REAL(x);
pout = REAL(out);
for (int i = 0; i < n; i++) {
pout[i] = px[i] + 2;
}
UNPROTECT(1);
return out;
")
add_two(as.numeric(1:10))
library(microbenchmark)
x <- as.numeric(1:1e6)
microbenchmark(
add_one(x),
add_two(x)
)
On my computer, add_two is about twice as fast as add_one for a million element vector. This is a common idiom in the R source code.
Strings and lists are more complicated because the individual elements are SEXPs not C-level data structures. You can use STRING_ELT(x, i) and VECTOR_ELT(x, i) to extract individual components of strings and lists respectively. To get a single C string from a element in a R character vector, use CHAR(STRING_ELT(x, i)). Set values in a list or character vector with SET_VECTOR_ELT and SET_STRING_ELT.
String vectors are a little more complicated. As discussed earlier, a string vector is a vector made up of pointers to immutable CHARSXPs, and it's the CHARSXP that contains the C string (which can be extracted using CHAR). The following function shows how to create a vector of fixed values:
abc <- cfunction(NULL, '
SEXP out;
PROTECT(out = allocVector(STRSXP, 3));
SET_STRING_ELT(out, 0, mkChar("a"));
SET_STRING_ELT(out, 1, mkChar("b"));
SET_STRING_ELT(out, 2, mkChar("c"));
UNPROTECT(1);
return out;
')
abc()
Things are a little harder if you want to modify the strings in the vector because you need to know a lot about string manipulation in C (which is hard, and harder to do right). For any problem that involves any kind of string modification, you're better off using Rcpp.
The following function just makes a copy of a string, so you can at least see how all the pieces work together.
copy <- cfunction(c(x = "character"), '
SEXP out;
int n = length(x);
const char* letter;
PROTECT(out = allocVector(STRSXP, n));
for (int i = 0; i < n; i++) {
letter = CHAR(STRING_ELT(x, i));
SET_STRING_ELT(out, i, mkChar(letter));
}
UNPROTECT(1);
return out;
')
copy(letters)
There also a few helper functions if you want to turn the first element of an R vector into a C scalar:
asLogical(x): INTSXP -> intasInteger(x): INTSXP -> intasReal(x): REALSXP -> doubleCHAR(asChar(x)): STRSXP -> const char*
And similarly it's easy to turn a C scalar into a length-one R vector:
ScalarLogical(x): int -> LGLSXPScalarInteger(x): int -> INTSXPScalarReal(x): double -> REALSXPmkString(x): const char* -> STRSXP
These all create R-level objects, so need to be PROTECTed.
You must be very careful when modifying an object that the user has passed into the function. The following function has some very unexpected behaviour:
add_three <- cfunction(c(x = "numeric"), '
REAL(x)[0] = REAL(x)[0] + 3;
return(x);
')
x <- 1
y <- x
add_three(x)
x
y
Not only has it modified the value of x, but it has also modified y! This happens because of the way that R implements it's copy-on-modify philosophy. It does so lazily, so a complete copy only has to be made if you make a change: x and y point to the same object, and the object is only duplicated if you change either x or y.
To avoid problems like this, always duplicate() inputs before modifying them:
add_four <- cfunction(c(x = "numeric"), '
SEXP x_copy;
PROTECT(x_copy = duplicate(x));
REAL(x_copy)[0] = REAL(x_copy)[0] + 4;
UNPROTECT(1);
return(x_copy);
')
x <- 1
y <- x
add_four(x)
x
y
R hides a few details of the underlying datastructures it uses. In some places in R code, it looks like you're working with a list (VECSXP), but behind the scenes R you're actually modifying a pairlist (LISTSXP). These include attributes, calls and ....
Pairlists differ from lists in the following ways:
-
Pairlists are linked lists, a data structure which does not have any easy way to get to an arbitrary element of the list
-
Pairlists have
tags, notnames, and tags are symbols, not strings.
Because you can't easily index into a specified location in a pairlist, R provides a set of helper functions to moved along the linked list. The basic helpers are CAR which extracts the first element of the first, and CDR which extracts the rest. These can be composed to get CAAR, CDAR, CADDR, CADDR, CADDDR. As well as the getters, R also provides SETCAR, SETCDR etc. (Or you can just use )
car <- cfunction(c(x = "ANY"), 'return(CAR(x));')
cdr <- cfunction(c(x = "ANY"), 'return(CDR(x));')
cadr <- cfunction(c(x = "ANY"), 'return(CADR(x));')
x <- quote(f(a = 1, b = 2))
car(x)
cdr(x)
car(cdr(x))
cadr(x)
You can make new pairlists with CONS or LCONS (if you want a call object). A pairlist is always terminated with R_NilValue.
new_call <- cfunction(NULL, '
SEXP out;
out = LCONS(install("+"), LCONS(
ScalarReal(10), LCONS(
ScalarReal(5), R_NilValue)));
return out;
')
new_call();
Similarly, you can loop through all elements of a pairlist as follows:
count <- cfunction(c(x = "ANY"), '
SEXP el, nxt;
int i = 0;
for(nxt = x; nxt != R_NilValue; el = CAR(nxt), nxt = CDR(nxt)) {
i++;
}
return(ScalarInteger(i));
')
count(quote(f(a, b, c)))
count(quote(f()))
TAG and SET_TAG allow you to get and set the tag (aka name) associated with an element of a pairlist. The tag should be a symbol. To create a symbol (the equivalent of as.symbol or as.name in R), use install.
Attributes are also pairlists behind the scenes, but come with the helper functions setAttrib and getAttrib to make access a little easier:
set_attr <- cfunction(c(obj = "ANY", attr = "character", value = "ANY"), '
const char* attr_s = CHAR(asChar(attr));
duplicate(obj);
setAttrib(obj, install(attr_s), value);
return(obj);
')
x <- 1:10
set_attr(x, "a", 1)
There are some (confusingly named) shortcuts for common setting operations: classgets, namesgets, dimgets and dimnamesgets are the internal versions of the default methods of class<-, names<-, dim<- and dimnames<-.
tags <- cfunction(c(x = "ANY"), '
SEXP el, nxt, out;
int i = 0;
for(nxt = CDR(x); nxt != R_NilValue; nxt = CDR(nxt)) {
i++;
}
PROTECT(out = allocVector(VECSXP, i));
for(nxt = CDR(x), i = 0; nxt != R_NilValue; i++, nxt = CDR(nxt)) {
SET_VECTOR_ELT(out, i, TAG(nxt));
}
UNPROTECT(1);
return out;
')
tags(quote(f(a = 1, b = 2, c = 3)))
tags(quote(f()))
R's NA is a subtype of NaN so IEEE 754 arithmetic should handle it
correctly. However, it is unwise to depend on such details, and is better to deal with missings explicitly:
-
In
REALSXPs, use theISNAmacro,ISNAN, orR_FINITEmacros to check for missing, NaN or non-finite values. Use the constantsNA_REAL,R_NaN,R_PosInfandR_NegInfto set those values -
In
INTSXPs, compare/set values toNA_INTEGER -
In
LGLSXPs, compare/set values toNA_LOGICAL -
In
STRSXPs, compare/setCHAR(STRING_ELT(x, i))values toNA_STRING.
For example, a primitive implementation of is.NA might look like
is_na <- cfunction(c(x = "ANY"), '
SEXP out;
int n = length(x);
PROTECT(out = allocVector(LGLSXP, n));
for (int i = 0; i < n; i++) {
switch(TYPEOF(x)) {
case LGLSXP:
LOGICAL(out)[i] = (LOGICAL(x)[i] == NA_LOGICAL);
break;
case INTSXP:
LOGICAL(out)[i] = (INTEGER(x)[i] == NA_INTEGER);
break;
case REALSXP:
LOGICAL(out)[i] = ISNA(REAL(x)[i]);
break;
case STRSXP:
LOGICAL(out)[i] = (STRING_ELT(x, i) == NA_STRING);
break;
default:
LOGICAL(out)[i] = NA_LOGICAL;
}
}
UNPROTECT(1);
return out;
')
is_na(c(NA, 1L))
is_na(c(NA, 1))
is_na(c(NA, "a"))
is_na(c(NA, TRUE))
There are a few other special values:
nil <- cfunction(NULL, 'return(R_NilValue);')
unbound <- cfunction(NULL, 'return(R_UnboundValue);')
missing_arg <- cfunction(NULL, 'return(R_MissingArg);')
x <- missing_arg()
x
If the user provides different input to your function to what you're expecting (e.g. provides a list instead of a numeric vector), it's very easy to crash R. For this reason, it's a good idea to write a wrapper function that checks arguments are of the correct type, or coerces them if necessary. It's usually easier to do this at the R level. For example, going back to our first example of C code, we might rename it to add_ and then write a wrapper function to check the inputs are ok:
add_ <- cfunction(signature(a = "integer", b = "integer"), "
SEXP result;
PROTECT(result = allocVector(REALSXP, 1));
REAL(result)[0] = asReal(a) + asReal(b);
UNPROTECT(1);
return(result);
")
add <- function(a, b) {
stopifnot(is.numeric(a), is.numeric(b), length(a) == 1, length(b) == 1)
add_(a, b)
}
Or if we wanted to be more accepting of diverse inputs:
add <- function(a, b) {
a <- as.numeric(a)
b <- as.numeric(b)
if (length(a) > 1) warning("Only first element of a used")
if (length(a) > 1) warning("Only first element of b used")
add_(a, b)
}
To coerce objects at the C level, use PROTEXT(new = coerceVector(old, SEXPTYPE)). This will return an error if the SEXP can not be converted to the desired type. Note that these coercion functions do not use S3 dispatch.
To check if an object is of a specified type, you can use TYPEOF, which returns a SEXPTYPE:
is_numeric <- cfunction(c("x" = "ANY"), "
return(ScalarLogical(TYPEOF(x) == REALSXP));
")
is_numeric(7)
is_numeric("a")
Or you can use one of the many helper functions. They all return 0 for FALSE and 1 for TRUE:
-
For atomic vectors:
isInteger,isReal,isComplex,isLogical,isString. -
For combinations of atomic vectors:
isNumeric(integer, logical, real),isNumber(integer, logical, real, complex), isVectorAtomic (logical, interger, numeric, complex, string, raw) -
Matrices (
isMatrix) and arrays (isArray) -
For other more esoteric object:
isEnvironment,isExpression,isList(a pair list),isNewList(a list),isSymbol,isNull,isObject(S4 objects),isVector(atomic vectors, lists, expressions)
Note that some of these functions behave differently to the R-level functions with similar names. For example isVector is true for any atomic vector type, lists and expression, where is.vector is returns TRUE only if its input has no attributes apart from names.
In many R functions you'll find code like .Internal(mean(x)) or .Primitive("sum"). That means that most of the function is implemented at the C-level. There are two steps to finding the corresponding C source code:
-
First, open src/main/names.c and search for the name of the function. You'll find an entry that tells you the name of the function (which always starts with
do_) -
Next, search the R source code for the name of that function. To make it easier to find where it's defined (rather than everywhere it's used), you can add
(SEXP. e.g. to find the source code forfindInterval, search fordo_findinterval(SEXP.
R_ShowMessage
warning
error
Rprintf
R_CheckUserInterrupt
An alternative to using .Call is to use .External. It is used almost identically, except that the C function will recieve a single arugment containing a LISTSXP, a pairlist from which the
arguments can be extracted. This makes it possible to write functions that take a variable number of arguments.
inline does not currently support .External functions.
If you're putting your code in a package, it's generally a good idea to stop using inline and revert back to separate R and C functions. At a minimum, you'll need:
- R files in a
R/directory - C files in
src/directory - A
DESCRIPTIONin the main directory - A
NAMESPACEfile containinguseDynLib(packagename), which can be generated using aroxygen2tag:@useDynLib packagename
Running load_all(path/to/package) will automatically compile and reload the code in your package.
Your C code will need these headers:
#include <R.h>
#include <Rinternals.h>