Edmonton R User Group meeting, April 26, 2013

1. Example Parallel Overview snow fork Summary
Parallel Computing with R
Péter Sólymos
Edmonton R User Group meeting, April 26, 2013

2. Example Parallel Overview snow fork Summary
Ovenbird example from 'detect' package
> str(oven)
'data.frame': 891 obs. of 11 variables:
$ count : int 1 0 0 1 0 0 0 0 0 0 ...
$ route : int 2 2 2 2 2 2 2 2 2 2 ...
$ stop : int 2 4 6 8 10 12 14 16 18 20 ...
$ pforest: num 0.947 0.903 0.814 0.89 0.542 ...
$ pdecid : num 0.575 0.562 0.549 0.679 0.344 ...
$ pagri : num 0 0 0 0 0.414 ...
$ long : num 609343 608556 607738 607680 607944 ...
$ lat : num 5949071 5947735 5946301 5944720 5943088 ...
$ observ : Factor w/ 4 levels "ARS","DW","RDW",..: 4 4 4 4 4 4 4 4 4 4 ...
$ julian : int 181 181 181 181 181 181 181 181 181 181 ...
$ timeday: int 2 4 6 8 10 12 14 16 18 20 ...

3. Example Parallel Overview snow fork Summary
NegBin GLM with bootstrap
> library(MASS)
> m <- glm.nb(count ~ pforest, oven)
> fun1 <- function(i) {
+ id <- sample.int(nrow(oven), nrow(oven), replace = TRUE)
+ coef(glm.nb(count ~ pforest, oven[id, ]))
+ }
> B <- 199
> system.time(bm <- sapply(1:B, fun1))
user system elapsed
26.79 0.02 27.11
> bm <- cbind(coef(m), bm)
> cbind(coef(summary(m))[, 1:2], `Boot. SE` = apply(bm, 1, sd))
Estimate Std. Error Boot. SE
(Intercept) -2.177 0.1277 0.1229
pforest 2.674 0.1709 0.1553

4. Example Parallel Overview snow fork Summary
Parallel bootstrap
> library(parallel)
> (cl <- makePSOCKcluster(3))
socket cluster with 3 nodes on host 'localhost'
> clusterExport(cl, "oven")
> tmp <- clusterEvalQ(cl, library(MASS))
> t0 <- proc.time()
> bm2 <- parSapply(cl, 1:B, fun1)
> proc.time() - t0
user system elapsed
0.00 0.00 11.06
> stopCluster(cl)

5. Example Parallel Overview snow fork Summary
High performance computing (HPC)
ˆ Parallel computing,
ˆ large memory and out-of-memory data,
ˆ interfaces for compiled code,
ˆ proling tools,
ˆ batch scheduling.
CRAN Task View: High-Performance and Parallel Computing with R

6. Example Parallel Overview snow fork Summary
Parallel computing
Embarassingly parallel problems:
ˆ bootstrap,
ˆ MCMC,
ˆ simulations.
Can be broken down into independent pieces.1
1Schmidberger et al. 2009 JSS: State of the Art in Parallel Computing with R

7. Example Parallel Overview snow fork Summary
Parallel computing
ˆ explicit (distributed memory),
ˆ implicit (shared memory),
ˆ grid,
ˆ Hadoop,
ˆ GPUs.

8. Example Parallel Overview snow fork Summary
Starting a cluster
library(snow)
cl - makeCluster(3, type = SOCK)
Cluster types:
ˆ SOCK, multicore
ˆ PVM, Parallel Virtual Machine
ˆ MPI, Message Passing Interface
ˆ NWS, NetWorkSpaces (multicore grid)
Error: invalid connection

9. Example Parallel Overview snow fork Summary
Distribute stu, evaluate expressions
clusterExport(cl, oven)
clusterEvalQ(cl, library(MASS))
[[1]]
[1] MASS methods stats graphics
[5] grDevices utils datasets base
[[2]]
[1] MASS methods stats graphics
[5] grDevices utils datasets base
[[3]]
[1] MASS methods stats graphics
[5] grDevices utils datasets base

10. Example Parallel Overview snow fork Summary
Random Number Generation (RNG)
library(rlecuyer)
tmp - clusterEvalQ(cl, set.seed(1234))
clusterEvalQ(cl, rnorm(5))
[[1]]
[1] -1.2071 0.2774 1.0844 -2.3457 0.4291
[[2]]
[1] -1.2071 0.2774 1.0844 -2.3457 0.4291
snow:::clusterSetupRNG(cl)
[1] RNGstream
clusterEvalQ(cl, rnorm(5))
[[1]]
[1] -1.14063 -0.49816 -0.76670 -0.04821 -1.09852
[[2]]
[1] 0.7050 0.4821 -1.2848 0.7198 0.7386
Important when calculating indices or doing simulations.

11. Example Parallel Overview snow fork Summary
Apply operations: split
parallel:::parLapply
function (cl = NULL, X, fun, ...)
{
cl - defaultCluster(cl)
do.call(c, clusterApply(cl, x = splitList(X, length(cl)),
fun = lapply, fun, ...), quote = TRUE)
}
bytecode: 0x04c1eba8
environment: namespace:parallel
snow:::splitList(1:10, length(cl))
[[1]]
[1] 1 2 3 4 5
[[2]]
[1] 6 7 8 9 10

12. Example Parallel Overview snow fork Summary
Apply operations: evaluate and combine
f - function(i) i * 2
(res - clusterApply(cl, snow:::splitList(1:10, length(cl)),
+ f))
[[1]]
[1] 2 4 6
[[2]]
[1] 8 10 12 14
[[3]]
[1] 16 18 20
do.call(c, res)
[1] 2 4 6 8 10 12 14 16 18 20

13. Example Parallel Overview snow fork Summary
Apply operations: load balancing
f - function(i) i * 2
unlist(parallel:::parLapplyLB(cl, 1:10, f))
[1] 2 4 6 8 10 12 14 16 18 20

14. Example Parallel Overview snow fork Summary
Implicit parallelism
No need to distribute stu, only evaluate on child processes.
mclapply(X, FUN, mc.cores)

15. Example Parallel Overview snow fork Summary
Summary
Parallel computing is not hard on a single computer.
Diculty comes in when using large, shared, and heterogeneous
resources.
stopCluster(cl)