Commit b3f2abeb authored by Fabien Priotto's avatar Fabien Priotto
Browse files

default.R added

parent 8bd41e40
Distance Calculator V 0.0:
This is to determinate distances beetween reef and cities. The entrie is a list of reefs given by number.
distance_calculator-FP.R: This script is to determinate distances beetween reef and cities. The entrie is a list of reefs given by number.
default.R: this is a short script to compare mono-thread and multi-thread computing (mcapply Vs parLapply)
#######################################################
# R parallel
# Fabien Priotto 2017
#######################################################
# This is a test to use the parLapply function from the parallel library (snow forked).
# The calcul is a simple sorting a list of "num numbers" between 0 and "roof"
#
# parLapply <- function (cl = NULL, X, fun, ...)
# {
# cl <- defaultCluster(cl)
# do.call(c, clusterApply(cl, x = splitList(X, length(cl)),
# fun = lapply, fun, ...), quote = TRUE)
# }
#######################################################
# Note the splitList(X, length(cl)). This will split the tasks into even portions and send them onto the workers.
#######################################################
rm(list=ls());
list.of.packages <- c("parallel","utils")
working_directory_init=getwd()
setwd(working_directory_init)
result_file=paste(working_directory_init,"/default.txt",sep="")
trace_file = paste(working_directory_init,"/default.log",sep="")
tr <- file(trace_file, open = "wt")
############## STARTING DEFUN TIME ###############################
# Multi-thread Shared environnement
# Cursors to set the calcul stronger
num <- 100000
roof <- 1000000
# Generate a list of "num" integers between 0 and "roof"
mylist <- floor(runif(num, min=0, max=roof))
mysize <- object.size(mylist)
op <- options(digits.secs = 12)
options(op)
now <- format(Sys.time(), "%a %b %d %X %Y")
cat ("See logs in",trace_file,"\n")
# Start writing to the log file
tr <- file(trace_file, open = "wt")
sink(tr, append=TRUE)
sink(tr, append=TRUE, type="message")
cat("---", c(now), "job started ---\n")
cat ("See results in",result_file,"\n")
new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
if(length(new.packages)) install.packages(new.packages, repos="https://cran.r-project.org/")
require(parallel)
require(utils)
no_cores <- detectCores()
############## END OF DEFUN TIME ###############################
cat("#######################################################\n")
# FORK cluster
cat("--- Start new job on using cluster type Fork:\n")
cl <- makeCluster(no_cores, type="FORK", outfile = trace_file)
############## STARTING RUN TIME ###############################
# The formal function form to call with parlapply:
todo <- function (X) {
X*X
}
# Exported env
clusterExport( cl, c("roof", "num", "result_file") )
cat ("Data size=")
print(mysize, units = "auto", standard = "SI")
cat("#######################################################\n")
cat("Compute a list of",num,"numbers between 0 and",roof,"USING parLapply:\n")
# If you need to check the list firtly:
## capture.output(cat("My list:",mylist), file = result_file, append = FALSE )
## You can compare bench times:
cat("############ Using mclapply: \n")
system.time( capture.output( mclapply( mylist, todo, mc.cores=no_cores ), file = result_file, append = FALSE) )
cat("############ Using parLapply: \n")
system.time( capture.output( parLapply( cl, mylist, todo ), file = result_file, append = TRUE ) )
cat("#######################################################\n")
stopCluster(cl)
sink(type = "message")
sink()
file.show(trace_file)
gc()
R version 3.3.3 (2017-03-06) -- "Another Canoe"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R est un logiciel libre livré sans AUCUNE GARANTIE.
Vous pouvez le redistribuer sous certaines conditions.
Tapez 'license()' ou 'licence()' pour plus de détails.
R est un projet collaboratif avec de nombreux contributeurs.
Tapez 'contributors()' pour plus d'information et
'citation()' pour la façon de le citer dans les publications.
Tapez 'demo()' pour des démonstrations, 'help()' pour l'aide
en ligne ou 'help.start()' pour obtenir l'aide au format HTML.
Tapez 'q()' pour quitter R.
[Sauvegarde de la session précédente restaurée]
> #######################################################
> # R parallel
> # Fabien Priotto 2017
> #######################################################
> # This is a test to use the parLapply function from the parallel library (snow forked).
> # The calcul is a simple sorting a list of "num numbers" between 0 and "roof"
> #
> # parLapply <- function (cl = NULL, X, fun, ...)
> # {
> # cl <- defaultCluster(cl)
> # do.call(c, clusterApply(cl, x = splitList(X, length(cl)),
> # fun = lapply, fun, ...), quote = TRUE)
> # }
> #######################################################
> # Note the splitList(X, length(cl)). This will split the tasks into even portions and send them onto the workers.
> #######################################################
> rm(list=ls());
>
> list.of.packages <- c("parallel","utils")
>
> working_directory_init=getwd()
> setwd(working_directory_init)
>
> result_file=paste(working_directory_init,"/default.txt",sep="")
> trace_file = paste(working_directory_init,"/default.log",sep="")
> tr <- file(trace_file, open = "wt")
>
> ############## STARTING DEFUN TIME ###############################
> # Multi-thread Shared environnement
>
> # Cursors to set the calcul stronger
> num <- 100000
> roof <- 1000000
>
> # Generate a list of "num" integers between 0 and "roof"
> mylist <- floor(runif(num, min=0, max=roof))
> mysize <- object.size(mylist)
>
> op <- options(digits.secs = 12)
> options(op)
>
> now <- format(Sys.time(), "%a %b %d %X %Y")
>
> cat ("See logs in",trace_file,"\n")
See logs in /media/boss/SSD250GO/DEV/R/rforkparl/default.log
>
> # Start writing to the log file
> tr <- file(trace_file, open = "wt")
> sink(tr, append=TRUE)
> sink(tr, append=TRUE, type="message")
>
> cat("---", c(now), "job started ---\n")
> cat ("See results in",result_file,"\n")
>
> new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
> if(length(new.packages)) install.packages(new.packages, repos="https://cran.r-project.org/")
>
> require(parallel)
> require(utils)
>
> no_cores <- detectCores()
> ############## END OF DEFUN TIME ###############################
> cat("#######################################################\n")
>
> # FORK cluster
> cat("--- Start new job on using cluster type Fork:\n")
> cl <- makeCluster(no_cores, type="FORK", outfile = trace_file)
>
> ############## STARTING RUN TIME ###############################
> # The formal function form to call with parlapply:
> todo <- function (X) {
+ X*X
+ }
>
> # Exported env
> clusterExport( cl, c("roof", "num", "result_file") )
>
> cat ("Data size=")
> print(mysize, units = "auto", standard = "SI")
>
> cat("#######################################################\n")
> cat("Compute a list of",num,"numbers between 0 and",roof,"USING parLapply:\n")
>
> # If you need to check the list firtly:
> ## capture.output(cat("My list:",mylist), file = result_file, append = FALSE )
>
> ## You can compare bench times:
>
> cat("############ Using mclapply: \n")
> system.time( capture.output( mclapply( mylist, todo, mc.cores=no_cores ), file = result_file, append = FALSE) )
>
> cat("############ Using parLapply: \n")
> system.time( capture.output( parLapply( cl, mylist, todo ), file = result_file, append = TRUE ) )
>
> cat("#######################################################\n")
> stopCluster(cl)
> sink(type = "message")
> sink()
> file.show(trace_file)
--- mer. mars 29 19:46:45 2017 job started ---
See results in /media/boss/SSD250GO/DEV/R/rforkparl/default.txt
Le chargement a nécessité le package : parallel
Warning message:
fermeture de la connexion inutilisée 3 (/media/boss/SSD250GO/DEV/R/rforkparl/default.log)
#######################################################
--- Start new job on using cluster type Fork:
Data size=781.3 Kb
#######################################################
Compute a list of 1e+05 numbers between 0 and 1e+06 USING parLapply:
############ Using mclapply:
utilisateur système écoulé
2.880 5.079 22.869
############ Using parLapply:
utilisateur système écoulé
2.735 5.563 23.644
#######################################################
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 245008 13.1 592000 31.7 350000 18.7
Vcells 458003 3.5 1483186 11.4 1445893 11.1
>
> proc.time()
utilisateur système écoulé
6.015 10.708 46.937
......@@ -116,4 +116,4 @@ sink()
file.show(trace_file)
# Cleaning garbage collector
gc()
\ No newline at end of file
gc()
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment