Error In Serializedata Node$con Error Writing To Connection
Contents |
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the error in serialize(data, node$con, xdr = false) : error writing to connection workings and policies of this site About Us Learn more about Stack
R Error In Unserialize(socklist[[n]]) : Error Reading From Connection
Overflow the company Business Learn more about hiring developers or posting ads with us Stack Overflow Questions Jobs calls:
R Error Writing To Connection
to connection My educated guess is that maybe the connection that I open using the commands below, has expired: ## Register Cluster ## cores<-8 cl <- makeCluster(cores) registerDoParallel(cl) Looking at makeCluster man page I see that by default the connections expires only after 30 days! I could set options(error=recover) in order to check, on the fly, if the connection is opened or not when the code halts, but I decided to post this general question before. IMPORTANT: 1) the error is really intermittent, sometimes I re-run the same code and get no errors. 2) I run everything on the same multi-core machine (Intel/8 cores). So it is not a communation (network) problem among the clusters. 3) I am a heavy user of CPU and GPU parallelization, on my laptop and desktop (64 cores) Unfortunately, it is the first time that I am getting this type of error. Is anybody having the same type of error? As requested I am providing my sessionInfo(): > sessionInfo() R version 2.15.3 (2013-03-01) Platform: x86_64-w64-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_Uni
Sign in Pricing Blog Support Search GitHub This repository Watch 56 Star 251 Fork 95 stan-dev/rstan Code Issues 53 Pull requests 6 Projects 0 Wiki Pulse Graphs New error in unserialize error reading from connection issue Error with "unserialize(node$con) : error reading from connection" #243 Open rBatt opened makecluster outfile this Issue Dec 13, 2015 · 34 comments Projects None yet Labels None yet Milestone No milestone
Error In Summary.connection(connection) : Invalid Connection
Assignees No one assigned 3 participants rBatt commented Dec 13, 2015 EDIT: The manifestation of this problem has morphed over time. Currently, models won't run with large data sets at all, http://stackoverflow.com/questions/17015598/error-calling-serialize-r-function even if on 1 core. I only get this error when I'm trying to analyze the full version of my data set using multiple cores. If I analyze the full data set on 1 core (1 chain, 4 iterations), it runs. Similarly, if I run the model for a very small fraction of the data set on multiple cores (4 cores, 4 chains, 4 https://github.com/stan-dev/rstan/issues/243 iterations), it also runs fine. Below are the errors for a remote Linux machine that had ~133 GB RAM available at the time. The schools example runs fine over there (both on 1 core, and 4, and 10). However, I also get similar errors on my local Mac. Sometimes I get the same error message, other times R just crashes (the console just goes away; this is not a 'hang'!). Looking at the resources, neither system appears to be having any trouble lead up to the crash/ error. starting worker pid=34649 on localhost:11567 at 17:34:31.565 starting worker pid=34658 on localhost:11567 at 17:34:31.798 starting worker pid=34667 on localhost:11567 at 17:34:32.032 starting worker pid=34676 on localhost:11567 at 17:34:32.268 SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 1). SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 2). SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 3). SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 4). Error in unserialize(node$con) : error reading from connection Calls: stan ... FUN -> recvData -> recvData.SOCK0node -> unserialize Session info for the Linux: > sessionInfo() R version 3.2.2 (2015-08-14) Platform: x86_64-redhat-linux-gnu (64-bit) Running under: CentOS release 6.6 (Final) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] L
in threaded view ♦ ♦ | Report Content as Inappropriate ♦ ♦ Intermittent error using clusterR Hi, I wrote a code which uses clusterR to run a function on multiple CPUs [see this http://r-sig-geo.2731867.n2.nabble.com/Intermittent-error-using-clusterR-td7585360.html thread showing part of the code, that has fixed based on Robert's help: https://stat.ethz.ch/pipermail/r-sig-hpc/2011-March/000934.html http://r-sig-geo.2731867.n2.nabble.com/cluster-error-with-clusterR-and-calc-td7585148.html]. Intermittently, my code stops with this error (along with its traceback): Error in serialize(data, node$con) : error writing to connection serialize(data, node$con) sendData.SOCKnode(con, list(type = type, data = value, tag = tag)) sendData(con, list(type = type, data = value, tag = tag)) postNode(con, "EXEC", list(fun = fun, args = args, return = return, error in sendCall(cl[[i]], clusfun, list(fun, i), tag = i) clusterR(s, fun1, filename = f) at calculateCloudH.R#33 calculateCloudH(current.h, t.cloudbase, t.cloud, file.cloudH, at fmask.R#332 fmask(mtl, filename = ofile, nodes = 8, overwrite = TRUE) Sometimes it doesn't stop, some other times it does. It may be similar to this same problem in which they attribute it to a call to a C function through .Call() [http://stackoverflow.com/questions/17015598/error-calling-serialize-r-function]. As in that thread, error writing to showConnections() show my connections are still opened and ready for read/write (but, there are 6 connections instead of 8: maybe this is due to this call I use when I initialize the cluster, on.exit(endCluster()), and which didn't do is job completely...?) Has anyone else ever experienced this same problem with clusterR? If so, I would appreciate your advice on how you fixed it if you did. Thanks! >sessionInfo(): R version 3.0.2 (2013-09-25) Platform: x86_64-w64-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=French_Canada.1252 LC_CTYPE=French_Canada.1252 LC_MONETARY=French_Canada.1252 [4] LC_NUMERIC=C LC_TIME=French_Canada.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] igraph_0.6.6 snow_0.3-13 fmask_0.1 RSAGA_0.93-6 plyr_1.8 [6] shapefiles_0.7 foreign_0.8-55 gstat_1.0-18 data.table_1.8.10 rgdal_0.8-14 [11] raster_2.1-68 sp_1.0-14 roxygen2_3.0.0 loaded via a namespace (and not attached): [1] brew_1.0-6 codetools_0.2-8 digest_0.6.3 grid_3.0.2 intervals_0.14.0 lattice_0.20-23 [7] spacetime_1.0-9 stringr_0.6.2 tools_3.0.2 xts_0.9-7 zoo_1.7-10 juejack Threaded Open this pos
[ignore, fixed] Re: SNOW Issues: Error in writing to connection (on localhost) Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Hi All- I am having trouble on a new machine with a fresh install of R and the requisite SNOW packages (snow, foreach, doSNOW). When I attempt to start 2 instances on localhost and use the %dopar% utility, I get the following message: Error in serialize(data, node$con) : error writing to connection Here's that systems' details: OS: Linux Mint 10 (Ubuntu 10.10) SessionInfo (relevant packages): other attached packages: [1] doSNOW_1.0.3 foreach_1.3.0 codetools_0.2-8 [4] iterators_1.0.3 fogbank_2.1.0 XML_3.2-0 [7] fExoticOptions_2110.77 fOptions_2110.78 snow_0.3-3 When I look at the created cluster object: > cl.tmp [[1]] $con description class "<-localhost.localdomain:10187" "sockconn" mode text "a+b" "binary" opened can read "opened" "yes" can write "yes" *************** On another system, the actual hostname of the system is shown (and I'm able to run the code successfully): > cl.tmp2 [[1]] $con description class "<-fogb-chi-cj1:10187" "sockconn" mode text "a+b" "binary" opened can read "opened" "yes" can write "yes" Here's what I'm trying to run. Unfortunately, the data is too large to attach: xj = colnames(HistoricalYields) require(snow) require(doSNOW) require(foreach) cl.tmp = makeCluster(rep("localhost",2), type="SOCK") registerDoSNOW(cl.tmp) parallel.arima <- function(data) { library(forecast) fit = auto.arima(ts(HistoricalYields[,data]), approximation=TRUE, allowdrift=TRUE, stepwise=TRUE) } system.time(res <- foreach(dat=xj) %dopar% parallel.arima(dat)) ** Error (as above) One more note, this appears to work for the single case. I'm not quite sure what the ramifications would be if I tried running this with distributed nodes