From cl@rkfitzg @ending from gm@il@com  Thu Sep 13 19:31:36 2018
From: cl@rkfitzg @ending from gm@il@com (Clark Fitzgerald)
Date: Thu, 13 Sep 2018 10:31:36 -0700
Subject: [Rd] Environments and parallel processing
In-Reply-To: <c0004f24-fed9-f9ff-9d2f-e6b7ee91163b@daqana.com>
References: <173ff2cb39cd47bfb5cc82158e9df0bc@keygene.com>
 <CABtg=KkZwe8avLGrn84GEA13Z-h_rVWG6ehLFnKe3xD+Z=Ct6g@mail.gmail.com>
 <c0004f24-fed9-f9ff-9d2f-e6b7ee91163b@daqana.com>
Message-ID: <CAE_rN=31Sdrqrb=w2HS+5RkSN3MWg7vahmm65j0s7dGHpv3M_Q@mail.gmail.com>

+1 to what Gabor and Ralf said.

In this case the memory address can be misleading. My understanding is that
the environments in all the processes, 1 parent and 2 child, have the
*same* memory address, but once you write to them the operating system
makes copies and maps the address to a *different* physical address. This
is the copy-on-write UNIX fork model.

The example below shows 3 processes that all use the same address for the
environment env1, yet there must be 3 different environments, because
env1$x has 3 different values simultaneously.

-Clark


library(parallel)
env1 <- new.env()
env1$x = 0
cl <- makeCluster(2, type="FORK")

# Now we write to the environment, and env1$x has two distinct values
clusterEvalQ(cl, env1$x <- rnorm(1))
# [[1]]
# [1] -1.296702
#
# [[2]]
# [1] -0.4001104

# The environments in the 2 child processes still have the same address,
# which is the same as the original address
parLapply(cl, 1:4, function(x) capture.output(str(env1)))
env1

# The original x is unchanged
env1$x

stopCluster(cl)


On Wed, Sep 12, 2018 at 11:31 AM Ralf Stubner <ralf.stubner at daqana.com>
wrote:

> On 12.09.2018 20:20, G?bor Cs?rdi wrote:
> > This is all normal, a fork cluster works with processes, that do not
> > share memory.
>
> And if you are after shared-memory parallelism, you can try the 'Rdsm'
> package: https://cran.r-project.org/package=Rdsm
>
> Greetings
> Ralf
>
> --
> Ralf Stubner
> Senior Software Engineer / Trainer
>
> daqana GmbH
> Dortustra?e 48
> 14467 Potsdam
>
> T: +49 331 23 61 93 11
> F: +49 331 23 61 93 90
> M: +49 162 20 91 196
> Mail: ralf.stubner at daqana.com
>
> Sitz: Potsdam
> Register: AG Potsdam HRB 27966 P
> Ust.-IdNr.: DE300072622
> Gesch?ftsf?hrer: Prof. Dr. Dr. Karl-Kuno Kunze
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

	[[alternative HTML version deleted]]



[Rd] Environments and parallel processing
Clark Fitzgerald