From MEC at stowers.org  Tue Nov 18 18:00:53 2014
From: MEC at stowers.org (Cook, Malcolm)
Date: Tue, 18 Nov 2014 17:00:53 +0000
Subject: [Rd] ShortRead::FastqStreamer and parallelization
Message-ID: <D4772401B9D976478C0895769BE3E7920F27A42C@MBSRV02.sgc.loc>

Hi,

I understand ShortRead::FastqStreamer will read chunks in parallel depending on the value of ShortRead:::.set_omp_threads

I see this discussed here:  https://stat.ethz.ch/pipermail/bioc-devel/2013-May/004355.html and nowhere else.

It probably should be documented in ShortRead.

Possibly this has already changed for I am using still R 3.1.0.   I thought I'd check.

Oh, and, in my hands/hardware, the value of this FastqStreamer's use of srapply's parallelization is negligible, at least if the consumer of successive yields is in the main process.  I see that the new bpiterate appears to take advantage of yielding in forked processes, which sounds promising.  Is that the idea?

Looking forward....

Malcolm Cook



[Rd] ShortRead::FastqStreamer and parallelization
Cook, Malcolm