From edd @end|ng |rom deb|@n@org  Tue Aug  8 04:16:59 2023
From: edd @end|ng |rom deb|@n@org (Dirk Eddelbuettel)
Date: Mon, 7 Aug 2023 21:16:59 -0500
Subject: [Rd] 
 Detecting physical CPUs in detectCores() on Linux platforms
In-Reply-To: <0DCAA0B2-CE51-4518-B7A5-95B737D1281F@R-project.org>
References: <CAObhMffcPOLHN_ko1VQEu-taHjCiuc5kPCA9ybS1O9r_UOn1GQ@mail.gmail.com>
 <25808.57266.337153.29417@rob.eddelbuettel.com>
 <20c86758cf57baa0c045aa39d99bc950a8cb8b31.camel@uni-jena.de>
 <CC836864-15F7-4D44-BED6-20CA02FCD8BA@R-project.org>
 <25809.34633.249825.74893@rob.eddelbuettel.com>
 <0DCAA0B2-CE51-4518-B7A5-95B737D1281F@R-project.org>
Message-ID: <25809.42395.29355.809081@rob.eddelbuettel.com>


On 8 August 2023 at 13:17, Simon Urbanek wrote:
| To be honest I think the motivation of this thread is dubious at best: it is a bad idea to use detectCore() blindly to specify parallelization and we explicitly say it's a bad idea - any sensible person will set it according to the demands, the hardware and the task. The number of cores is only partially relevant - e.g. if any I/O is involved you want to oversubscribe the CPU. If you have other users you want to only use a fraction etc. That doesn't mean that the we couldn't do a better job, but if you have to use detectCores() then you are already in trouble to start with.

As I often say, "life's a bitch and then you die".  Detecting hardware
capabilties at run-time is no small task.

When 1 1/2 decades ago I filled in maintaining slurm for Debian (and
interfacing open-mpi) that latter project added an entire library for that
scope (libhwloc).  For all I know it is still best practices.  So in that
sense no point grepping / reading /proc/cpuinfo as that has clear limits.

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org



[Rd] Detecting physical CPUs in detectCores() on Linux platforms
Dirk Eddelbuettel