By default parallel::makeCluster() uses type = "PSOCK" for its parallelism. But changing the type to "FORK" is more efficient on non-Windows systems. I suggest wherever makeCluster() is used the code be changed to
if (.Platform$OS.type == "windows") {
cl <- parallel::makeCluster(n_cores, type = "PSOCK", outfile = "")
} else {
cl <- parallel::makeCluster(n_cores, type = "FORK", outfile = "")
}
This also significantly reduces memory usage on systems where forking is available.
With my quick testing of 300 cells with 5000 genes and ks = 1:10 I saw the PSOCK version taking ~90 seconds and the FORK version taking ~60 seconds.
The downside would be that forks do not parallelise across clusters.
By default
parallel::makeCluster()usestype = "PSOCK"for its parallelism. But changing the type to"FORK"is more efficient on non-Windows systems. I suggest wherevermakeCluster()is used the code be changed toThis also significantly reduces memory usage on systems where forking is available.
With my quick testing of 300 cells with 5000 genes and
ks = 1:10I saw the PSOCK version taking ~90 seconds and the FORK version taking ~60 seconds.The downside would be that forks do not parallelise across clusters.