Question: Does MEDI operate on a per-sample or per-dataset basis?

[yenkaiC]

Hi, 

Some HPCs have a time-limit on the high-memory partitions. 

I’m wondering if MEDI predicts on a per-sample or per-dataset basis?
If it’s per-sample, I think there shouldn’t be any issue with the time limit since I can process a subset of the data.

However, if it’s a per-dataset basis, are there any suggestions for this?

Best,

[cdiener]

Hi, that's a bit of a tricky question. In theory it's per sample in the sense that MEDI results for one sample do not depend on other samples. In practive the DB caching trick we use requires several samples to be processed in a batch. The pipeline already applies a time limit on this step which is: 2h + 0.5h * n_batch. So maybe choose the batch size based on this and the limit of your HPC system. The batch size can be supplied with the --batchsize argument.