Multicore Parallel Processing for Large Datasets

[chinthanishanth]

i am using matchit to calculate propensity scores for large data (say approx ~ 500 thousand records) it is taking approx 2 hours to get results. it will be great if this package support muticore parallel processing i.e., using all available cores in processor, so that computation time can be reduced significantly.

[ngreifer]

This is something we will look into, but it is not straightforward how to make nearest neighbor matching without replacement run in parallel. It is an iterative process where each step depends on the steps taken before it. This makes it not "embarrassingly parallel". This may be possible with matching with replacement, however. We will continue to examine this.

[ginnydang]

I have the same problem working with dataset up to 400,000 entries. I wonder if you come up with some ideas or not. Thank you so much!

[buhtz]

Some things need to be serial. 2 hours is not much. Take a longer lunch break.