Mixing sequential and parallel promises

Preface and introduction may be important, but I would prefer to show you the code first.

const R = require('rambdax')
const { execCommand } = require('./execCommand')

async function lintFiles(cwd = process.cwd(), files) {
  console.time('lintFolder')
  
  await R.mapFastAsync(async chunkOfFiles => {
    for (const filePath of chunkOfFiles) {
      
      await execCommand(
        R.multiline(`
          prettier 
          --no-semi
          --single-quote
          --trailing-comma
          --write
          ${filePath}
        `)
      }
    }
  }, R.splitEvery(5, allowedFiles))

  console.timeEnd('lintFiles')
}

The task is to lint list of files. As this list can grow beyond 100 files, it is challenging task to do in the most optimal way.

If I use simple Promise.all I am risking putting the OS under too much pressure.

If I use sequential run of each lint command, that would be noticeably slow.

So I needed to mix both options for best result, i.e.running Promise.all on sequential list of promises.

This is exactly what happens in the example.

The list of files is split to 5 quasi-equal parts with R.splitEvery and each part is passed to R.fastAsync, which uses Promise.all underneath.

It is easy to make comparison between the described logic and sequential run of promises, if we change R.mapFastAsync to R.mapAsync .

With list of 203 files the results are:

Sequential promises = 320 seconds
Promises.all with chunks = 130 seconds