Job's not starting sometimes but only when bull-board running. hmm.

[pkellner]

I understand that bull-board is only reading data from Redis so I assume that what is causing my queued job's not to start can't be a bull-board issue, but still hoping to find some insight there on what it could be the actual problem. I'm currently building an IMAP processor for iCloud mail because Apple's is so lame at processing rules. I'm only testing so there is not much load, but still I can repeatably start and stop the dashboard (all running locally on my beefy MacBook M3) and observe that some jobs will not start when bull-board is running. What is weird is I get no error I can find, just nothing in my log showing anything happened.

My code is very simple. It's just what I have. below. What will happen is that in my new Worker(... code, I get the message about "about to process email", but then no log entry from inside await processEmail. I'm setting the jobId with a guid so I know it's not rejected as a duplicate job, but still don't get how this can happen (again, only when bull-board is running).

Two questions as a BullMQ/bull-board newby. 1) is this a good place for this discussion? 2) if not, what's a better place? I hate to create an issue on bullmq because it's hard to make it 100% repeatable, and .. I'm a newby here.

Thanks

const generalQueueName = "general-queue";
const generalQueue = new Queue(generalQueueName, {
  connection: {
    host: redisHost,
    port: redisPort,
  },
  prefix: process.env.REDIS_QUEUE_PREFIX ?? "email-queue-prefix-for-redis",
});

const generalWorker = new Worker(
  generalQueueName,
  async (job: Job) => {
    if (consoleLogging) {
      console.log("/src/app.ts: generalWorker: job.data.email: (about to call processEmail) ", job.data.email, " job.data: ", job.data);
    }
    await processEmail(job.data.email, job);
  },
  {
    concurrency: generalQueueConcurrency,
    connection: {
      host: redisHost,
      port: redisPort,
    },
    prefix: process.env.REDIS_QUEUE_PREFIX ?? "email-queue-prefix-for-redis",
  },
);

generalWorker.on("active", (job: Job) => {
  job?.log(`Job with id ${job.id} has started`);
});

generalWorker.on("completed", (job: Job) => {
  job?.log(`Job with id ${job.id} has been completed`);
});

generalWorker.on("error", (err) => {
  // log the error
  if (consoleLogging) {
    console.error("/src/app.ts: generalWorker: error: ", err);
  }
});

generalWorker.on("failed", (job?: Job) => {
  if (consoleLogging) {
    job?.log(`Job with id ${job?.id} has failed with ${job?.failedReason}`);
  }
});

[felixmosh]

Hi, it is probably not related to bull-board (as you've mentioned).

Maybe it is related to some connection limitation?

Bull-board doesn't related to the worker at all, so the only thing that makes sense is some connection (to redis) limitation.