TypeError: Cannot read properties of null (reading 'points') using mongoose #216
Comments
|
code setup: const ratelimit_db = mongoose.createConnection(`mongodb+srv://${config.DB_USER}:${config.DB_PASSWORD}@${config.ENDPOINT}/ratelimiter?retryWrites=true&w=majority`, options={useNewUrlParser: true});
var ratelimiter = new RateLimiterMongo({
storeClient: ratelimit_db,
points: 10,
duration: 15 * 60 //15 minutes
});
async function ratelimitPage(req,res,next) {
ratelimiter.consume(req.ip, 2).then((ratelimitResponse)=>{
res.locals.ratelimited = false;
next();
})
.catch((ratelimitResponse)=>{
console.log(ratelimitResponse);
res.locals.ratelimited = true;
res.locals.ratelimit = ratelimitResponse.msBeforeNext;
next();
})
} |
|
@Kinuseka Hi, please take a look at this Wiki page https://github.com/animir/node-rate-limiter-flexible/wiki/Mongo#note-about-buffering-in-mongoose. It is about Buffering and connection. Also, check this part https://github.com/animir/node-rate-limiter-flexible/wiki/Mongo#connection-to-mongo-and-errors. |
|
This seems pretty difficult situation to deal with, 1var mongo_ratelimit = mongoose.createConnection(`mongodb+srv://${config.DB_USER}:${config.DB_PASSWORD}@${config.ENDPOINT}/ratelimiter?retryWrites=true&w=majority`, options={useNewUrlParser: true});
mongo_ratelimit.set('bufferCommands', false);
mongo_ratelimit.set('autoCreate', false);still results in the same error 2var mongo_ratelimit = mongoose.createConnection(`mongodb+srv://${config.DB_USER}:${config.DB_PASSWORD}@${config.ENDPOINT}/ratelimiter?retryWrites=true&w=majority`, options={useNewUrlParser: true, bufferCommands: false, autoCreate: false});or var mongo_ratelimit = async ()=>{
return await mongoose.createConnection(`mongodb+srv://${config.DB_USER}:${config.DB_PASSWORD}@${config.ENDPOINT}/ratelimiter?retryWrites=true&w=majority`, options={useNewUrlParser: true, bufferCommands: false, autoCreate: false});
} Results in an error: 3according to this it seems that disabling buffering is not recommended. I also tried the unpopular solution mongoose.createConnection(`mongodb+srv://${config.DB_USER}:${config.DB_PASSWORD}@${config.ENDPOINT}/ratelimiter?retryWrites=true&w=majority`, options={useNewUrlParser: true, bufferCommands: false, autoCreate: false, bufferMaxEntries:0});
} Which just results in: okayIf anything, these feels like a bandaid solution than a permanent fix. |
|
@Kinuseka Could you start your server after the connection is established? |
|
Found the root cause, mongoDB in the examples works flawlessly due to Unlike Although this solution above works, it is a stretch especially when you already have your workflow arranged. my solution to this problem is to simply wait for the database to connect before creating a RateLimitMongo instance //mongo.js
var mongo_db_rt = mongoose.createConnection(`mongodb+srv://${config.DB_USER}:${config.DB_PASSWORD}@${config.ENDPOINT}/ratelimiter?retryWrites=true&w=majority`, options);
async function DB_wait(db) {
function sleep(ms) {
return new Promise((resolve) => {
setTimeout(resolve, ms);
});
}
/*
0: disconnected
1: connected
2: connecting
3: disconnecting
*/
var state = { 0: "Disconnected", 1: "Connected", 2: "Connecting", 3: "Disconnecting" };
while (db.readyState !== 1) {
console.log(`Waiting for connection on db: ${db.name} | State: ${state[db.readyState]}`);
await sleep(1000);
}
console.log(`Connection established with: ${db.name} | State: ${state[db.readyState]}`);
return db;
}
var mongo_ratelimit = DB_wait(mongo_db_rt); // this assigns the variable into an unresolved promise
module.exports = {mongo_ratelimit};const {mongo_ratelimit} = require('./database/mongo');
var ratelimiter = async ()=>{
await mongo_ratelimit //since this is a promise, we wait for it to become resolved
return new RateLimiterMongo({
storeClient: mongo_ratelimit,
points: 10,
duration: 10 * 60 //10 minutes
});
}
async function ratelimitPage(req,res,next) {
(await ratelimiter()).consume(req.ip, 2).then((ratelimitResponse)=>{
next();
}
...
} |
|
I have read past issues, and this mistake happens quite pretty often. I feel like it is worth noting the differences between |
|
jumping on this question because im seeing a similar issue and was wondering if it had to do with buffering. I see error Is this a bug with the rate limiter that isnt handling null from the value properly on the first call, it should expect that the first call will not have any entry in the database. edit: adding context using rate limiter with the following setup @animir for vis |
|
I have encountered the same error mentioned by @o-ali . My rate limiter middleware looks like this: const mongoConn = mongoose.connection;
const options = new RateLimiterMongo({
storeClient: mongoConn,
dbName: ENV.DATABASE_NAME,
keyPrefix: "middleware",
points: 2, // 10 requests
duration: 1, // per 1 second by IP
tableName: "rate_limits", // Name of the collection to use for storing rate limit data
});
const rateLimiterMiddleware = async (req, res, next) => {
try {
const rateLimiterRes = await options.consume(req.ip); // Consume 1 point for each request
log.debug("RateLimit-Limit Response .....");
console.log(rateLimiterRes);
res.setHeader("Retry-After", rateLimiterRes.msBeforeNext / 1000);
res.setHeader("X-RateLimit-Limit", options.points);
res.setHeader("X-RateLimit-Remaining", rateLimiterRes.remainingPoints);
res.setHeader("X-RateLimit-Reset", new Date(Date.now() + rateLimiterRes.msBeforeNext).toISOString());
next();
// eslint-disable-next-line unicorn/catch-error-name
} catch (rateLimiterRes) {
if (rateLimiterRes instanceof RateLimiterRes) {
log.warning("RateLimit-Limit Error .....");
console.log(rateLimiterRes);
res.setHeader("Retry-After", rateLimiterRes.msBeforeNext / 1000);
res.setHeader("X-RateLimit-Limit", options.points);
res.setHeader("X-RateLimit-Remaining", rateLimiterRes.remainingPoints);
res.setHeader("X-RateLimit-Reset", new Date(Date.now() + rateLimiterRes.msBeforeNext).toISOString());
log.error("rate-limiter-flexible : ", "Too Many Requests");
res.status(429).send("Too Many Requests");
} else {
// Handle other types of errors
console.error(rateLimiterRes);
res.status(500).send("Internal Server Error");
}
}
};
module.exports = rateLimiterMiddleware; |
|
my issue was fixed in 4.0.1, see: #251. |
|
I'm still experiencing this issue with
This is what the result looks like in As you can see result.value is null, so the
|
|
@alert-justin Could you share your code? |
|
@animir Here is a minimal example that can reproduce the issue: const mongoose = require('mongoose');
const { RateLimiterMongo } = require('rate-limiter-flexible');
const DATABASE_URL = "";
const RATE_LIMIT_OPTIONS = {
points: 10000,
duration: 3600,
storeClient: mongoose.connection,
tableName: 'RateLimitEntries',
keyPrefix: 'data-rate-limit'
};
const rateLimiter = new RateLimiterMongo(RATE_LIMIT_OPTIONS);
const main = async() => {
const args = process.argv.slice(2);
try {
await mongoose.connect(DATABASE_URL);
} catch (err) {
console.log('Error connecting to mongoose');
console.log(err);
return;
}
console.log('Successfully connected to mongoose');
try {
const res = await rateLimiter.consume('test-id');
console.log('Successfully called consume', res);
} catch (err) {
console.log('Error calling consume');
console.log(err);
}
};
main().then(() => {
process.exit(0);
}).catch((err) => {
console.log(err);
process.exit(-1);
});"dependencies": {
"mongoose": "^6.12.0",
"rate-limiter-flexible": "^5.0.4"
}Node: v20.17.0 Run this script once to see the error, run it again to see it work. Delete the entry in the DB or change the ID to produce the error again. |
|
Investigating this further, there is an error being thrown in I think this is because of my use of const upsertOptions = {
upsert: true,
returnDocument: 'after',
returnOriginal: false
}; |
|
@alert-justin Thanks. |
Getting an error issue with ratelimiter, this error happens once if the IP connects for the first time and any subsequent connection the rate-limiter will work correctly.
I am using mongoose 6.5.3 under serverless instance, (do note that this issue also occurs on shared instances.)
The text was updated successfully, but these errors were encountered: