MongoDB MongooseError: The `collection.aggregate()` method cannot be called until the initial connection is fully established

Check out my personal website that I created using NextJS and hosted on Vercel. The site connects to a MongoDB database through Mongoose in my NodeJS API.

Recently, I've been encountering a strange error affecting only about 1% of the users:

MongooseError: Cannot call 'hotels.aggregate()' before initial connection is complete if 'bufferCommands = false'. Make sure you 'await mongoose.connect()' if you have 'bufferCommands = false'.
at NativeCollection.<computed> [as aggregate] (/var/task/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:193:15)
at /var/task/node_modules/mongoose/lib/aggregate.js:998:18
at /var/task/node_modules/kareem/index.js:23:7
at processTicksAndRejections (internal/process/task_queues.js:77:11)

(hotels represents the collection being displayed, with the .aggregate() function being called within my API)

I am unable to consistently replicate this error, although it has happened to me as well.

I'm adhering to the recommended method from NextJS for connecting to MongoDB by referring to their example code snippet:

import mongoose from 'mongoose'

const MONGODB_URI = process.env.MONGODB_URI

// Code snippets continue...

Despite following the correct structure, the issue persists. Here's how my API starts:

import dbConnect from "utils/dbConnect";
import HotelModel from "models/HotelModel";

export default async function handler(req, res) {
  await dbConnect();

      try {
        const pipeline = {}; // this is a big aggregation pipeline

        const hotels = await HotelModel.aggregate(pipeline);

        return res.status(200).json(hotels);
      } catch{
        ... 
      }
}

Any suggestions on what might be causing this unexpected behavior?

Answer №1

Get rid of

const settings = {
      disableBuffering: false,
}

or specifically define disableBuffering: true.

Vercel operates on AWS Lambda, which can sometimes lead to issues with cached connections. A contributor from Vercel raised concerns about this issue in a mongoose PR thread https://github.com/Automattic/mongoose/issues/9239#issuecomment-659000910

In response, it was suggested in the discussion to steer clear of disableBuffering: false, leading to its removal from the official mongoose recommendations a year later: docs/lambda.md

Answer №2

Include this in the connection options:

 bufferMaxEntries:0

bufferMaxEntries - The MongoDB driver has its own buffering system that activates when the driver disconnects. By setting this option to 0 and turning off bufferCommands on your schemas, you can ensure that your database operations will fail immediately if the driver is not connected, rather than waiting for a reconnection.

Read more in the documentation

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Error: The schema configuration is incorrect: `U` is an invalid type specified at path `0`

const accountSchema = new mongoose.Schema({ username: { type: String, required: true, unique: true }, password: { type: String, ...

Attempting to send a POST request, only to be informed by the form that it is devoid of

I have been struggling with this problem for some time now. I implemented the category_create_post functionality in the categoryController, and everything seems to be set up correctly. I also configured the category_form.ejs to accept user input. However, ...

The aframe module is unable to detect the embedded .gltf scene within the <a-gltf-model/> tag, resulting in a 404 Not Found error when using node.js

When using aframe, I encountered an issue where the embed .gltf scene was not being recognized inside 'a-gltf-model' - returning a 404 Not found error when accessed through node.js. However, placing the src attribute into 'a-emity' resu ...

When utilizing a Javascript event listener within ReactJS with NextJS, the dynamically imported module without server-side rendering fails to load

Hello everyone, I am new to ReactJS and NextJS and would really appreciate some advice on the issue below. Thank you in advance! Here is my current tech stack: Node v16.6.1 React v17.0.2 Next.js v10.0.4 I'm working on implementing a carousel and si ...

Setting custom headers for an HTML document in Playwright involves configuring the necessary HTTP headers to achieve

I'm looking to customize headers in Playwright specifically for an HTML document for unique identification purposes. While browserContext.setExtraHTTPHeaders and page.setExtraHTTPHeaders can set headers for all requests on a page, I am seeking a way ...

Make sure to add the local private NPM dependency before running the prepublish script

Within my application package.json file, I am referencing a local private NPM dependency like this: "core-module": "file:///Users/myuser/Documents/projects/core_module" My goal is to have the local private dependencies (such as core-module) automatically ...

What is the best method for displaying private API-endpoint JSON data secured by Auth0-JWT within an Android application?

Having developed an Android application with a NodeJS backend, I have implemented a private API endpoint protected by Auth0. The NodeJS code for the private API looks like this: app.get('/api/private', jwtCheck, function(req, res) { res.json({ ...

Client-side libraries need to be installed via npm in a NodeJS environment

Currently, I am facing an issue with a nodejs application that needs to serve js assets to the client. After installing modules via npm (such as primus or socket.io), I aim to transmit these js files to the browser. If I use jade and have a public folder ...

There is no Method available when requiring Node.js/Express.js Middleware

Encountering an issue with requiring Express.js middleware in my code: ... var middlewares = require('./middlewares'); ... routes = require('./routes')(app, config, crypto, middlewares, models, oauth2); ... The above code snippet requ ...

Commands for Yarn, such as type checking, testing, and formatting with Prettier, are encountering

I have been encountering errors in my Jenkins builds for the past 3 weeks. Despite numerous attempts, I cannot seem to get these commands to work within the Jenkins pipeline code using our shared libraries. The same commands work fine locally. The errors ...

Setting environment variables for React scripts can be done by leveraging the process.env object

Currently, I am in the process of developing a project that encompasses both backend and frontend components. It struck me as a great idea to have both sets of code housed within the same directory, complemented by another npm script responsible for runnin ...

User error: next.js couldn't find the specified resource /api/ on vercel

I am currently working on a Next.js website that I plan to deploy on Vercel. As part of my project, I have created a Next.js API endpoint "/api/contact" which is responsible for sending emails using Nodemailer. The code works perfectly fine when tested on ...

Having trouble connecting to the online redis database

I am currently working on establishing a connection to my online redis database. const redis = require('redis'); const client = redis.createClient({ password: '<password>', socket: { host: <host> port: <p ...

PhpStorm encountered an issue while attempting to compile scss files to css with the error message "Error env: node: No such file or directory" on MacOS

After successfully installing npm, node-sass, and JDK on my mac, I managed to compile the file correctly from the console: https://i.stack.imgur.com/vMrKm.png However, when I tried compiling it in PhpStorm, an error occurred: https://i.stack.imgur.com/n ...

Guide to implementing error handling middleware in Express 4.x

Despite setting process.env.NODE_ENV ='development', I attempted to use middleware errorhandler, but it failed to work. Here is the server code: var express = require('express'); var app = express(); var errorhandler = require('e ...

Handlers for routing in Express

Hey there, I'm diving into the world of nodejs and express. Currently going through the express guide, but struggling to grasp the idea behind using route handlers as an array of functions. Let's take a look at this example: Consider these two ...

Is ExpressJs used as a server in web development?

I am currently learning about Node.js. Can you explain the role of Express in NodeJs? Is it primarily utilized as a server for NodeJs applications? If yes, how can we deploy our code using Express? Thank you! ...

When using node.js with express, the req.on('end') event is triggered, but the req.on('data') event does not fire

When using body parser, you have the option of either: application/x-www-form-urlencoded body parser or json body parser Both options yield the same results. This is how the API is being called: $.ajax({ type:'post', url:'/ ...

Issue encountered while trying to install http-server: Unable to locate a compatible version for ecstatic@^3.0.0

As a newcomer to front end development, I am currently tasked with maintaining an AngularJS application. My current challenge involves trying to set up http-server for testing the app. However, when I attempt to install it using the command npm install htt ...

Using Node.js and Express to retrieve data from a MySQL database and update a table

My .ejs file contains a form that sends data to a MySQL database and then displays two tables with the retrieved data on the same page. However, I am facing an issue where the tables do not refresh after submitting the form, requiring me to restart the ser ...