Is it possible to horizontally scale socket connections using a combination of node.js, socket.io, and redis in an

I am diving into the world of node.js for the first time and seeking some guidance:

Here are the programs I have installed on my server:

  • node.js v0.11.3-pre
  • express v3.3.4
  • socket.io v0.9.14
  • connect-redis v1.4.5
  • Redis server v=2.6.14
  • redis-cli 2.6.14

To begin, I created an express app:

express testApplication

In the generated "package.json" file, I specified all the necessary dependencies.

Right from the start, I established a cluster for vertical scaling (multiple processes) in a file named "cluster.js":

var cluster = require('cluster');

if( cluster.isMaster ) {
        var numOfWorkers = process.env.NODE_WORKERS || require('os').cpus().length;

        console.log("Workers found: " + numOfWorkers);

        for (var i = 0; i < numOfWorkers; i += 1) {
                cluster.fork();
        }
} else {
        require('./app.js');
}

cluster.on('exit', function(worker, code, signal) {
        var exitCode = worker.process.exitCode;

        console.log('worker' + worker.process.pid + ' died (' + exitCode + '). restarting...');

        if( typeof cluster.workers[worker.id] != "undefined" )
                cluster.workers[worker.id].delete();

        cluster.fork();
});

In my "app.js" file, I configured REDIS for storing socket.io information:

  io.set('store', new RedisStore({
            redisPub: pub,
            redisSub: sub,
            redisClient: client
          }));

Everything is running smoothly so far.

When a client connects to the socket.io server, the cluster manages the connections with different workers.

My goal is to enable clients to send messages to particular individuals. This involves storing all socket ids for every user in an array, then selecting the relevant socket ids when sending a message by retrieving the sockets using those ids.

This method works well for a single-server socket.io application. Now, I aim to set up another server with identical programs, modules, and packages. Load balancing will likely be facilitated by HAProxy. Consequently, socket.io connections will be managed between Server A and Server B.

Illustrative scenario:

User A connects to Server A, while User B connects to Server B. User A's socket resides on Server A and User B's socket on Server B.

How can the application recognize that it needs to find User B's socket on Server B to dispatch the message? It won't locate the socket on Server A as it was created on Server B.

Thank you very much!

Answer №1

Expanding horizontally necessitates the sharing of a common datastore among your servers. Fortunately, Redis fits this need perfectly. Rather than storing socket mappings in an array, you should push them into Redis and perform lookups from there.

Subsequently, you have the option to either have servers directly communicate with each other or, for better scalability, route messages through Redis as well. In this setup, each server would check its Redis queue to determine which messages need to be sent to specific sockets. Upon receiving a message, a server would store it in Redis directed towards the appropriate server for delivery. If maintaining the sequence of messages is crucial, I suggest exploring https://github.com/learnboost/kue as a layer built on top of Redis.

Answer №2

Remember to always have NGINX in front of your NodeJS server and utilize PM2 for better performance!

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Delivering HTML files to construct a basic single page application with Node.js/Express

I am interested in developing a Single Page Application for my website. From my understanding, a SPA involves sending a single HTML file as the entry point to the application. To achieve this, I plan to use Node and Express to serve the main page, and th ...

I am experiencing issues with my local MongoDB database not properly storing data when using Express and Mongoose

I am encountering an issue where my code is functioning correctly in the console but it is not saving data to the database. Every time I restart the server, the data gets reset. While I can read data from the database without any problem, the issue arise ...

Switching the endpoint renders the middleware ineffective

I've encountered a puzzling issue with my NodeJs - Express server, which serves as the backend for my mobile application. The problem arises when I send post requests to certain endpoints like checkmail and checkusername using axios from the frontend ...

Encountering a CORS issue when utilizing passport-facebook for Facebook authentication in my MEAN application

I'm currently working on implementing a Facebook login feature for my web app, built with express js, mongodb, and angular 2. The client-side of the app is generated using angular-cli (running on port 4200), and I'm utilizing the proxy config pr ...

Having trouble getting webpack and babel to start using npm

Greetings, wonderful people of the internet! I am a newcomer to the enchanting world of programming and I am facing a perplexing issue. Although Webpack is trying to guide me towards the solution, I seem to be struggling with fixing it on my own. const pa ...

Error message: "We apologize, the name must only consist of characters that are compatible with URLs."

Currently, I am following a tutorial on Angular 12 that can be found at . However, I am facing an issue with executing the command below: npm install express cors body-parser mongoose Here is the code snippet: PS D:\VArdhan\projects\node-an ...

Analyzing User Input and Database Information with Mongodb

Here's the HTML form I'm working with: <form id="contact-form" method="POST" action="/search"> <label for="company">Phone company</label> <input type="text" name="company" value=""> &l ...

Is it possible to retain various delimiters after dividing a String?

In the code below, the someString gets split into an array using specified delimiters in separators var separators = ['\\.', '\\(', '\\)', ':', '\\?', '!&apos ...

Using the app.use method in Node.js with Express

Currently, I am working on deciphering an old codebase: var express = require('express'); var app = express.createServer(); app .use('/run!', getUrl('app.sys', '/run')) .use('/stat', getUrl('app.sys& ...

sharing data between two node.js servers using http

I am currently working on integrating two node.js/express servers that communicate with each other using HTTP. One of the servers, known as server A, is responsible for handling file upload requests from the browser. My goal is to seamlessly transfer any u ...

Is this a problem with npm or JavaScript?

Hi everyone, I'm trying to figure out if this issue is related to JavaScript or npm. If there's a problem with my JS code, could someone please help me identify it? PLEASE NOTE I used some code to get the current uid from Firebase. I'm ...

Having trouble updating an array in a mongoose document?

Need help with updating an array in a document by adding or replacing objects based on certain conditions? It seems like only the $set parameter is working for you. Here's a look at my mongoose schema: var cartSchema = mongoose.Schema({ mail: Stri ...

Node(Meteor) experiencing a memory leak due to setTimeout

I have encountered an unusual memory leak associated with the use of setTimeout. Every 15 seconds, I execute the following code using an async function that returns an array of promises (Promise.all). The code is supposed to run again 15 seconds after all ...

error encountered in node.js express route.get() function

I have used express-generator and installed the necessary dependencies. The only thing I've changed is the app.js file, which I have included here. Upon running npm start, I encountered an error. I have provided both the error message and the app.js ...

npm will only update when modifications are made to index.js

Recently diving into the world of React, I've encountered an issue with updating my website when making changes to files within my project. While modifications to the index.js file reflect on the site, any adjustments made to imported apps do not get ...

Issues with saving data to MongoDB using NodeJS, ExpressJS, and EJS database connection

I am in the process of developing a childcare management web application using NodeJS with ExpressJS, mongoose, and EJS. The application features a module for adding participants and another module to track each participant's daily meal count four tim ...

Electron and React: Alert - Exceeded MaxListenersWarning: Potential memory leak detected in EventEmitter. [EventEmitter] has 21 updateDeviceList listeners added to it

I've been tirelessly searching to understand the root cause of this issue, and I believe I'm getting closer to unraveling the mystery. My method involves using USB detection to track the connection of USB devices: usbDetect.on('add', () ...

Remove route with problem: undefined parameters

I have encountered an error while working on a delete route that is causing issues with req.params.bookid. This is not a common problem for me. The console shows the error message: "TypeError: Cannot read property 'bookid' of undefined"... I atte ...

Can the Node DNS module be incorporated into Angular code?

I am facing a requirement where I must perform a DNS lookup from client-side code built in Angular. Is it feasible to utilize the DNS module from https://nodejs.org/api/dns.html in Angular? ...

Retrieve the array from the response instead of the object

I need to retrieve specific items from my database and then display them in a table. Below is the SQL query I am using: public async getAliasesListByDomain(req: Request, res: Response): Promise<void> { const { domain } = req.params; const a ...