Is there a way to serve server-side rendered content exclusively to search engine crawlers like Google Bot, without SSR for regular users in Next.js?

With a staggering number of users visiting the site every day, we are making strides to enhance our visibility on Google by implementing SSR (which may sound unbelievable) and achieving a richer preview when content is shared on messaging platforms.

As the current setup utilizes a client-side React app, concerns arise about how the server will handle the transition to Next.js.

Distinguishing between human users and bots should be easily achievable through user agent detection.

So, what's the most efficient solution here?

Could nginx potentially customize URL parameters based on user agents – for instance, using '/page' for regular users and '/page@ssr' for bots? This approach would involve having pages/page.js without getInitialProps for users, and 

pages/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="5424353331142727267a3e27">[email protected]</a>
for bots. These two files would fetch data differently but ultimately render the same component with corresponding data. How does that sound to you?

javascriptreactjsoptimizationnext.js

Answer №1

Have you considered exploring ISR (Incremental Static Generation)?

If you're worried about the server getting overwhelmed during peak times, ISR might be a solution for you. By implementing ISR for both web crawlers and users in the same manner, you can simplify the system and potentially improve response times.

I've written a detailed guide on how to use ISR in Next.js at What is the difference between fallback false vs true vs blocking of getStaticPaths with and without revalidate in Next.js?. Essentially, you just need to include the following in your getStaticPaths function:

{
  fallback: true, // or 'blocking'
  revalidate: <integer>
}

This setup allows the server to render once every revalidate seconds and cache the result. If multiple requests are made before the timeout, the page won't be re-rendered (potentially displaying slightly outdated data).

To handle user-specific data (like content specific to logged-out users), you can make an additional API request after the initial load to update the user's view accordingly.

For example, check out this implementation of a Medium clone using ISR and user-specific data updates: Example Implementation. By optimizing specific parts of the page that require constant updates, you can provide a more seamless user experience.

In reality, the performance and server usage differences between SSR and ISR may not be noticeable until you have a higher volume of daily requests. Ultimately, it's best to benchmark and analyze your individual needs before making a decision.

Answer №2

In the scenario where you have a custom express server, one option to consider is utilizing the SSR For bots solution that I developed for similar situations!

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Show brief tags all on one line

This image showcases the functionality of the site, specifically in the "Enter a code" column where users can input data using TagsInput. I am seeking to enhance this feature by ensuring that shorter tags are displayed on a single line. While I am aware th ...

javascriptcssreactjsbootstrap-tags-input

JavaScript is throwing an error stating that the requestData is undefined, even though the

Hello, I am attempting to retrieve weather information based on the country and city using the openweather api. JSON is a new concept for me in coding, so please bear with me through this learning process. Below is the code snippet that I have been workin ...

javascriptjqueryjson

Having trouble retrieving the exported state variable from a custom hook

I am struggling with a custom hook that handles light/dark mode using MUI. The hook exports a state variable that I pass to MUI's createTheme, but when I try to access the state variable outside of the hook, it returns undefined in the console. Can an ...

reactjsreact-hooksnext.jsmaterial-ui

Fetching Data Using Cross-Domain Ajax Request

Seeking assistance with a cross-domain Get request via Ajax. The code for my ajax request is as follows: var currency_path = "http://forex.cbm.gov.mm/api/latest"; $.ajax({ url: currency_path, crossDomain:true, type:"GET", dataTyp ...

javascriptajaxcross-domain

Axios GET request encounters Error 401 - Access Denied

In my current ReactJS project, I am utilizing axios to fetch data from MongoDB. Within this project, I have a login component that invokes a startLogin function. This function includes two axios requests: one POST request to send user data and another GET ...

reactjsapirestaxioshttp-status-code-401

Is it possible for three.js to integrate information from REST APIs?

Currently, I am in the process of learning three.js and have successfully created basic 3D models using hardcoded values. My goal now is to retrieve these values from a database that I have set up in MSSQL Server. These x, y, and z parameters are stored ...

node.jsreactjsspring-bootrestthree.js

A Comprehensive Guide on Incorporating Nodejs Modules in a Subdirectory

Within my AWS Lambda layer, I have a directory structure that resembles the following: main > nodejs > node_modules Prior to uploading to AWS, I require a test.js file for functionality testing purposes. The question at hand is whether it's fe ...

javascriptnode.jsaws-lambda-layers

After removing the timezone from the timestamp log, why is it now showing as one day behind?

Within my programming, I store a timestamp in the variable 'var timeStamp = finalData.obj.followers[0].timestp;', which currently outputs '2020-04-15T00:00:00.000Z.' To extract only the date and remove the time zone information, I util ...

javascriptdatetimegoogle-apps-scripttimezonegoogle-apps-script-editor

Modify MUI's ListItemText component by targeting a specific span to implement customized styles using the useStyle hook

It's quite perplexing that although this is a straightforward task in regular CSS, it must be accomplished through MUI's useStyles for some peculiar reason. Essentially, I possess a ListItem containing a ListItemText. It appears as follows: cons ...

javascriptcssreactjsmaterial-ui

In order to manage this file type properly, you might require a suitable loader, such as an arrow function within a React

Currently, I am in the process of creating an npm package and have encountered a difficulty with the following code: You may need an appropriate loader to handle this file type. | } | | holenNummerInSchnur = Schnur => { | if (this.beurte ...

javascriptreactjsnpmpackage

Interface circular dependency is a phenomenon where two or more interfaces

In my MongoDB setup, I have created an interface that defines a schema using mongoose as an ODM. import mongoose, { model, Schema, Model, Document } from "mongoose"; import { IUser } from "./user"; import { IPost } from "./posts&q ...

typescriptmongodbmongoosenext.js

What is the best way to incorporate polling into the code provided below? I specifically need to retrieve data from the given URL every 60 seconds. How should I go about achieving this

Can you assist me in integrating polling into the code below? <!DOCTYPE html> <html> <head> <script src="https://code.jquery.com/jquery-3.1.1.min.js"></script> <script src="https://code.highcharts.com/highcharts.js"> ...

javascripthtmlajaxhighcharts

Utilizing Angular to integrate with an external API

I have encountered an issue while trying to connect to the Expedia API. In order to do so, I need an API key and ID. Initially, I utilized JSONP for this connection but ran into a bug causing problems. Additionally, storing my API key in JavaScript poses ...

javascriptphpangularjsajax

What is the best way to keep an image fixed at the bottom, but only when it's out of view in the section

There are two buttons (images with anchors) on the page: "Download from Google Play" and "Download from App Store". The request is to have them stick to the bottom, but once the footer is reached they should return to their original position. Here are two ...

javascriptjqueryhtmlcssfrontend

retrieve the class name of a reusable component in HTML

I have been assigned a web scraping project and I am required to scrape data from the following website for testing: The HTML: <div class="quote" itemscope itemtype="http://schema.org/CreativeWork"> <span class="text& ...

javascripthtmlweb-scrapingcomponents

Switch between two tabs with the convenient toggle button

I have implemented 2 tabs using Bootstrap as shown below: <ul class="nav nav-tabs" role="tablist"> <li role="presentation" class="active"><a href="#home" role="tab" data-toggle="tab">CLient</a></li> <li role="presentatio ...

javascriptjquerytwitter-bootstrap

Tips for calculating the canvas-relative mouse position on a CSS 3D transformed canvas

Recently decided to challenge myself by experimenting with drawing on 3D transformed canvases. After some trial and error, I managed to get it partially working. const m4 = twgl.m4; [...document.querySelectorAll('canvas')].forEach((canvas) =& ...

javascripthtmlcsscanvascss-transforms

Learning how to track mouse wheel scrolling using jQuery

Is there a way to track mouse scrolling using jquery or javascript? I want the initial value to be 0 and increment by 1 when scrolling down and decrement by 1 when scrolling up. The value should always be positive. For example, if I scroll down twice, the ...

javascriptjqueryhtmlcss

Guide to successfully implement a POST action using jquery/ajax Colorbox! Check out the demo here

I stumbled upon a colorbox tutorial and attempted to replicate it with a POST action, but unfortunately, it just continues to load. Check out this fiddle The button represents the problematic POST action, while the text link is the original example that ...

javascriptphpjqueryajaxcolorbox

What is the best way to trigger a Quasar dialog from an outside component?

Currently, I am working with Vue.js 2.x + Quasar 1.x using http-vue-loader (no build tools required). I have placed a q-dialog in a separate component named MyComponent. However, when I try to include it in a parent component like this: <my-component&g ...

javascriptvue.jsvuejs2frontendquasar-framework