Currently, I have a routing system set up in my application. Whenever a user navigates to site.com/page, the route triggers a call to an SQL database to retrieve data which is then parsed and returned as JSON. The retrieved data is then passed to the view using the following code:
res.render('route/to/view', {data: result, moredata: resultTwo})The issue arises when dealing with large datasets as it significantly delays the rendering process, impacting the overall performance of the page. I have noticed that by reducing the dataset size, the speed at which the page loads increases. While I acknowledge that how the data is displayed also plays a role in performance, my main focus is on optimizing the speed of passing data from the route to the view.
My question is whether breaking down the dataset into smaller chunks for passing would be more efficient than transferring one large chunk at once. Is gradual passing of small chunks the only solution to improve this process?