Using rxjs for exponential backoff strategy

Exploring the Angular 7 documentation, I came across a practical example showcasing the usage of rxjs Observables to implement an exponential backoff strategy for an AJAX request:

import { pipe, range, timer, zip } from 'rxjs';
import { ajax } from 'rxjs/ajax';
import { retryWhen, map, mergeMap } from 'rxjs/operators';

function backoff(maxTries, ms) {
 return pipe(
   retryWhen(attempts => range(1, maxTries)
     .pipe(
       zip(attempts, (i) => i),
       map(i => i * i),
       mergeMap(i =>  timer(i * ms))
     )
   )
 );
}

ajax('/api/endpoint')
  .pipe(backoff(3, 250))
  .subscribe(data => handleData(data));

function handleData(data) {
  // ...
}

Even though I grasp the essence of Observables and backoff mechanisms, I'm still struggling to comprehend how retryWhen calculates the time intervals for resubscribing to the ajax source.

In particular, I wonder about the inner workings of zip, map, and mapMerge in this framework. How do they contribute to the process?

Furthermore, what does the emitted attempts object entail when passed into retryWhen?

Despite consulting their documentation extensively, I find myself unable to fully grasp this concept.

Answer №1

After dedicating a significant amount of time to researching this topic (for educational purposes), I am prepared to provide a comprehensive explanation of how this code operates.

Let's start by examining the original code along with annotations:

import { pipe, range, timer, zip } from 'rxjs';
import { ajax } from 'rxjs/ajax';
import { retryWhen, map, mergeMap } from 'rxjs/operators';

function backoff(maxTries, ms) {
  return pipe(
   retryWhen(attempts => range(1, maxTries)
     .pipe(
       zip(attempts, (i) => i),
       map(i => i * i),
       mergeMap(i =>  timer(i * ms))
     )
   )
 );
}

ajax('/api/endpoint')
  .pipe(backoff(3, 250))
  .subscribe(data => handleData(data));

function handleData(data) {
  // ...
}
  1. Creating a custom operator called backoff using the existing retryWhen operator allows us to apply it within the pipe function later on.
  2. The pipe method in this context returns a customized operator.
  3. Our customized operator is essentially an altered version of the retryWhen operator. It accepts a function as an argument which will be executed once, specifically when this retryWhen function is first triggered. The retryWhen only comes into play when the source observable encounters an error, halting further error propagation and resubscribing to the source. If the source produces a non-error result, the retryWhen is bypassed.

    With regards to attempts, it is an observable created exclusively for the retryWhen function and is used to react to each failed subscription attempt to the source observable.

    So, here is what we're going to do.

    Initially, we generate range(1, maxTries), an observable that provides an integer for every potential retry. Although range can emit all its numbers immediately, we need to synchronize it with the retries. This is why we...

  4. ...zip it with the attempts, pairing each emitted value of attempts with a respective value from range.

    At the moment this function is executed, attempts has fired a single next event — indicating the initial failed subscription. Thus, our zipped observables have produced just one value at this stage.

    Regarding the values in the combined observable, the function determines this: (i) => i. To clarify, it could be written as

    (itemFromRange, itemFromAttempts) => itemFromRange
    . Since the second argument is unused, it is omitted, and the first is renamed as i.

    Essentially, we ignore the actual values emitted by attempts and solely focus on acknowledging their occurrence while progressing through the sequence of range observable...

  5. ...and squaring the integers for the exponential increase part of the backoff strategy.

    Subsequently, whenever a (re-)subscription to the source fails, we possess a growing integer sequence at our disposal (1, 4, 9, 16...). How do we convert these integers into time delays until the next re-subscription?

    It is crucial to note that the current function must yield an observable using attempts as input. This resulting observable is constructed only once. retryWhen then subscribes to this output and retries subscribing to the source observable whenever this derived observable emits a next event; terminates or triggers an error on the source observable based on the outcome of the derived observable.

  6. In brief, we require the retryWhen to pause momentarily. While the delay operator might suffice, setting up an exponentially growing delay would likely be cumbersome. Consequently, the mergeMap operator emerges as a solution.

    mergeMap effectively merges two operators - map and mergeAll. Firstly, map transforms each ascending integer (1, 4, 9, 16...) into a timer observable that delivers a next signal after a designated number of milliseconds. Subsequently, mergeAll ensures that retryWhen truly subscribes to the timer observables. Without this step, the resultant observable would immediately emit a next event containing the timer instance.

  7. At this juncture, we have successfully formulated a custom observable to guide the behavior of retryWhen concerning the timing of subsequent attempts to reconnect to the source observable.

As currently structured, there are two dilemmas associated with this approach:

  • Upon the final emission of the last next event by our derived observable (triggering the final reconnection attempt), an immediate complete event is generated. Unless the source observable promptly yields a response (assuming the ultimate retry proves successful), this outcome will be disregarded.

    This stems from the fact that following the receipt of a complete signal from the derived observable, the retryWhen invokes complete on the source observable, potentially while it remains engaged in an AJAX request.

  • If all retries prove unsuccessful, the source instead signals complete as opposed to the more logical option of signaling an error.

To address both issues, my proposed remedy incorporates firing an error right at the conclusion of the process, granting the final retry a reasonable window to fulfill its objectives.

Below is the revised implementation including adjustments for the deprecation of the zip operator in the latest version of rxjs v6:

import { delay, dematerialize, map, materialize, retryWhen, switchMap } from "rxjs/operators";
import { concat, pipe, range, throwError, timer, zip } from "rxjs";

function backoffImproved(maxTries, ms) {
    return pipe(
        retryWhen(attempts => {
            const observableForRetries =
                zip(range(1, maxTries), attempts)
                    .pipe(
                        map(([elemFromRange, elemFromAttempts]) => elemFromRange),
                        map(i => i * i),
                        switchMap(i => timer(i * ms))
                    );
            const observableForFailure =
                throwError(new Error('Could not complete AJAX request'))
                    .pipe(
                        materialize(),
                        delay(1000),
                        dematerialize()
                    );
            return concat(observableForRetries, observableForFailure);
        })
    );
}

Following extensive testing, this modification appears to perform reliably under various scenarios. I won't delve into detailed explanations at this moment, assuming most readers may overlook the lengthy discourse above.

Special thanks to @BenjaminGruenbaum and @cartant for guiding me towards the correct path in resolving these complexities.

Answer №2

Here is an alternate version that allows for easy customization and expansion:

import { Observable, pipe, throwError, timer } from 'rxjs';
import { mergeMap, retryWhen } from 'rxjs/operators';

export function customRetry(maxAttempts = 5): (_: Observable<any>) => Observable<any> {
  return pipe(
    retryWhen(errors => errors.pipe(
      mergeMap((error, attempt) => {
        const retryCount = attempt + 1;
        if (retryCount > maxAttempts) {
          return throwError(error);
        } else {
          const delayInMs = retryCount * retryCount * 1000;
          console.log(`Attempt ${retryCount}: Retrying in ${delayInMs}ms`);
          return timer(delayInMs);
        }
      }),
    ))
  );
};

Reference: retryWhen documentation

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Angular 2 - Can a Content Management System Automate Single Page Application Routing?

I am interested in creating a single-page application with an integrated content management system that allows users to edit all aspects of the site and add new pages. However, I have found it challenging to configure the SPA to automatically route to a n ...

When running `npm install`, it attempts to install version 1.20.0 of grpc even though version 1.24.2 is specified in

After running npm install on my React-Native project, an error occurred stating that it was attempting to install gRPC version 1.20.0, but my package.json and package-lock.json specified gRPC version 1.24.1. I attempted to fix the issue by adjusting the v ...

Discovering and revising an item, delivering the complete object, in a recursive manner

After delving into recursion, I find myself at a crossroads where I struggle to return the entire object after making an update. Imagine you have an object with nested arrays containing keys specifying where you want to perform a value update... const tes ...

jQuery not being applied to dynamically added dropdown element

I am currently utilizing bootstrap and jquery within my react project. I have a button that, when clicked, should transform into a dropdown field. The dropdown functions properly when placed statically, but the functionality is lost once it is dynamically ...

Identify all td inputs within a TR element using jQuery

Is there a way to retrieve all the input values within each table cell (td) of a table row (tr) using jQuery? Suppose I have a tr with multiple td elements, and some of these tds contain inputs or select elements. How can I extract the values from these in ...

Getting data from an API using a Bearer Token with React Hooks

I am currently developing a React application that is responsible for fetching data from an API. This API requires Bearer Token Authorization. To handle this, I have implemented useState() hooks for both the token and the requested object. Additionally, th ...

What exactly are AngularJS module dependencies and how do they work together?

After exploring the tutorial example provided on AngularJs's site ( here) (The main HTML appears to be quite minimal with only ng-view and ng-app=phonecatApp included) Within the app.js file, we find: var phonecatApp = angular.module('phoneca ...

Error: Undefined object while trying to access 'injection' property

After updating to React 16.x, I encountered the following error: Uncaught TypeError: Cannot read property 'injection' of undefined at injectTapEventPlugin (injectTapEventPlugin.js:23) at eval (index.js:53) at Object.<anonymous> ...

My code seems to be malfunctioning

I'm attempting to conceal the chatname div. Once hidden, I want to position the chatid at the bottom, which is why the value 64px is utilized. However, I encountered an error stating The function toggle3(); has not been defined. <div id="chat ...

How can JavaScript allow access to files outside of a web server environment? Could .htaccess provide a

Currently, I'm facing a challenge with my local server on Mac 10.8. I am trying to serve files such as videos and images from an external hard drive. The path for the external hard drive is HD: /Volumes/ while the web server's path is /Library/Se ...

Having trouble getting Jquery Ajax Post to work properly when using JinJa Templating?

Objective: My goal is simple - to click a button and post information to a database. Problem: Unfortunately, clicking the button doesn't seem to be posting to the database as expected. Setup: I am working with Flask Framework, Jquery, and Jinja Temp ...

What sets apart `lib.es6.d.ts` from `lib.es2015.d.ts` in TypeScript?

I'm a bit confused about ES6 and ES2015. In TypeScript, there are two type declarations available for them: lib.es6.d.ts and lib.es2015.d.ts. Can someone explain the difference between these two? And which one is recommended to use? ...

The Validators.pattern in Angular fails to match when comparing two different versions

I encountered a unique scenario where I need to validate either a datetime format or an empty string. Both should be accepted inputs, but any malformed or incomplete datetimes should fail validation. myForm = this.form.group({ ... ts: [&apos ...

Utilizing URL-based conditions in Reactjs

Currently, I am working with Reactjs and utilizing the Next.js framework. My goal is to display different text depending on whether the URL contains "?id=pinned". How can I achieve this? Below is the snippet of my code located in [slug.js] return( ...

Minifying Angular using grunt leads to 'Error initializing module' issue

Currently, I have multiple controllers, models, and services set up as individual files for preparation before minification. My goal is to combine and minify all these files into one JS file for production. To illustrate how my files are structured, here ...

My selection of jQuery multiselect is experiencing issues with enabling disabled options

Incorporating the chosen jQuery plugin into my project has presented me with a challenge. The issue at hand is listed below. I have implemented a dropdown menu that includes both continents and countries in the same list. The specific scenario I am encou ...

Adjust the range slider's color depending on its value

I'm looking to customize the color of a range slider as its value increases, switching from red to green. Below is the code I've tried, but it's not quite working as intended. The goal is for the color to change based on the value of masterR ...

Utilizing the Jquery hover feature to reveal or conceal an element

My Hover function is designed to display and hide sub menus when a person hovers on them. The issue I'm facing is that the menu disappears when I move the mouse down towards it. Can someone help me identify what I am doing wrong here? ...

"Need help passing an API key in the header of a Vue.js project? I recently encountered this issue while using a

How can I include an API key in the header of a Vue.js request? I am using DRF pagination. methods: { getPostData() { axios .get("http://192.168.43.126:8000/api/?page=" + this.currentPage, { headers: { &q ...

Using ExtJS to populate a panel with data from various sources

I am currently working with an Ext.grid.GridPanel that is connected to a Ext.data.JsonStore for data and Ext.grid.ColumnModel for grid specifications. Out of the 10 columns in my grid, 9 are being populated by the json store without any issues. However, I ...