Efficiently integrating Firebase Functions with external sub path imports beyond the project's

Question

Efficiently integrating Firebase Functions with external sub path imports beyond the project's

I encountered an issue in my firebase functions project with typescript. The problem arises when I use types from outside the project with sub path imports, causing the build files to become distorted.

Instead of having main:lib/index.js, I have main:lib/functions/src/index.js.

This is the file structure:

petertoth@Peters-MBP-2 lib % tree .
.
├── functions
│   └── src
│       ├── index.js
│       ├── index.js.map
│       ├── journalLogs.type.js
│       ├── journalLogs.type.js.map
│       └── util
│           ├── audiFiles.js
│           ├── audiFiles.js.map
│           ├── db.js
│           ├── db.js.map
│           ├── getJournalSettings.js
│           ├── getJournalSettings.js.map
│           ├── prompt.js
│           ├── prompt.js.map
│           ├── storage.js
│           └── storage.js.map
└── types
    ├── firebase
    │   ├── CreateFullRecording.request.js
    │   ├── CreateFullRecording.request.js.map
    │   ├── generateJournal.requests.js
    │   └── generateJournal.requests.js.map
    ├── firestore
    │   ├── JournalSettingsDoc.js
    │   └── JournalSettingsDoc.js.map
    └── openai
        ├── language.js
        └── language.js.map

Here's a snippet from package.json:

{
  "name": "functions",
 ...
 "imports": {
    "#types/*": ["../types/*"]
  },
  "engines": {
    "node": "16"
  },
  "main": "lib/functions/src/index.js",
  "private": true
  ...
}

And this is tsconfig:

{
  "compilerOptions": {
    "module": "commonjs",
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "outDir": "lib",
    "sourceMap": true,
    "strict": true,
    "target": "es2017",
    "moduleResolution": "nodenext",
    "paths": {
      "#types/*": ["../types/*"]
    }
  },
  "baseUrl": ".",
  "compileOnSave": true,
  "include": [
    "src"
  ]
}

During the local development, everything works perfectly fine. However, when I attempt to deploy using firebase deploy --only functions, the following error occurs:

i  deploying functions
Running command: npm --prefix "$RESOURCE_DIR" run build

> build
> tsc

✔  functions: Finished running predeploy script.
i  functions: preparing codebase default for deployment
i  functions: ensuring required API cloudfunctions.googleapis.com is enabled...
i  functions: ensuring required API cloudbuild.googleapis.com is enabled...
i  artifactregistry: ensuring required API artifactregistry.googleapis.com is enabled...
✔  functions: required API cloudfunctions.googleapis.com is enabled
✔  artifactregistry: required API artifactregistry.googleapis.com is enabled
✔  functions: required API cloudbuild.googleapis.com is enabled
i  functions: preparing ./functions/ directory for uploading...
i  functions: packaged XX (98.13 KB) for uploading
✔  functions: ./functions/ folder uploaded successfully
i  functions: updating Node.js 16 function generateJournal(europe-west1)...
i  functions: updating Node.js 16 function migrageJournal(europe-west1)...
i  functions: updating Node.js 16 function getCollectionNames(europe-west1)...
i  functions: updating Node.js 16 function createFullRecording(europe-west1)...
Build failed: > build
> tsc

src/index.ts(22,44): error TS2307: Cannot find module '#types/firebase/CreateFullRecording.request' or its corresponding type declarations.
src/index.ts(25,40): error TS2307: Cannot find module '#types/firebase/generateJournal.requests' or its corresponding type declarations.
src/util/getJournalSettings.ts(1,36): error TS2307: Cannot find module '#types/firestore/JournalSettingsDoc' or its corresponding type declarations.
src/util/prompt.ts(1,36): error TS2307: Cannot find module '#types/firestore/JournalSettingsDoc' or its corresponding type declarations.
src/util/prompt.ts(2,30): error TS2307: Cannot find module '#types/openai/language' or its corresponding type declarations.
src/util/prompt.ts(15,31): error TS18046: 'journalBullet' is of type 'unknown'.
src/util/prompt.ts(16,34): error TS18046: 'journalBullet' is of type 'unknown'.
src/util/prompt.ts(48,10): error TS7053: Element implicitly has an 'any' type because expression of type 'LanguageCode' can't be used to index type '{ en: string; da: string; sv: string; no: string; de: string; }'.; Error ID: 1a2262f3

I suspect that there might be an issue in the configuration related to packaging files for deployment. Unfortunately, I'm unsure how to further debug this problem.

node.js typescript google-cloud-functions firebase-tools

Answer 1

Answer №1

Make sure your Firebase CLI is up to date: npm install -g firebase-tools

In the latest update, Google Cloud Functions now automatically runs npm run build during deployment. Unfortunately, this seems to be causing some issues.

However, take note that according to a comment by colerogers, the Firebase team has addressed this problem in firebase-tools v11.27.0 by disabling the mentioned feature.

When you upgrade to at least that version, you should no longer encounter this error. The issue will be resolved without any additional steps required.

Answer 2

Make sure your Firebase CLI is up to date: npm install -g firebase-tools

In the latest update, Google Cloud Functions now automatically runs npm run build during deployment. Unfortunately, this seems to be causing some issues.

However, take note that according to a comment by colerogers, the Firebase team has addressed this problem in firebase-tools v11.27.0 by disabling the mentioned feature.

When you upgrade to at least that version, you should no longer encounter this error. The issue will be resolved without any additional steps required.

Answer 3

Answer №2

In my experience, I have encountered a situation where the files under lib/functions/src/index.js seemed to be moving around, resulting in a confusing mix of the "src" folder with the build output. This usually happens when you reference imports from above the TypeScript build folder without using a "reference" to another TypeScript project. It appears that this occurs because TSC (TypeScript compiler) tries to find the root common path for all imports and ends up shifting the project folder upwards. As a consequence, the "src" folder is not considered special anymore but instead regarded as just a subfolder. Obviously, this is not an ideal outcome for your build - even if it seems to work locally.

To resolve this issue, you can add another tsconfig file in the "types" folder to treat it as its own TypeScript project. Then, include a project reference to this new tsconfig file in your main tsconfig:

  "references": [
    {
      "path": "../types"
    }
  ]

Additionally, adjust your TypeScript build command to use the --build flag in your package.json:

    "build": "tsc --build ",

By doing this, when you rebuild your project, the main project output will move back up two folders to where it should be in the lib directory, specifically in lib/index.js and lib/util/*, etc. (You may need to delete the existing lib folder first for a clean rebuild).

With these changes, it should fix the peculiar build issue that you encountered. However, there is still an important consideration when deploying your project: the lib/types folder will not be uploaded because, as Doug explains, they are no longer within the functions folder and are only referenced locally.

I have faced this problem before and found a solution without resorting to a private npm repository or other complex methods. Instead, I used a pre-deploy build script that packages the lib project into a GZ file and adds it as a file dependency to the main project.

In the package.json file of your "types" library project, include the name, version, and a build script:

{
  "name": "my-lib",
  "version": "1.0.0",
  "scripts": {
    "build": "npm pack"
  },

Running npm run build will generate a zipped up version of the library like: my-lib-1.0.0.gz.

You then need to copy this file from /lib to /functions/ (or any desired subfolder).

Finally, require the correct version in your main project's package.json file:

  "main": "lib/index.js",
  "dependencies": {
    "#types": "file:my-lib-1.0.0.tgz"
  },

When deploying the functions, the GZ file will be uploaded and resolved during the deployment build from the local file in the container.

I have written a bash script that handles these steps before deploying, which also includes an idiot-proof version check:

echo Packaging the latest my-lib and importing it to gcp/functions...
echo Note: the functions package.json must require the new version.
#read -p "Press any key..."

cd libs/my-lib;
# Detect the current version
LIB_VERSION=`node -pe "require('./package.json').version"`

echo Version \($LIB_VERSION\) detected OK?
read -p "Press any key..."

# Package the latest version
LIB_FILENAME=my-lib-${LIB_VERSION}.tgz
npm run build;
# Move it into the Cloud Function folder so the tarfile can be deployed.
mv $LIB_FILENAME ../../gcp/functions;
# Reinstall it locally too
cd ../../gcp/functions;
npm i $LIB_FILENAME
cd ..

Based on your question, I am uncertain how the accepted answer could have provided a resolution for your specific issue.

Answer 4

In my experience, I have encountered a situation where the files under lib/functions/src/index.js seemed to be moving around, resulting in a confusing mix of the "src" folder with the build output. This usually happens when you reference imports from above the TypeScript build folder without using a "reference" to another TypeScript project. It appears that this occurs because TSC (TypeScript compiler) tries to find the root common path for all imports and ends up shifting the project folder upwards. As a consequence, the "src" folder is not considered special anymore but instead regarded as just a subfolder. Obviously, this is not an ideal outcome for your build - even if it seems to work locally.

To resolve this issue, you can add another tsconfig file in the "types" folder to treat it as its own TypeScript project. Then, include a project reference to this new tsconfig file in your main tsconfig:

  "references": [
    {
      "path": "../types"
    }
  ]

Additionally, adjust your TypeScript build command to use the --build flag in your package.json:

    "build": "tsc --build ",

By doing this, when you rebuild your project, the main project output will move back up two folders to where it should be in the lib directory, specifically in lib/index.js and lib/util/*, etc. (You may need to delete the existing lib folder first for a clean rebuild).

With these changes, it should fix the peculiar build issue that you encountered. However, there is still an important consideration when deploying your project: the lib/types folder will not be uploaded because, as Doug explains, they are no longer within the functions folder and are only referenced locally.

I have faced this problem before and found a solution without resorting to a private npm repository or other complex methods. Instead, I used a pre-deploy build script that packages the lib project into a GZ file and adds it as a file dependency to the main project.

In the package.json file of your "types" library project, include the name, version, and a build script:

{
  "name": "my-lib",
  "version": "1.0.0",
  "scripts": {
    "build": "npm pack"
  },

Running npm run build will generate a zipped up version of the library like: my-lib-1.0.0.gz.

You then need to copy this file from /lib to /functions/ (or any desired subfolder).

Finally, require the correct version in your main project's package.json file:

  "main": "lib/index.js",
  "dependencies": {
    "#types": "file:my-lib-1.0.0.tgz"
  },

When deploying the functions, the GZ file will be uploaded and resolved during the deployment build from the local file in the container.

I have written a bash script that handles these steps before deploying, which also includes an idiot-proof version check:

echo Packaging the latest my-lib and importing it to gcp/functions...
echo Note: the functions package.json must require the new version.
#read -p "Press any key..."

cd libs/my-lib;
# Detect the current version
LIB_VERSION=`node -pe "require('./package.json').version"`

echo Version \($LIB_VERSION\) detected OK?
read -p "Press any key..."

# Package the latest version
LIB_FILENAME=my-lib-${LIB_VERSION}.tgz
npm run build;
# Move it into the Cloud Function folder so the tarfile can be deployed.
mv $LIB_FILENAME ../../gcp/functions;
# Reinstall it locally too
cd ../../gcp/functions;
npm i $LIB_FILENAME
cd ..

Based on your question, I am uncertain how the accepted answer could have provided a resolution for your specific issue.

Efficiently integrating Firebase Functions with external sub path imports beyond the project's

Answer №1

Answer №2

Similar questions

How to build custom middleware with parameters in Node.js

NEXT JS 13 experiencing an infinite loop when using State, encountering an error with Params, and facing issues with hook definition

Error in Node and Express: Unable to access route

Is it possible to nest the newly added element beneath its sibling in Cheerio?

Tips on automatically changing the background image every few seconds

What is the best way to extract the text from a class only when it is nested within a particular header tag?

Triggering event within the componentDidUpdate lifecycle method

Establishing the parameters for a list that is not empty using a specific data type

Now that connect no longer utilizes the parseCookie method, what is the alternative method for accessing session data in express?

Poorly packaged library - Custom Angular library - Node Package Manager

What is the best way to store an audio Blob in the repository file system?

ValueError: Unable to perform slicing operation on this data

React JS displayed the string of /static/media/~ instead of rendering its markdown content

Travis is experiencing issues with retrieving the API key after logging in with npm

When executing `npm install` within a Docker container, the error message `cb() never called!` may be encountered

Fetching information from external API prior to displaying it

Attempting to establish a basic socket connection utilizing socket.io

Encountering an error when attempting to connect a nodejs REST API to an HTTPS URL using port 844

Dealing with a Nodejs/Express and Angular project - Handling the 404 error

Node.js sessions cookies are not being generated properly as req.cookies['connect.sid'] is not being created, and req.sessionID is changing every time the page is refreshed