Encountering an SSH issue while attempting to deploy to Digital Ocean using Gitlab's CI/CD functionality

Hey there, I've been attempting to deploy to Digital Ocean using a Gitlab CI/CD pipeline. However, every time I run the pipeline, I encounter the following error message: "chmod: /root/.ssh/id_rsa: No such file or directory $ chmod og= ~/.ssh/id_rsa Cleaning up file based variables 00:00 ERROR: Job failed: exit code 1"

It seems like it's not recognizing the user I set up for deployment and is defaulting to root. Oddly though, when I check the ssh key on my server using the cat command, it appears under both the root and deployer users. Below is the content of my .yml file.

before_script:
  - echo $PATH
  - pwd
  - whoami 
  - mkdir -p ~/.ssh
  - cd ~/.ssh
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' > id_rsa
  - echo "$SSH_PUBLIC_KEY" | tr -d '\r' > id_rsa.pub
  - chmod 700 id_rsa id_rsa.pub
  - cp id_rsa.pub authorized_keys
  - cp id_rsa.pub known_hosts
  - ls -ld *
  - cd -

stages:
  - build
  - publish
  - deploy

variables:
  TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest
  TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHORT_SHA

build:
  image: node:latest
  stage: build
  script:
    - npm install
    - echo   "ACCOUNT_SID=$ACCOUNT_SID" >> .env
    - echo   "AUTH_TOKEN=$AUTH_TOKEN" >> .env
    - echo   "API_KEY=$API_KEY" >> .env
    - echo   "API_SECRET=$API_SECRET" >> .env
    - echo   "PHONE_NUMBER=$PHONE_NUMBER" >> .env
    - echo    "sengrid_api=$sengrid_api" >> .env

publish:
  image: docker:latest
  stage: publish
  services:
    - docker:dind
  script:
    - docker build . -t $TAG_COMMIT -t $TAG_LATEST 
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
    - docker push $TAG_COMMIT
    - docker push $TAG_LATEST

deploy:
  image: alpine:latest
  stage: deploy
  tags:
    - deployment
  script:
    - whoami
    - uname -a
    - echo "user $SERVER_USER"
    - echo "ip $SERVER_IP"
    - echo "id_rsa $ID_RSA"
    - (which ifconfig) || (apt install net-tools)
    - /sbin/ifconfig
    - touch blah
    - find .
    - apk update && apk add openssh-client
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY"
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker pull $TAG_COMMIT"
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker container rm -f my-app || true"
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker run -d -p 80:3000 --name my-app $TAG_COMMIT"
  environment:
    name: production
    url: http://167.172.225.124
  only:
    - master

node.jsshellexpressgitlabgitlab-ci-runner

Answer №1

Finally, after hours of effort and overcoming various errors:

Made progress by executing the following commands: - cat id_rsa.pub >> authorized_keys: resolved permission denied (public key,password) issue - ssh-keyscan gitlab.com >> authorized_keys: successfully fixed connection refused error. Here is the complete .yml file that is now working like a charm.

# ssh-keyscan gitlab.com >> authorized_keys: use this command to add gitlab ssh keys to sever. Run on server terminal
# cat id_rsa.pub >> authorized_keys Run this command on the sever on the terminal. 
# Both COMMANDS ABOVE ARE necessary.

stages:
  - build
  - publish
  - deploy

variables:
  TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest
  TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHORT_SHA

build:
  image: node:latest
  stage: build
  script:
    - npm install
    - echo   "ACCOUNT_SID=$ACCOUNT_SID" >> .env
    - echo   "AUTH_TOKEN=$AUTH_TOKEN" >> .env
    - echo   "API_KEY=$API_KEY" >> .env
    - echo   "API_SECRET=$API_SECRET" >> .env
    - echo   "PHONE_NUMBER=$PHONE_NUMBER" >> .env
    - echo    "sengrid_api=$sengrid_api" >> .env

publish:
  image: docker:latest
  stage: publish
  services:
    - docker:dind
  script:
    - docker build . -t $TAG_COMMIT -t $TAG_LATEST 
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
    - docker push $TAG_COMMIT
    - docker push $TAG_LATEST

deploy:
  image: ubuntu:latest
  stage: deploy
  tags:
    - deployment
  before_script:
    ##
  ## Install ssh-agent if not already installed, it is required by Docker.
  ## (change apt-get to yum if you use an RPM-based image)
  ##
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'

  ##
  ## Run ssh-agent (inside the build environment)
  ##
  - eval $(ssh-agent -s)
  ##
  ## Create the SSH directory and give it the right permissions
  ##
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  ##
  ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  ## We're using tr to fix line endings which makes ed25519 keys work
  ## without extra base64 encoding.
  ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
  ##
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa
  - echo "$SSH_PUBLIC_KEY" | tr -d '\r' > ~/.ssh/id_rsa.pub
  - chmod 600 ~/.ssh/*
  - chmod 644 ~/.ssh/*.pub
  - ssh-add
 

  ##
  ## Use ssh-keyscan to scan the keys of your private server. Replace gitlab.com
  ## with your own domain name. You can copy and repeat that command if you have
  ## more than one server to connect to.
  ##
  - ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts
  - ls -ld ~/.ssh/*
  - cat ~/.ssh/*
  ##
  ## Alternatively, assuming you created the SSH_SERVER_HOSTKEYS variable
  ## previously, uncomment the following two lines instead.
  ##
  #- echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'
  #- chmod 644 ~/.ssh/known_hosts

  ##
  ## You can optionally disable host key checking. Be aware that by adding that
  ## you are suspectible to man-in-the-middle attacks.
  ## WARNING: Use this only with the Docker executor, if you use it with shell
  ## you will overwrite your user's SSH config.
  ##
  #- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

  ##
  ## Optionally, if you will be using any Git commands, set the user name and
  ## email.
  ##

  script:
    
    - ssh -v -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY"
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker pull $TAG_COMMIT"
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker container rm -f my-app || true"
    - ssh  -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP "docker run -d -p 80:3000 --name my-app $TAG_COMMIT"
  environment:
    name: production
    url: http://167.172.225.124
  only:
    - master

Answer №2

If you are working through the steps outlined in the DigitalOcean tutorial, you will need to have a non-root user with sudo privileges, as well as a user account on GitLab that has access to a container registry.

When installing the gitlab-runner service using script.deb.sh, you will be prompted for the password of a non-root user to proceed.

Furthermore, it is important to create a dedicated user specifically for deployment tasks, and then configure a CI/CD pipeline to use this user to log in to the server.

This means that the gitlab-ci process should not run as root, as root privilege is unnecessary and should not be involved in any part of the process.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Is there a way to set a custom URL for Firebase Realtime Database without using the "default-rtdb" prefix?

My real-time database URL looks something like Upon starting my node.js server, I encounter the following error message : @firebase/database: FIREBASE WARNING: Firebase error. Please make sure that the URL of your Firebase Realtime Database instance is ...

node.jsfirebase-realtime-database

Encountering a problem while trying to upload an image using multer in a Node.js application

import bodyParser from "body-parser"; import express from "express"; import mongoose from "mongoose"; import ejs from "ejs"; import multer from "multer"; const app = express(); const port = 3000; app.set( ...

node.jsmongodbexpressmulterbody-parser

Communication between the Node development server and the Spring Boot application was hindered by a Cross-Origin Request

Here is the breakdown of my current setup: Backend: Utilizing Spring Boot (Java) with an endpoint at :8088 Frontend: Running Vue on a Node development server exposed at :8080 On the frontend, I have reconfigured axios in a file named http-common.js to s ...

javascriptjavanode.jsspringcross-domain

Encountered a failure while attempting to substitute environment variables in configuration files with the help of Bash and

Trying to integrate a private NPM module into my application is proving to be quite challenging. I need to configure the NPM access tokens correctly so that third-party tools like Heroku and CI can access and install the module. In my ~/.bash_profile, I h ...

node.jsbashherokunpm

cycle through several handlebars entities

Hey friends, I could really use your help right now. I'm attempting to iterate through these objects using handlebars. RowDataPacket { idUser: 1, username: 'xxxxxx', password: 'xxxxx', fullname: 'Julian Rincon'}, RowDat ...

javascriptnode.jsexpresshandlebars.js

I am encountering difficulties displaying the image and CSS files from another folder in my HTML with Node.js

I am facing difficulty in establishing a connection between my front-end and back-end using node.js. As a result, the website only displays the HTML code without linking to the CSS or image files. Here is the folder structure: root -src •pi ...

javascripthtmlcssnode.jsexpress

User experiencing data loss following JWT refresh

Before diving into the issue, let me share the pertinent code snippets. This is my middleware for authenticating JWT: module.exports.authenticateToken = (req, res, next) => { const authHeader = req.headers["authorization"]; const token = authHeader ...

node.jsexpressjwtauthorization

``There seems to be an issue with the Express app when trying to access it

I have set up an express app that I want other devices on the same WIFI network to access without requiring internet connectivity. The main computer hosting the app is assigned with a fixed IP address: 192.168.1.60 In my server.js file, I have included t ...

javascriptnode.jsexpresslocalhostlocal

Is it possible to dynamically add nodes to the palette without the need to restart the server?

Is there a way to dynamically add custom nodes to the node-red palette without relying on the standard "npm install node service" or manual server restart? If so, what is the best approach to achieve this? ...

node.jsnpmnode-red

A guide on compiling Sass without using gulp and Laravel Elixir [SOLUTION]

If you encounter an error similar to this one: https://i.stack.imgur.com/ZqVeV.png You have come to the right place :) ...

phpnode.jslaravelnpmsass

Upon triggering a GET request to the URL "http://localhost:3000/search", a "404 (Not Found)" error response was received. Interestingly

Can Someone assist me please? I'm having trouble creating a website similar to YouTube and encountering the following error: GET http://localhost:3000/search 404 (Not Found) Uncaught (in promise) Error: Request failed with status code 404 at createEr ...

javascriptnode.jsreactjsaxios

Utilizing cheerio to set outerHTML in HTML

Could someone kindly assist me with setting the outerHTML of an element using cheerio? I seem to be encountering some issues with this process. For example, let's consider the following HTML structure: <div class="page-info"> <s ...

javascripthtmlnode.jsnpmcheerio

Comparing the Distinctions of jQuery Post and HTML Form Post

After researching extensively, I am still confused about the difference between a Jquery $.post method and submitting an HTML form using post. Here is my code for the jQuery implementation: $.post( "/test", query , function( data ) { console.log(data) ...

jqueryhtmlnode.jsexpress

Unexpected Undefined Return in Request Parameters

Hey everyone, I'm currently working on setting up a mock API server using a JSON file with some dummy data. The first route I created is functioning perfectly: const express = require("express"); const router = express.Router(); const data = requir ...

javascriptnode.jsjsonexpressmockserver

What is the best way to retrieve routes from a custom REST API using React Native?

Working on developing an app for my company has been an exciting challenge for me. Being new to programming, I decided to take a react-native course and a nodeJs course to enhance my skills. One particular question I have is related to fetching data from m ...

node.jspostgresqlreact-nativeexpress

Issue: Trying to access a property called 'name' on an undefined variable in NodeJS

I'm currently experiencing an issue with the post statement in Node.js and have been struggling to find a solution. The error message I am encountering is as follows: TypeError: Cannot read property 'name' of undefined at C:\x&bsol ...

node.jsmongoose

Issue updating BrowserList using NPM

Whenever I attempt to update my list of browsers using npx update-browserslist-db@latest, an error message appears. /home/tburton/.npm/_npx/2662cbd72e57abb6/node_modules/update-browserslist-db/cli.js:39 throw e ^ SyntaxError: Unexpected end of ...

node.jsnpmnpxcaniuse-lite

Struggling to locate a suitable mock server that can deliver JSON responses for a specific URL that has been predetermined

I have encountered a challenge in my frontend app development using vue.js. I need to find a mock backend server (without mocking it on the front end). My app is capable of making HTTP requests, specifically GET and PATCH are the methods of interest. I am ...

node.jsjsonexpressmockingjson-api

Unable to establish headers after they have been already sent through socket.io

Hey there, I'm a beginner in nodeJs and I've run into an issue while using socket.io. The error message I'm getting is "Can't set headers after they are sent". Here's the code I've been working on: var app = require('exp ...

node.jsexpresssocket.io

How to Implement Dropdowns with a vue.js Router-Link

I have set up a vue.js router and I am currently using the provided structure to insert links from an array. These links are displayed horizontally on the page. However, I am interested in changing these simple links into dropdown menus instead. Is it po ...

node.jsvue.jsvue-router