Node corrupting images during upload

Question

Node corrupting images during upload

I've been facing an issue with corrupted images when uploading them via Next.js API routes using Formidable.

When submitting a form from my React component, I'm utilizing the following functions:

    const fileUpload = async (file: File) => {
        const url = '/api/image'
        const formData = new FormData()
        formData.append('file', file)
        const config = {
            headers: {
                'content-type': 'multipart/form-data',
            },
        }
        const response = await axios.post(url, formData, config)
        const { data } = response
        return data
    }

    const handleSubmit = async (event: React.SyntheticEvent) => {
        const url = '/api/post'
        if (files) {
                        // ignore 
            fileUpload(files[0]).then((response) =>
                console.log('submit response', response)
            )
        }
        event.preventDefault()
    }

The structure of the API route in Next.js is as follows:

import formidable from 'formidable'
const fs = require('fs')
const { BlobServiceClient } = require('@azure/storage-blob')

if (process.env.NODE_ENV !== 'production') {
    require('dotenv').config()
}

const AZURE_STORAGE_CONNECTION_STRING =
    process.env.AZURE_STORAGE_CONNECTION_STRING

export const config = {
    api: {
        bodyParser: false,
    },
}

const getBlobName = (originalName) => {
    const identifier = Math.random().toString().replace(/0\./, '')
    return `${identifier}-${originalName}`
}

export default async (req, res) => {
    const form = new formidable.IncomingForm()
    form.keepExtensions = true
    form.uploadDir = './public/static/uploads'

    form.parse(req, async (err, fields, files) => {
        if (err) {
            return
        }
        return res.json({ fields, files })
    })

    form.on('file', async (name, file) => {
        const blobServiceClient = await BlobServiceClient.fromConnectionString(
            AZURE_STORAGE_CONNECTION_STRING
        )
        const containerClient = await blobServiceClient.getContainerClient(
            'images'
        )
        const buff = fs.readFileSync(file.path)
        const data = buff.toString('base64')

        const blobName = getBlobName(file.name)
        const blockBlobClient = containerClient.getBlockBlobClient(blobName)

        blockBlobClient.upload(data, data.length)
    })
}

Upon inspection, the image stored locally appears to be corrupted and resembles a TV screen showing static. It seems like there might be an issue with how I am encoding it — Could it be related to ContentType or string encoding?

node.js reactjs typescript azure-blob-storage next.js

Answer 1

Answer №1

It seems like the issue arises from converting the data into a base64 encoded string:

const data = buff.toString('base64')

Since the uploaded file is already being saved somewhere on the server (accomplished by using the formidable package), consider trying something like this:

    const blobName = getBlobName(file.name)
    const blockBlobClient = containerClient.getBlockBlobClient(blobName)
    blockBlobClient.uploadFile(file.path)

Reference for the uploadFile() method: https://learn.microsoft.com/en-gb/javascript/api/@azure/storage-blob/blockblobclient?view=azure-node-latest#uploadfile-string--blockblobparalleluploadoptions-.

UPDATE

Please try out this code snippet. I tested it and successfully uploaded a file without any corruption issues.

  form.on('file', async (name, file) => {
    const blobServiceClient = await BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);
    const containerClient = await blobServiceClient.getContainerClient('images');
    const blobName = file.name;
    const contentType = file.type;
    const filePath = file.path;
    const blockBlobClient = containerClient.getBlockBlobClient(blobName);
    const uploadBlobResponse = await blockBlobClient.uploadFile(file.path);
  });

UPDATE #2

Here's my complete code implementation:

<form id="form1" method="post" action="/upload" enctype="multipart/form-data">
  <input name="file" type="file" id="file1" accept="image/*"/>
  <input type="button" id="button1" value="Upload" />
</form>


<script src="https://code.jquery.com/jquery-1.12.4.js" integrity="sha256-Qw82+bXyGq6MydymqBxNPYTaUXXq7c8v3CwiYwLLNXU=" crossorigin="anonymous"></script>

<script>
  $(document).on('ready', function() {

  });
  $('#button1').on('click', function(e) {
    data = new FormData();
    console.log($('#file1')[0].files[0]);
    data.append('file', $('#file1')[0].files[0]);
    console.log(data);
    $.ajax({
            url: '/upload',
            data: data,
            processData: false,
            contentType: false,
            type: 'POST',
            success: function ( data ) {
                alert( data );
            }
        });
    e.preventDefault();
    e.stopPropagation();
  });
</script>

var express = require('express'),
    path = require('path'),
    fs = require('fs'),
    formidable = require('formidable');
const { BlobServiceClient } = require('@azure/storage-blob');

var app = express();

app.set('port', (process.env.PORT || 5000));

// Configure express to serve static files from specific directories
app.use(express.static('public'));
app.use('/uploads', express.static('uploads'));

app.listen(app.get('port'), function() {
    console.log('Express started at port ' + app.get('port'));
});

app.get('/', function (req, res) {
    res.sendFile(path.join(__dirname, 'index.html'));
});


app.post('/upload', function(req, res) {
  const connectionString = 'DefaultEndpointsProtocol=https;AccountName=account-name;AccountKey=account-key;EndpointSuffix=core.windows.net;';
  const container = 'container-name';

  let form = new formidable.IncomingForm();
  form.keepExtensions = true;
  form.uploadDir = './public/static/uploads';
  form.parse(req, async function (err, fields, files) {
  });

  form.on('file', async (name, file) => {
    const blobServiceClient = await BlobServiceClient.fromConnectionString(connectionString);
    const containerClient = await blobServiceClient.getContainerClient(container);
    const blobName = file.name;
    const contentType = file.type;
    const filePath = file.path;
    console.log(file);
    const blockBlobClient = containerClient.getBlockBlobClient(blobName);
    const uploadBlobResponse = await blockBlobClient.uploadFile(file.path);
    console.log(uploadBlobResponse);
  });
});

Answer 2

It seems like the issue arises from converting the data into a base64 encoded string:

const data = buff.toString('base64')

Since the uploaded file is already being saved somewhere on the server (accomplished by using the formidable package), consider trying something like this:

    const blobName = getBlobName(file.name)
    const blockBlobClient = containerClient.getBlockBlobClient(blobName)
    blockBlobClient.uploadFile(file.path)

Reference for the uploadFile() method: https://learn.microsoft.com/en-gb/javascript/api/@azure/storage-blob/blockblobclient?view=azure-node-latest#uploadfile-string--blockblobparalleluploadoptions-.

UPDATE

Please try out this code snippet. I tested it and successfully uploaded a file without any corruption issues.

  form.on('file', async (name, file) => {
    const blobServiceClient = await BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);
    const containerClient = await blobServiceClient.getContainerClient('images');
    const blobName = file.name;
    const contentType = file.type;
    const filePath = file.path;
    const blockBlobClient = containerClient.getBlockBlobClient(blobName);
    const uploadBlobResponse = await blockBlobClient.uploadFile(file.path);
  });

UPDATE #2

Here's my complete code implementation:

<form id="form1" method="post" action="/upload" enctype="multipart/form-data">
  <input name="file" type="file" id="file1" accept="image/*"/>
  <input type="button" id="button1" value="Upload" />
</form>


<script src="https://code.jquery.com/jquery-1.12.4.js" integrity="sha256-Qw82+bXyGq6MydymqBxNPYTaUXXq7c8v3CwiYwLLNXU=" crossorigin="anonymous"></script>

<script>
  $(document).on('ready', function() {

  });
  $('#button1').on('click', function(e) {
    data = new FormData();
    console.log($('#file1')[0].files[0]);
    data.append('file', $('#file1')[0].files[0]);
    console.log(data);
    $.ajax({
            url: '/upload',
            data: data,
            processData: false,
            contentType: false,
            type: 'POST',
            success: function ( data ) {
                alert( data );
            }
        });
    e.preventDefault();
    e.stopPropagation();
  });
</script>

var express = require('express'),
    path = require('path'),
    fs = require('fs'),
    formidable = require('formidable');
const { BlobServiceClient } = require('@azure/storage-blob');

var app = express();

app.set('port', (process.env.PORT || 5000));

// Configure express to serve static files from specific directories
app.use(express.static('public'));
app.use('/uploads', express.static('uploads'));

app.listen(app.get('port'), function() {
    console.log('Express started at port ' + app.get('port'));
});

app.get('/', function (req, res) {
    res.sendFile(path.join(__dirname, 'index.html'));
});


app.post('/upload', function(req, res) {
  const connectionString = 'DefaultEndpointsProtocol=https;AccountName=account-name;AccountKey=account-key;EndpointSuffix=core.windows.net;';
  const container = 'container-name';

  let form = new formidable.IncomingForm();
  form.keepExtensions = true;
  form.uploadDir = './public/static/uploads';
  form.parse(req, async function (err, fields, files) {
  });

  form.on('file', async (name, file) => {
    const blobServiceClient = await BlobServiceClient.fromConnectionString(connectionString);
    const containerClient = await blobServiceClient.getContainerClient(container);
    const blobName = file.name;
    const contentType = file.type;
    const filePath = file.path;
    console.log(file);
    const blockBlobClient = containerClient.getBlockBlobClient(blobName);
    const uploadBlobResponse = await blockBlobClient.uploadFile(file.path);
    console.log(uploadBlobResponse);
  });
});

Node corrupting images during upload

Answer №1

Similar questions

Response Looping Function

What is the proper method for setting up handlers in functional React components?

NodeJS tool for interacting with a simulated DOM and coordinating updates with a web browser

Updating a single .jshintrc option for a folder

Utilize the up and down arrow keys to scroll through a description list in React

When attempting to retrieve information using the findById(''), the process became frozen

Achieving selective exclusion of specific keys/values while iterating through an array and rendering them on a table using Angular

Utilize app.js variables within routes without relying on global express

Instead of watching the video player, the puppeteer automatically downloads the video file

Error in communication: Microsoft Bot framework encountered an issue with receiving a security token during the ChatConnector process

When attempting to transfer data from a client to a server, a "Bad Request" error is encountered with the message: `POST http://localhost:5000/api

Using NPM version 7 to pass flags to a script is a convenient feature

A guide on setting up fixed row numbers in MUI-X DataGrid

Updating a label dynamically in Angular

Trouble with defining variables in EJS

In React Native on Android, the permission to WRITE_EXTERNAL_STORAGE is consistently denied

The reducer I have is inexplicably returning undefined even though I'm certain it was added to combineReducers

The error message "Cannot bind to 'ngForOf' because it is not recognized as a valid property of the element."

Tips for obtaining an array of JSON objects instead of Mongoose documents

Knejx Js is the embodiment of guarantees