How can I compress any data in client-side JavaScript before sending it to the server?
E.g.
- Text
- JSON
- Binary data
What kind of compression formats can be used?
Today, web browsers support compression natively in JavaScript.
- You can compress any data using common compression methods like Gzip
- For images, you can export PNG out of a canvas
- Web browsers support Streams API that with CompressionStream
- Node.js has zlib built-in that provides some common compression methods (although its interface is very archaic and needs to be tweaked for async).
- In web, you might need to base64 encode and decode the compressed payload because embedding it in formats like JSON does not support binary
- A good compression format is
gzip
as it is universally supported
CompressionStream
supports only deflate (zip) and Gzip.
An example is how to compress a string (e.g. JSON) on the client-side JavaScript.
Example compress.js
:
/**
* Compress strings and Blobs as gzip.
*
* Based on https://dev.to/ternentdotdev/json-compression-in-the-browser-with-gzip-and-the-compression-streams-api-4135
*
* @param {string | Blob} data Plain-text data as a string or Blob for binary data
*
* @param {boolean} base64 Set true to output base64 encoded (can be embedded in JSON, query parameters, etc.), otherwise return JavaScript blob
*
* @returns {Promise<string | Blob>} Compressed binary or
*/
export async function compressStringGzip(data, base64) {
// Convert incoming string to a stream
let stream;
if(typeof data == "string") {
stream = new Blob([data], {
type: 'text/plain',
}).stream();
} else {
// Assume blog
stream = data.stream();
}
// gzip stream
const compressedReadableStream = stream.pipeThrough(
new CompressionStream("gzip")
);
// create Response
const compressedResponse = await new Response(compressedReadableStream);
// Get response Blob
const blob = await compressedResponse.blob();
if(base64) {
// Get the ArrayBuffer
const buffer = await blob.arrayBuffer();
// convert ArrayBuffer to base64 encoded string
const compressedBase64 = btoa(
String.fromCharCode(
...new Uint8Array(buffer)
)
);
return compressedBase64;
} else {
return blob;
}
}
Example decompress.js:
import * as zlib from "zlib";
import * as util from "util";
/**
* Decompress a string or a Buffer as gzip.
*
* Assume the encoded data compresses as UTF-8 text.
*
* Only for server-side Node.js.
*
* @param {*} data Data to decompress, either Buffer or a string
*
* @param {*} plainText Set true if the encoded data is utf-8 and you want to decode the stirng
*
* @returns {Promise<string | Buffer>} Decompressed data as a Buffer object or a string if plainText is set
*/
export async function decompressStringGzip(data, plainText) {
const gunzip = util.promisify(zlib.gunzip);
// Assume base64 encoded if we are given data as a string
if(typeof data == "string") {
data = Buffer.from(data, 'base64');
}
const raw = await gunzip(data);
if(plainText) {
return raw.toString("utf-8");
}
return raw;
}