I wanted to create an function/API in node which will help me calculate the desktop and mobile page sizes for a given URL and also provide the compression/encoding strategy which was used by the server.
I found a way of doing this using axios so I’m sharing my solution here. Any further suggestions are appreciated.
We can use axios for this purpose as below
-
First we need to define
User-Agent
request header for desktop and mobile devices. Here I’m using aisMobile
as a dummy flag which you can pass in your function:const headers = { 'User-Agent': isMobile ? 'Mozilla/5.0 (Windows NT 10.0; Android 13; SM-G981B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Mobile Safari/537.36' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36' , 'Accept-Encoding': 'gzip, deflate, br' }
-
Then we can use axios to get the page size as below:
const response = await axios.get(url, { headers, responseType: 'arraybuffer', validateStatus: () => true, decompress: false, }); const sizeInBytes = response.data.length; const encoding = response.headers['content-encoding'] || 'none';
Note:
- You can use the response header
content-length
to get the page size but there are many cases when this header is not provided by the server, hence I have used a response type of arraybuffer and calculated the length of the buffer. - if you don’t use the
decompress: false
flag then axios/node will by default decompress the response and also youcontent-encoding
header will not be present due to which you won’t be able to identify the encoding type.