We’re using RabbitMQ as our message broker, when I emit a message I’n using this piece
import { Connection, Channel, ConsumeMessage, Options, Replies, connect } from 'amqplib';
import * as msgpack from '@msgpack/msgpack';
async emit(eventName: string, data: any, route: string = '*'): Promise<boolean> {
let serializedMessage: Uint8Array | null = null;
let buffer: Buffer | null = null;
let channel: Channel | undefined;
try {
const messageData = { eventName, data };
serializedMessage = msgpack.encode(messageData);
channel = await this.getChannel();
if (!channel) {
throw new Error('Channel is not available');
}
buffer = Buffer.from(serializedMessage);
const result = channel.sendToQueue(route, buffer); // THIS LEAKS !!
log.debug(`Memory usage after sending message: ${JSON.stringify(process.memoryUsage())}`);
return result;
} catch (error) {
log.error('Failed to emit message', error);
throw error;
} finally {
serializedMessage = null;
if (buffer) {
buffer.fill(0); // Clear buffer content
buffer = null;
}
if (global.gc) {
global.gc();
}
}
}
However this code causes my POD to reach 4GB+ of RAM (and it never stops),
and noticed that it happens when sending payload of 1MB and above.
On the other hand I’ve tried to understand where is happens, and found it:
async emit(eventName: string, data: any, route: string = '*'): Promise<boolean> {
let fakeSerializedMessage: Uint8Array | null = null;
// Fake
const fakeMessageData = {
eventName,
data: {
key1: {
subKey1: 'key.....',
config: {
foo: 'barbur',
},
},
},
};
fakeSerializedMessage = msgpack.encode(fakeMessageData);
try {
const channel: Channel = await this.getChannel();
const result = channel.sendToQueue(route, Buffer.from(fakeSerializedMessage));
log.debug(`Memory usage after sending message: ${JSON.stringify(process.memoryUsage())}`);
return result;
} catch (error) {
log.error('Failed to emit message', error);
throw error;
} finally {
// empty
}
}
When changing emit() to be as the 2ND form (sending a fake message), no memory leak.
We’ve tried multiple ways to prevent it even using the GC, but I don’t like this so called solution.
Any idea how to solve this memory leak ?