Thiết kế website giá rẻ

Question

for the sake of academic interest I am developing my own protocol similar to GRPC. I chose NodejS as the language because it’s easy and pleasant for me to do quite a lot of things.

I developed my own data format, and to be precise, it’s like this

Message type (1 byte):

0x01 – Request
0x02 – Answer

Method Length (1 byte):

The length of the string containing the name of the method to be called.

Called Method (variable length):

A string containing the name of the method to call.

Number of Parameters (1 byte):

An integer value indicating the number of parameters.

Parameters (variable length):

Parameter type (1 byte):

0x01 – Integer (4 bytes)
0x02 – String
0x03 – Boolean value (1 byte)
0x04 – Object (recursively serialized)

Parameter value (variable length):

Depending on the parameter type.

Trailing byte (1 byte):

0xFF – Indicates the end of the message.

I have developed serialization/deserialization algorithms, but deserialization throws an error

This is the data structure i want to serialize

{
  "messageType": "request",
  "callMethod": "PetService.createPet",
  "parameters": [
    {
      "id": 12,
      "name": "john",
      "owner": {
        "id": 1,
        "name": "beria"
      }
    },
    14
  ]
}

Here is the enum code in which I specify the bytes

export enum MessageTypeBytes {
  REQUEST = 0x01,
  RESPONSE = 0x02,
}

export enum DataTypeBytes {
  INTEGER = 0x01,
  STRING = 0x02,
  BOOLEAN = 0x03,
  OBJECT = 0x04,
}

export enum ServiceBytes {
  EOL = 0xFF,
}

This is the serialization algorithm

import { Schema, MessageType, MessageTypeBytes, DataTypeBytes } from "@koda-rpc/common";
import { match } from "ts-pattern";
import { validateParams } from "./validation";

export interface ISerializeOptions {
  callMethod: string;
  messageType: MessageType;
  parameters: Array<unknown>;
  schema: Schema;
}

export const serialize = async ({
  callMethod,
  messageType,
  parameters,
  schema,
}: ISerializeOptions) => {
  console.log(JSON.stringify({
    messageType,
    callMethod,
    parameters,
  }, null, 2));

  await validateParams(
    parameters,
    schema,
    callMethod,
  );

  let buffer: Buffer = Buffer.alloc(0);

  // Message type
  let msgTypeByte = match<MessageType>(messageType)
    .with(MessageType.REQUEST, () => MessageTypeBytes.REQUEST)
    .with(MessageType.RESPONSE, () => MessageTypeBytes.RESPONSE)
    .exhaustive();
  
  buffer = Buffer.concat([buffer, Buffer.from([msgTypeByte])]);

  // Method length
  const methodLengthBuffer = Buffer.alloc(1);
  methodLengthBuffer.writeUint8(callMethod.length);
  buffer = Buffer.concat([buffer, methodLengthBuffer]);

  // Called method
  buffer = Buffer.concat([buffer, Buffer.from(callMethod)]);

  // Parameters count (in symbols)
  const paramsCountBuffer = Buffer.alloc(1);
  paramsCountBuffer.writeUint8(parameters.length);
  buffer = Buffer.concat([buffer, paramsCountBuffer]);

  // Parameters
  parameters.forEach(parameter => {
    if (typeof parameter === 'number') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.NUMBER])]);
      const numberValueBuffer = Buffer.alloc(4);
      numberValueBuffer.writeUint32LE(parameter);
      buffer = Buffer.concat([buffer, numberValueBuffer])
    } else if (typeof parameter === 'string') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.STRING])]);
      const stringLengthBuffer = Buffer.alloc(1);
      stringLengthBuffer.writeUInt8(parameter.length);
      buffer = Buffer.concat([buffer, stringLengthBuffer]);
      buffer = Buffer.concat([buffer, Buffer.from(parameter)]);
    } else if (typeof parameter === 'boolean') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.BOOLEAN])]);
      const boolValueBuffer = Buffer.alloc(1);
      boolValueBuffer.writeUInt8(parameter ? 0x01 : 0x00);
      buffer = Buffer.concat([buffer, boolValueBuffer]);
    } else if (typeof parameter === 'object') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.OBJECT])]);
      buffer = Buffer.concat([buffer, serializeObject(parameter)]);
    } else {
      throw new Error(`Unsupported parameter type: ${typeof parameter}`);
    }
  });

  return buffer;
};

const serializeObject = (obj: object): Buffer => {
  let buffer = Buffer.alloc(0);

  Object.entries(obj).forEach(([key, value]) => {
    // Длина ключа
    const keyLengthBuffer = Buffer.alloc(1);
    keyLengthBuffer.writeUInt8(key.length);
    buffer = Buffer.concat([buffer, keyLengthBuffer]);
    // Ключ
    buffer = Buffer.concat([buffer, Buffer.from(key)]);
    
    if (typeof value === 'number') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.NUMBER])]);
      const numberValueBuffer = Buffer.alloc(4);
      numberValueBuffer.writeInt32LE(value);
      buffer = Buffer.concat([buffer, numberValueBuffer]);
    } else if (typeof value === 'string') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.STRING])]);
      const stringLengthBuffer = Buffer.alloc(1);
      stringLengthBuffer.writeUInt8(value.length);
      buffer = Buffer.concat([buffer, stringLengthBuffer]);
      buffer = Buffer.concat([buffer, Buffer.from(value)]);
    } else if (typeof value === 'boolean') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.BOOLEAN])]);
      const boolValueBuffer = Buffer.alloc(1);
      boolValueBuffer.writeUInt8(value ? 0x01 : 0x00);
      buffer = Buffer.concat([buffer, boolValueBuffer]);
    } else if (typeof value === 'object') {
      buffer = Buffer.concat([buffer, Buffer.from([DataTypeBytes.OBJECT])]);
      buffer = Buffer.concat([buffer, serializeObject(value)]);
    } else {
      throw new Error(`Unsupported value type: ${typeof value}`);
    }
  });

  return buffer;
}

This is a deserialiaztion alghoritm

import { DataTypeBytes, MessageType, MessageTypeBytes, Schema } from "@koda-rpc/common";
import { validateParams } from "./validation";
import { match } from "ts-pattern";
import { IDeserializedData } from "./types";

interface IDeserializeOptions {
  buffer: Buffer;
  messageType: MessageType;
  schema: Schema;
}

export const deserialize = async ({
  buffer,
  schema,
}: IDeserializeOptions): Promise<IDeserializedData> => {
  // Распаковываем данные с использованием zlib
  const uncompressedBuffer = buffer;

  let index = 0;

  // Читаем тип сообщения
  const msgTypeByte = uncompressedBuffer[index++] as MessageTypeBytes;
  const msgType = match<MessageTypeBytes>(msgTypeByte)
    .with(MessageTypeBytes.REQUEST, () => MessageType.REQUEST)
    .with(MessageTypeBytes.RESPONSE, () => MessageType.RESPONSE)
    .otherwise(() => '');

  // Читаем длину метода
  const methodLength = uncompressedBuffer[index++];
  // Читаем вызываемый метод
  const callMethod = uncompressedBuffer.slice(index, index + methodLength).toString();
  index += methodLength;

  // Читаем количество параметров
  const parametersCount = uncompressedBuffer[index++];
  const parameters: unknown[] = [];

  // Читаем параметры
  for (let i = 0; i < parametersCount; i++) {
    // Читаем тип параметра
    const parameterType = uncompressedBuffer[index++];
    if (parameterType === DataTypeBytes.NUMBER) {
      const numberValue = uncompressedBuffer.readInt32LE(index);
      parameters.push(numberValue);
      index += 4;
    } else if (parameterType === DataTypeBytes.STRING) {
      const stringLength = uncompressedBuffer[index++];
      const stringValue = uncompressedBuffer.slice(index, index + stringLength).toString();
      parameters.push(stringValue);
      index += stringLength;
    } else if (parameterType === DataTypeBytes.BOOLEAN) {
      const booleanValue = uncompressedBuffer[index++] === 0x01;
      parameters.push(booleanValue);
    } else if (parameterType === DataTypeBytes.OBJECT) {
      const { parsedObject, newIndex } = deserializeObject(uncompressedBuffer.slice(index));
      parameters.push(parsedObject);
      index += newIndex;
    } else {
      throw new Error(`Unsupported parameter type: ${parameterType}`);
    }
  }

  // Проверяем наличие завершающего байта
  const endByte = uncompressedBuffer[index];
  if (endByte !== 0xFF) {
    throw new Error('Invalid end byte');
  }

  await validateParams(
    parameters,
    schema,
    callMethod,
  );

  return {
    messageType: msgType as MessageType,
    callMethod,
    parameters,
  };
}

const deserializeObject = (buffer: Buffer): { parsedObject: object; newIndex: number } => {
  let index = 0;
  const parsedObject: { [key: string]: any } = {};

  while (index < buffer.length) {
    // Читаем длину ключа
    const keyLength = buffer[index++];
    // Читаем ключ
    const key = buffer.slice(index, index + keyLength).toString();
    index += keyLength;

    // Читаем тип значения
    const valueType = buffer[index++];
    if (valueType === DataTypeBytes.NUMBER) {
      const numberValue = buffer.readInt32LE(index);
      parsedObject[key] = numberValue;
      index += 4;
    } else if (valueType === DataTypeBytes.STRING) {
      const stringLength = buffer[index++];
      const stringValue = buffer.slice(index, index + stringLength).toString();
      parsedObject[key] = stringValue;
      index += stringLength;
    } else if (valueType === DataTypeBytes.BOOLEAN) {
      const booleanValue = buffer[index++] === 0x01;
      parsedObject[key] = booleanValue;
    } else if (valueType === DataTypeBytes.OBJECT) {
      const { parsedObject: nestedObject, newIndex } = deserializeObject(buffer.slice(index));
      parsedObject[key] = nestedObject;
      index += newIndex;
    } else {
      throw new Error(`Unsupported value type: ${valueType}`);
    }
  }

  return { parsedObject, newIndex: index };
}

Serialization works well, produces this format

01145065 74536572 76696365 2E637265 61746550 65740204 02696400 0C000000 046E616D 6502046A 6F686E05 6F776E65 72040269 64000100 0000046E 616D6502 05626572 6961000E 000000

But the deserialization algorithm throws an error

Error: Unsupported value type: 0

I’m fairly new to the world of binary formats, so please help me with this. I just can’t achieve a working result. It would be great if you show me exactly where and how to fix it =(