I have a FIFO queue set up in AWS SQS. When a user registers for my Next.js app, a message is sent to the queue including their email address (this works perfectly). I also have an API route that will eventually be triggered by a cron job, that consumes that queue, and for each message, sends a request to the mailchimp API to send an email template to that user. At the moment, I am hitting that API route manually using the browser.

When I do this on localhost, it works perfectly. Every message is picked up, an email template is sent via mandrill, and the message is deleted from the queue. When I run this on our Vercel-hosted preprod branch however, it only works for about 50% of newly registered users. The other 50% of the time, the call to the SQS client to receive messages returns a message for a user that has already been processed and deleted, and no email is sent. The message for the current user that I would expect to be handled, remains unaffected in the queue.

const clientMailchimp = mailchimpClient(process.env.MAILCHIMP_API_KEY!);

export async function GET(
  req: NextRequest,
): Promise<NextResponse<{ data: any } | { message: string; error?: unknown }>> {

  const sqsClient = new SQSClient({
    region: process.env.MY_AWS_REGION,
    credentials: {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
    },
  });

  try {
    const input = {
      QueueUrl: process.env.QUEUE_URL,
      MessageAttributeNames: [
        'Action',
        'DynamicsId',
        'Auth0Id',
        'Email',
        'StemRole',
        'Name',
      ],
      MaxNumberOfMessages: 10,
      WaitTimeSeconds: 20,
    };
    const command = new ReceiveMessageCommand(input);
    const response = await sqsClient.send(command);
  
    if (!response.Messages || response.Messages?.length === 0) {
      return NextResponse.json({
        message: `No SQS messages found`,
      });
    }

    if (response.Messages) {
      for (const message of response?.Messages) {
        const { MessageAttributes, ReceiptHandle } = message;

        if (MessageAttributes?.Action?.StringValue === 'WELCOME') {
          const {
            Email: { StringValue: email },
            StemRole: { StringValue: stemRole },
            Name: { StringValue: name },
          } = MessageAttributes;

          const decryptedEmail = AES.decrypt(
            email!,
            process.env.ENCRYPTION_KEY!,
          ).toString(enc.Utf8);
          const decryptedFirstName = AES.decrypt(
            name!,
            process.env.ENCRYPTION_KEY!,
          ).toString(enc.Utf8);

          const mailchimpResults = await clientMailchimp.messages.sendTemplate({
            template_name: templates[stemRole!],
            template_content: [],
            message: {
              subject: 'Welcome',
              from_email: '[email protected]',
              to: [{ email: decryptedEmail }],
              merge_vars: [
                {
                  rcpt: decryptedEmail,
                  vars: [
                    {
                      name: 'first_name',
                      content: decryptedFirstName,
                    },
                  ],
                },
              ],
            },
          });

          const emailSent = mailchimpResults?.[0]?.status === 'sent';

          if (emailSent === true && ReceiptHandle) {
            const command = new DeleteMessageCommand({
              QueueUrl: process.env.QUEUE_URL,
              ReceiptHandle: ReceiptHandle,
            });

            await sqsClient.send(command);
          }
        }
      }
    }

    return NextResponse.json(
      {
        message: `SQS messages have been processed`,
      },
      { status: 200 },
    );
  } catch (error) {
    return NextResponse.json(
      { message: `Error processing SQS messages: ${error}` },
      { status: 500 },
    );
  }
}

I have been adjusting, adding and removing both the wait time and the visibility timeout parameters, as I am assuming there is some kind of timing issue happening. I also tried the “empty cache and hard reload” on the browser to check it wasn’t a browser caching issue.