feat/add global SQS mode with single-queue-per-event and payload size control

[nolramaf]

Introduce a global SQS configuration mode with unified queue provisioning and event routing, add large-payload handling via S3 fallback, and enhance queue management abstractions in SqsController.

New Features:

• Introduce a global SQS mode to automatically provision and publish events to queues across all instances under a common prefix
• Add S3 fallback storage for oversized SQS messages when payloads exceed MAX_PAYLOAD_SIZE and S3 storage is enabled

Enhancements:

• Refactor SqsController to dynamically choose between global and per-instance event lists and queue naming based on configuration
• Validate message payload size and switch dataType to ‘s3’ with automatic upload when exceeding SQS limits
• Enable content-based deduplication for globally managed FIFO queues
• Streamline queue listing, creation, and removal abstractions using a unified prefixName parameter

Build:

• Extend environment configuration to include SQS_GLOBAL_ENABLED, SQS_GLOBAL_PREFIX_NAME, SQS_MAX_PAYLOAD_SIZE, and per-event flags
• Expose HttpServer.NAME in config for richer message metadata

Summary by Sourcery

Implement a global SQS configuration mode with unified queue management and add large-payload handling via S3 fallback, while refactoring SqsController to simplify dynamic queue routing and message handling.

New Features:

• Add global SQS mode for unified queue provisioning and event routing across instances
• Introduce S3 fallback storage for oversize messages exceeding MAX_PAYLOAD_SIZE

Enhancements:

• Refactor SqsController to select queues and naming dynamically based on global or per-instance mode
• Validate message payload sizes and switch to S3 storage when exceeding SQS limits
• Enable content-based deduplication on globally managed FIFO queues
• Streamline queue listing, creation, and removal by using a unified prefix parameter

Build:

• Extend configuration with SQS_GLOBAL_ENABLED, SQS_GLOBAL_PREFIX_NAME, MAX_PAYLOAD_SIZE, and per-event flags
• Expose HttpServer.NAME in environment configuration

[sourcery-ai]

Reviewer's Guide

This PR enhances SqsController by adding a global SQS mode with unified queue provisioning, refactoring event routing and queue management to use a common prefix, enforcing payload size validation with S3 fallback for oversized messages, updating FIFO queue deduplication, and extending environment configuration to support new SQS global flags and server metadata.

Sequence diagram for SQS message publishing with S3 fallback for large payloads

sequenceDiagram
    participant SqsController
    participant SQS
    participant S3Service
    actor Logger
    SqsController->SQS: sendMessage(message)
    alt Payload size <= MAX_PAYLOAD_SIZE
        SQS->SqsController: Message accepted
        SqsController->Logger: Log success
    else Payload size > MAX_PAYLOAD_SIZE & S3 enabled
        SqsController->S3Service: uploadFile(message)
        S3Service->SqsController: fileUrl
        SqsController->SQS: sendMessage({dataType: 's3', data: {fileUrl}})
        SQS->SqsController: Message accepted
        SqsController->Logger: Log success
    else Payload size > MAX_PAYLOAD_SIZE & S3 disabled
        SqsController->Logger: Log error (payload too large)
    end

Loading

Class diagram for updated SqsController queue management and event routing

classDiagram
    class SqsController {
      +init(): Promise<void>
      +set(instanceName: string, data: EventDto): Promise<any>
      +sendData(instanceName: string, event: string, data: any, ...): void
      +saveQueues(prefixName: string, events: string[], enable: boolean)
      +listQueues(prefixName: string)
      +removeQueuesByInstance(prefixName: string)
      -sqs: SQS
      -logger
      -monitor
      -prisma
      -configService
    }
    SqsController <|-- EventController
    SqsController --> SQS
    SqsController --> S3Service
    SqsController --> ConfigService
    SqsController --> Logger

Loading

File-Level Changes

Change	Details	Files
Async init and global queue provisioning	Converted init() to async and removed redundant Promise wrapper Loaded global SQS config and invoked saveQueues with prefix and events when GLOBAL_ENABLED is true	src/api/integrations/event/sqs/sqs.controller.ts
Unified event filtering and queue naming for global and per-instance modes	Merged global and per-instance event list selection into a single logic branch Determined prefixName based on GLOBAL_ENABLED flag or instanceName Constructed queueName and queueUrl dynamically using prefixName	src/api/integrations/event/sqs/sqs.controller.ts
Validate message size and fallback to S3 for oversized payloads	Measured JSON payload size against MAX_PAYLOAD_SIZE Uploaded oversized payloads to S3 when S3.ENABLE is true Updated message.dataType to 's3' and replaced data with fileUrl on fallback	src/api/integrations/event/sqs/sqs.controller.ts
Enhanced FIFO queue deduplication logic	Enabled ContentBasedDeduplication for globally managed FIFO queues Conditionally added MessageDeduplicationId only when global mode is disabled	src/api/integrations/event/sqs/sqs.controller.ts
Refactor queue management methods to use common prefixName	Renamed listQueuesByInstance to listQueues and accept prefixName Updated saveQueues and removeQueuesByInstance to generalize on prefixName Replaced instanceName-based prefixes and adjusted log messages accordingly	src/api/integrations/event/sqs/sqs.controller.ts
Extend environment configuration with global SQS flags and server metadata	Added SQS_GLOBAL_ENABLED, SQS_GLOBAL_PREFIX_NAME, SQS_MAX_PAYLOAD_SIZE, and EVENTS map to env.config Exposed HttpServer.NAME for richer message metadata Updated ConfigService to parse and expose all new variables	src/config/env.config.ts

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey there - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[DavidsonGomes]

Please fix the lint with npm run lint