What features does SQS provide for retrying and reprocessing messages?

SQS Features That Help Retry and Reprocess Messages

Amazon SQS offers several features that help with retrying and reprocessing messages:

1. Visibility Timeout – When a message is received from an SQS queue, it enters a “visibility timeout” period where it is not visible to other consumers. This allows the initial consumer time to process the message. If the message is not deleted within the visibility timeout period, it will become visible again and another consumer can process it.
2. Message Retention Period – Messages remain in the SQS queue for a configurable retention period, up to 14 days. This ensures that messages are available for retry even if the initial processing attempt fails.
3. Long Polling – By using long polling, consumers will only be notified when a message is actually available. This reduces empty receives and ensures messages are available for processing when a consumer is ready.
4. Dead Letter Queues (DLQs) – If messages are not processed successfully within the maximum receive count (for example, after 5 attempts), they can be sent to a DLQ. The DLQ acts as a “poison pill” queue where problematic messages can be inspected and retried manually.
5. Message Attributes – Attributes can be set on messages to track things like the number of receives and processing attempts. Consumers can use these attributes to implement their own retry logic.
6. Handlers for request errors – By implementing retry and backoff logic, consumers can automatically retry failed requests to receive and process messages. This helps ensure messages are eventually processed successfully.
7. Idempotency – Consumers should implement idempotent processing to ensure messages can be processed more than once without any adverse effects. This allows for retry and reprocessing.

In summary, SQS provides many features that help with retrying and reprocessing messages in the event of initial failures. The combination of visibility timeouts, long polling, retries, DLQs, and message attributes give consumers the tools they need to build robust, fault-tolerant systems.

Sources

1. https://docs.aws.amazon.com/lambda/latest/operatorguide/sqs-retries.html
2. https://aws.amazon.com/sqs/features/
3. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/working-with-messages.html