Fix MaxRetriesExceededError: celery.exceptions.MaxRetriesExceededError: Can't retry app.tasks.send_email[abc123] max retries exceeded in Celery

This error occurs when a Celery task has failed and exhausted all configured retry attempts. The underlying error keeps recurring across retries, typically because of a persistent external service failure. Fix it by implementing exponential backoff, catching the MaxRetriesExceeded exception to handle the final failure gracefully, and adding a dead-letter mechanism for failed tasks.

Reading the Stack Trace

Here's what each line means:

• File "/app/app/tasks.py", line 18, in send_email: The task's retry call triggers after catching an exception from the email service.
• File "/app/venv/lib/python3.12/site-packages/celery/app/task.py", line 763, in retry: Celery's retry mechanism detects that the maximum retry count (default: 3) has been reached.
• celery.exceptions.MaxRetriesExceededError: Can't retry app.tasks.send_email[abc123] max retries exceeded: All retry attempts have failed. The task is marked as permanently failed unless this exception is handled.

Common Causes

@celery.task(bind=True, max_retries=3)
def send_email(self, to, subject, body):
    try:
        smtp.send(to, subject, body)
    except SMTPException as exc:
        self.retry(exc=exc)  # retries 3 times, then crashes

@celery.task(bind=True, max_retries=3)
def send_email(self, to, subject, body):
    try:
        smtp.send(to, subject, body)
    except SMTPException as exc:
        self.retry(exc=exc)  # no countdown, retries instantly

# No try/except around self.retry()
# MaxRetriesExceededError propagates as an unhandled error

The Fix

Add exponential backoff by calculating the countdown as 60 * 2^retries, giving the external service increasing time to recover. Catch MaxRetriesExceededError to log the permanent failure and alert the operations team so the issue can be investigated.

@celery.task(bind=True, max_retries=3)
def send_email(self, to, subject, body):
    try:
        smtp.send(to, subject, body)
    except SMTPException as exc:
        self.retry(exc=exc)

from celery.exceptions import MaxRetriesExceededError

@celery.task(bind=True, max_retries=5, default_retry_delay=60)
def send_email(self, to, subject, body):
    try:
        smtp.send(to, subject, body)
    except SMTPException as exc:
        try:
            self.retry(exc=exc, countdown=60 * (2 ** self.request.retries))
        except MaxRetriesExceededError:
            # Log to dead-letter queue and alert ops
            log_failed_email(to, subject, str(exc))
            notify_ops(f'Email to {to} failed after {self.max_retries} retries: {exc}')
            raise

Testing the Fix

import pytest
from unittest.mock import patch, MagicMock
from celery.exceptions import MaxRetriesExceededError
from app.tasks import send_email
from app import create_app

@pytest.fixture
def app():
    app = create_app()
    app.config['CELERY_ALWAYS_EAGER'] = True
    app.config['CELERY_EAGER_PROPAGATES'] = True
    app.config['TESTING'] = True
    return app

@patch('app.tasks.smtp')
def test_successful_email(mock_smtp, app):
    with app.app_context():
        result = send_email('user@example.com', 'Hello', 'Body')
        mock_smtp.send.assert_called_once()

@patch('app.tasks.log_failed_email')
@patch('app.tasks.smtp')
def test_max_retries_logs_failure(mock_smtp, mock_log, app):
    mock_smtp.send.side_effect = Exception('SMTP down')
    with app.app_context():
        with pytest.raises(MaxRetriesExceededError):
            send_email('user@example.com', 'Hello', 'Body')
        mock_log.assert_called_once()

Run your tests:

pytest tests/ -v

Pushing Through CI/CD

git checkout -b fix/celery-retry-exceeded-handling,git add app/tasks.py tests/test_tasks.py,git commit -m "fix: add exponential backoff and dead-letter handling for retries",git push origin fix/celery-retry-exceeded-handling

Your CI config should look something like this:

name: CI
on:
  pull_request:
    branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    services:
      redis:
        image: redis:7-alpine
        ports:
          - 6379:6379
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v --tb=short
        env:
          CELERY_BROKER_URL: redis://localhost:6379/0

The Full Manual Process: 18 Steps

Here's every step you just went through to fix this one bug:

1. Notice the error alert or see it in your monitoring tool
2. Open the error dashboard and read the stack trace
3. Identify the file and line number from the stack trace
4. Open your IDE and navigate to the file
5. Read the surrounding code to understand context
6. Reproduce the error locally
7. Identify the root cause
8. Write the fix
9. Run the test suite locally
10. Fix any failing tests
11. Write new tests covering the edge case
12. Run the full test suite again
13. Create a new git branch
14. Commit and push your changes
15. Open a pull request
16. Wait for code review
17. Merge and deploy to production
18. Monitor production to confirm the error is resolved

Total time: 30-60 minutes. For one bug.

pip install bugstack

import bugstack

bugstack.init(api_key=os.environ["BUGSTACK_API_KEY"])

Deploying the Fix (Manual Path)

1. Run pytest locally to verify retry behavior and dead-letter logging.
2. Open a pull request with the retry and error handling changes.
3. Wait for CI checks to pass on the PR.
4. Have a teammate review and approve the PR.
5. Merge to main and monitor retry counts and dead-letter logs in staging.

Frequently Asked Questions