Fix Sidekiq::Shutdown: Sidekiq worker killed due to excessive memory usage (RSS: 1.2GB) in Sidekiq

This error means your Sidekiq process consumed too much memory and was killed. Memory leaks in Sidekiq jobs are often caused by accumulating objects in class variables, not releasing large data structures, or loading too many records without batching. Use tools like derailed_benchmarks to profile memory and ensure jobs release resources after processing.

Reading the Stack Trace

Here's what each line means:

• sidekiq (7.2.1) lib/sidekiq/manager.rb:95:in `check_memory': Sidekiq's memory watchdog detected the process exceeded the configured memory limit.
• app/jobs/report_generation_job.rb:35:in `perform': The report generation job accumulates data in memory without releasing it.
• sidekiq (7.2.1) lib/sidekiq/processor.rb:88:in `kill': The processor kills the current job and signals the worker for shutdown.

Common Causes

class ReportGenerationJob
  include Sidekiq::Job

  def perform
    @results = []
    Order.find_each do |order|
      @results << build_report_row(order)  # Grows unbounded
    end
    save_report(@results)
  end
end

class DataProcessingJob
  include Sidekiq::Job
  @@cache = {}

  def perform(key)
    @@cache[key] = expensive_computation(key)  # Grows forever
  end
end

class ImportJob
  include Sidekiq::Job

  def perform(file_url)
    content = URI.open(file_url).read  # Reads entire file into memory
    process(content)
  end
end

The Fix

Stream report data to a temporary file instead of accumulating it in memory. Use find_each with a batch size to limit how many records are loaded at once. The temp file is automatically cleaned up after the block.

class ReportGenerationJob
  include Sidekiq::Job

  def perform
    @results = []
    Order.find_each do |order|
      @results << build_report_row(order)
    end
    save_report(@results)
  end
end

class ReportGenerationJob
  include Sidekiq::Job

  def perform
    Tempfile.create(['report', '.csv']) do |file|
      csv = CSV.new(file)
      csv << report_headers
      Order.find_each(batch_size: 500) do |order|
        csv << build_report_row(order)
      end
      file.rewind
      upload_report(file)
    end
  end
end

Testing the Fix

require 'rails_helper'
require 'sidekiq/testing'

RSpec.describe ReportGenerationJob do
  before { Sidekiq::Testing.inline! }

  it 'generates report without excessive memory' do
    create_list(:order, 10)
    before_memory = GetProcessMem.new.mb
    ReportGenerationJob.perform_async
    after_memory = GetProcessMem.new.mb
    expect(after_memory - before_memory).to be < 50
  end

  it 'produces a valid CSV report' do
    create_list(:order, 5)
    expect_any_instance_of(ReportGenerationJob).to receive(:upload_report)
    ReportGenerationJob.perform_async
  end
end

Run your tests:

bundle exec rspec spec/jobs/report_generation_job_spec.rb

Pushing Through CI/CD

git checkout -b fix/sidekiq-memory-leak,git add app/jobs/report_generation_job.rb,git commit -m "fix: stream report to file to prevent memory leak in Sidekiq job",git push origin fix/sidekiq-memory-leak

Your CI config should look something like this:

name: CI
on:
  pull_request:
    branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
        ports: ['5432:5432']
      redis:
        image: redis:7
        ports: ['6379:6379']
    steps:
      - uses: actions/checkout@v4
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: '3.3'
          bundler-cache: true
      - run: bin/rails db:setup
      - run: bundle exec rspec

The Full Manual Process: 18 Steps

Here's every step you just went through to fix this one bug:

1. Notice the error alert or see it in your monitoring tool
2. Open the error dashboard and read the stack trace
3. Identify the file and line number from the stack trace
4. Open your IDE and navigate to the file
5. Read the surrounding code to understand context
6. Reproduce the error locally
7. Identify the root cause
8. Write the fix
9. Run the test suite locally
10. Fix any failing tests
11. Write new tests covering the edge case
12. Run the full test suite again
13. Create a new git branch
14. Commit and push your changes
15. Open a pull request
16. Wait for code review
17. Merge and deploy to production
18. Monitor production to confirm the error is resolved

Total time: 30-60 minutes. For one bug.

gem install bugstack

require 'bugstack'

Bugstack.init(api_key: ENV['BUGSTACK_API_KEY'])

Deploying the Fix (Manual Path)

1. Refactor jobs to stream data instead of accumulating in memory.
2. Remove class-level caches or add size limits.
3. Profile memory usage with get_process_mem.
4. Open a pull request.
5. Merge and monitor Sidekiq RSS in production.

Frequently Asked Questions