ℹ️ Select 'Choose Exercise', or randomize 'Next Random Exercise' in selected language.

Choose Exercise:
Timer 00:00
WPM --
Score --
Acc --
Correct chars --

Redis Lua Job Queue Processor

Redis Lua

Goal -- WPM

Ready
Exercise Algorithm Area
1local function processJob(queue_name, failed_queue_name, max_retries)
2-- Processes a single job from a Redis list-based queue.
3-- Args:
4-- queue_name: The Redis key for the primary job queue.
5-- failed_queue_name: The Redis key for the failed jobs queue.
6-- max_retries: The maximum number of times a job can be retried.
7-- Returns:
8-- The job data if processed successfully, or nil if no job was available or processing failed permanently.
9
10if not queue_name or not failed_queue_name or not max_retries then
11return redis.error_reply('Invalid arguments: queue_name, failed_queue_name, and max_retries are required.')
12end
13
14-- Atomically pop a job from the left of the queue.
15-- BLPOP is blocking, but in a script we usually use LPOP and handle empty queue.
16local job_data_array = redis.call('LPOP', queue_name)
17
18if job_data_array == nil then
19-- No jobs available in the queue.
20return nil
21end
22
23-- Job data is expected to be a string, potentially JSON.
24local job_data = job_data_array
25
26-- Simulate job processing.
27-- In a real scenario, this would involve complex logic.
28-- For this script, we'll assume a simple success/failure based on a marker.
29-- Example: Job data might be a JSON string like '{"id": 123, "payload": {...}, "retries": 0}'
30
31local job_info = cjson.decode(job_data) -- Assuming job_data is JSON
32if not job_info then
33-- Malformed job data, move to failed queue.
34redis.call('RPUSH', failed_queue_name, job_data)
35return nil
36end
37
38local current_retries = job_info.retries or 0
39
40if current_retries >= max_retries then
41-- Exceeded max retries, move to failed queue.
42redis.call('RPUSH', failed_queue_name, job_data)
43return nil
44end
45
46-- Simulate processing logic here.
47-- Let's assume a job fails if its ID is odd (for demonstration).
48local job_id = job_info.id
49local processing_successful = true
50if job_id and job_id % 2 ~= 0 then
51processing_successful = false
52end
53
54if processing_successful then
55-- Job processed successfully.
56-- In a real system, you might delete the job or mark it as done.
57-- For this example, we just return the data.
58return job_data
59else
60-- Job processing failed. Increment retries and re-queue.
61job_info.retries = current_retries + 1
62local updated_job_data = cjson.encode(job_info)
63redis.call('RPUSH', queue_name, updated_job_data)
64return nil -- Indicate failure to process this attempt
65end
66end
67
68-- Note: This script assumes the presence of a JSON library (like cjson) available in the Redis environment.
69-- Example usage:
70-- local queue = 'my_jobs'
71-- local failed_queue = 'my_failed_jobs'
72-- local retries = 3
73-- local processed_job = processJob(queue, failed_queue, retries)
74-- if processed_job then
75-- print('Successfully processed job: ' .. processed_job)
76-- else
77-- print('Job processing attempt finished (may be requeued or failed).')
78-- end
Algorithm description viewbox

Redis Lua Job Queue Processor

Algorithm description:

This Redis Lua script manages a job queue, allowing workers to atomically fetch, process, and handle jobs. It uses Redis Lists to store pending jobs and a separate list for failed jobs. Jobs are processed one by one, with logic to retry failed jobs up to a maximum number of times before moving them to the failed queue. This pattern is essential for building reliable background processing systems.

Algorithm explanation:

The `processJob` function takes the names of the main queue, a failed queue, and the maximum retries. It uses `redis.call('LPOP', queue_name)` to atomically remove and retrieve a job from the left of the queue. If the queue is empty, it returns `nil`. It then decodes the job data (assuming JSON) and checks for malformed data or if the job has exceeded `max_retries`. In these cases, the job is moved to the `failed_queue_name` using `RPUSH`. If the job is valid and within retry limits, it simulates processing. If processing fails, the job's retry count is incremented, and it's re-enqueued using `RPUSH` to the main queue. If successful, the job data is returned. The time complexity is O(1) for `LPOP`, `RPUSH`, `ZCARD` (if used for retries), and `DEL` (if used for completion). JSON encoding/decoding can add overhead, but is typically considered constant for typical job sizes. Space complexity is O(J) where J is the size of a single job.

Pseudocode:

FUNCTION processJob(queue_name, failed_queue_name, max_retries):
  job_data_array = Redis.LPOP(queue_name)

  IF job_data_array IS null THEN
    RETURN null
  END IF

  job_data = job_data_array
  job_info = JSON.DECODE(job_data)

  IF job_info IS null THEN
    Redis.RPUSH(failed_queue_name, job_data)
    RETURN null
  END IF

  current_retries = job_info.retries OR 0

  IF current_retries >= max_retries THEN
    Redis.RPUSH(failed_queue_name, job_data)
    RETURN null
  END IF

  -- Simulate processing
  processing_successful = TRUE
  IF job_id IS odd THEN
    processing_successful = FALSE
  END IF

  IF processing_successful THEN
    RETURN job_data
  ELSE
    job_info.retries = current_retries + 1
    updated_job_data = JSON.ENCODE(job_info)
    Redis.RPUSH(queue_name, updated_job_data)
    RETURN null
  END IF
END FUNCTION