Handling Overlapping Execution in Scheduled Python Scripts
What You’ll Need
- n8n Cloud or self-hosted n8n for orchestrating scheduled tasks
- Hetzner VPS or Contabo VPS for hosting Python background workers
- DigitalOcean as an alternative cloud instance provider
- Python 3.10 or higher installed on your server environment
Table of Contents
- The Problem: Race Conditions in Scheduled Python Scripts
- Strategy 1: OS-Level File Locking with fcntl
- Strategy 2: Distributed Locking with Redis
- Strategy 3: Atomic Lock Decorator with Graceful Exit
- Orchestration Strategies and Systemd Controls
- Getting Started
The Problem: Race Conditions in Scheduled Python Scripts
Every developer who manages background automation eventually runs into overlapping executions. You configure a cron job or a scheduler to trigger a script every five minutes. The script usually takes thirty seconds to complete. Everything runs smoothly until external factors intervene: an API endpoint slows down, a database query hits an unindexed lock, or network latency spikes.
When the next scheduled interval arrives, your operating system spawns a second instance of the script. Now two processes are reading from the same message queue, updating identical database records, or sending duplicate webhooks to external API consumers.
If your task involves external operations like handling rate limits when querying Anthropic models, an overlapping instance can instantly exhaust your quota or trigger cascading failure modes across your worker cluster.
Time --->
[Cron Trigger 10:00] ---> [ Script Instance A Starts ]........................[ Complete 10:07 ]
[Cron Trigger 10:05] ---> [ Script Instance B Starts ] -------> [ OVERLAP FAILS / DUPLICATES DATA ]
To eliminate race conditions, you need determinism. The incoming execution must detect that a previous process is still running and either exit immediately or wait gracefully without corrupting state.
Strategy 1: OS-Level File Locking with fcntl
When running scheduled scripts on a single host like a Hetzner VPS, operating system locks provide the fastest and most dependable mechanism to block overlapping runs. Linux and Unix systems provide the fcntl module natively inside Python standard library.
Using fcntl.flock, we can request an exclusive non-blocking lock on a specific file descriptor. If another process holds the lock, fcntl raises an OSError (specifically EWOULDBLOCK or EAGAIN). The second script catches this error and terminates immediately.
Here is the complete implementation of a process lock manager using file locking.
import fcntl
import sys
import time
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] PID %(process)d: %(message)s",
handlers=[logging.StreamHandler(sys.stdout)]
)
class SingleInstanceFileLock:
def __init__(self, lock_file_path: str):
self.lock_file_path = lock_file_path
self.fp = None
def __enter__(self):
self.fp = open(self.lock_file_path, "w")
try:
fcntl.flock(self.fp.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
self.fp.write(str(sys.argv[0]))
self.fp.flush()
logging.info(f"Acquired lock on file: {self.lock_file_path}")
return self
except (OSError, IOError):
logging.warning(f"Another instance is already running with lock file: {self.lock_file_path}")
self.fp.close()
sys.exit(0)
def __exit__(self, exc_type, exc_val, exc_tb):
if self.fp and not self.fp.closed:
try:
fcntl.flock(self.fp.fileno(), fcntl.LOCK_UN)
self.fp.close()
logging.info(f"Released lock on file: {self.lock_file_path}")
except (OSError, IOError) as err:
logging.error(f"Failed to release lock file: {err}")
def run_long_task():
logging.info("Starting task execution sequence...")
for step in range(1, 6):
logging.info(f"Processing step {step} of 5...")
time.sleep(2)
logging.info("Task execution sequence finished successfully.")
if __name__ == "__main__":
LOCK_PATH = "/tmp/my_scheduled_python_job.lock"
with SingleInstanceFileLock(LOCK_PATH):
run_long_task()
When you trigger this script twice in parallel, the second instance attempts to secure an exclusive lock on /tmp/my_scheduled_python_job.lock. Because the file descriptor is already locked by PID A, PID B catches the exception, logs a warning, and exits cleanly with return code 0.
💡 Fast-Track Your Project: Don’t want to configure this yourself? I build custom n8n pipelines and bots. Message me with code SYS3-HUGO.
Strategy 2: Distributed Locking with Redis
File locking works perfectly for a single host. However, modern deployment architectures split work across multiple application nodes on hosts like DigitalOcean or Contabo VPS. If your cron triggers run inside multiple Kubernetes pods or autoscaling server instances, local file locks will fail to prevent execution overlaps.
In distributed setups, we use Redis as an atomic lock manager. We acquire locks using SET key value NX EX seconds, which guarantees that the key is only created if it does not already exist, while automatically setting an expiration timeout (TTL) to avoid deadlocks if a worker node crashes unexpectedly.
To release the lock safely without deleting a lock acquired by a subsequent run, we execute an atomic Lua script that compares the stored random identifier with our instance identifier.
When handling distributed worker failures, combining this pattern with robust recovery mechanisms like implementing circuit breakers for outbound webhook delivery ensures downstream endpoints remain healthy when retries occur.
Here is the complete Python script for distributed locking using Redis:
import time
import uuid
import sys
import logging
import redis
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] PID %(process)d: %(message)s"
)
LUA_RELEASE_SCRIPT = """
if redis.call("get", KEYS[1]) == ARGV[1] then
return redis.call("del", KEYS[1])
else
return 0
end
"""
class RedisDistributedLock:
def __init__(self, redis_client: redis.Redis, lock_name: str, expire_seconds: int = 60):
self.redis_client = redis_client
self.lock_key = f"lock:{lock_name}"
self.expire_seconds = expire_seconds
self.token = str(uuid.uuid4())
self.release_lua = self.redis_client.register_script(LUA_RELEASE_SCRIPT)
def acquire(self) -> bool:
acquired = self.redis_client.set(
name=self.lock_key,
value=self.token,
nx=True,
ex=self.expire_seconds
)
return bool(acquired)
def release(self) -> bool:
result = self.release_lua(keys=[self.lock_key], args=[self.token])
return bool(result)
def __enter__(self):
if not self.acquire():
logging.warning(f"Could not acquire distributed lock for key: {self.lock_key}. Terminating script.")
sys.exit(0)
logging.info(f"Acquired distributed lock: {self.lock_key} with token: {self.token}")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
released = self.release()
if released:
logging.info(f"Successfully released distributed lock: {self.lock_key}")
else:
logging.error(f"Failed to release lock or lock expired before task completion: {self.lock_key}")
def process_distributed_batch():
logging.info("Executing distributed payload synchronization...")
for i in range(1, 4):
logging.info(f"Syncing item chunk {i}/3...")
time.sleep(3)
logging.info("Distributed payload synchronization completed.")
if __name__ == "__main__":
redis_conn = redis.Redis(host="localhost", port=6379, db=0, decode_responses=True)
LOCK_NAME = "sync_user_subscriptions"
LOCK_TIMEOUT = 30
with RedisDistributedLock(redis_client=redis_conn, lock_name=LOCK_NAME, expire_seconds=LOCK_TIMEOUT):
process_distributed_batch()
Strategy 3: Atomic Lock Decorator with Graceful Exit
To keep your core application logic clean, you can abstract concurrency management into a reusable Python decorator. This enables you to wrap any standard task function with execution guarantees without cluttering the function body.
This production decorator supports configurable retry attempts, custom lock file destinations, and clean POSIX signal handlers to ensure locks release when processes encounter SIGTERM or SIGINT.
import os
import sys
import time
import fcntl
import signal
import functools
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s]: %(message)s"
)
def prevent_concurrent_execution(lock_file_name: str):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
lock_dir = "/tmp/script_locks"
os.makedirs(lock_dir, exist_ok=True)
lock_path = os.path.join(lock_dir, lock_file_name)
lock_fd = open(lock_path, "w")
try:
fcntl.flock(lock_fd.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except (OSError, IOError):
logging.warning(f"Skipping execution of '{func.__name__}': another instance is actively running.")
lock_fd.close()
return None
def cleanup_signal_handler(signum, frame):
logging.info(f"Received signal {signum}. Cleaning up lock file...")
try:
fcntl.flock(lock_fd.fileno(), fcntl.LOCK_UN)
lock_fd.close()
except Exception as e:
logging.error(f"Error closing lock on signal: {e}")
sys.exit(128 + signum)
signal.signal(signal.SIGINT, cleanup_signal_handler)
signal.signal(signal.SIGTERM, cleanup_signal_handler)
try:
logging.info(f"Lock acquired for '{func.__name__}'. Executing workload.")
result = func(*args, **kwargs)
return result
finally:
try:
fcntl.flock(lock_fd.fileno(), fcntl.LOCK_UN)
lock_fd.close()
logging.info(f"Lock released for '{func.__name__}'.")
except Exception as e:
logging.error(f"Failed releasing lock for '{func.__name__}': {e}")
return wrapper
return decorator
@prevent_concurrent_execution(lock_file_name="hourly_data_export.lock")
def export_hourly_analytics_data():
logging.info("Gathering analytics records from primary database...")
time.sleep(4)
logging.info("Analytics data compiled and sent to storage bucket.")
if __name__ == "__main__":
export_hourly_analytics_data()
Orchestration Strategies and Systemd Controls
While code-level locks prevent concurrent runs, moving schedule logic out of raw crontabs and into dedicated service orchestrators provides built-in concurrency controls, native failure visibility, and detailed metric logs.
If you are running stand-alone scripts on server infrastructure, Linux systemd timers offer built-in prevention against overlapping runs via Systemd service definitions.
By default, if a systemd service unit is already active when its corresponding timer triggers, systemd skips the invocation automatically without raising system errors.
Systemd Configuration Example
Create the service unit file at /etc/systemd/system/py_job.service:
[Unit]
Description=Scheduled Data Processing Job
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/automation
ExecStart=/usr/bin/python3 /opt/automation/worker.py
Restart=no
Create the corresponding timer unit file at /etc/systemd/system/py_job.timer:
[Unit]
Description=Run Data Processing Job Every 5 Minutes
[Timer]
OnCalendar=*:0/5
Persistent=true
[Install]
WantedBy=timers.target
Reload systemd daemon configurations and enable the timer:
systemctl daemon-reload
systemctl enable --now py_job.timer
If your Python script takes 8 minutes to execute, the timer trigger at minute 5 will be safely ignored because py_job.service is still in an active state.
For complex workflows involving multiple dependent steps, offloading scheduling to a dedicated workflow engine is often superior to managing raw scripts. You can learn how to set up an enterprise orchestrator in our guide on how to self host Kestra on cloud VPS.
Similarly, automation platforms like n8n Cloud allow you to configure execution policies directly in the interface, choosing between queueing subsequent runs or cancelling concurrent executions entirely.
Getting Started
To implement robust execution locking in your Python infrastructure, set up your development or production nodes with these hosting providers:
- Spin up dedicated cloud compute instances using a high-performance Hetzner VPS or Contabo VPS.
- Deploy scalable cloud infrastructure on DigitalOcean for distributed worker setups.
- Orchestrate script execution, manage API retries, and prevent overlapping schedules using n8n Cloud.
Outsource Your Automation
Don’t have time? I build production n8n workflows, WhatsApp bots, and fully automated YouTube Shorts pipelines. Hire me on Fiverr, mention SYS3-HUGO for priority. Or DM at chasebot.online.
Want to automate this yourself?
Start with n8n Cloud (free tier available) or self-host on a Hetzner VPS for full control.
Want this engine running on your own VPS?
This blog publishes itself — daily, unattended, on free API tiers. The full engine, Hugo theme, and setup guide are available as System 3.
Get System 3