Setting Up Logrotate for Custom Linux Server Logs
What You’ll Need
- A Linux server hosted on Hetzner VPS or Contabo VPS running Ubuntu, Debian, Rocky Linux, or Ubuntu Server
- A DigitalOcean droplet as an alternative hosting environment
- System administrative access with standard sudo privileges
- A domain name managed via Namecheap if you are routing web traffic to your server application
Table of Contents
- Understanding Logrotate Architecture on Modern Linux Systems
- Step 1: Preparing Custom Application Directories and Log Output
- Step 2: Writing a Production Logrotate Configuration File
- Step 3: Managing File Descriptors and Signal Handling
- Step 4: Debugging, Testing, and Systemd Automation
- Getting Started
Understanding Logrotate Architecture on Modern Linux Systems
Running microservices, custom APIs, or background workers on a cloud server like a Hetzner VPS requires proper log maintenance. Custom applications that emit unstructured standard output or structured JSON files can consume available disk space within days if left unchecked. Unbounded log files lead to filesystem exhaustion, kernel panics, and unexpected application failures.
Logrotate is the standard system utility designed to parse, compress, rename, and remove aging system log files. Rather than running continuously as a heavy background daemon, logrotate is typically invoked on a schedule by standard system automation utilities like cron or systemd timers.
The master configuration file lives at /etc/logrotate.conf. This file sets global defaults such as default rotation frequency, file creation permissions, and compression rules. At the bottom of /etc/logrotate.conf, an inclusion directive reads external configuration files:
include /etc/logrotate.d
This modular architecture allows system administrators to isolate application settings. Instead of modifying the global configuration file, you create modular definition files inside the /etc/logrotate.d/ directory. When logrotate executes, it aggregates every file inside this folder, evaluates their matching paths against filesystem state, and executes rotation actions according to your specific rules.
Understanding how logrotate interacts with open file descriptors is essential. Linux systems track open files using inodes rather than path strings. If logrotate simply renames /var/log/custom-app/api.log to /var/log/custom-app/api.log.1, your application process will keep holding the file descriptor for inode pointing to api.log.1. The application continues writing data to the renamed file, frustrating your cleanup efforts. Logrotate offers two strategies to solve this: process signaling via postrotate hooks, or copy-truncation via the copytruncate directive.
Step 1: Preparing Custom Application Directories and Log Output
To demonstrate log management, let us set up a dedicated workspace for a multi-threaded web application. We will create a non-root system user, build dedicated logging directories, set appropriate access control permissions, and construct a Python application script that streams high-volume logs.
Executing custom worker scripts is common when designing idempotent webhook receivers for distributed systems. High throughput endpoints generate massive event trails that require structured log rotation rules.
First, create the dedicated system user and log directories on your server:
sudo useradd --system --shell /bin/false sys3app
sudo mkdir -p /var/log/sys3app
sudo chown -R sys3app:sys3app /var/log/sys3app
sudo chmod 0750 /var/log/sys3app
Now create a Python script located at /usr/local/bin/sys3_worker.py that continuously generates simulated API log entries into two separate files: access.log and error.log.
#!/usr/bin/env python3
import time
import os
import sys
from datetime import datetime
LOG_DIR = "/var/log/sys3app"
ACCESS_LOG = os.path.join(LOG_DIR, "access.log")
ERROR_LOG = os.path.join(LOG_DIR, "error.log")
def write_log(filepath, message):
timestamp = datetime.utcnow().isoformat()
with open(filepath, "a") as f:
f.write(f"[{timestamp}] {message}\n")
def main():
if not os.path.exists(LOG_DIR):
sys.exit(1)
counter = 0
while True:
counter += 1
write_log(ACCESS_LOG, f"INFO processing request status=200 path=/api/v1/resource id={counter}")
if counter % 5 == 0:
write_log(ERROR_LOG, f"ERROR database connection timeout query_id={counter}")
time.sleep(0.1)
if __name__ == "__main__":
main()
Make the script executable and set up a systemd service file at /etc/systemd/system/sys3-worker.service so the application runs continuously in the background:
[Unit]
Description=Sys3 Custom Application Worker
After=network.target
[Service]
Type=simple
User=sys3app
Group=sys3app
ExecStart=/usr/usr/bin/python3 /usr/local/bin/sys3_worker.py
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
Enable and start your newly created service using systemctl:
sudo systemctl daemon-reload
sudo systemctl enable --now sys3-worker.service
Verify that log streams are accumulating inside your log directory:
ls -la /var/log/sys3app/
tail -n 5 /var/log/sys3app/access.log
tail -n 5 /var/log/sys3app/error.log
💡 Fast-Track Your Project: Don’t want to configure this yourself? I build custom n8n pipelines and bots. Message me with code SYS3-HUGO.
Step 2: Writing a Production Logrotate Configuration File
Now that our application streams data into /var/log/sys3app/access.log and /var/log/sys3app/error.log, we must define explicit rotation policies. Create a new logrotate configuration file located at /etc/logrotate.d/sys3-app.
/var/log/sys3app/*.log {
daily
rotate 14
missingok
notifempty
compress
delaycompress
dateext
dateformat -%Y%m%d-%s
create 0640 sys3app sys3app
sharedscripts
copytruncate
}
Let us analyze each directive used in this production configuration:
/var/log/sys3app/*.log: Target path pattern. Applying wildcards ensures all files ending in.logwithin this directory are evaluated.daily: Triggers log rotation once per calendar day. Alternative interval directives includehourly,weekly, ormonthly.rotate 14: Keeps 14 rotated log files before purging the oldest archives. Combined withdaily, this maintains a two-week historical rolling audit window.missingok: Prevents system errors. If the log directory is missing or empty during execution, logrotate skips processing without raising fatal error codes.notifempty: Skips file processing if the log file is zero bytes. This avoids generating empty archive files.compress: Compresses rotated log files using standardgzipcompression to reduce disk space overhead.delaycompress: Postpones compression of the most recent rotated file until the next rotation cycle. The active log becomesaccess.log.1as plain text, whileaccess.log.2.gzis compressed. This accommodates applications that continue writing to an old file handle immediately after rotation.dateext: Appends a date string extension to rotated log files instead of using sequential numbers like.1or.2.dateformat -%Y%m%d-%s: Defines the timestamp pattern appended bydateext. Using standard Year-Month-Day followed by Unix epoch timestamp ensures unique filenames during manual or sub-daily force rotations.create 0640 sys3app sys3app: Instructs logrotate to immediately create a fresh, empty log file after rotation, applying Unix permissions0640owned by usersys3appand groupsys3app.sharedscripts: Ensures that pre-rotation or post-rotation hook scripts execute only once per cycle, rather than once for every file matching the target glob pattern.copytruncate: Copies the active log file content to a backup location, then truncates the original file in-place to zero bytes. This strategy allows long-running daemon processes to continue writing to their open file handle without requiring a process restart or signal handling logic.
Step 3: Managing File Descriptors and Signal Handling
While copytruncate works for simple applications, high-performance production services like web servers, proxy layers, and system daemons require direct signal handling to rotate files cleanly without IO race conditions.
During heavy log output, copytruncate can lose log lines between the copying phase and the truncation phase. To solve this, servers emit a POSIX signal (typically SIGHUP or SIGUSR1) instructing the running application to close its current file descriptor and re-open the file path on disk.
When implementing SSL setups covered in our Nginx SSL certificate renewal automation guide, process signaling ensures web servers drop old log handles cleanly without interrupting active HTTPS traffic.
Below is a configuration file demonstrating process signaling via postrotate hooks instead of copytruncate. Create this configuration file at /etc/logrotate.d/sys3-signaled:
/var/log/sys3app/access.log
/var/log/sys3app/error.log {
weekly
size 100M
rotate 8
missingok
notifempty
compress
delaycompress
su sys3app sys3app
create 0640 sys3app sys3app
sharedscripts
postrotate
/usr/bin/systemctl reload sys3-worker.service > /dev/null 2>&1 || true
endscript
}
Key features introduced in this configuration block:
- Explicit block target lists: Path definitions listed across separate lines scope rules specifically to listed paths.
size 100M: Overrides standard schedule rules. Rotation triggers whenever a log file reaches 100 Megabytes in size, regardless of whether a week has passed.su sys3app sys3app: Explicitly instructs logrotate to switch user and group privileges tosys3appbefore touching log files. This avoids permission mismatches when accessing paths secured by tight Access Control Lists (ACLs).postrotate ... endscript: Execution block containing custom shell commands executed immediately after log files are moved. The line/usr/bin/systemctl reload sys3-worker.servicesends an instruction to your system daemon to flush buffers and re-open file descriptors.
Background scheduling tools, such as scripts created when executing scheduled Python tasks using APScheduler, benefit from signal hooks by releasing file handles cleanly during automated jobs.
Step 4: Testing, Debugging, and Systemd Automation
Before leaving logrotate configs in production, you must validate syntax and test execution mechanics using built-in debug commands.
Validating Configuration Syntax
Run logrotate in debug mode using the -d (or --debug) flag. Debug mode prints execution steps without modifying files, renaming logs, or executing script blocks:
sudo logrotate -d /etc/logrotate.d/sys3-app
Output should show matching target files and planned rotation steps:
reading config file /etc/logrotate.d/sys3-app
Reading state from /var/lib/logrotate/status
Allocating hash table for state file, 1000 entries
Handling 1 logs
rotating pattern: /var/log/sys3app/*.log after 1 days (14 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/sys3app/access.log
Now: 2026-03-30 12:00
Last rotated at 2026-03-30 00:00
log does not need rotating (log has already been rotated)
considering log /var/log/sys3app/error.log
Now: 2026-03-30 12:00
Last rotated at 2026-03-30 00:00
log does not need rotating (log has already been rotated)
Executing a Force Rotation
To test actual file system modifications, compression creation, and postrotate script execution, trigger a forced rotation using the -f (or --force) flag alongside the verbose -v flag:
sudo logrotate -fv /etc/logrotate.d/sys3-app
Verify that rotated log files exist in your target log directory:
ls -la /var/log/sys3app/
Expected filesystem output:
total 24
drwxr-xr-0 2 sys3app sys3app 4096 Mar 30 12:05 .
drwxr-xr-x 8 root root 4096 Mar 30 12:00 ..
-rw-r----- 1 sys3app sys3app 1240 Mar 30 12:05 access.log
-rw-r----- 1 sys3app sys3app 9812 Mar 30 12:05 access.log-20260330-1743336300
-rw-r----- 1 sys3app sys3app 320 Mar 30 12:05 error.log
-rw-r----- 1 sys3app sys3app 1940 Mar 30 12:05 error.log-20260330-1743336300
Checking Systemd Timers and Status Persistence
Modern Linux distributions manage standard logrotate scheduling using systemd timers rather than legacy crontab files. Check the status of your logrotate timer using systemctl:
sudo systemctl status logrotate.timer
To view next scheduled execution windows across system tasks, query systemd timer listings:
sudo systemctl list-timers --all | grep logrotate
Logrotate tracks rotation history inside a state tracking file located at /var/lib/logrotate/status (or /var/lib/logrotate/logrotate.status on older Debian releases). Inspect state records for your custom application using grep:
grep "sys3app" /var/lib/logrotate/status
Output:
"/var/log/sys3app/access.log" 2026-3-30-12:05:0
"/var/log/sys3app/error.log" 2026-3-30-12:05:0
Getting Started
Deploying custom applications on standard server infrastructure requires proactive log management. By setting up logrotate on your cloud instances hosted with Hetzner VPS or DigitalOcean, you prevent unexpected disk exhaustion, maintain reliable system logs, and ensure consistent application performance. If you need domain routing for your public-facing apps, configure DNS management via Namecheap.
Outsource Your Automation
Don’t have time? I build production n8n workflows, WhatsApp bots, and fully automated YouTube Shorts pipelines. Hire me on Fiverr, mention SYS3-HUGO for priority. Or DM at chasebot.online.
Want to automate this yourself?
Start with n8n Cloud (free tier available) or self-host on a Hetzner VPS for full control.
Want this engine running on your own VPS?
This blog publishes itself — daily, unattended, on free API tiers. The full engine, Hugo theme, and setup guide are available as System 3.
Get System 3