Redirecting Output Streams For Better Crontab Logging

Understanding Crontab Output Redirection

The crontab utility allows users to schedule periodic jobs on Unix-based operating systems. By default, crontab jobs send their standard output and standard error streams to the syslog service. However, relying solely on syslog has limitations for logging crontab activity.

Syslog collects messages from many system services, making it difficult to isolate crontab logs. Additionally, syslog rotations may overwrite relevant crontab history. To maintain separate, complete logs for cron jobs, administrators need to redirect crontab output streams to dedicated log files.

Overview of Crontab and Standard Output Behaviors

The cron daemon activates crontab jobs at scheduled intervals. When active, jobs automatically receive three standard streams:

Table of Contents

Standard input (stdin): Provides data and commands to the job process
Standard output (stdout): Outputs job results to syslog by default
Standard error (stderr): Outputs job error messages to syslog by default

Administrators can redirect both stdout and stderr to log files for enhanced crontab logging and monitoring.

Issues with Default Crontab Logging to Syslog

Since syslog captures all system logs, crontab job logs can easily get lost in syslog data. Finding relevant cron logs requires digging through many unrelated syslog messages. Syslog also rotates logs based on size rather than time intervals. This can cause cron logs to overwrite crucial historical data.

By redirecting output streams to dedicated log files, admins can avoid these issues and retain complete crontab logs.

Capturing Crontab Logs

Crontab logs provide increased visibility into scheduled jobs. Capturing this data involves redirecting stdout and stderr streams to log files when defining cron jobs. This section covers key log file considerations.

Redirecting stdout and stderr to a File

To redirect output streams to a file, use the following syntax when defining a cron job:

* * * * * command > /path/to/logfile 2>&1

This redirects stdout to the specified logfile. The “2>&1” syntax links the stderr stream to the file handle for stdout, sending both streams to the same log file. Now all command output and errors will write to /path/to/logfile rather than syslog.

Appending Versus Overwriting the Log File

By default, output streams overwrite log files when cron jobs run. To append instead, use “>>”:

* * * * * command >> /path/to/logfile 2>&1

Now, each cron job will add output to the end of the file. This preserves historical log data.

Log File Permissions to Consider

Restrict log file permissions to protect sensitive data that may appear in command output. Use chmod to set permissions for the owner, group, and all system users appropriately. Additionally, set the cron job user as the log file owner to enable output stream writing.

When writing logs to a centralized system, ensure the logging application has write access. Log collection services often operate under separate users and groups.

Rotating Crontab Logs

Although appending output prevents overwriting, log files still grow in size over time. Logrotate allows admins to automatically rotate crontab logs on a periodic basis.

Preventing Overly Large Log Files

Without logrotation, appended logs will expand infinitely. This wastes disk space over time. Logrotate allows admins to set size thresholds that trigger the creation of new log files. For example, setting a 100MB threshold renames the original log file and replaces it with an empty file after it reaches 100MB. The rotated log gets compressed or archived as needed.

Setting Up Logrotate for Crontab Logs

Installing logrotate creates a master configuration file at /etc/logrotate.conf. This enables global logrotate settings. Additionally, individual configuration files stored in /etc/logrotate.d define per-application settings. For crontab logs, create a crontab config here with details like:

Log file path
Log file permissions
Compress options
Rotation frequency
Number of old log files to preserve

With this configuration, logrotate will now handle crontab log rotation automatically based on the parameters.

Example Crontab Logrotate Configuration

Here is an example logrotate crontab configuration file.

/var/log/crontab.log {

  daily
  size 100M
  compress
  delaycompress
  copytruncate
  create 0600 root root
  
  rotate 7
  
}

This rotates the /var/log/crontab.log file daily if it grows beyond 100MB in size. Old rotated logs get compressed to save space and retain 7 days worth of previous logs.

Monitoring Crontab Operations

Properly redirecting and rotating logs enables easy monitoring of cron jobs. This section covers ways to track crontab activity using log files.

Viewing Real-time Crontab Logs with Tail

The tail command prints the tail end of log files to stdout. Using tail -f tracks logs in real-time as new lines get added. This provides visibility into live cron jobs. For example:

tail -f /var/log/crontab.log

Now admins can monitor progress and errors as cron jobs execute.

Getting Notified of Crontab Activity

In addition to live monitoring, admins can configure active notifications for certain cron events like job failures. This allows quick awareness and troubleshooting of issues. Notification scripts can trigger on keywords or return codes from log file data.

Integrating with Centralized Logging Systems

For larger environments, centralized log aggregation solutions like Splunk, Logstash, and Graylog process logs from many servers. While logrotate handles local storage, forwarding crontab logs to these systems enables robust searching, analysis, archival, and monitoring across an infrastructure.

Most aggregation systems provide agent daemons capable of tailing live log files. Installing an agent on each server with crontab logs allows continuous streaming of this data to the central system.