Solutions For Infinite Bash History

Storing Unlimited Bash History

The default Bash shell settings limit the amount of command history that is saved between sessions. Specifically, the HISTSIZE and HISTFILESIZE variables control the number of commands stored in memory during a session and written to the ~/.bash_history file when a session ends. The default values are 500 and 1000, respectively, meaning only the most recent 500 commands are available in history searches and only the last 1000 commands used are saved to disk.

Increasing these limits has several productivity and troubleshooting benefits for power users and developers. Having your entire command history available allows finding previously used commands with more context. Rather than just seeing the last 500 commands, you can search through weeks or months of past work to better understand what you were doing at the time. This can help jog your memory when retracing previous tasks.

Additionally, unlimited history serves as documentation of everything done on a system for auditing or forensic purposes. And saving more history across sessions reduces frustration from losing valuable context and reference points between logins.

Table of Contents

Enabling Unlimited Bash Sessions

Saving unlimited history can be easily achieved by editing Bash’s configuration files. Here are some common settings to apply:

# /etc/bash.bashrc or ~/.bashrc

# Unlimit Bash history size
HISTSIZE=-1  

# Append all history, rather than overwriting
shopt -s histappend

# Write out unlimited history file size
HISTFILESIZE=-1

The HISTSIZE=-1 config removes any limit on the number of commands stored in memory during an active session. The histappend option tells Bash to append new history to the file, rather than overwriting each time. And HISTFILESIZE=-1 eliminates any cap on the ~/.bash_history size.

With these simple tweaks, Bash will remember all commands line by line, restricted only by available disk space. Although saving everything isn’t always necessary, essentially infinite history can be invaluable for advanced users.

Rolling Over Old Entries

A downside of unlimited Bash history is the ~/.bash_history file grows forever, eventually consuming a lot of disk space. One solution is to periodically cull old entries.

This one-liner checks the file size and keeps only the most recent 50,000 lines if over 5 MB:

cat ~/.bash_history | tail -50000 > ~/.bash_history.tmp && mv ~/.bash_history.tmp ~/.bash_history

Of course the “50000” and “5” values can be adjusted as needed. A more advanced approach runs this automatically when closing Bash sessions:

# Keep only 500KB of history
HISTSIZE=500000
# Truncate history file on logout  
trap 'echo $(date) > ~/.bash_history_archive; >| ~/.bash_history' EXIT

This will archive logs to an incremental dated file before truncating history on exit. The desired length can also be archived across sessions, simply reducing storage creep over time.

Persisting History Across Sessions

A secondary downside to Bash’s default history implementation is that sessions are siloed. The known history Within a login session lasts until you exit the shell or terminal. Then it disappears unless manually saved via the ~/.bash_history file.

To persist commands across all interactive shells and subshells, we can append to a shared file continuoously. For example:

# Save history across sessions
HISTFILE=~/.bash_eternal_history  

# Append history log in real-time
PROMPT_COMMAND="history -a; history -n;$PROMPT_COMMAND"

Now Bash will unconditionally append each command to ~/.bash_eternal_history when entered. And history -n avoids duplicates with the current session’s memory. The $PROMPT_COMMAND variable ensures other prompt customizations still apply.

With this approach, you essentially have a permanent running log of all activity. The history is always available via searching or grepping the single historical record.

Searching Through Extensive History

While unlimited history is useful, finding commands in gigabytes or terabytes of logs can be challenging. Fortunately, Bash provides somecapable tools for searching and filtering huge datasets quickly.

The built-in Ctrl+R reverse incremental search is invaluable with long histories. Just start typing a command fragment and matching results popup instantly. Ctrl+S forward searches from the current position too.

For more advanced queries, piping history through grep finds matches anywhere in the archive:

history | grep 'apt-get install python'

And timestamps can help narrow down periods of activity:

  
history | grep '2022-10'

Consider setting handy short aliases like hgrep too:

alias hgrep='history | grep'

With these tools, having unlimited history is not a hinderance. The more historical command data available, the more potential value when troubleshooting or reusing past work.