Advanced Techniques For Merging Bash Histories

Preserving Command History Across Sessions

Being able to preserve the bash command history between sessions and logins can be invaluable for auditing, debugging, and productivity. Here we explore some methods to export the current history to a file, import history files automatically during login, and configure seamless command history persistence in bash.

Exporting the current history to a file

The most straightforward approach for saving your current bash history is to use the history command. The syntax is simple:

  
    history > my_history_file.txt

This will overwrite my_history_file.txt with all commands in your current in-memory command history. The file contains one command per line, without timestamps or other metadata. You can also append the history to an existing file without overwriting with this notation:

Table of Contents

  
    history >> existing_history.txt

For basic session-to-session history preservation, simply export your history before logging out or disconnecting and import it when creating your next bash session.

Importing a history file at login

To load a saved bash history file when starting a new login shell or session, use the history -r command, specifying the file to read:

  
    history -r my_previous_history.txt

This will append the contents of my_previous_history.txt to your current in-memory bash history, allowing you to access previous commands.

To import history automatically for interactive shells, add the history -r command to your ~/.bashrc script. For example:

  
    # At end of ~/.bashrc
    if [ -f ~/my_bash_history.txt ]; then
      history -r ~/my_bash_history.txt
    fi

This will load ~/my_bash_history.txt into your history every time you start a login or interactive shell session.

Configuring automatic history persistence

Rather than manually importing and exporting history files, you can configure bash to automatically read and write history on startup and shutdown.

In ~/.bash_profile, add the following to load history on startup:

    
    # Load history if it exists
    [ -f ~/.myhistory ] && history -r ~/.myhistory

And in ~/.bash_logout, add this code to save history on exit:

  
    # Save history before logging out
    history -a 
    history -c
    history -r ~/.myhistory

This will continually update ~/.myhistory with yourbash history across sessions, truncate the current session’s history, and reload history from the file on startup.

Strategies for Large-Scale History Merging

When accumulating bash history across multiple systems and users over long periods of time, specialized techniques are needed to handle duplicates and maintain usability of large, shared history files.

Handling duplicate entries during merges

If the same command appears in multiple imported history files, bash history will contain duplicate entries. This clutters completion results and search output.

To prevent duplicate entries when merging histories, add this to your ~/.bashrc:

  
    # Remove duplicates
    export HISTCONTROL=ignoredups:erasedups

Bash will not add commands that match recent history, keeping only the most recent execution.

Pruning old or unwanted commands

Long-running bash history files can accumulate crufty commands. To filter unwanted history, use tools like history | grep to find and sed to remove matching lines before saving. For example:

  
    # Prune history file of large data transfers
    history | grep -v 'scp .*100M' > cleaned_history.txt

You can also periodically cull old history entries if only recent commands are needed. For example, to keep only the last month of history:

  
    # Keep only past 30 days 
    history | grep "$(date +%Y-%m --date='30 days ago')..." > trimmed_history.txt

Maintaining hierarchy when histories span machines

When aggregating history across multiple systems, it can be useful to track which commands originated from which machine.

This can be accomplished by prepending machine names when saving history:

  
    # Tag history with machine name
    history | sed "s/^/server1 /" > server1_history.txt

The machine name prefix on each command also serves as a namespace, avoiding collisions between identical commands run on different systems.

Best Practices for Multi-User History Sharing

When history files are shared between users, access controls and contamination prevention techniques should be applied. Here we review some best practices.

Setting permissions for shared history files

If a bash history file is world-readable, it may expose sensitive information entered on the command line. Use Unix permissions to control access:

    
    # User can read/write, group can read, no access for others
    chmod 660 /shared/bash_history
    chown user:group /shared/bash_history

This allows sharing within a trusted group while preventing unapproved access.

Avoiding contamination from other users

When many users contribute to a shared history file, commands from other users can clutter your own history unless filtered. To avoid pollution on import:

  
    # Only import our own commands 
    history -r shared_history | grep "^$(whoami) " > my_history.txt

The grep filters for lines starting with our username. This ensures you only import your personal contribution to the shared history.

Enabling selective opt-in for history sharing

For shared multi-user machines, an opt-in configuration can allow users to selectively contribute history. Add this to the global /etc/bash.bashrc:

  
    # Shared history optional per user  
    [ -f ~/.optin ] && history -a /shared/history

Users that want to participate can simply touch ~/.optin, enabling history export. Others automatically opt out, avoiding unintentional sharing.

Extending Bash History Functionality

Bash command history serves basic recall and repetition needs. By augmenting it with additional metadata, context, and tools, we can greatly enhance convenience and analytical capabilities.

Enhancing searching and filtering

The standard history command offers basic substring searching with Ctrl-R. Tools like thefuck and fzf enable more advanced regex and interactive filtering of history entries.

For example, to fuzzily search history, pipe to fzf:

  
    history | fzf

You can then interactively filter and select entries. Integrating these tools improves discovery and reuse of previous commands.

Adding timestamps for better diagnostics

By recording command execution timestamps (with milliseconds), history files can enable precise performance auditing and debugging of long-running processes and pipelines. For example:

  
    1581034441.912 ls -al ~
    1581034486.524 grep -c foo large_file | wc -l

To automatically add timestamps to history as it is accumulated:

  
    # Timestamp format 
    export HISTTIMEFORMAT="%s.%N "

Millisecond precision timestamps before each command will improve diagnostics.

Custom logging to augment built-in history

For additional flexibility beyond bash’s history, you can log all command output to an external file using a custom prompt command, PROMPT_COMMAND. For example, to log bash interactions:

    
    export PROMPT_COMMAND='tee -a ${HOME}/.bash_logfile'

This will duplicate stdin/stdout to a log before displaying the prompt. The file collects a verbatim transcript of shell activity, complementing history’s command-only storage.