Using Stow To Manage Locally Compiled Software On Debian

The Package Management Problem

The Debian package management system provides a convenient way to install, upgrade, configure and remove software on a Debian system. However, sometimes a user may need or want to compile a piece of software from source rather than using a package. Reasons for this include:

Needing a newer version of some software than what is available in the Debian repositories
Needing to customize the compilation options for performance, features, etc.
Not having an appropriate package available for the software at all

Manually compiling and managing this locally installed software can cause problems down the road:

The software can conflict with packaged files
Upgrades can overwrite it
It can be difficult to track everything that was installed

Stow is a system designed to manage locally installed software in a way that avoids these problems.

Table of Contents

Understanding Stow

Stow is a program that manages sets of software packages installed into separate folders through the use of symlinks. The directory structure setup by Stow looks something like this:

/path/to/stow (Stow directory)
├── bin (Package 1 - binaries)
│   └── package1 
├── lib (Package 2 - libraries)   
│   └── package2
└── share (Package 3 - shared files)
    └── package3

The key thing that Stow does is create symlinks from files within the package subdirectories into locations like /usr/local/bin, /usr/local/lib, etc. This allows locally installed packages to integrate with the rest of the system.

An example set of Stow commands to set up an environment might look like:

# Install package1 source into /path/to/stow/bin/package1
./configure --prefix=/path/to/stow/bin/package1
make && make install

# Install package2 source into /path/to/stow/lib/package2  
./configure --prefix=/path/to/stow/lib/package2
make && make install  

# Install package3 source into /path/to/stow/share/package3
./configure --prefix=/path/to/stow/share/package3
make && make install
    
# Stow the packages into place    
stow bin 
stow lib
stow share

Now all three locally compiled packages will work together with the rest of the system software.

Installing Stow

Stow can be easily installed from the Debian repositories:

sudo apt update
sudo apt install stow

The stow package has several dependencies including:

perl-modules
libmodule-build-perl
libclass-accessor-perl

So installing stow will bring in other necessary Perl components as well. No further setup should be necessary after installing the Debian stow package.

Using Stow for Local Packages

Once Stow is installed, using it to manage local packages generally follows this workflow:

Obtain and compile source packages into Stow package subdirectories
Run Stow on each subdirectory to symlink it into place
Update, reinstall etc by modifying the source subdirectories
Unstowing and restowing subdirectories easily rolls back changes

For example, to compile the latest Emacs and make it available system-wide:

# Create directory for emacs under Stow root  
mkdir -p ~/software/emacs/emacs-27.1  

# Compile source there
./configure --prefix=$HOME/software/emacs/emacs-27.1
make && make install  

# Stow into place  
stow emacs

And Vim can be installed similarly:

# Create Vim subdirectory 
mkdir -p ~/software/vim/vim-8.2

# Compile source there
./configure --prefix=$HOME/software/vim/vim-8.2 
make && make install

# Stow vim directory  
stow vim

Now both Emacs 27.1 and Vim 8.2 are installed and managed by Stow outside of the Debian package system. They can be removed or upgraded by updating the source subdirectories and restowing.

Advanced Stow Usage

Stow provides additional functionality to aid managing more complex installations:

unstow – Undo a stow operation, removing symlinks.
restow – Stow a package again, updating symlinks.
-d – Dry run option shows symlinks that would be created.
-nvR – Restow all managed packages verbosely.
-S – Stow package to specified target.

Some examples of advanced usage:

  
# Unstow vim, removing symlinks
unstow vim   

# Restow vim to update configs  
stow -Rnv vim  

# Restow all packages, see changes   
stow -nvR */  

# Stow different Vim version elsewhere 
stow -S ~/apps/vim-8.3 vim

Being able to easily update and rollback changes through restow and unstow commands helps maintain control over complex local installations.

Troubleshooting Stow Issues

Sometimes issues can come up when using Stow to manage local packages including:

File conflicts between packages
Unable to create symlinks due to permissions
Breakage caused by upstream software changes
Manually deleted symlinks

File conflicts will cause Stow to abort with errors like:

  
stow: Cannot install vim into target directory 
    '/usr/local/bin': File exists

This is often caused by collisions between the packaged software paths and local paths. It can usually be resolved by strategic use of Stow target directories via the -S option.

Permissions issues symlinking can happen when Stow is run as a normal user that cannot write to system directories. Using sudo to run the Stow command as root will typically resolve this.

Upstream software changes like config file locations changing with a Vim upgrade can break existing Stow symlinks. Running Stow in dry run mode first via -d will show issues. A forced restow -Rnv can then rectify problems.

And careful use of unstow and restow commands allows rolling back any issues from broken symlinks or bad configurations.

Conclusion

Stow provides an extremely flexible system for managing complex installations of locally compiled software on Debian. The benefits of Stow include:

Avoids collisions between local and packaged installs
Locally compiled software integrates cleanly with system
Simple commands abstract symlink complexity
Easily update, remove or rollback installations

Following the best practices laid out here like utilizing package subdirectories, stowing software in the correct system locations, and leveraging advanced Stow features will enable smooth management of any locally compiled Debian software.