Scripting

Sed is a powerful text processing tool that allows matching input text against regular expression patterns. However, improperly sanitizing user input before using it in sed can introduce security vulnerabilities and enable code injection attacks. This article covers techniques for properly escaping arbitrary user input to prevent exploitation when using sed. Why Escape User Input?…

Escaping Regular Expressions Natively Properly escaping regular expressions in shell scripts can be challenging. Metacharacters like ., *, ^, $, [], {}, () carry special meaning and will not match literally by default. Usually external utilities like sed or awk are used to escape these characters. However, native regular expression escaping built into shells like…

The Problem with Unescaped Metacharacters in sed Briefly explain issue of unescaped regex metacharacters causing unintended behavior in sed scripts when variables are interpolated. Why Metacharacters Need Escaping Explain how metacharacters are interpreted by sed engine and can trigger actions or patterns unintentionally. Examples of Problems Caused by Unescaped Metacharacters Provide concrete examples of errors…

Why Backticks Persist in Shell Scripts Despite being deprecated in favor of $( ) syntax, backticks continue to have a stubborn persistence in shell scripts. There are several key reasons this antiquated command substitution syntax has yet to fade away: There is a massive amount of legacy shell scripts that rely on backticks for command…

Making Scripts Portable Across Unix Platforms The Need for Portable Scripts Unix operating systems have many differences between shells, builtin commands, and external utilities. Scripts written for one Unix may not work on another due to discrepancies in shells, tools, and filesystem layouts. Writing portable scripts that function across Unix platforms can save considerable time…

Why Backticks Fail in Nested Command Substitution Command substitution allows you to run commands and capture their output to be used as an argument or part of a larger command. The output text substitutes the command substitution construct when the command is run. The most common forms of command substitution are using backticks (`) or…

Legacy Shell Scripts Still Using Backticks Despite the widespread adoption of $(…) command substitution in most major shells like Bash, Zsh, and modern versions of CSH/TCSH, backticks (`) remain widely used in legacy shell scripts. Systems that have not been upgraded often contain old shell scripts that still rely on backticks for command substitution. Upgrading…

Bash scripting provides immense power and flexibility for automating tasks and workflows on Linux systems. However, this power comes with complexity around order of operations. Understanding how Bash handles brace expansion, variable expansion, and evaluation order will help you write clearer, more robust Bash scripts. Understanding Brace Expansion Brace expansion allows generating arbitrary strings for…

Bash brace expansion provides a convenient shortcut to generate sequences of characters or numbers. However, its usefulness diminishes when working with large or dynamic data sets. The bash parser imposes limitations that make brace expansion ill-suited for anything beyond simple static ranges. Attempting to use brace expansion with one million values like {1..1000000} will likely…

The Problem with Unquoted Brace Expansion Brace expansion in Bash allows convenient generation of arbitrary strings. However, unquoted brace expansions can lead to unexpected and dangerous behaviors. When brace expansions are left unquoted, the resulting strings may be improperly split into multiple arguments and improperly ordered. This can cause commands to act on the wrong…