Shell history is one of the simplest productivity tools in an operator’s environment. It is also an unstructured record of what was run, where it was run, and—when discipline slips—what secret was pasted into a command.
Periodically rewriting the history file with a clever deduplication script solves the least important part of the problem. What I want is useful recall without preserving more sensitive data than the job requires.
Prevent duplicate noise at the source
For Bash, these settings provide a practical baseline:
shopt -s histappendHISTCONTROL=ignoreboth:erasedupsHISTSIZE=10000HISTFILESIZE=20000PROMPT_COMMAND="history -a${PROMPT_COMMAND:+; $PROMPT_COMMAND}"Add them to ~/.bashrc, then start a new shell. histappend appends entries instead of overwriting the history file when a shell exits. erasedups removes older matching entries from the active history list, while HISTSIZE and HISTFILESIZE bound in-memory and on-disk retention. The PROMPT_COMMAND addition writes new entries after each prompt instead of waiting for shell exit.
Review an existing PROMPT_COMMAND before combining values. Frameworks and terminal integrations may already use it, and a malformed replacement can silently disable their behavior.
There is still a tradeoff: multiple active shells can produce surprising ordering. If exact, multi-session auditability matters, shell history is the wrong mechanism. Use centralized session recording or privileged-access tooling designed for that purpose.
Remove a sensitive entry deliberately
If a secret reaches the current shell history, identify and delete the entry before writing the history file:
historyhistory -d 1234history -wReplace 1234 with the history entry number identified by the first command.
That only addresses the local history file. The value may also exist in terminal logs, process telemetry, remote-session recordings, backups, or the destination system. Assume exposure and rotate the secret when the blast radius is uncertain.
Why I no longer recommend bulk rewrite one-liners
A pipeline that sorts, deduplicates, and replaces ~/.bash_history looks efficient, but it creates avoidable failure modes:
- It can destroy ordering that explains an operational sequence.
- It can overwrite the only copy before validation.
- It handles multiline and timestamped history poorly.
- It solves duplicate storage after the fact instead of improving capture behavior.
If a one-time cleanup is necessary, make a protected backup, work on a copy, and inspect the result before replacement. For routine use, configure retention behavior once and let the shell do the work.
A random password generated from the shell can still leak through the next command that uses it. The generation method and the command environment have to be considered together.
Lessons learned
I used to approach this as a cleanup problem. The one-liner reduced duplicates, but it did nothing about what had already been captured and could discard ordering, timestamps, or multiline context. Changing capture and retention behavior proved safer and more predictable than repeatedly rewriting the file.
Shell history is both a convenience and a record of administrative activity. Decide what it is for, who can read it, how long it should remain, and what happens when a secret lands there. Then test the settings with concurrent shells, duplicate commands, multiline commands, and a known excluded entry. The resulting file should preserve the context you meant to keep—and omit what you meant to exclude.
How does your organization preserve useful administrative evidence without turning shell history into an unmanaged source of credentials or sensitive context?
Sources and disclosures
- GNU Bash manual: Bash history facilities
- GNU Bash manual: history builtins
- GNU Bash manual: the
shoptbuiltin
This field note reflects the author’s operational practice. Exact behavior depends on the Bash version, startup files, terminal tooling, and session-recording controls in the environment.
