How the bash history works

I have been using Unix for about thirty years. That means that I have also used Unix shells for thirty years. I started using csh as my interactive shell, then moved to the KornShell and after that to the Bourne Again Shell. I tried out zsh for a while and then moved back to bash.

Apart from csh, I never fully learned how to use any of the shells. I did indeed learn a lot about them and certainly enough for comfortable everyday use. That is why I never felt the need to fully learn them, I guess. Throughout the years of using bash, though, I have stumbled across mainly two things related to the command line history (I think) that have annoyed me, but that have not annoyed me more than allowing me to simply ignore them. Today I would like to find out what those things actually are, how to avoid them or possibly exploit them.

When working at the shell's command line, I usually do one of these:

  1. type a command and run it
  2. hit r which is an alias for fc -s to rerun the previous command
  3. hit ESC and k a few times to get to a previous command and possibly edit it before running it
  4. hit / to search for a previous command and possibly edit it before running it

This has worked fine, but eventually I have done something that has interrupted my workflow. This will probably not be very exciting, but let me use this as a log while finding out what it is all about.

Two annoying features

The args: prompt

As far as I remember it, sometimes a prompt containing the word args, possibly some parentheses and a colon, appears.

This is apparently the readline arguments feature which is supposed to pass a numeric argument to a readline command. It is triggered using the meta key which allows a number to be entered. The manual uses an example with emacs style editing commands, so how this is actually triggered for vi mode may be a question of semantics as ESC (which in emacs mode is a way to metafy) is used to enter the command mode. When in vi mode, this seems to be little more than a reminder that a repeat count to the following editing command is being entered. It also proves how very little I have been using repeat counts on the command line throughout the years. Furthermore, this has nothing to do with the history. The actual prompt looks like (arg: 1). All of this should have been clear to me if I had spent just a few minutes reading the manual.

Vanishing commands

I have sometimes found myself in a situation where I have entered and executed a long command and then later searched or navigated to it in the history in order to execute it again with some modification. However, I must have done something when editing the command because all of a sudden it has vanished from my current command line and it also seems to vanish from the history itself. Could it be that the history itself is mutable?

The history is indeed mutable using the -c and -d options of the history command. However, this is not what I have been doing when my commands have vanished. Something else must be going on.

Actually, there seems to be many things going on.

Command vanishing while editing the history

Consider the following session.


      $ history -c
      $ echo foo
      foo
      $ echo bar
      bar
      $ echo baz
      baz
      $ echo quux
      quux
      $ history
          1  echo foo
          2  echo bar
          3  echo baz
          4  echo quux
          5  history
      $
Hitting ESCkkkdd or ESC/bazRETdd will of course delete the line echo baz. The line will be empty and you will not be able to find anything when searching for baz (stay tuned for what happens when you search). If you hit Return now and run the history command, you will see the following. 

    $
    $ history
        1  echo foo
        2  echo bar
        3  echo baz
        4  echo quux
        5  history
        6  history
    $

Command being marked as modified

If, instead of hitting Return after deleting echo baz in the session above, you first hit k before hitting Return, you will see the following: 


      $ history -c
      $ echo foo
      foo
      $ echo bar
      bar
      $ echo baz
      baz
      $ echo quux
      quux
      $ history
          1  echo foo
          2  echo bar
          3  echo baz
          4  echo quux
          5  history
      $ echo bar
      bar
      $ history
          1  echo foo
          2  echo bar
          3* 
          4  echo quux
          5  history
          6  echo bar
          7  history
      $

In fact, any modification of the line will be saved in the history and marked, just like stated in the manual for the history command. The purpose of this does not yet seem clear to me. Nor is it clear why the modified command is saved only when the cursor is on another command in the history.

Command not being marked as modified

Again, consider the following session.


      $ history -c
      $ echo foo
      foo
      $ echo bar
      bar
      $ echo baz
      baz
      $ echo quux
      quux
      $ history
          1  echo foo
          2  echo bar
          3  echo baz
          4  echo quux
          5  history
      $

Now do the following:

  1. Hit ESCkkk to go to the line echo baz
  2. Hit dd to delete it
  3. Type /fooRET to go to the line echo foo
  4. Hit jj to go back to the line echo baz which is still there because you left it using a search command (see below)
  5. Hit dd to delete it again
  6. Hit k to move to the line echo bar
  7. Type /fooRET to go to the line echo foo again
  8. Hit enter to execute the command echo foo
  9. Run the command history 

      $ echo foo
      foo
      $ history
          1  echo foo
          2  echo bar
          3  
          4  echo quux
          5  history
          6  echo foo
          7  history
      $

Look at that! The command is gone but not marked as modified. I have no clue about what is going on here. It is worth mentioning that hitting return on step 6 above and skipping step 7 will cause the command to marked as modified. Likewise, skipping steps 3 through 5 will also make the command marked.

Commands reappearing after searches while editing the history

One last time, consider the following session.


      $ history -c
      $ echo foo
      foo
      $ echo bar
      bar
      $ echo baz
      baz
      $ echo quux
      quux
      $ history
          1  echo foo
          2  echo bar
          3  echo baz
          4  echo quux
          5  history
      $

If we now go to the line echo baz and delete it with dd, we can verify that it is deleted by moving up and down in the history with k and j, noticing that the line is in fact empty. However, if you, instead of using the keys j and k, move away from the line using the search command, for example /foo and then move back to the deleted line, you will see that it has now reappeared.

Other things that I have come across worth mentioning