This blog has been silent for a long time but I will try to write a few articles about some bash tricks or other things related to programming.
I just read the article by Julia Evans about editing files using ed for doing batch editions that are more complex than simple search and replace https://jvns.ca/blog/2018/05/11/batch-editing-files-with-ed/ !
I didn’t even know that you could write an ed script, and I wanted to share how I usually deal with these kinds of tasks by using a very simple script that uses only vim, which lets me use the commands I am used to (which are the vim command i for insert, a for append, …)
So lets take a very simple file which I called fruits.txt
foo: - bar - baz - bananas
The objective is to change it to add the line ` – elephant` just after the baz line, to have following file :
foo: - bar - baz - elephant - bananas
It is not so easy to do that with common commands, and I don’t know awk or sed well enough to do it within less than 5 minutes.
To do that, I use following bash one liner :
cat fruits.md | vimpipe 'norm /baz^Myypfbcwelephant' | sponge fruits.md
How does it work ?
- The cat fruits.md shows the full file.
- The vimpipe is the command that will edit the input.
- The sponge command writes to the file « at the end ».
Here usually, we would be tempted to do >fruits.md to redirect the stdout of the second command to fruits.md, however if you do that, you will be quite surprised to see that your file will be empty, because the file is both written and read from simultaneously (I think it is possible that the file could grow until your disks are full too). sponge is a command that will wait for the stdin to be closed, and then write the output to fruits.md (See following link from moreutils, which has developped sponge https://joeyh.name/code/moreutils/)
The argument to vimpipe describes the changes that I want to make, in the syntax of vim commands (The most common command is the s command, which is usually used with :s/pattern/replacement/g). Here we are using the norm command, which means that we want to execute the next letters as tough we were in normal mode inside vim. In our case, we want to search for baz, so we use /baz, but since we want to hit enter to « validate our search », we press Control-V and then Enter, which will pass the Enter Key to vim (this is shown in most terminals as ^M, which is a single character representing a newline). Then we do yyp which means yank the whole line, and paste it below the current line (which will duplicate the line), then we want to change the word baz by the word elephant, so we first go to the b letter by hitting fb, and then cw meaning change the word below the cursor, then elephant to change the word to elephant (we are in insert mode due to the c command).
What is the content of this vimpipe script ?
vimpipe is the following script, there’s no magic voodoo in it
\vim – -u NONE -es ‘+1’ « +$* » ‘+%print’ ‘+:qa!’ | tail -n +2
\vim will run vim but not using any aliases that could be currently set by the user , I put it because I have aliased vim to neovim (nvim)
The first – means that we want to read input from stdin and not from a file.
-u NONE means don’t load anything from vimrc, this speeds up startup time and avoids unwanted sideeffects caused by plugins, but you can omit it if you want to use some plugins for example
-es means start vim in ex mode (eg process commands, do not start a visual interface)
Then, all remaing arguments are vim commands (starting with +)
+1 means go to the first line of the file
+$* will execute all commands given in the arguments of vimpipe
+%print will run the print command on each line (so this will actually show the current file)
:qa! will exit vim without saving the file
The tail -n +2 is to ignore the first line which is `Vim: Reading from stdin…`
Vim can be used « almost anywhere »
I find this script quite useful (I probably use it everyday), especially if you already know vim commands well but would like to automate things without going interactive.
Here are a few examples that I don’t have the time to explain, but might give you ideas :
To remove the last argument of all functions called foobar : from foobar(a,b,c) to foobar(a,b) : cat test.code | vimpipe ‘g/foobar/norm f)F,dt)’ | sponge test.code
To remove the first line containing ‘teststring’ : cat test | vimpipe ‘/teststring^Mdd’ | sponge test