KenFallon.com

I often have deep directory trees where I am only interested in the deepest layers and not the supporting sub directories. An example of this might be a music collection where you have moved all your albums into directories based on artist name, then album, then disk, then songs etc. I wanted a script to be able to display all the deepest directories in a directory tree. Take this tree as an example:

$find -type d
.
./x
./x/y
./x/y/z
./a
./a/b
./a/b/c
./1
./b
./b/b
./c
./c/c
./c/c/c
./c/c/c/c
./c/c/c/c/c
./c/c/c/c/c/c
./11
./11/2
./11/2/3
./11/1
./11/1/1
./11/1/1/1
./11/1/11
./11/11
./11/11/11
./z

I put this script together which cycles through the output of the find command looking for directories only and then compares this line to the previous one.

$ cat twigs
#!/bin/bash
LASTTWIG=""
for LINE in `find -type d`;do
 if [ ! "${LASTTWIG}" = "" ];then
   echo ${LINE} | grep ${LASTTWIG}/ > /dev/null
   if [ "$?" != "0" ]; then
     echo ${LASTTWIG}
   fi
  fi
  LASTTWIG=${LINE}
done
echo ${LASTTWIG}

The last echo is necessary to get the last directory which will always be a end of chain directory. The output gives something like this.

[user@localhost x]$ ./twigs
./x/y/z
./a/b/c
./1
./b/b
./c/c/c/c/c/c
./11/2/3
./11/1/1/1
./11/1/11
./11/11/11
./z
[user@localhost x]$

One strange issue I ran across when using a find -type d | while read LINE;do iteration was that the pipe caused the last line not to work as the variable was stored in a sub process as the pipe | executed the while loop in a sub shell. The post Variable Scope in Bash helped to clear it up.

In a follow on from my last post, I want to talk about replacing spaces with underscores. To do this you can use the tr command. From the man page tr will “Translate, squeeze, and/or delete characters from standard input, writing to standard output.”. This is a small script that I run after bashpodder runs that will replace spaces with underscores.

#!/bin/bash
ls | while read -r FILE
do
mv -v “$FILE” `echo ${FILE} | tr ‘ ‘ ‘_’ `
done

Simple but effective …. well that was until I started coming across file names that have an acute, circumflex, umlaut etc in them. Transferring these files to my mp3 player caused it problems so I modified my script to convert all the file names to the ASCII character set first and then replace the spaces.

#!/bin/bash
ls | while read -r FILE
do
mv -v “$FILE” `echo ${FILE} |  iconv -f utf-8 -t ASCII -c | tr ‘ ‘ ‘_’ `
done

The magic is handled by the iconv and from the man page “iconv – Convert encoding of given files from one encoding to another”. In the script the program will convert form utf-8 encoding to ASCII. The only problem is that it will fail with an error if there is no equivelent charachter in the translating to encoding scheme. For that reason I use the -c option which will “Omit invalid characters from output.” So in the case of a file named Über Schön.ogg the iconv pipe will convert the filename to “ber Schn.ogg” and the tr pipe will convert that to “ber_Schn.ogg”.

I think you’ll agree that that is not ideal but then this is supposed to be a quick and dirty fix.

I’ve decided to post this one as it’s one of those things that I have come across so often but each time I froget the solution. The problem arises when you are trying to script using the for command and you run up against filenames that have spaces. No matter what way you try and quote them the for command will split them out. Assume we have two files in a directory. One with spaces in it’s name and one without.

ken@berta:~/x$ ls -1
file-name-one.txt
file name two.txt

Say we want to rename the files using the mv command. We could use the following for loop:

ken@berta:~/x$ for FILE in *.txt;do mv ${FILE} ${FILE}.bak;done
mv: target `two.txt.bak’ is not a directory
ken@berta:~/x$ ls -1
ken@berta:~/x$ file-name-one.txt.bak
ken@berta:~/x$ file name two.txt

There was no problem with the first file as it has no spaces. Rather than passing one variable “file name two.txt” to the mv command, the for command split it up into three separate variables using the space character as a delimiter. What this actually expanded to was this:

ken@berta:~/x$ mv file name two.txt file name two.txt.bak
mv: target `two.txt.bak’ is not a directory

Here the move command (mv) is trying to move all the files named ‘file’, ‘name’, ‘two.txt’, ‘file’ and ‘name’ into a directory called ‘two.txt.bak’. Obviously the files don’t exist but as the mv command first checks for the existance of the destination directory ‘two.txt.bak’, it gives the error that there is no directory by that name.

I’ve found a few ways to get around this but the one I like the best is to use the while loop instead. The while loop is nice as you can also use it to read in from a text file.

ken@berta:~/x$ ls -1
file-name-one.txt
file name two.txt
ken@berta:~/x$ ls -1 *.txt| while read FILE;do mv -v “${FILE}” “${FILE}”.bak ;done
`file-name-one.txt’ -> `file-name-one.txt.bak’
`file name two.txt’ -> `file name two.txt.bak’
ken@berta:~/x$ ls -1
file-name-one.txt.bak
file name two.txt.bak
ken@berta:~/x$

Here we pipe the output of the ls command into the while command loop. You need to put the double quotes around the variable or it will be treated as multiple variables.