Archiving your DV tapes

I was listening to a podcast about archiving your data and I decided it was time to archive my Digital Video tapes. Good thing I did because although the oldest were only four (4) years old they were giving read errors playing them. Not good ! Warning to all – Copy your DV tapes ASAP.

A 60 minute DV tape takes about 15Gb of disk space in raw dv format which means my 100 tape collection would reach 1.5Tb. 15Tb. In a year or two it will be possible for me to afford a 13Tb drive but for now re-encoding is the only answer. I looked around for tips on what the best format to use was and I decided to use H264 as it seems to be supported widely. I found this excellent post by Elias Torres entitled “How do I manage my personal videos?” and I just copied and pasted the mencoder settings into my script. See the post itself for more information on the mencoder settings.

The tool to use for dv capture on the command line is dvgrab which is part of the Kino project. This is a great tool and allows me to dump out each clip to a different file with the start time as the file name.

dvgrab –opendml –size 0 –autosplit -t

The only gripe is that (on my camera) it doesn’t detect the end of tape which means the program never terminates and so it’s difficult to build a bash script around it. I did some searches for work arounds and I remember a post from IBM developerworks Linux tip: Controlling the duration of scheduled jobs. This shows how you can start a process in the background and make note of the process ID, go to sleep for a period of time and then wake and kill the process. As all my tapes are an hour long stopping the capture process after 70 minutes seemed like a good setting. As for the rest I just copied and pasted.

This is the full script that would transfer a tape to disk and then encode it using mencoder.

#!/bin/bash
runtime=${1:-70m}
mypid=$$
# Run dvgrab in background
dvgrab –opendml –size 0 –autosplit -t &
dvgrabpid=$!
sleep $runtime
kill -s SIGTERM $dvgrabpid
for file in *.avi
do
mencoder $file -ofps 30000/1001 -nosound -ovc x264 -x264encopts pass=1:bitrate=5500:turbo=1:me=umh:nodct_decimate:interlaced:no8x8dct:threads=4:fast_pskip:nobrdo:trellis=0:nr=350:turbo=1:nomixed_refs:noglobal_header:qp_min=10:qp_max=51:nobime:keyint=290:keyint_min=29:frameref=1:bframes=0:nob_adapt:nob_pyramid:noweight_b:subq=5:chroma_me:nocabac:nodeblock -passlogfile ./h264.log -o /dev/null
mencoder $file  -ofps 30000/1001 -oac mp3lame -lameopts abr:br=140:aq=4:vol=1.5:mode=1:highpassfreq=0:lowpassfreq=0 -ovc x264 -x264encopts pass=2:bitrate=5500:me=umh:nodct_decimate:interlaced:no8x8dct:threads=4:fast_pskip:nobrdo:trellis=0:nr=350:nomixed_refs:noglobal_header:qp_min=10:qp_max=51:nobime:keyint=290:keyint_min=29:frameref=1:bframes=0:nob_adapt:nob_pyramid:noweight_b:subq=5:chroma_me:nocabac:nodeblock -passlogfile ./h264.log -o H264-$file
mv $file ${file}.done
done

I actually split the transfer and conversion into two different processes. I had my laptop transfer the tapes and move the rawdv files to a big raid0 (yes that’s raid zero) temp directory on my server. The server then processed the files 24×7.

I wrote a looping script on my laptop so that once I pressed a key it would use the program dvcont rewind to rewind the tape. After 5 minutes that would be terminated and then dvgrab would run as above. The only difference was that instead of converting the files I moved them to the server as *.avi.temp first and when they were over I renamed them to just *.avi. That prevented the looping conversion script from starting to encode a partially transferred video.

Edit: Thanks Vladimir