Microwave Biscuit

Perl Script to List Files

Posted in Linux by microwavebiscuit on February 21, 2007

I have a ton of .flac and .shn files and it can be a challenge to keep up with them all. I have written a Perl script to recurse through a directory structure and write all of the files to a text file. I then use that text file in a second script to load it all to a MySQL database. You may find all or part of this useful – also it is not specific to any given file type, you could use it to create a listing of any type of files. Here’s the first script (note the Perl Modules I’m using – you may need to get them):


#!/usr/bin/perl
# listfiles.pl
use strict;
use File::Find;
use Time::localtime;
use File::Basename;
@ARGV = qw(.) unless @ARGV;
open(TEMPFL, '>/home/me/music.txt') or die "$0: Cannot open /home/me/music.txt for output!\n";
select TEMPFL;
find sub {
next if $_ eq "." or $_ eq ".." or -d $_;
my $dir = $File::Find::dir;
my $age = (stat($_))[9];
my $tm = localtime($age);
my $ext = (fileparse($_, '\..*'))[2];
$ext =~ s/^\.//;
print "$dir:$_:$ext:";
print -s if -r and -f;
printf (":%04d-%02d-%02d\n", $tm->year+1900, $tm->mon+1, $tm->mday);
}, @ARGV;
close(TEMPFL);

The usage is: perl listfiles.pl pathtodirectory. The output file looks like this:
/mnt/sda1/Archive/Music/GD/1974/gd74-05-14.sbd.murphy.1823.sbeok.shnf_shn : gd74-05-14d1t01.shn : shn : 40247417 : 2006-11-01
/mnt/sda1/Archive/Music/GD/1974/gd74-05-14.sbd.murphy.1823.sbeok.shnf_shn : gd74-05-14d1t02.shn : shn : 23120657 : 2006-11-01
/mnt/sda1/Archive/Music/GD/1974/gd74-05-14.sbd.murphy.1823.sbeok.shnf_shn: gd74-05-14d1t03.shn : shn : 38838225 : 2006-11-01

Now, I need to parse those “records” so that MySQL can read ’em and then load it to the database. Note: You need to have a MySQL database setup to accept the records. My database is called files_list and has the following fields in it: folder, filename, filetype, filesize, filedate


#!\usr\bin\perl
#insert.pl
use warnings;
use strict;
use DBI;
my ($dbh, $sth, $folder, $filename, $filetype, $filesize, $filedate, $rows);
$dbh=DBI->connect('dbi:mysql:test', 'user', 'password') ||
die "Error opening Database: $DBI::errstr\n";
$sth=$dbh->do ("TRUNCATE `files_list` ") || die "Couldn't empty db first"; #empty db first
$sth=$dbh->prepare
("INSERT INTO files_list (folder, filename, filetype, filesize, filedate)
VALUES (?, ?, ?, ?, ?)");
$rows=0;
while() {
chomp;
($folder, $filename, $filetype, $filesize, $filedate) = split(/:/);
$sth->execute ($folder, $filename, $filetype, $filesize, $filedate) ||
die "Couldn't insert record : $DBI::errstr";
$rows+=$sth->rows();
}
print "$rows new rows have been added to db\n";
#update unknown filetypes to be text
$sth=$dbh->do ("UPDATE `files_list`
SET `filetype`='txt'
WHERE (
`filetype` NOT LIKE 'shn'
AND `filetype` NOT LIKE 'flac'
AND `filetype` NOT LIKE 'txt'
AND `filetype` NOT LIKE 'md5'
AND `filetype` NOT LIKE 'jpg'
AND `filetype` NOT LIKE 'ffp'
AND `filetype` NOT LIKE 'gif'
)
") || die "Couldn't update db after";
$dbh->disconnect || die "Failed to disconnect\n";

Now, I’m sure that some of the quote marks, etc. will get screwed up in the formatting here so if it doesn’t work for you let me know and I can send you a complete script file. Also, I’m not a Perl wizard and I would guess there are probably other ways to do this maybe some better than me, although this works so I’m happy. The only thing I might like to do is to combine the scripts such that I don’t create the text file first -just load the records directly into the database as I create them.

Advertisements

3 Responses

Subscribe to comments with RSS.

  1. Michael Heukels said, on February 22, 2007 at 2:28 pm

    Sometimes I wonder if you need to get out more.

  2. ojiwa said, on March 2, 2007 at 1:14 am

    Hi, i find this script very useful. However, i can’t use it for now because I can’t figure out where to get the Perl modules that you have used like warnings, strict, DBI. Where can I find these modules???.

  3. microwavebiscuit said, on March 2, 2007 at 11:06 am

    Strict and Warnings are not Perl Modules. They are settings for Perl. DBI and other perl modules can be downloaded via cpan.

    Open a terminal and type sudo cpan. This should open up cpan and you may need to answer some questions. After you do that, type install DBI which should install the DBI module. You can install any modules you need that way including the others in my script. Hope this helps!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: