Microwave Biscuit

Why can’t buying a new car be more like buying a PC? Here’s what I’d like to do - go to a manufacturer website, configure a vehicle exactly as I want it, pay the fair/final price and have UPS deliver it. How about it? If you think about it, outside of the price scale it’s not that different a transaction. When buying a PC you select how much horsepower you want (Processor), Interior space (RAM, HDD), accessories (video card, sound, etc). Then you get shown a price. If you like it, you buy it and it comes to your door in a few days. When buying a PC you are stuck with a limited set of choices (see all the recent posts about Dell and Linux floating around). If you visit the Dell, Gateway, etc sites they have differing levels of performance available and each one has a given set of options/upgrades. And just as most factory car stereos suck which are ripped out to be replaced by better units, you can get rid of the mandatory Windows install and replace it with Linux. The major difference in buying a PC is that at the end of the day you don’t get stuck with a deep down feeling that you may have gotten screwed.

As it stands now, you can go to an auto manufacturers website and do all kinds of building/configuring but then you get a “MSRP” which probably has very little to do with what you might actually pay. You also obviously have no ability to purchase. I’m hoping that sometime in the near future this may become a reality. Over the last 3 months I’ve been involved with 3 separate dealerships (with 3 different manufacturers - Honda, Toyota and Nissan) and have ended up not purchasing a vehicle. It comes down to not feeling a sense of value and fairness in the transaction.

I am obviously a big fan of Ubuntu. One of the many reasons that I like it is that generally speaking, it just works. Last night I took a few pictures with my Panasonic Lumix digital camera and I wanted to upload them to my pc. I had never connected this camera to this PC so I was wondering what would happen. I plugged in the USB cable, turned on the camera and Ubuntu popped up a window - Camera Found - Import Pictures? (or something along those lines). I clicked on import and it read in my pictures. It even created a folder based on the date/time so that I have a “roll” folder. I understand that this is how computers are supposed to work and yes, if I had plugged it into a Windows PC I probably would have gotten a similar result. However, having used Linux for many years and dealt with a myriad of hardware related issues I’m so very impressed with Ubuntu’s ability to detect and manage hardware. Situations like this are what make mainstream acceptance of Linux that much easier. Your everyday non-geek PC user wants to be able to just have things work without building things from sources or writing drivers. Of course I fall into the “geek” category of computer users and I do get some satisfaction out of building something from sources, etc. but if Linux is going to go further into the mainstream it needs to be able to “just work” out of the box too.

Yesterday, I noticed an incoming link from WordPress on my blog stats. Apparently yesterday I was the 46th fastest growing blog in the WordPress blogosphere. Thanks!

In the previous post I described “why” I convert legally obtained music to a lossy format. Here’s a “how-to” post. First and foremost, you need the “etree scripts” package. I have it listed in synaptic under multimedia but I’m not positive which repository it is from. If you don’t have it listed, here’s a link to the sourceforge site: http://etree-scripts.sourceforge.net/. This is a set of Perl scripts that makes living with flac and shn files way easy. There are a few dependencies that you need to get as well.

First: Shorten - This is necessary to deal with .shn files. You can get it here. You will need to build it from the sources.
Second: Lame - Used to encode to mp3. It may be in your repos or you can get it here
Third: Flac -Free Lossless Audio Codec - Also in your repos, if not go here.
Fourth: Ogg Vorbis - Again, in the repos or here.

You should also get a couple extra Perl Modules that aren’t required but make encoding easier.

Date::Parse - open a terminal and type cpan. Answer any questions that cpan has, then type: install Date::Parse. This should build and install Date::Parse which makes the parsing of date/time info easier for the scripts.

Audio::Wav - Used to deal with WAV files. Get it the same way as Date::Parse. You may already have it.

Now, all dependencies satisfied I’ve written the following script to do some magic:


#!/bin/bash
# Purpose: Convert Downloaded Music
# Script written by Microwave Biscuit
# Script Name: converts
before=”$(date +%s)”
for name in $(pwd | sed ’s/ /____/g’)
do
oldname=”$(echo $name | sed ’s/____/ /g’)”
newname=”$(echo $name | sed ’s/____/_/g’)”
if [ "$oldname" != "$newname" ] ; then
echo “renaming \”$oldname\” to $newname”
cp -r “$oldname” “$newname”
fi
done
cd $newname
md5check;
result=$?
if [ $result -ne 0 ]; then
echo “Something is wrong with the MD5 Sums”;
exit 1;
fi
shn2mp3 `pwd`
after=”$(date +%s)”
elapsed_seconds=”$(expr $after - $before)”
elapsed_minutes=”$(expr $elapsed_seconds / 60)”
echo “Conversion Time: ” $elapsed_minutes ” minutes for: ” $newname | mailx -s “Conversion Finished” me@mypc


Save this script as converts and make it executable (chmod +x).

What this script does is:

1. Gets a starting time so that we can figure out how long the process took.
2. It figures out if there are any blank spaces in the name of the directory that contains the flac/shn files. The Etree script shn2mp3 doesn’t like folders with spaces.
3. If there are spaces, it creates a new directory that replaces the spaces with underscore characters and copies all of the files into it.
4. Runs the md5check script. This ensures that all of the files are complete and match their md5 signatures.
5. If it fails the check, it will exit the script and report the error to you.
6. Next, it runs the shn2mp3 Etree script which will encode all of the files to mp3 or ogg (your choice), parse the info file if it exists and update all of the tags on the newly created files.
7. Figures out how long the process took and emails me that it is complete and lets me know how long it took.

This process works for 99% of the files that I get from etree. There are a few files that don’t follow the etree info file conventions which means that they don’t get their tag data done correctly. I also occasionally get md5 check errors. This is almost always due to the person who created the md5 file having extra files in the directory, i.e. the original WAV files or something else unnecessary. If it fails the md5 check and reports that a wav file is missing or thumbs.db is missing, I just manually run the shn2mp3 script on the directory. If there are other errors in the md5 check then you likely have a problem with your download or the source flac/shn files.

Another very useful script in the etree scripts package is burn-shns. This script will take a directory of flac/shn files and create a CD from it. If your downloaded files have multiple disks in one directory (usually identified with filenames like d1t1, d2t1, etc) create a new directory for each disk and copy all of the d1 files to the new folder and then run burn-shns in each folder.

Some may have issues with using the mp3 format - feel free to use ogg. Also, I believe that shorten is a proprietary format but it’s free to use for non-commercial usage. Please respect copyrighted music and especially trade friendly artists. If there is an artist that you like who is trade friendly you should definitely buy their commercial releases so that you support their work.

This post by Eclectic - Converting flac files to ogg « Eclectic - seems innocent enough - just a tip on how to encode music files from flac to ogg. However this brings up a little bit of controversy. I have tons of shn and flac files as previously noted. I get them predominantly via e-tree which is a bit torrent tracker for “trade friendly” artists. They are very careful to not include artists that have any issues with having their music traded and they also do not allow any commercial or copyrighted material. In any case, I often find in the information file of a recording something like this:

*****************************

DON’T CONVERT TO LOSSY FORMAT

*****************************

NOT FOR SALE

*****************************

I can certainly get behind the “not for sale” part. If an artist allows people to record their live performances and trade the music people should respect that. I’m sure that some of you have purchased “bootlegs” or “imports” at some stores and basically that is ripping off a lot of people. First of all, the artist doesn’t get any money out of the deal, and secondly the person that recorded it probably didn’t intend on having it show up in a record store for sale. I do doubt that this particular warning in an info file would prevent someone from selling it, however as I said I’m behind keeping it not for sale.

The “Don’t convert to lossy format” part is what kind of gets to me. I do always keep the original flac and or shn files so that when I share the music it retains it’s original state, however even if my iPod did play .flac (which I don’t think it does) I would certainly not keep it in a flac format. For those of you who don’t know, flac stands for free lossless audio codec. Flac files are not nearly as big a WAV files (which are uncompressed) but they are way bigger than ogg or mp3 files. I always convert the files to a “lossy” format before moving them to my iPod.

I’ll be posting another post on the process I use to convert flac and shn later.

As you may be able to tell I like Perl. You may also notice that I’m not terribly good at it but I manage to get done what I usually need to. In learning Perl I got 3 different books. They are:

• Beginning Perl by Simon Cozens
• Programming Perl by Larry Wall, Tom Christiansen and Jon Orwant (The Camel book)
• Perl Cookbook by Tom Christiansen and Nathan Torkington

The listing is pretty much my order of preference on the books. Here’s a brief overview of each:

Beginning Perl - This is the book for me that I use most often. I have no idea where I got the recommendation for it or even when I bought it. It’s just on my bookshelf. I have a tendency to just pick out the most important things about a programming language and try and hack into the rest. With Perl, I decided to take a different approach. I went through this book starting on page 1 and wrote the “hello world” examples. In fact, I wrote 99% of the examples in the book just to do it. This book is not for advanced users really, nor is it as comprehensive as the “camel” book but for me it was a nice mix of basic and intermediate stuff.

Programming Perl - As indicated this is the “Camel” book and the book that most people recommend to anyone trying to learn Perl. This is the book that I turn to second if I’m trying to do something and I can’t figure it out from Beginning Perl. It is a much more comprehensive book and covers just about everything there is to know. Generally I use this book by the index, meaning, I turn to the Index to find out where something is documented as opposed to running through the book front to back.

Perl Cookbook - As you wold expect with that title, it’s basically a book of Perl recipes. It’s not as instructional as the previous two and it certainly expects you to have a good working knowledge. Like Programming Perl, I use this by the Index looking for topics of interest or when I need a solution.

I’m sure many people could get by with just Programming Perl but for me it was a bit easier to test the waters with Beginning Perl.

Batteries not included posts about Desktop Search here: Beagle and desktop search on Ubuntu « Batteries Not Included which brings me back to my previous post about using locate. In that post I talked about using locate to configure and use Yakuake. If I use Beagle to search for yakuake it returns only one result which is the yakuake executable itself. I don’t find that terribly useful, in that I was looking for configuration files, documentation, etc. and I wanted to see where all of that stuff was.

At work I do quite a bit of database stuff. One of the databases I use puts out an error file for unmatched items. The problem is, the unmatched item could appear in 200, 2,000 or a million records and sorting through it to figure out the real issues can be a challenge.

Here’s an example of the error log:

\\ Member foo Not Found In Database
“US$” “Working Scenario” “FY06″ “PC” “foo” “Jan” 100

\\ Member bar Not Found In Database
“US$” “Working Scenario” “FY06″ “PC” “bar” “Jan” 100

\\ Member foo bar Not Found In Database
“US$” “Working Scenario” “FY06″ “PC” “foo bar” “Jan” 100

\\ Member bar foo Not Found In Database
“US$” “Working Scenario” “FY06″ “PC” “bar foo” “Jan” 100

So, the file is organized with \\ indicating the beginning of the message, the second row is the entire record, then there is a new line. In the above example, I have only 4 error records and each one contains a unique item however in reality I could have 500 or more records for “foo” which would have 500 messages in this log.

Here’s the Perl script that I wrote to deal with it so that I get a unique, sorted list of items that aren’t found. Note: many of the items in this script are “hard coded” and if I were to make this a more general script I would accept the file name at runtime, etc. I also know that I’m probably using too many temp files here and I could probably go from the input file to an output file without the intermediary temp file, however when I did it like that I got weird results. It works perfectly for my task as is so I’m good with it.


#!/usr/bin/perl
# dataerrs.pl
use warnings;
use strict;
my %seen;
my @uniq;
my @list;
my $item;
# Find Errors in Error Log
open FILE, “dataload.err” || die “Can’t find file”;
open(TEMPFL, ‘>/home/me/dataerrs.txt’) || die “Can’t open output file”;
$_ = ;
s/\r\n/\n/g; # replace Windows new line with Unix just in case
while () {
chomp;
if (/^\\\\/) { # row needs to begin with \\
s/\\\\//; # get rid of \\
s/Member //; # get rid of “Member”
s/Not Found In Database//; # get rid of “Not Found in Database”
s/^\s+//;
s/\s+$//g;
print TEMPFL “$_\n” # All that should be left is item not found - write it to a temp file
}
}
close TEMPFL;
open INFILE, “dataerrs.txt” || die “Can’t find file”; #open temp file
open(OUTFILE, ‘>/home/me/finalerrs.txt’) || die “Can’t open output file”; # open final output file
@list=;
%seen = ();
foreach $item (@list) {
push (@uniq, $item) unless $seen{$item}++;
}
my @sorted = sort {lc($a) cmp lc($b) } @uniq;
print OUTFILE @sorted; #print unique and sorted records to the outfile
close OUTFILE;


In this example, the result would be a file “finalerrs.txt” with the following in it:

foo
bar
foo bar
bar foo

I know that this particular script is only really suited for the log file that I am dealing with but portions of it and the concepts are certainly reusable.

Previously I mentioned that I use logcheck to peruse the system logs and e-mail me anomalies. It often sends me notes about odd occurrences. Here’s a note from today:


System Events
=-=-=-=-=-=-=
Feb 22 11:34:01 my-pc nss_wins[13110]: gethostby*.getanswer: asked for “btack.info.nyud.net IN AAAA”, got type “39″
Feb 22 11:34:01 my-pc nss_wins[13110]: gethostby*.getanswer: asked for “btack.info.nyud.net IN AAAA”, got type “39″
Feb 22 11:34:01 my-pc nss_wins[13110]: gethostby*.getanswer: asked for “btack.info.nyud.net IN A”, got type “39″
Feb 22 12:02:09 my-pc gconfd (root-23336): starting (version 2.16.0), pid 23336 user ‘root’
Feb 22 12:02:09 my-pc gconfd (root-23336): Resolved address “xml:readonly:/etc/gconf/gconf.xml.mandatory” to a read-only configuration source at position 0
Feb 22 12:02:09 my-pc gconfd (root-23336): Resolved address “xml:readwrite:/root/.gconf” to a writable configuration source at position 1
Feb 22 12:02:09 my-pc gconfd (root-23336): Resolved address “xml:readonly:/etc/gconf/gconf.xml.defaults” to a read-only configuration source at position 2
Feb 22 12:02:09 my-pc gconfd (root-23336): Resolved address “xml:readonly:/var/lib/gconf/debian.defaults” to a read-only configuration source at position 3
Feb 22 12:02:09 my-pc gconfd (root-23336): Resolved address “xml:readonly:/var/lib/gconf/defaults” to a read-only configuration source at position 4


Two things of interest. First is the wins note about gethostby*.getanswer. It seems that those messages are happening as a result of Azureus running. Somehow there are unrecognized requests being set. I’m not really happy about it, but searching Google didn’t produce much useful information. Secondly is notes regarding gconfd. Honestly I have NO IDEA what is causing that but I get it just about every day. It doesn’t appear to be an error but I’d like to make it stop.

Any hints/help is appreciated.

Locate is a tool that I use almost every day. It’s a great way to find files that have gone missing or to find out where a new package was installed. In a previous post I discussed using Tilda and Yakuake. When I installed them only Yakuake ended up with a menu item. Tilda didn’t end up visible anywhere. Even though Yakuake did end up in the menu I had some difficulty configuring it as the default keybinding is F12 which I had previously bound to something else. So, in order to attempt to reconfigure Yakuake and find/configure Tilda I went to a terminal and used locate.

First, being that I had just installed new packages I needed to update the locate database. Easy to do - simply enter:

sudo udpatedb

Which then updates the locate database with any new items. Then, I searched for yakuake by typing:

locate yakuake

Which returned:


/var/cache/apt/archives/yakuake_2.7.5-0ubuntu1_amd64.deb
/var/lib/dpkg/info/yakuake.postinst
/var/lib/dpkg/info/yakuake.list
/var/lib/dpkg/info/yakuake.postrm
/var/lib/dpkg/info/yakuake.md5sums
/home/me/.kde/share/config/yakuakerc
/usr/bin/yakuake
/usr/share/app-install/desktop/yakuake.desktop
/usr/share/app-install/icons/yakuake.png
/usr/share/applications/kde/yakuake.desktop
/usr/share/apps/yakuake
/usr/share/apps/yakuake/default
/usr/share/apps/yakuake/default/tabs.skin
/usr/share/apps/yakuake/default/title
/usr/share/apps/yakuake/default/title/config_down.png
/usr/share/apps/yakuake/default/title/back.png
/usr/share/apps/yakuake/default/title/config_over.png
/usr/share/apps/yakuake/default/title/config_up.png
/usr/share/apps/yakuake/default/title/focus_down.png
/usr/share/apps/yakuake/default/title/focus_over.png
/usr/share/apps/yakuake/default/title/focus_up.png
/usr/share/apps/yakuake/default/title/left.png
/usr/share/apps/yakuake/default/title/quit_down.png
/usr/share/apps/yakuake/default/title/quit_over.png
/usr/share/apps/yakuake/default/title/quit_up.png
/usr/share/apps/yakuake/default/title/right.png
/usr/share/apps/yakuake/default/tabs
/usr/share/apps/yakuake/default/tabs/right_corner.png
/usr/share/apps/yakuake/default/tabs/back_image.png
/usr/share/apps/yakuake/default/tabs/left_corner.png
/usr/share/apps/yakuake/default/tabs/minus_down.png
/usr/share/apps/yakuake/default/tabs/minus_over.png
/usr/share/apps/yakuake/default/tabs/minus_up.png
/usr/share/apps/yakuake/default/tabs/plus_down.png
/usr/share/apps/yakuake/default/tabs/plus_over.png
/usr/share/apps/yakuake/default/tabs/plus.png
/usr/share/apps/yakuake/default/tabs/plus_up.png
/usr/share/apps/yakuake/default/tabs/selected_back.png
/usr/share/apps/yakuake/default/tabs/selected_left.png
/usr/share/apps/yakuake/default/tabs/selected_right.png
/usr/share/apps/yakuake/default/tabs/unselected_back.png
/usr/share/apps/yakuake/default/tabs/separator.png
/usr/share/apps/yakuake/default/manual.readme
/usr/share/apps/yakuake/default/title.skin
/usr/share/apps/yakuake/default/install.sh
/usr/share/doc/yakuake
/usr/share/doc/yakuake/copyright
/usr/share/doc/yakuake/README
/usr/share/doc/yakuake/TODO
/usr/share/doc/yakuake/AUTHORS
/usr/share/doc/yakuake/changelog.Debian.gz
/usr/share/doc/yakuake/changelog.gz
/usr/share/icons/hicolor/16×16/apps/yakuake.png
/usr/share/icons/hicolor/32×32/apps/yakuake.png
/usr/share/locale/da/LC_MESSAGES/yakuake.mo
/usr/share/locale/de/LC_MESSAGES/yakuake.mo
/usr/share/locale/it/LC_MESSAGES/yakuake.mo
/usr/share/locale/nl/LC_MESSAGES/yakuake.mo
/usr/share/locale/pt/LC_MESSAGES/yakuake.mo
/usr/share/locale/pt_BR/LC_MESSAGES/yakuake.mo
/usr/share/locale/ru/LC_MESSAGES/yakuake.mo
/usr/share/locale/sv/LC_MESSAGES/yakuake.mo
/usr/share/locale/tr/LC_MESSAGES/yakuake.mo
/usr/share/man/man1/yakuake.1.gz
/usr/share/menu/yakuake

In this list of files I found two things that were useful - First, there is a man page entry available (/usr/share/man/man1/yakuake.1.gz) which meant that I could type “man yakuake” and get some help. Secondly, I saw that there was a file yakuakerc in my home directory. This was the configuration file.

Amongst other things using locate helped me to get a new program running the way I wanted it to. Locate does have a bunch of options including the ability to limit the amount of results, enter a regex, etc.