ripping compact discs to audio files

Perhaps it is not necessary for another article about how to convert cd’s to a format that can be used on our mp3 or ogg music players.  I have used k3b program for extracting ogg files in the past.

I suspect that there are also a number of similar programs on the windows side that are just as easy to use.  I would have used this method but I had a small stack of disc’s that needed to be ripped and I would prefer to just kick off a command line to do the task.

There already exists a fabulous script called “abcde” for doing most of this work.  This script extracts the audio data as wav data and then uses the desired codec for converting this wav data into the new format (ie mp3 or ogg).

The abcde script also downloads the names of the songs as well as the album art (for abcde version 2.7 and higher).

I knocked together the following script which will extract either ogg or mp3 files and rename the temporary directory into the name of the album.  By default, this script will call abcde and convert the ripped files to ogg format.  If this script is called with mp3 it will then rip the files and convert them to mp3.

FORMAT=ogg
if [ $# -eq 1 ]
then
   FORMAT=$1
fi

TMPDIR=abcde.`cd-discid | sed 's/ .*//'`

abcde -N -g -d /dev/sr0 -a read,encode,tag,getalbumart,clean -o $FORMAT -x

cd $TMPDIR

CNT=$((`cat cddbchoices | wc -l` - 2))
IDX=1

while [ $CNT -gt 1 ]
do
  TRACK=`tail -$CNT cddbchoices | head -1`
  TRACK=`echo $TRACK | sed 's/^..: //'`
  TRACK=`echo $TRACK | sed 's/^.: //'`

  TRACK=`echo $TRACK | sed 's/ /_/g' | sed 's/[^a-zA-Z0-9_-]//g'`

  FILENAME=$IDX
  if [ $IDX -lt 10 ]
  then
    FILENAME=0$IDX
  fi

  DEST_FILENAME="$FILENAME - $TRACK.$FORMAT"
  SRC_FILENAME=track${IDX}.$FORMAT

  if [ -f $SRC_FILENAME ]
  then
     echo mv "$SRC_FILENAME" "$DEST_FILENAME"
     mv "$SRC_FILENAME" "$DEST_FILENAME"
  fi

  CNT=$(($CNT - 1))
  IDX=$(($IDX + 1))
CNT=1
done

ALBUM=`head -2 cddbchoices | tail -1 | sed 's/ /_/g' | sed 's/[^a-zA-Z0-9_]//g'`
ALBUM=`echo $ALBUM | sed 's/^_//g'`
ALBUM=`echo $ALBUM | sed 's/_$//g'`

cd ..
mv $TMPDIR $ALBUM
echo $ALBUM

To be honest all of this scripting should be unnecessary.  Logic exists for doing all of this for you.  Simply change the configuration file to select the output format.

The standard configuration file /etc/abcde.conf allows for a lot of individual configuration.  Simply uncomment the OUTPUTFORMAT variable and change it to fit your needs.

#OUTPUTFORMAT='${ARTISTFILE}-${ALBUMFILE}/${TRACKNUM}.${TRACKFILE}'

For reasons unknown, this didn’t work for me.  Nor did creating a configuration file .abcde.conf in my home directory.

There is a lot information on the internet about how to use or configure and use this utility.

http://www.andrews-corner.org/linux/abcde/index.html

https://www.mankier.com/1/abcde

One of the more interesting things about the abcde script is that you can create multiple output files at the same time.  It is possible to create both mp3 and ogg files at the same time.  If both types of files are required then this is actually the fastest way to convert into both types of data.

Simply create a list of all formats when running the script.

-o mpg,ogg

Note: After writing this small script I discovered it didn’t work.  Well, it worked just fine for CD’s that only had one IMDB entry.  Even not using my script, the abcde script is fabulously useful.

Posted in programming | Tagged , | Leave a comment

software development the easy way …

Is there really an easy way for software development?  Over time there has been an increasing number of different attempts at providing the silver bullet to faster better development.  The solutions have ranged from the tools such as adding syntax highlighting to editors, code generation and even fancy integrated development environments.

In one sense, software development is not a boring field as there is a constant addition of new languages and tools.

Back in the day, I was not terribly interested in designing web pages so I kept focused on developments in more traditional computing.  The goal to improve the lot of internet people everywhere continued without me.  It started with the creation of javascript (in 95) and later the creation of jquery(06) actually did make the task of javascript and HTML development easier.

The syntax for creating a table is actually really simple.  You can create a header and footer, if you are really clever you can use the footer to display totals.

<table id="example" class="display" style="width:100%">
 <thead>
  <tr>
    <th>Name</th> 
    <th>Position</th> 
    <th>Start date</th> 
    <th>End date</th> 
    <th>days in office</th> 
 </tr>
 </thead>
 <tfoot>
  <tr>
    <th>Name</th> 
    <th>Position</th> 
    <th>Start date</th>
    <th>End date</th> 
    <th>days in office</th> 
 </tr>
 </tfoot>

But the result of the simple html table is really really boring as well as feature poor.

Yet, there has been additional work to help make the process easier – especially when working with tables.  This solution comes in the form of additional javascript libraries.  These datatables  libraries (yup there are more than one) are available from datatables.net.

They are actually like magic.  With simply adding the tiniest amount of code you can get fully functional tables that look good and are very feature rich.

All that is necessary is the addition of the following table definition.

<script>
   $(document).ready(function() {
      $('#example').DataTable( {
      });
   } );
</script>

It is possible to define how many lines are displayed at one time, special buttons for exporting excel or csv data as well as defining the location of the buttons.

<script>
   $(document).ready(function() {
      $('#example').DataTable( {
         "lengthMenu": [[7, 14, 28, -1], [7, 14, 28, "All"]]
         ,"dom": '<"top"l>rt<"bottom"Bip>'
         ,buttons: [
         'copyHtml5','excelHtml5','pdfHtml5', 'csvHtml5', 'print'
      ]
      });
   } );
</script>

 

This is just a tiny piece of what functionality is possible using this datatables library.  Displaying a table is pretty tame.  It is possible to make the table editable.  This actually makes it possible to create “proper” applications inside your browser.

I am hoping to do a few blog postings showing off how powerful this library is.

Posted in programming | Leave a comment

AWS – CLI

I am rather a command line person myself so in retrospect I was a bit surprised that I didn’t leap to the AWSCLI command line program.  This tool lets you do from your shell what you might have had to do with various AWS screens.

Not quite intuitive

The AWS documentation makes it look pretty simple.  Just type the following command.

pip install awscli --upgrade --user

This actually does make the assumption that you already have python and pip installed on your machine.

I am not a python developer but it sounded like this would all be a downhill slide as pip actually already comes installed with python since 3.4.

I did have some false starts for a lot of reasons.  The first is that if you are installing this on windows you should take the windows install package.  It may have been an error but I tried the 64 bit embedded version which did not seem to have pip, nor did the 32 bit embedded version. I was a bit disappointed but i then tried a windows install that still didn’t quite work.

This was probably my fault as I assumed that this was some sort command to be interpreted by the python interpreter – nope. If you do that, you will end up watching youtube videos and reading webpages that basically lead you in the wrong direction.

Eventually I found the correct web page that pointed out that the pip command is essentially an external program that needs to be run, just as seen above.

However, I found this out after reverting to python 3.5.4 and making sure that both the installed directory as well as its Scripts sub-directory was in the path.  Yup, that makes everything work a lot easier.  But I am not really willing to install the more up to date 3.6.3 python – well at the moment.

AWSCLI

Well, the aws command is exactly like the pip command, you simply run this from the command line and then everything works just fine, well, sorta.

You need to first do one little configuration and amazon again does step you through it.  Of course you do need to either setup a user in the IAM or use an existing user with the necessary privilages.

aws configure

You will be asked four questions and most of these answers are already available from the IAM. What I thought would be the tricky question was what is my aws_access_key_id and the aws_secret_access_key.  The other questions seem pretty tame, what format do you want you output and what region are you in.

I got the hard part right but somehow got my region wrong.  My attempts to manually fix these two files (.aws/credentialls and .aws/config) only seemed to make things worse.  My example command seemed simple enough.

aws ec2 describe-instances

I either ended up having some fairly odd problems with the explanation that my profile could not be found or the program would hang. In the end, I could not manuallly fix these files so I simply deleted the two configuration files and re-ran the configure with a minor change.

aws configure --profile myuser

This worked a lot better and then my test program almost worked.  I realized that I was running my test using a different region (Frankfurt) versus what I should have been US East (Ohio).

I think that I was saved by that mistake.  Once I changed my region I started to get a much more reasonable error.

Invalid endpoint: https://ec2.us east.amazonaws.com

Yes, I made a pretty bad guess at my region but it did help to yield this very badly formatted URL and google came to my assistance.  I turned out that I was nowhere near correct with my region name

(quick side rant)

The amazon documentation has a hyperlink everywhere when you are reading about almost any topic.  It is almost impossible to go through from start to end on a topic without visiting a dozen other document links.  Guess how many links about regions they had on this page.  Yup, none.

Anyway, Amazon also has a list of all of their regions which was very helpful.  I changed my region to one from this list and then my aws test command worked.

I never got a chance to get back to my studies of Aws Lambda, but I will get to that in the morning.

 

Python

https://www.python.org/downloads/release/python-354/

Posted in Setup From Scratch | Tagged , , | Leave a comment

Simplicity in development or syntactic sugar

Computing is a pretty big field so it is not always possible to stay on top of every little (or big) framework or enhancement that comes down the line.  I am happy to be involved in software development so I can get a quick heads up when something really cool comes down the line.

Unfortunately I found out about a new thing (for me) in the worst possible way.  I had just joined a new project and was coming up to speed on some of the technologies that was different from my last project.

  • Java (that isn’t end of life)
  • Git
  • Spring Toolset
  • Hibernate
  • New data model
  • Different business area

In short there were no limit to things that I needed to spend my time coming up to speed on when I did the fateful command.

git pull

Yup, all I did was to refresh my workspace with the most recent code from our server.

Oh, sure, the code compiled once I installed the tool Lombok into my IDE and added it to my classpath.  The surprise was that the code that compiled no longer ran.

The culprit

What does Lombok do?  It is actually a really interesting use of the annotations that were added all those years ago in java 1.5. The purpose of Lombok it to reduce the amount of boilerplate code that needs to be manually entered that is added to all (well a lot) of projects by virtually everyone.  Not only that, it is possible to use the methods that Lombok creates even though they are not physically stored in the source code file.

Basically, this allows you to see and focus on the really interesting parts of your code that add value and not be distracted by some of the less interesting boilerplate code that you also added.

I am not going to cover everything that Lombok supports as that is all nicely documented.  I will just focus on a few of the annotations.

  • @Getter
  • @Setter
  • @Data

The first two actually do pretty much what they sound like.  If you add this annotation before your variable then Lombok will automatically generate the getter and setter for you.  Not only that but it will add these methods to the outline tab in your editor.

On the left is my own code and on the right is the outline showing the methods and variables for that class.  If you look closely enough you can see that no “get” method has been created for the iq or age variables and no “get” or “set” have been created for the internalvalue variable.

This code is obviously much smaller and would allow the programmer to be more focused on the real logic.   So this new project actually does add value.

If the Java object is really just an abstract object then there is an easier way than adding all of these “@Setter” and “@Getter” annotations it is possible to add a single @Data annotation.

@NoArgsConstructor
public <strong>@Data</strong> class ComplicatedObject 
{
	private String First;
	private String Last;
	private int age;
	private String Address;
	private String City;
	private String State;
	private int zipcode;
	private int iq;
	private int internalvalue;

	public static void main(String[] args) 
	{
		ComplicatedObject person = new ComplicatedObject();
		
		person.setFirst("Max");
		person.setLast("Musterman");
		person.setAddress("123 Main street");
		person.setCity("Smalltown");
		person.setState("IA");
		person.setZipcode(12345);
		person.setIq(125);
		
		System.out.println(person.toString());
	}
}

Installation

The installation process is actually super simple.  Simply run the lombok.jar file and it will open up a small installer.  First it searches for all installed IDE’s and presents them as a list of locations to install lombok.

Once the IDE has been selected it takes hardly a second to finish the setup of lombok.  In order to use it you simply need to restart the IDE and add the lombok.jar file to any projects that you wish to use this new functionality.

 

The good

The software both installs very simply and is limited to a single jar file. This makes it really easy to add to a project and to later remove if necessary.

 

The bad

I don’t really have any “hard proof” that Lombok itself was problematic to my work project.  I downloaded lombok and did a small test at home.  I didn’t have any difficulties and the functionality is pretty neat.

Thus there is nothing bad to report but this functionality is mainly to simplify the definitions of getters and setters for some plain old java class that is just an abstract data type.

If this is the extent of what is necessary you could go back to the olden days and just have an object and address each field directly.

package com.company.de;

public class ComplicatedObject2 
{
	String First;
	String Last;
	int age;
	String Address;
	String City;
	String State;
	int zipcode;
	int iq;

	public String toString()
	{
		return First + " " + Last + " " + Address + " " + City + " " + State + " " + zipcode + " " + iq;
	}
	public static void main(String[] args) 
	{
		ComplicatedObject2 person = new ComplicatedObject2();

		person.First = "Max";
		person.Last = "Musterman";
		person.Address = "123 Main street";
		person.City = "Smalltown";
		person.State = "IA";
		person.zipcode = 12345;
		person.iq = 125;

		System.out.println(person.toString());
	}
}

I am not saying you should do this but this is essentially what adding all getters and setters to a simple java class does.

The ugly

We did have a few problems with Lombok in our project but I think that this might have been due to a misconfiguration.  We are still looking into that.

Posted in programming | Tagged | Comments Off on Simplicity in development or syntactic sugar

AWS – Fun with S3 Buckets

My last post was discussing the S3 bucket which can be used as more than a simple dumb disk for storing family photos or videos.

One of the more interesting things that can be done with an S3 bucket is to use it as a web server.

Web Server

Don’t get overly excited.  Indeed S3 can be configured to perform as a web server but it is not a full fledged server that you might be used to.  That is to say that it will serve up static content but not run server side scripts.

Once you have a S3 bucket setup it is trivial to create the static web site. Simply go into the properties for the bucket and enable “static website hosting” and provide the actual html script or scripts to be used.

However, just like in my previous article AWS – S3 buckets permissions are the tiny bug-a-boo that could cause some behavior problems in your website.

This simple example uses exactly one html page and two graphic files but because the permissions for one of the graphic files was not set to public the contents are not displayed when someone visits the web page.  This is a common permission problem that needs to be kept in mind when creating static web pages on S3.

I am not a html programmer but I would suppose that the code (below) is virtually identical to what you might see in your normal website.

I would have loved to put the actual code here but was having formatting problems.  The code is actually fairly simple html, if you are really curious you can download it here.

Download files

I am not expecting any problems with the Amazon S3 bucket but this web site does have one little flaw.  If for some reason this S3 service was not available, it might be difficult to move it to another bucket to run.  The reason for this is that the references to the various resources are using the actual path to the regional bucket (s3.us-east-2.amazonaws.com) and not something more generic like supercoolwebsite.com.  However, this could probably be corrected by using route 53 and your own domain – it would still leave you to design a highly tolerant solution.  You might want to replicate your files to another bucket in another region just in case.

 

Posted in Setup From Scratch | Tagged , , | Comments Off on AWS – Fun with S3 Buckets

AWS – S3 and Simple Notifications

Individual Amazon web services are actually quite amazing all on their own but they are really neat when you start to use them together.

There probably isn’t anyone who in their life hasn’t had a sdcard or hard disk fill up who just needed a small bit of extra space to complete their action.  Of course in that situation in the real world you have to stop what you are doing and free up (ie delete something) so you can continue.

Amazon S3 doesn’t have that problem.  If you have the budget, you can keep adding data to it until you are satisfied you are finished.  Sure you have to keep on eye on the costs but that is a completely separate issue.

However, it isn’t any one service that makes AWS so cool as much as how they all seem work together.  One such example of that is the ability to send notifications based on some event that has occurred.  This might be an event you setup to monitor your costs or it might be used as a trigger to an application.

In my case, I am going to demonstrate an email notification going out if a file ends up in a S3 bucket.  If you were some sort of picture site, this might instead be triggering a lambda function to create thumbnails or perhaps this information might be put into a queue for some other processing (or both).

In my previous article, AWS – Simple Notification Service, I discussed everything that needs to be done in order to setup a Simple Notification Service topic but in this blog entry I will actually use put the AWS SNS to use.

In addition to sending out messages, I will also add replication to a s3 bucket.  This will provide a cool way to put data into one bucket and watch it being replicated to the second bucket.  This replication is pretty much as the name says.  If you add data to the source bucket, it will be added to the destination bucket but if you delete it from the source bucket it will also be deleted from the destination bucket.

Replication

The replication is actually quite simple. The setup is essentially from where do you want to copy what, and where should it be placed.  Replication setup is available from the management tab on the bucket.

source

It is also possible, actually recommended that you check the checkbox to encrypt the objects that are replicated.

Destination

The replicated box must be in another region.  It is also possible to change the storage class in the destination.  By selecting a class with a different type it may be possible to reduce the cost of the storage in this additional location.

Permissions

I did not create this IAM rule, nor name myself – I had it created for me.  This is necessary in order to allow the proper read and writing for these two S3 buckets.

Review

Once the replication setup has been saved then new data that is saved or deleted from the source bucket will be replicated a short while later in the destination bucket.

Notifications

The actual notifications on the S3 bucket are trivial to setup.  Simply go to the properties of the bucket and setup an “event”.

Delete event setup

It is possible to be notified depending on which type of action occurred, ranging from simple create or even just simple HTTP commands.

However, it might not be quite that simple as it appears on first look.

You might get an error if the permissions are not properly setup in the simple notification service.

Original notification setup

 

Proper – notification setup

Once you setup the proper permissions in the simple notification setup you can then create and save other rules as long as they do not overlap in functionality (ie two different types of create)

create event setup

Posted in Setup From Scratch | Tagged , , | Comments Off on AWS – S3 and Simple Notifications

More posts to love

Just in case you haven’t noticed it yet the Paranoid professor has doubled our staff. My youngest wanted to balance the computing and other world news with news of sport.

We will still cover all the usual technical positions but now we will see a few more posts on the world of soccer.

Posted in Soapbox | Comments Off on More posts to love

Belgium vs. Japan

Hi guys did you guys watch the game? I did. The ball was more on the Japanese side for the first 20. minutes.Japan scored a goal in the 48. minute and in the 52.minute. And Belgium in the 69. minute. and in the 73.minute and in the additional time. And Belgium won.

 

Posted in Sports | Comments Off on Belgium vs. Japan

Sweden vs. Switzerland

Sweden is OK. Switzerland I don’t now. For who where you? I was for nobody. For the first 50 minutes the ball was mostly on the side of the Swiss. Sweden fouls a lot. Sweden scored in the 66. minute.sweden wins 1-0.

Posted in Sports | Comments Off on Sweden vs. Switzerland

Columbia vs. England

Did you guys watch it I did?Columbia fouls a lot.England scored a penalty.Columbia got a goal in the additional time.England won 4-5.yeah.

Posted in Sports | Comments Off on Columbia vs. England