AWS – Cloud computing with Amazon

I recently started looking at cloud computing by looking at OpenStack.  OpenStack allows you to take a lot of common hardware and create your own cloud server on your own hardware.  Once the software is setup it is easy for the user to setup his or her own little server or network.

The only problem is that I don’t have a bunch of intel I7 multi core servers full with ram sitting around for creating such a cloud.  I did have a five year old AMD 8 core server but unfortunately creating your own cloud server is very resource intensive.  Much more intensive than my poor old computer could handle.

I actually didn’t finish the tutorials because working with OpenStack was too slow with the equipment that I had available to me – yet it wasn’t their fault.  I wanted to do more with cloud computing so I decided to give Amazon Web Services a chance.  My thought was that Amazon has some amazing infrastructure around the world and so I should be able to use theirs without waiting an excessive amount of time.

Wikapedia says this about Cloud computing

Cloud computing is an information technology (IT) paradigm, a model for enabling ubiquitous access to shared pools of configurable resources (such as computer networks, servers, storage, applications and services), which can be rapidly provisioned with minimal management effort, often over the Internet.

Cloud computing

Cloud computing is a pretty big topic.  Cloud computing is almost like virtual computing except that clould computing takes a disparate resources and makes them available.  However, in addition to making them available it is done so by letting the user allocate what he or she wants rather than wait for IT to create the VM.

Yet that isn’t really cloud computing.  When a user can allocate their own “servers” from a pool of resources that is really only bordering on the edge of cloud computing as that is simply virtual computing – ie. take a server and run it on virtual hardware.

Clould computing is one step further.  It allows you to configure the setup so that multiple servers can be automatically brought on line up as the demand requires it but they can also be automatically shutdown when they are no longer needed. Additionally cloud computing can be configured to be smart enough to replace servers that are no longer responding and or even virtualize away the networking away from the physical hardware.

To create your own cloud based VM you need a network (VPC) and the computer (EC2).  In the next series articles I will create the network and then a virtual machines to run on it.  I will also discuss about setting up auto scaling and adding a load balancer as well as touching on some of the interesting services that Amazon web services offers.

Posted in Setup From Scratch | Tagged , , | Comments Off on AWS – Cloud computing with Amazon

Customer advocacy or just smarting off without consequences

Don’t ever dare Murphy’s law.

I guess I did put the hit out on our washing machine by bragging how old it was.  Within weeks, it broke. We have children and cannot be without a wash machine for more than a few days.  The internet came to the rescue.

The internet is a great place, you can research and find the best deals all without changing out of your pajamas.  We went to the web site of a large home merchandise and it really didn’t take too long to find a replacement.  It was ordered and we were informed it could be delivered by the end of the week – awesome.

The company called and made an appointment for when we they could install it but it was when they showed up that things started to fall apart.  They delivered the washing machine but we needed a safety plate for the top of the machine as it was to be built under the counter in the kitchen.

No plate, no install.  This was a problem as we paid for the installation but also because we generate a lot of dirty laundry.  The delivery guys were professionals and they knew what could be safely done and what not.

It was looking like we would have a wash machine sitting in the middle of the room for days until we could sort this out.  I guess that created an extra load of stress as I could feel it over the phone line as I was explaining everything to her.  A short time later, my wife called back and reminded me that the old machine was the same brand and perhaps we could reuse that plate.

We were lucky.  I spoke with the installation guys who were game enough but while we were fooling around trying to install this my spouse called the support line. You really wouldn’t believe what happened – I didn’t.

The squeeky wheel gets the grease … kinda

Due to an amazing set of events I thought I would try and enlighten the suits in the C-suite with just how much glory they were covered in.  Who would be the best person to contact?  Store manager – nah, doesn’t really get the word out.  The CEO of the store chain – close, but I was upset and my wife couldn’t hardly string together words.  I finally found the correct person – I wrote to the CEO of the holding company for both that chain and another similarly large one.  I am not certain but I think those two chains is the majority of home items and electronics in all of Germany.

It was nice to receive a response from their public relations group but I don’t really believe that they are all that worried by a single dissatisfied customer.  The 30 Euros gift certificate that they sent was a nice gesture, small but nice.  It represented less than 4% of the total purchase, I am not sure that they were all that concerned.

The only silver lining was that I can work from home so in this particular case I didn’t have to worry about taking off more time from work to wait for a delivery of a wash machine.

My Letter

April 2017

Dear <big boss man>

I have just purchased a washing machine from Big Box Store and I am having a hard time trying to contain my dissatisfaction.

I have gone to your webpage which seemed to be well written and offered quite a bit of assistance when trying to compare the various models against each other. This machine was replacing one that was already built into our kitchen and so we selected that we wanted the old machine to be taken away and the new machine was to be installed by your firm or your designated workers.

Your web site did not bother to mention that additional hardware would be necessary to safely install this machine. We found this out when the machine was delivered to our house by which time it was too late to correct. The Hermes people who did the delivery were totally knowledgable, polite and helpful in this problem that was not of their making.

In my opinion this situation would have been bad luck if it were not for two factors. The lesser is that Big Box Store is a large multinational corporation earning hundreds of millions in yearly profits selling these types of devices for years – you should have known better. I was almost willing to consider this an oversight until my wife called up to discuss this with your support personnel.

How she was handled by your staff, Rude Customer Service Dude, is probably textbook reading. First discuss the problem and then when it starts to get hard to explain why you cannot fulfill your end of the purchase decide you won’t speak with the client. It turns out that my name was on the purchase order so your man decided he could stop speaking with my wife in the middle of the conversation. It is a rather curious type of customer service that your firm practices. Perhaps also some additional guidance would be helpful for your support staff who are currently only tarnishing your companies image.

You cannot imagine her surprise to find out your employee Rude Customer Service Dude not even willing to discuss this problem with her. The reason she was given is that she did not purchase the machine.

I guess this is part of new policy of Big Box Store is pursuing to convince people not to purchase things from their web site. This does make a small amount of sense if you are trying to limit your sales opportunities strictly to stores that must both display the merchandise and pay sales people to attempt to sell this merchandise.

It would be better to simply get your software developers to add an additional optional item that needs to be purchased when the user decides on “unterbaufähig” devices like washers as well as dealing with any similar situations.

I will most likely be telling this story for years. I would like to know if this was a simple oversight that is being corrected and how? Failing any feedback would force me to consider that either this is proof that your company cannot even write a simple web site or perhaps that Big Box Store is a good example of an old bricks and mortar company that will be replaced with by some cyber company in the coming years.

Sincerely,

Max Mustermann

Their response

Dear Mr. Mustermann, 

We are very sorry to hear about your unsatisfactory purchasing experience.

First of all we want to ensure you that this does not correspond to our corporate philosophy.

We can understand your anger regarding the missing information about the necessary installation accessory. Of course we informed our specific department to find any solution for a better understanding regarding this products. Furthermore we want to apologize for the unfriendly and inappropriate behavior of the Hotline employee. We also will clarify this situation to prevent such customer experiences for the future.

As compensation for this inconveniences we will send you a gift card worth 30 €, which you can use in our Onlineshop as well as in our stores.

We’re looking forward to welcome you again and hope we can convince you of the contrary to your bad experience.

Yours sincerely

<big boss man>

The real response

I have had more than six months to cool down from this experience but I am feeling my blood pressure again going up.  It is not simply because I went back to the infamous website and this particular oversight does not seem to be fixed – but that is part of it.

No the real reason is that while speaking with my wife she reminded me of a very similar story.  The problem was not a washing machine but a dishwasher but it was the exact same issue.  This time it was the other large home and electronics store that made the mistake.

If that weren’t amazing enough, they made the mistake with my wife (girlfriend at the time) and I was at work about 10 years back.

I kinda wonder just how often this has happened in the last decade …

Posted in Soapbox | Tagged | Comments Off on Customer advocacy or just smarting off without consequences

using docker

Although docker does have quite a few command options a small subset is all you need for general usage.  The number of individual tasks you need is limited the following.

  • download an image
  • run the image
  • remove the image
  • build a new image
  • monitoring images

Docker has a store that contains both their official containers as well as community supplied containers.  The official containers contain a lot of very large well known software programs.

  • Microsoft SQL server
  • Oracle database
  • Oracle Java 8
  • WordPress
  • Tomcat
  • Owncloud
  • gcc

The store is important as it contains a list of the containers that are available and what their name is, however, there is one more really important thing in the store.  Each of the containers that are listed have the name of the container to be downloaded but more importantly they contain helpful notes.

Each container shows the command necessary for pulling down the image to your machine.  This is important but other details are listed that might be helpful when actually running the container.

This might be how to run the image, how to extend the existing container to make a new container, descriptions or perhaps the license for this software.

The good news is that all of the entries in the store show the command for retrieving that specific image.

The only inconvenience is that you need elevated privileges to run the docker commands.  This is actually as easy as using sudo.  Easy but still somewhat inconvenient.  Yet this too can be overcome with the tiniest of changes. Quite some time back Docker was changed to have the group docker for the docker daemon.  The daemon has ownership of the read/writable for that socket.

Thus the solution is to simply add your user to the docker group.  This is either done when creating your user or if your user already exists simply use the usermod command to add the user to the group.  This is explained in a really good post at the howtogeek website.

General usage

One of the challenges for production environments as well development environments is to ensure that the exact identical setup exists.  The neat thing about docker is that you can pull down all the specific tools by version. You simply provide a host and pull down your containers. If for any reason you need a different version or multiple versions you can run them on the same machine as a container regardless of if this would be possible as a simple install.

Docker has a lot of different command options which can be used to monitor the containers but the most important one is actually running containers. Simply use the run parameter and then the container is run as a normal process.  This actually isn’t very different from some installed program running but the containerization makes this a bit different.

Containers are kept segregated from the host and from each other which means that they cannot actually do very much unless you allow the container to interact with system.  Docker was not only done really well but also very granular.  It is possible to allow the container to access a file, directory or even a port.

The most simple container does not need any input to the outside world, but those types of jobs would be pretty rare.  An example of this is the hello-world container provided by docker.

This example doesn’t save anything to the file system nor read any input from a port.

A more common use case would be to either access a directory or to map a configuration file from the host into a container.  This is done by passing in the mapping or mappings by using the -v argument.

-v <host source directory>:<container dest directory>

This can be done with either a directory or just a simple file – in either case the syntax is the same.  The same format is used for mapping a port from the host machine to the container.

-p <host port>:<container port>

It is possible to also pass through environment variables to the container as well.

-e <host variable>:<container variable>

Interactive container

In my opinion one of the neatest uses of containers is to run a graphical program like Eclipse. However as the container itself is not persistent Eclipse in a container is not very useful unless it is exposed to the host.

In this case a directory from the host is mapped to the container workspace directory. Simply by mapping the local directory to the container makes Eclipse behave exactly as if it was a normal install on the host machine.

Below is a small script to run the container and map a few directories and an environment variable into the container.  This script below is fairly similar to the one in the docker store for this container.

cd ~cdock
mkdir `pwd`/.eclipse-docker >/dev/null 2>&1

myip=`hostname -I | awk -e '{ print ($1)}'`
xhost + $myip

export DISPLAY=$ip:0

CMD="docker run -d --rm -e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v `pwd`/.eclipse-docker:/home/developer \
-v /home/cdock/workspace:/home/developer/workspace \
fgrehm/eclipse:v4.4.1 "

$CMD 

It is not alwtays obvious what host resources need to be mapped into a container.  This particular container is just another example why it is useful to look at the notes that are in the store.

The eclipse container is a great example of an user interactive task but not all containers have a gui.  It is possible to run a program interactively but that program could also be bash.

docker run -i -t fgrehm/eclipse:v4.4.1 /bin/bash

Monitoring Docker

There are two different tasks to monitor.  The first is what containers are downloaded to your computer.

This will however only show the images not which containers are actually running.  The “ps” argument, much like the Linux ps command, is used to see which containers are actually running.

The tasks that are running are probably a daemon or other process that runs in the background.  Stopping the process is actually as simple as starting it.  Simply ask docker to stop the process for the container id.

docker stop ae527eb50499

After stopping a container it will no longer show up in the list of running tasks.  However, if you ask right, you can see a list of the containers that have already been stopped.

It is possible to clean up this list by removing the stopped processes.

If you were limited to pre-created docker containers from the docker store this would still be a very powerful tool, but it is actually possible to either create your own containers or even extend existing containers.

I will talk about that in my next article on docker.

Posted in programming | Tagged | Comments Off on using docker

lightweight virtualization – docker

Virtual machines can allow you better utilize the resources or make it possible to run multiple incompatible versions of the same software or even different operating systems. Virtual machines sound like the perfect solution, instead of a server that runs only database server or web server you can have a single server that runs both of them and more.

There is actually one little downside to all of these virtual machines.  Each one is actually running an entire operating system.  This takes up more ram and more disk space than is strictly necessary.  A single windows 7 installation could take 10gb disk space but four could take 40gb or more.

It doesn’t take a mathematics genius to see this downside to just using virtual machines fairly clearly – the solution was to bring down virtualization to the application level.

Containers

This “application virtualization” was created as containers.  These containers contain not only the application but it also bundles other required resources.  These containers then are separated from each other in the operating system through the implementation of both namespaces and cgroup‘s inside the Linux kernel.

The difference between container technology and virtual machines is that the containers use the host operating system for the application while the virtual machine has its own operating system.  Thus the resources used by a container should always be less than any virtual machine.

Docker

Linux did create its own container technology (LXC/LXD) but for most people a more recognizable container solution is Docker. Docker is a friendly command line container program which is more performent than using a VM but more importantly it also has a repository of containers that have been already been built. It is quite possible that the program that you want or need is already in a docker container.  There are two different sets of precreated containers, official docker containers and community containers.

Docker may have been originally developed on Linux but this awesome container technology can be run on either Linux or Windows – well, Windows 10 and Windows server 2016.

Docker is basically just a command line program and because of that you can use all of the same commands on Windows as you do on Linux.  Actually, if you really are a Windows aficionado you will be happy to learn that you can also use powershell command line as well with Docker.

The installation process is really strait forward, rather than duplicate the instructions here are a few links for installing docker on Linux.

How to install Docker and run Docker containers on Linux Mint 18/18.1

How to install Docker Machine on Linux Mint 18 and 18.1

I plan on doing a few more blog posts on actually using Docker, but for right now I will put up a link for a history of Linux containers.  The solution of a program utilizing the host operating system but yet being (somewhat) segregated from all the other processes is not new.  The solution for segregating a an application used to be a chroot.

History

https://dzone.com/articles/evolution-of-linux-containers-future

Posted in programming | Tagged , | Comments Off on lightweight virtualization – docker

Support your favorite …

The open source community is an amazing place.  No matter what software you want there is an extremely good chance that it exists. You can use Gimp for manipulating graphic images, LibreOffice or Apache Office for word processing, piviti for editing videos or possibly Blender for creating videos.

Today I was a bit saddened when I read that Linux Journal will be shuttering its doors.  This is a magazine that has been around for 20 years and has sharing a lot of technical know how.  Yet size is no refuge.

There is so much software that you don’t need to spend a dime on anything but your hardware.  This is pretty cool but most of the world needs money for things.  The big popular projects do seem to manage to get funding but it is important to support those projects with time and assistance but also with money.

Most of the open source projects are not making their developers rich – they are doing so most likely out of a labor of love.

In my opinion, to keep the Linux ecosystem alive and thriving we need to open up our wallets and donate to the cause or project that you care about.  This might be a donation to our favorite distribution or it might be more general – giving some money to the Electronic Frontier Foundation.

There are other Linux magazines still left but I can imagine that there is a lot of pressure.  It was not very long ago Linux Voice merged with Linux Magazine

Showing support with any Linux group, event or organization helps keep the ecosystem alive.

Posted in Soapbox | Tagged , | Comments Off on Support your favorite …

fighting the last war – playing Blu-ray movies on Linux

How exactly did the product meeting go.

We never had a graphical desktop and so lets just continue on with a 25 x 80 character green text only screen, the customer always loved those.

Today a personal computer or even a smart phone that did not have a multi core processor, graphical environment and gigabytes of memory would be an anachronism.  Coming out with a device that supported less than this would begin a slow death of the company that produced it.

A few years ago, I wanted to create a multi media personal computer at home to show off and provide excellent functionality.  I also wanted to do it using open source software. How hard could this really be?  There are a lot of popular well done open source programs created by talented people that I can use to fulfill my dreams.

My wife, who must be bucking for sainthood, took all of this in stride.  Sometimes it is easy to overlook just how usable consumer grade products are.  All of the big name companies do a heck of a job.  There are no problems with font sizes on menus and users don’t need to think about installing codecs.

In the end, we did have a multimedia PC but it ended up being powered by windows 7. It was simply not possible to create a complete solution up to the standard of the rest of the family.  This was partly due to the inability to play Blu-ray disks and partly due to other services requiring silverlight.

Lets try again

The computer still works but due to some technical difficulties I was going to have to reinstall windows. Perhaps now that a few years have passed a lot of other Linux people people have Blu-ray’s that they want to watch.  Perhaps playing them is no longer a problem.

It doesn’t seem to be the case.  There are plenty of people who discuss their solution to playing a Blu-ray but they all have caveats.

  • plays Blu-ray’s produced before 2012
  • plays some Blu-ray’s
  • needs to rip the Blu-ray to disk and then play
  • plays some Blu-ray but without the actual menu

The real solution in the current century is to support streaming.  You might have a few dvd’s laying around at home but new movies won’t be picked up at the rental store but via Amazon, Netflix or some other streaming service.

I looked into trying to get streaming to work but that was more difficult than coming up with a blu-ray solution.

Streaming from Amazon

Amazon streaming requirements

Streaming Netflix

Posted in Soapbox | Comments Off on fighting the last war – playing Blu-ray movies on Linux

The perception of power (of 9 volts)

When travelling I want to arrive at my destination ontime and unhurt.  I do appreciate that the world’s governments are just as interested and do take steps to ensure that dangerous people don’t hijack the plane or even worse.

Unfortunately sometimes I think that these agencies designed to protect us sometimes feel that they must constantly change something in order to show that they are still keeping us safe – even if the change doesn’t really help.  They think, as do I, that perception matters and if everyone sees them trying really hard then they will think they are being very successful.

That isn’t really the case.  I realize that bringing box cutters on the plane is a bad idea but it is fairly obvious to most rational people that bringing on fingernail clippers is fine.  It is being kind to call some of these “safety” choices as a bit overzealous.

That last sentence was echoing in my head when I read a quote from a TSA official Camille Morris about batteries.  She seems to suggest that a single AA battery is fine as is a single AAA battery, but a common 9 volt battery a big problem.

Actually 9 volt batteries may be a big problem. I cannot say which is the most beloved battery in home made explosives but unfortunately for Miss Morris batteries with smaller voltages can be combined in series to create a battery with a higher voltage.

If a 9volt battery is serious, then, I am rather curious what the TSA thinks about laptops and other larger devices that contain even more powerful batteries.  Let us hope not. What are people supposed to do with their devices.  Laptops filled with lithium batteries.  These are coincidentally the same batteries that the airplane companies work so hard at to ensure they are safely incorporated into and transported using planes.

The FAA Is Freaked Out About Lithium-Ion Batteries on Planes

Crash: UPS B744 at Dubai on Sep 3rd 2010, cargo fire

I might be even more worried about a plane with a belly full of laptops and other assorted battery carrying devices than seeing a tablet in the hand of the guy sitting next to me.

Christmas will be here soon, hopefully there will not be a new batch of “keep us safe” suggestions from the TSA.  We will see if I am still able to board with with a tablet or if that device goes into the naughty stack.

Posted in Soapbox | Tagged | Comments Off on The perception of power (of 9 volts)

coding like it is 1996

It is a sad fact but I don’t always get to pick the tools I use at work.  Just today I felt like my colleague and I woke up in 1996 – back when java 1.6 was released.

The client wanted to keep track of all record changes in one of the fields – the history field.  I am not sure how the other consultancy firm convinced them to do that instead of having either a log file or putting the logging information into a separate table.

The plan was to store all the record changes in the history field in JSON format.  The JSON format is actually great for holding the information in an easy to use format but that is true only with a good parsing class.

That is where our problems began.  We were given the task of dealing with JSON data but the boss said we couldn’t use anything that didn’t come standard with Java 1.6.  This meant that we couldn’t use google’s solution GSON or any other external libraries.

We kicked  a few ideas around before we decided we could use the StringTokenizer class to split up the JSON strings.  Our solution works but only because there is only a single array of records.

We did some rudimentary parsing of the data, but it still felt like going back to pre-internet where you had to write every function yourself for formatting every little thing.

package my.json;
 
import java.util.StringTokenizer;
 
public class TestJson {
 
       public String mydata = "[{\"Date\":\"2017-10-20 16:55\",\"User\":\"Interface\",\"Action\":\"Create\",\"Type\":\"INTERFACE\",\"UserComment\":\"Item created\",\"Level\":\"info\",\"Details\":\"Item created\",\"NotificationStatus\":\"\",\"NotificationReason\":\"\",\"NotificationMessage\":\"\"},{\"Date\":\"2017-10-20 17:35\",\"User\":\"Interface\",\"Action\":\"email\",\"Type\":\"INTERFACE\",\"UserComment\":\"Sending\",\"Level\":\"error\",\"Details\":\"Item created\",\"NotificationStatus\":\"Notification Overdue\",\"NotificationReason\":\"Overdue\",\"NotificationMessage\":\"\"},{\"Date\":\"2017-10-20 17:41\",\"User\":\"Interface\",\"Action\":\"email\",\"Type\":\"INTERFACE\",\"UserComment\":\"Sending\",\"Level\":\"error\",\"Details\":\"Item created\",\"NotificationStatus\":\"Notification Overdue\",\"NotificationReason\":\"Overdue\",\"NotificationMessage\":\"\"}]";
       public String newrec = "{\"Date\":\"2017-10-20 17:41\",\"User\":\"Interface\",\"Action\":\"email\",\"Type\":\"INTERFACE\",\"UserComment\":\"Sending\",\"Level\":\"error\",\"Details\":\"Item created\",\"NotificationStatus\":\"Notification Overdue\",\"NotificationReason\":\"Overdue\",\"NotificationMessage\":\"\"}";
   
       public void printValue(int idx, String datapair)
       {
             StringTokenizer tok = new StringTokenizer(datapair, "\":");
             String key = tok.nextToken();
             String value = "";
             if (tok.hasMoreTokens())
                    value = tok.nextToken();
 
             key = (key + "                   ").substring(0,22);
             System.out.println(idx++ + " " + key + "  " + value);;
       }
 
       public void printfields(String data)
       {
             System.out.println(data);
             StringTokenizer tok = new StringTokenizer(data, "[]{},");
             if (tok.hasMoreElements() == false)
                    return;
             int idx = 0;
             do
             {
                    String str = tok.nextToken();
                    printValue(idx,str);
                    idx++;
             }
             while (tok.hasMoreElements() == true);
       }
       public void process (String args[])
       {
             StringTokenizer tok = new StringTokenizer(mydata, "}");
             do
             {
                    String str = tok.nextToken();
                    printfields(str);
                    System.out.println("\n");
             }
             while (tok.hasMoreElements() == true);
       }
 
       public int ItemCount(String data)
       {
             int retval = 0;
 
             if (data.length() == 0)
                    return 0;
             data = data.replace("[", "").replace("]", "");
             String parts[] = data.split("}");
             retval = parts.length;
             return retval;
       }
       public String getItem(int item, String data)
       {
             String retval = "";
 
             try {
                    data = data.replace("[", "").replace("]", "");
                    if (data.subSequence(0, 1).equals(","))
                          data = data.substring(1);
                    String parts[] = data.split("}");
 
                    if (parts[item].substring(0,1).equals(","))
                          retval = parts[item].substring(1) + "}";
                    else
                          retval = parts[item] + "}";
             }
             catch (ArrayIndexOutOfBoundsException ex)
             {
                    retval = null;
             }
             return  retval;
       }
       public String getItem(int item)
       {
             return getItem(item,mydata);
       }
       public int ItemCount()
       {
             return ItemCount(mydata);
       }
 
       public TestJson()
       {
 
       }
       public TestJson(String record)
       {
             mydata = record;
       }
       static public void main (String args[])
       {
             TestJson x = new TestJson();
             System.out.println(x.ItemCount());
             System.out.println(x.ItemCount(x.newrec));
             System.out.println(x.ItemCount(""));
 
             System.out.println(x.getItem(0));
             System.out.println(x.getItem(1));
             System.out.println(x.getItem(2));
             System.out.println(x.getItem(3));
       }
}
Posted in programming | Tagged , | Comments Off on coding like it is 1996

windows batch processing

I really didn’t want to fool around with that little windows task that has been laying on my boss’s desk. It was planned that Robert would do it after the new year started but then he decided to quit. Now it has been given to me.

The task is actually pretty simple, migrate that setup from the Hartmut’s old computer to a virtual machine. The task itself is just to automate the extracting of some data from MQ and writing to the file system with it’s original filename. The marketing guys will deal with the files after that.

I would rather use cron and bash scripts but that isn’t an option on Windows server 2008, so instead I will get to use the Windows task scheduler and a few cmd scripts.  The Windows scheduler is just as powerful and is standard for that platform.

Task scheduler (taskschd.msc)

Most of the task scheduler is pretty easy to guess.  The main tricks is to use a user that has admin privileges and to select run whether or not the user is logged in – well assuming that is correct for your situation.

Triggers

New trigger

Action

New action

     

I did my testing on windows 7 and the task manager works just fine if you have enough privileges.

 

Cmd Shell

My script doesn’t need to do more than setup a class path and call our Java program.  The output will be written to the current directory and pretty much everyone will be happy.

Well, almost everyone.  Not all that long ago one of our Unix test environments filled up with junk and I received a lot of status emails.  I don’t want something similar to happen on windows, and if it does, I don’t want it to be because of our log files.

In Unix or Linux I would be using the find command to gather up a list of files to delete, while on windows the cmd shelll has the “forfiles” command.

Just like the Unix find command, you can gather up the files by regular expression and/or age and then run a command for each file. This command will basically loop over all files in the given directory that matches the file specification or date.

@echo off
rem deal with old log files
set RUNLOGDIR=c:\logdirectory\runlogdir
 
set DELCMD=del
set LOGFILES=*.log
 
cd %RUNLOGDIR%
forfiles /p %RUNLOGDIR% /m %LOGFILES% /c "cmd /c %DELCMD% @path" /d -90

During testing you can replace the del command on line 5 with echo to see what would be deleted.  This was fairly convenient during testing.

I still am not a fan of Windows but it is obvious that they do have a lot of the same type of functionality as their Unix brethren.

 

Powershell

Windows also has the powershell interpreter which can also be used to create batch tasks. This might be a better choice than the old DOS cmd shell mainly because it is a true programming language that has a lot of functionality to allow easy access to the operating system functions.

It is just as easy to create very reusable function to delete files older than a given number of days.

# delete old files 
Function DeleteFiles([string]$DelPath,[string]$regexpr,[int]days) 
{
        # files older than xx days
        $cutoffdate = (Get-Date).AddDays(-days)

        $filesToDelete = Get-ChildItem -Path "$DelPath" -Filter $regexpr | Where-Object {$_.LastWriteTime -lt $checkdate -and !$_.psiscontainer}
        ForEach($file in $filesToDelete) 
        {
                Remove-Item  $DelPath$file -force
        }
}

This is a very nice solution that will be essentially self documenting when you add that function call to the rest of your powershell script.

ie

DeleteFiles c:\temp\logfiles *.log 15

There is a lot more information about powershell programming on the internet as well as are couple of my own blog entries

Windows scripting sucks or does it?

Windows powershell scripting

Reference information

Forfiles syntax

FORFILES [/P pathname] [/M searchmask] [/S]
         [/C command] [/D [+ | -] {dd/MM/yyyy | dd}]

Description:
    Selects a file (or set of files) and executes a
    command on that file. This is helpful for batch jobs.

Parameter List:
    /P    pathname      Indicates the path to start searching.
                        The default folder is the current working
                        directory (.).

    /M    searchmask    Searches files according to a searchmask.
                        The default searchmask is '*' .

    /S                  Instructs forfiles to recurse into
                        subdirectories. Like "DIR /S".

    /C    command       Indicates the command to execute for each file.
                        Command strings should be wrapped in double
                        quotes.

                        The default command is "cmd /c echo @file".

                        The following variables can be used in the
                        command string:
                        @file    - returns the name of the file.
                        @fname   - returns the file name without
                                   extension.
                        @ext     - returns only the extension of the
                                   file.
                        @path    - returns the full path of the file.
                        @relpath - returns the relative path of the
                                   file.
                        @isdir   - returns "TRUE" if a file type is
                                   a directory, and "FALSE" for files.
                        @fsize   - returns the size of the file in
                                   bytes.
                        @fdate   - returns the last modified date of the
                                   file.
                        @ftime   - returns the last modified time of the
                                   file.

                        To include special characters in the command
                        line, use the hexadecimal code for the character
                        in 0xHH format (ex. 0x09 for tab). Internal
                        CMD.exe commands should be preceded with
                        "cmd /c".

    /D    date          Selects files with a last modified date greater
                        than or equal to (+), or less than or equal to
                        (-), the specified date using the
                        "dd/MM/yyyy" format; or selects files with a
                        last modified date greater than or equal to (+)
                        the current date plus "dd" days, or less than or
                        equal to (-) the current date minus "dd" days. A
                        valid "dd" number of days can be any number in
                        the range of 0 - 32768.
                        "+" is taken as default sign if not specified.

    /?                  Displays this help message.

Examples:
    FORFILES /?
    FORFILES
    FORFILES /P C:\WINDOWS /S /M DNS*.*
    FORFILES /S /M *.txt /C "cmd /c type @file | more"
    FORFILES /P C:\ /S /M *.bat
    FORFILES /D -30 /M *.exe
             /C "cmd /c echo @path 0x09 was changed 30 days ago"
    FORFILES /D 01/01/2001
             /C "cmd /c echo @fname is new since Jan 1st 2001"
    FORFILES /D +14/2/2017 /C "cmd /c echo @fname is new today"
    FORFILES /M *.exe /D +1
    FORFILES /S /M *.doc /C "cmd /c echo @fsize"
    FORFILES /M *.txt /C "cmd /c if @isdir==FALSE notepad.exe @file"

Posted in programming | Tagged , | Comments Off on windows batch processing

lo-resolution fun – setting up a minecraft server

I remember playing computer games on 640×480 resolution and it was a lot of fun. Computer graphics has now improved to the point that some of the games are virtually photo-realistic.

With all of this in mind it didn’t make any sense to me when my son asked for Minecraft.  I tried to be a good parent and tested it out on my sons tablet.  It took me about 5 minutes to get bored and stop testing.  It just wasn’t my cup of tea to create my own buildings and dig holes in the ground but it seemed harmless enough.

For reasons I cannot fathom Minecraft was a instant hit with both boys.  The only thing was that all of their buildings ended up on one tablet which was inconvenient if both didn’t want to play at the same time.

Of course they also found some other Minecraft servers on the internet which allowed them to do some shoot’em up in teams.

Closing the stable door after the horse has bolted

It would be nice if the boys built their own villages and cities here on my server where it would be easier to control.  Shoot’em up games didn’t hurt me but I think the creative mode where you can build your own world is more interesting.

Installation of Minecraft

Microsoft has made the process of installing and running your own server painless.  The task is as simple as download the jar file and run it.  To make things simple Microsoft tells you exactly how to run the jar file.

java -Xmx1024M -Xms1024M -jar minecraft_server.1.12.jar nogui

The current version of the Minecraft server is 1.12 and requires java 8.

The last parameter is so the gui is not started when the server is started.  Once the server is started it is essentially a command shell.  From this point it is possible to change various aspects about the world, summon additional creatures or even grant operator status to any of the players.

Minecraft commands

There are a lot of commands for controlling the Minecraft world.  It is literally beyond the scope of what I can research and document about Minecraft, here is a small sample of some of the commands that I thought were fun.

Name Description
/time set <int> A day is equal to 24000 ticks where daytime is 1000 and nighttime is 13000.
/time set [day|night] Sets the world time to daylight or night time.
/weather [thunder| clear|rain] This will change the weather to one of the three states.  A fourth state is snow.  Snow only occurs if it is raining and you are in a winter area.
/summon sheep x y z This will summon a sheep to the x,y,z coordinates in the world.

Note: you can get your coordinates from your map.

It was fun to do these commands when my son was playing Minecraft mainly because of what I heard from the other room.

I am sure that there are a lot of sites that fully describe and list these commands.  The one that I used to find these commands was digminecraft.com.

Upgrade of Minecraft server

The upgrade process is simply to shutdown the server, download the new file and then startup the server with the new jar file.

 

Posted in Setup From Scratch | Comments Off on lo-resolution fun – setting up a minecraft server