I remember quite some time ago when I was developing a an application when the personal computer was actually fairly slow. Of course I happened to pick a technology that worked just fine on a newer computer but it actually ran quite pokey on my machine.
The decision was made to never develop on a machine less than or equal to the target platform. Fast forward a few years to discover that I periodically get stuck with the slow development environment and there is nothing that I can do about it.
Well, I can’t speed up the machine but I can make my programs and scripts a bit smarter. The target of my focus was a small script that called some horribly inefficient java program. Not all the programs dawdled, just this one.
The problem was the script called this program even though in most cases the input directory was empty – not a problem on production but really slow on development.
The input directory can hold quite a few data files and the names are never the same. I ran through a few different ideas.
Idea #1
Well, it seems to be a pretty strait forward solution. Get a count of the number of files and then you will know how many things there are to process and thus if there is anything to process.
It almost makes sense until you try this. It works just great when data does exist but when no data exists then it doesn’t work very well.
#!/bin/bash
EXT=xml
INPUT=/data/input
CNT=`ls -1 $INPUT/*$EXT | wc -l`
if [ $CNT -gt 0 ]
then
echo $CNT items
else
echo "files don't exist"
fi
The count variable is set to zero but the the shell informs us that no data exists.
ls: cannot access /data/input/*xml: No such file or directory
Not a serious issue but depending on the person very distracting.
Idea #2
This error message can be easily gotten rid of. Simply redirect the error output to a file or to /dev/null.
#!/bin/bash
EXT=xml
INPUT=/data/input
CNT=`ls -1 $INPUT/*$EXT 2>/dev/null| wc -l`
if [ $CNT -gt 0 ]
then
echo $CNT items
else
echo "files don't exist"
fi
Not a bad idea if you know that the directory permissions for the input directory will always be OK. If not, then you will be (possibly) throwing away the only help to diagnosing an odd behavior.
testing existence – Extra credit edition
This second solution is quite acceptable and unless it is called a lot it shouldn’t be too much overhead. Yet there is a much easier solution that is probably even more efficient.
#!/bin/bash
EXT=xml
INPUT=/data/input
ls $INPUT/*$EXT >/dev/null 2>&1
if [ $? -eq 0 ]
then
echo $EXT files exist
else
echo "$EXT files don't exist "
fi
No, you don’t know the number of files, but you don’t have to create a pipe and fill it with a few (or many) files just to decide whether or not to run a program. Simply look at the Unix return code from our last command, if zero, then we know our files exist.
