Paranoid Professor

Most of the scripts that I need to write don’t actually need to manipulate a lot of data.  It is usually enough to pipe that data through a command or two and use the result.  When this isn’t acceptable then it is usually a small enough amount of data that I can save it in a temporary file with the same name.

Yet there might be a number of cases where truly unique names are necessary.  One example might be if the script gets run many times in parallel but still needs to store some temporary data.

There are a few different methods of generating a unique name.

Process Id

Each Unix process has a unique id and that can be used as part of the file name.  There are some special variables that can be accessed and one of them is $$.  This bash shell variable will be the actual process id of the current shell.

TEMPFILE=mytempdata.$$

This actually should be pretty safe for a lot of situations depending on the volume.

Timestamp

Another fairly classic method of creating a unique name would be to create a timestamp.  This is a terrible device for coming up with a unambiguous name unless the script in question is being run in serial.

TEMPFILE=mytempdata.`date '+%Y%m%d-%H%M%S'`

This example simply uses the date command to display the current date and time.  This date and time is redirected using the backtick.  This can also be done using the more modern method of redirecting the output.

TEMPFILE=mytempdata.$(date '+%Y%m%d-%H%M%S')

Operating system support

There exists the mktemp command which will create an absolutely unique name based on the names of the files or directories in the destination directory.  There is one “little” side effect of this command and that is that the temporary file or directory is created. The file is a zero length file while the directory is a normal directory.

Using the same technique as in the previous example it is possible to both create a unique name while also capturing the name in a variable.

TEMPFILE=$(mktemp /tmp/mytempdata.XXXX)

TEMPDIR=$(mktemp -d /tmp/mytempdata.XXXX)

The operating system cannot know how many free files you might need in your directory. The mktemp command will replace the capital letter X with the unique alphanumeric value.  If more than 999 files might be in use at a time then use four X’s.  The exact location of the capital X is not important.  It can be set as an extension or just a value within the name.

(flashback)

Our project was inching closer and closer to going live.  On the evening before the Easter break some clod started a massive report which tried to process all the data since the beginning of time. My colleague’s inefficient program gathered up all of the data from the database and wrote it to the data directory in a temp file.

The data directory was more than large enough for a test environment but it wasn’t a big enough for such a test.  We could have easily recovered if it had occurred while we were at work but instead the system attempted to process nightly batches over those four days without enough space and made a pretty big mess.

My boss Theodore was more upset than he should have been for a test environment and kept yammering on about what if this has been production.  He was right of course but one of the preconditions of the system is that enough resources are available.  It is our groups responsibility to write the programs but it is someone else’s to ensure that the system doesn’t run out of resources.

Anyway, we implemented the boss’ warning email feature.  Each time the program is run it checks for enough disk space when not enough space exists then send out a warning email and quits. To be on the safe side, my boss asked that I have my email address as one of the recipients.

(the present)

If I receive one more warning email from one of the test systems I am afraid I am going to kill someone.

Von: Automatically.Generated.Message@acme.com [mailto:Automatically.Generated.Message@acme.com] 
Gesendet: Dienstag, 7. Februar 2017 11:54
An: process_monitoring@acme.com; 
Betreff: Warning ... the end is near on acme-app1

An error has occurred, the copy app has not been launched 
because of insufficient disk space on following partition.

Filesystem             size   used  avail capacity  Mounted on
acme-app1_dpool/app    85G    85G     0K   100%    /appdir

Corrective action is required immediately.


The status of rest of machine is as follows.

Filesystem             size   used  avail capacity  Mounted on
/                       10G   7.7G   2.3G    77%    /
/dev                    10G   7.7G   2.3G    77%    /dev
proc                     0K     0K     0K     0%    /proc
ctfs                     0K     0K     0K     0%    /system/contract
mnttab                   0K     0K     0K     0%    /etc/mnttab
objfs                    0K     0K     0K     0%    /system/object
swap                   140G   400K   140G     1%    /etc/svc/volatile
fd                       0K     0K     0K     0%    /dev/fd
swap                   8.0G   700M   7.3G     9%    /tmp
swap                   140G    40K   140G     1%    /var/run
acme-app1_dpool/app    85G    85G     0K   100%    /appdir
acme-app1_dpool/acme_home   1.0G   353K   1.0G     1%    /appdir/home/gast
acme-app1_dpool/acme_samba   2.0G    36K   2.0G     1%    /appdir/samba
acme-app1_dpool/acme_scripts   2.0G   249M   1.8G    13%    /appdir/scripts

This is an automatically generated message for informational purposes.

The idea seemed ok; when no disk space exists then send out an email.  The underlying assumption was someone in IT would deal with the problem.

Apparently the idiot users turn off half of the system about a week back but not every process.  I came to work and found hundreds of emails clogging up my inbox.  Looking through them you could literally see the space filling up over time.

Well, hundreds of files are annoying but the general functionality is awesome.  A combination of the bash script and sendmail allows me to capture the important facts about our system and send it to someone.

Just look at the script.

#!/usr/bin/bash
SUBJECT="warning the end is near"
DF_Command=`df -h ${FILESYSTEM}`
FULL_DF=`df -h`
TO=process_monitoring@acme.com
FROM="automatically generated message"

HOSTMACHINE=acme-app1

( cat << !
To: ${TO}
From: ${FROM}
Subject: warning ... the end is near on $HOSTMACHINE

An error has occurred, the copy app has not been launched because of insufficient disk space on following partition.

${DF_Command}

Corrective action is required immediately.

The status of rest of machine is as follows.

${FULL_DF}

This is automatically generated message for informational purposes.
!
) | /usr/sbin/sendmail -t

Fill the variables with information ranging from a single word up to a lot of lines of text and then substitute them into your mail.  The bash shell will expand them before sending the mail out.

I guess that the moral of the story should be that more logic should be used because some idiot will inevitably trigger it on a non-production environment.  Well, that or just get rid of the idiots ….