Windows scripting sucks or does it?

I am not a fan of the big MS.  It started a long time ago when my funds were limited and I simply couldn’t afford to keep up with the MS upgrade schedule.  Not all of their software back in the day, in my opinion, was all that great.  I went cold turkey on as many of the MS products that I could.

There is a lot of open source software that could fill in the empty spots for me and since then most of my work life was on Unix which led to joys of shell scripting (ksh,bash,csh)

I love Unix shell scripting because the shell is basically just the glue to tie together a lot of small command line utilities which makes it easy to create and add extra commands, and Unix has a lot of command line tools.  Windows, at that time, had a very weak batch scripting language and a limited number interesting command line commands.

PowerShell

(Eventually) Those folks at MS decided to create their own scripting language called powershell.  They decided to take a slightly different route than the Unix crowd. Powershell is an interpreter that contains the logic for doing tasks.  This “shell” appears to be more of a replacement for the typical cmd.exe shell that has existed since the beginning (well it used to be command.com) as well as all of the command line programs.

The commands that exist from the powershell are called cmdlets.  Unlike Unix these are not a simple executables that can be executed from the cmd shell but are commands that are part of the powershell interpreter.  This might give a small bit of extra performance but it also means a new version of powershell is needed to add some new functionality.

The one good thing about the powershell is that it was installed on my computer by default. In addition, The big MS did want to show their security credentials and they have added a security level that must be set before you are able to run any powershell scripts.  This security level is essentially determining to which extent you trust powershell scripts with the focus being on either local or from other sources.

This is actually a pretty smart as it is possible to do quite a few things using powershell once you can run the scripts.  It would be pretty serious to have such power unlocked by default for any user who happens to execute a powershell script that is an attachment to an email.

However, even accidentally running a powershell script is a bit harder as MS has associated the extension .ps1 to be assigned to notepad.  Thus if you click on a script from windows explorer you will simply open up the script in notpad.

In order to run powershell scripts you need to first select an executable permission. There are four different permissions that can be selected in order to set the various rights at different levels.

Level Description
Restricted Can not run powershell scripts
RemoteSigned Can run local scripts but remote scripts need to be signed.
Unrestricted Any script can be run.
AllSigned Very secure, requires all scripts to be signed.
Undefined Removes existing level

Changing the level is actually pretty easy but you do need to be an administrator to do so. Simply use the set-executionpolicy cmdlet to set the permission to one of the described levels.

Signed Scripts

I personally am a fan of the unix “permissions” as it is really quite simple and very easy to modify. However, I wouldn’t say that they are the final word in preventing unauthorized scripts from being run.

I guess the final word is trying to protect your scripts is to use Microsoft’s method of creating recognized certificates while using a certification authority infrastructure.  If you want you can learn more about this by visiting Microsoft and downloading their best practices document.  I didn’t feel up to the task of going through the entire 67 page document so instead I decided to set my execution policy level to RemoteSigned.

This then let me run my fabulous powershell script.

$list = dir
echo $list

Actually this small script does quite a bit.  It does a directory listing and assigns the output to a variable.  Much like the bash shell that variable can be a list and echoed to the output device.

 

Note: It is possible to run any other programs from this powershell if they are in your path.

Unix friendly

It is possible that the big MS wanted to create a shell that would also appeal to the Unix crowd. I think this is the case as the powershell has the alias command.  This gave them the ability to create a bunch of commands that appear to include the common Unix commands.

The alias command links the name to an internal cmdlet which performs some action. One example of this is the “clear” alias being linked to the cmdlet clear-content.  The alias’s that they choose appear to look like they were picked to make Unix guys feel comfortable.

PS C:\Windows\system32> alias

CommandType     Name                            Definition
-----------     ----                            ----------
Alias           %                               ForEach-Object
Alias           ?                               Where-Object
Alias           ac                              Add-Content
Alias           asnp                            Add-PSSnapIn
Alias           cat                             Get-Content
Alias           cd                              Set-Location
Alias           chdir                           Set-Location
Alias           clc                             Clear-Content
Alias           clear                           Clear-Host
Alias           clhy                            Clear-History
Alias           cli                             Clear-Item
Alias           clp                             Clear-ItemProperty
Alias           cls                             Clear-Host
Alias           clv                             Clear-Variable
Alias           compare                         Compare-Object
Alias           copy                            Copy-Item
Alias           cp                              Copy-Item
Alias           cpi                             Copy-Item
Alias           cpp                             Copy-ItemProperty
Alias           cvpa                            Convert-Path
Alias           dbp                             Disable-PSBreakpoint
Alias           del                             Remove-Item
Alias           diff                            Compare-Object
Alias           dir                             Get-ChildItem
Alias           ebp                             Enable-PSBreakpoint
Alias           echo                            Write-Output
Alias           epal                            Export-Alias
Alias           epcsv                           Export-Csv
Alias           epsn                            Export-PSSession
Alias           erase                           Remove-Item
Alias           etsn                            Enter-PSSession
Alias           exsn                            Exit-PSSession
Alias           fc                              Format-Custom
Alias           fl                              Format-List
Alias           foreach                         ForEach-Object
Alias           ft                              Format-Table
Alias           fw                              Format-Wide
Alias           gal                             Get-Alias
Alias           gbp                             Get-PSBreakpoint
Alias           gc                              Get-Content
Alias           gci                             Get-ChildItem
Alias           gcm                             Get-Command
Alias           gcs                             Get-PSCallStack
Alias           gdr                             Get-PSDrive
Alias           ghy                             Get-History
Alias           gi                              Get-Item
Alias           gjb                             Get-Job
Alias           gl                              Get-Location
Alias           gm                              Get-Member
Alias           gmo                             Get-Module
Alias           gp                              Get-ItemProperty
Alias           gps                             Get-Process
Alias           group                           Group-Object
Alias           gsn                             Get-PSSession
Alias           gsnp                            Get-PSSnapIn
Alias           gsv                             Get-Service
Alias           gu                              Get-Unique
Alias           gv                              Get-Variable
Alias           gwmi                            Get-WmiObject
Alias           h                               Get-History
Alias           history                         Get-History
Alias           icm                             Invoke-Command
Alias           iex                             Invoke-Expression
Alias           ihy                             Invoke-History
Alias           ii                              Invoke-Item
Alias           ipal                            Import-Alias
Alias           ipcsv                           Import-Csv
Alias           ipmo                            Import-Module
Alias           ipsn                            Import-PSSession
Alias           ise                             powershell_ise.exe
Alias           iwmi                            Invoke-WMIMethod
Alias           kill                            Stop-Process
Alias           lp                              Out-Printer
Alias           ls                              Get-ChildItem
Alias           man                             help
Alias           md                              mkdir
Alias           measure                         Measure-Object
Alias           mi                              Move-Item
Alias           mount                           New-PSDrive
Alias           move                            Move-Item
Alias           mp                              Move-ItemProperty
Alias           mv                              Move-Item
Alias           nal                             New-Alias
Alias           ndr                             New-PSDrive
Alias           ni                              New-Item
Alias           nmo                             New-Module
Alias           nsn                             New-PSSession
Alias           nv                              New-Variable
Alias           ogv                             Out-GridView
Alias           oh                              Out-Host
Alias           popd                            Pop-Location
Alias           ps                              Get-Process
Alias           pushd                           Push-Location
Alias           pwd                             Get-Location
Alias           r                               Invoke-History
Alias           rbp                             Remove-PSBreakpoint
Alias           rcjb                            Receive-Job
Alias           rd                              Remove-Item
Alias           rdr                             Remove-PSDrive
Alias           ren                             Rename-Item
Alias           ri                              Remove-Item
Alias           rjb                             Remove-Job
Alias           rm                              Remove-Item
Alias           rmdir                           Remove-Item
Alias           rmo                             Remove-Module
Alias           rni                             Rename-Item
Alias           rnp                             Rename-ItemProperty
Alias           rp                              Remove-ItemProperty
Alias           rsn                             Remove-PSSession
Alias           rsnp                            Remove-PSSnapin
Alias           rv                              Remove-Variable
Alias           rvpa                            Resolve-Path
Alias           rwmi                            Remove-WMIObject
Alias           sajb                            Start-Job
Alias           sal                             Set-Alias
Alias           saps                            Start-Process
Alias           sasv                            Start-Service
Alias           sbp                             Set-PSBreakpoint
Alias           sc                              Set-Content
Alias           select                          Select-Object
Alias           set                             Set-Variable
Alias           si                              Set-Item
Alias           sl                              Set-Location
Alias           sleep                           Start-Sleep
Alias           sort                            Sort-Object
Alias           sp                              Set-ItemProperty
Alias           spjb                            Stop-Job
Alias           spps                            Stop-Process
Alias           spsv                            Stop-Service
Alias           start                           Start-Process
Alias           sv                              Set-Variable
Alias           swmi                            Set-WMIInstance
Alias           tee                             Tee-Object
Alias           type                            Get-Content
Alias           where                           Where-Object
Alias           wjb                             Wait-Job
Alias           write                           Write-Output

PS C:\Windows\system32>

This is NOT exactly like Unix. The “ls” alias does do a directory listing but unsurprisingly it does not accept the same options.

Microsofts cmdlets

PowerShell actually implements all the logic in the form of cmdlets.  You could find a book that described PowerShell programming along with (hopefully) a big reference section. That is one possible approach but the good news is that PowerShell has built in help.

The cmdlet get-help will return the information about any of the supported cmdlets.

Microsoft also has a very complete webpage that describes the different cmdlets. This information is grouped together by function.

https://technet.microsoft.com/en-us/library/dd772285.aspx

One final note about security

Well, I am glad that I didn’t spend a whole lot of time on setting up an elaborate system to deal with signed certificates.  It turns out that there are actually a number of different ways that you can run powershell scripts and some of them get around the existing security settings.

I didn’t actually come up with these methods myself, but instead happened upon the following webpage that elaborated a lot of ways to get around the execution policy.

15 Ways to Bypass the PowerShell Execution Policy

Of all the methods of bypassing the security I liked the method of piping the script through the interpreter and telling it to ignore the execution permissions by not using anyone’s profile.

Does windows scripting suck?  It is a bit too early to answer this question as the only script I have provided is equivalent to the dir command.  I will cover a bit more about powershell scripting in part II.

Posted in programming | Tagged , | Comments Off on Windows scripting sucks or does it?

high quality USB cables

He said          I don’t want a tablet

She said         They are really convenient.

I should probably check my wallet to see if I belong to some sort of anti-technology society. I actually didn’t have a tablet for the longest time and then eventually after a lot of thought I got a tablet.

Well, that’s a bit of a lie.  I agreed with my wife that it would be an ok for her to get me one as a Christmas gift – although what would better would be a nice pair of socks.  I didn’t get the socks and the tablet didn’t really displace any computers.  Yet, I did find the tablet to be useful for looking up things on the internet and doing some reading.

It must have wormed its way into my heart as I became a bit angry that it might need to be replaced after a few years.  The real problem was it wasn’t charging or couldn’t hold a charge.  It might take three days to get fully charged for just a few hours.  Did it get kicked?  One fall from the bed too many?

I should have suspected either the charger or the cable right away.  After all, sometimes when my call phone was charging I could hear it connect and disconnect quite a few times over the course of the evening.

I was tidying up my shelf while the table was in the middle of one of its marathon charging sessions.  I must have bumped it as I then heard it disconnect but before I could do anything it reconnected.

Well, now that interesting…

A small bit of upwards pressure on cable? (kaplink) Tablet charging.

A small bit of downwards pressure on cable? (kerplink) Tablet no longer charging.

repeat a few times

This didn’t rule out a problem with the tablet but it convinced me that the charger was ok.  A quick switch of cables ruled out a problem with the tablet itself.

I must live a pretty sheltered life as I haven’t had a problem with a USB cable not doing my bidding.

I try and buy good cables and good chargers and apparently so far I have been lucky. However, once I make the upgrade to USB 3.0 I will need more than to rely on good luck as those cables carry enough power to fry just about anything plugged into it.

Amazon bans faulty usb c cables – Google engineer reviewed hundreds

https://www.theguardian.com/technology/2016/mar/30/amazon-clamps-down-dangerous-usb-c-cables

Apparently, the cheapest isn’t always the best.  I guess you get what you pay for and in the future, saving a few pennies on a cable might get you the opportunity to purchase a new tablet or phone.

Posted in Soapbox | Comments Off on high quality USB cables

command line fun – finding more space

I may not know where I set down my keys or my latest copy of Linux Magazine, but I do know where my programs are installed.  If I cannot think of where one of my files is I can always use the find command.

The find utility can be used in a lot of interesting ways to help you find something.  I did briefly cover the find command for locating individual files, but it can also locate directories, old files, big files or even something specified by inode.

I also do a pretty good job of ensuring that when I install something on production it will look after itself as far as space consumed – old data and log files don’t normally hold a lot of added value.

Yet, some of the guys I work with apparently feel that the disks are large and they never feel the need to clean up after themselves.  I know that because I start to receive emails from IT that the system is filling up while leaving us to find out which directories are the culprit.

Disk Usage(du)

The disk usage tool actually will show how much space is being consumed by individual directories.  As long as those directories are no deeper than one level, you don’t need more than this.

du -h .

This will show the disk space used in human measurements (ie -h) so it is possible to see how many kilo, mega or giga bytes you are using.  By default the disk usage command will show how much space is used for the directories under the path given and also display how much space any other directories under that take.

This is both annoying and less than transparent if someone created a new sub-directory for each day or months processed data files.

3.1M ./purchaseord/processed/201505
302K ./purchaseord/processed/201506
563K ./purchaseord/processed/201507
1.7M ./purchaseord/processed/201505
271K ./purchaseord/processed/201508
255K ./purchaseord/processed/201509
96K ./purchaseord/processed/201510
65K ./purchaseord/processed/201511
175K ./purchaseord/processed/201512
1.5M ./purchaseord/processed/201503
21K ./purchaseord/processed/201601
8.9M ./purchaseord/processed
409K ./purchaseord/manual-processed

Finding only the top directories is harder as the number of directories, sub-directories and sub-sub directories increases in inconsistent ways.

The good news is that the find command can be used to not only find directories (ie -type d) but also limit the directories to those found at the top level (! -name . -prune).  This is actually not all that different from when I was trying to create a list of files or directories as an input to another program.

DATADIRECTOR=/vend/samba/configsys/interfaces/
find $DATADIRECTORY -type d ! -name . -prune | xargs du -hs

The find command will just generate a list of top directories.  This clever way of piping it allows us to take that list and pass it to the disk usage command one at a time by using the xargs utility.

1.0G  ./purchaseord
883K  ./calendars
10K   ./winhistory
63M   ./repository
2K    ./acctrpt
96M   ./regulatory

 

 

Posted in programming | Tagged , | Comments Off on command line fun – finding more space

review of Huawai Mate S

It is a bit of a simplification to say that all smart phones are the same but it isn’t too dramatic of a statement to say that most high end smart phones sold at any particular time are quite close in features and specifications.

Sure, someone might have a thinner phone or a bigger screen but most of the cell phones sold today are running one of two different operating systems.  Each release of IOS probably is more similar than different to the previous version and this is also most likely true for Android as well.

I needed a new phone and for a lot of different reasons didn’t go with Samsung.  Oddly enough it wasn’t due to their recent battery problems but more on price, and the previous version didn’t support adding an sdcard.

One thing led to another and I happened to find myself owning the Mate S from Hauwai.  I am not going to do a blow by blow account of every feature, there are professional magazines, advertising companies and real dedicated hardware types for that.  I am going to briefly list a few of the highlights based on my experiences so far.  The phone is still pretty new so there may be a follow up if anything unexpected occurs.

The good

The camera and the screen are really nice.  If you want to snap off a few photos of that special person you are stalking  your new girlfriend or pet.  The camera takes clear photos and does offer a lot of special filters just in case you really like seeing everything in black and white.

I was completely take by surprise not long after having this phone when I received an update.  I am not so familiar with Huawai that I can say if this is novel or normal.  I was just happy as I know that a lot of smart phones from many manufacturers don’t see many if any upgrades.

The phone came with apps installed including some that I would never install in this lifetime.  Yet I was surprised to see that it was possible to simply uninstall some but not all of those offending apps.  I didn’t have to root the phone nor do a lot of strange maneuvers I just dragged the app to the garbage can.  This is the smartest smart phone I have owned.  I am tired of being the victim of the added crap value apps that the manufacturer decides to provide based on the biggest fee they receive when bundling an app.

That last sentence is a lie.  I really don’t care what crappy applications they install, what bothers me is that normally you cannot uninstall those apps.

The less good

I don’t know if this had more to do with Android or the the image that Huawai was using.  After my upgrade I did have a small problem as the sdcard went from a name (i think it was sdard1) to the actual name of the card which was 8 digits.  This was a problem because some of my apps were now looking in the wrong place for their data files.

Despite the phone having a Android GUI everything did seem a bit unfamiliar when compared to the previous version I was using.  This is probably just a case of grumpy old man syndrome as it was not that different and within a short time I was using the phone without any difficulty.

The bad

There is really not too much negative to talk about for this phone but I did realize one thing.  You cannot change the battery.  This means that in a few years if my battery life isn’t good enough I will have to purchase a new phone rather than to extend the life of the phone with a new battery.

“I will have to purchase a new phone rather than to insert a new battery into this phone.”

It is a simple statement but not 100% accurate.  It is actually possible to replace the battery if you are willing to go through a fair amount of effort.  I did discover a you tube video that does describe this process.

From watching the video, the phone appears to be built in a manner that does allow for small parts to be replaced if necessary.  However, just the tool list will probably scare away all but the bravest DIY’er.

Not totally pleased with the battery situation but it does guarantee that I move on with newer technology at some point in the future.

This might require a change in thinking.  Who knows, perhaps I will even get rid of my rotary phone one day.

Posted in Review | Tagged | Comments Off on review of Huawai Mate S

Installing Eclipse on Linux

Eclipse is an amazing example of an open source project that easily dwarfs its commercial competition.  Does an IDE make you a weaker programmer?  I don’t think so.  If it makes you more productive, help you with syntax, or speeds up repetitive tasks then it is simply one more tool in your toolbox. Depending on what you are developing it may your go to tasks for debugging or refactoring code.

I wanted to do some restful web services and a number of the examples are described using eclipse.  In order to get a good starting point, I needed to install a copy Eclipse.

Depending on your level of familiarity with the Linux command prompt, the process of setting up Eclipse on your system is fairly trivial.

Unpack file

I wasn’t setting up eclipse for all users on a system but rather just one user for this virtual machine.  So I copied it to ~myuser/tmpdir and used both gunzip and tar to decompress and extract the files from the tar.

cd ~myuser/Downloads

mkdir ~myuser/tmpdir

mv ~myuser/Downloads/eclipse-jee-neon-2-linux-gtk.tar.gz .

gunzip eclipse-jee-neon-2-linux-gtk.tar.gz

tar -xvf eclipse-jee-neon-2-linux-gtk.tar

Move to final location

mv ~myuser/eclipse /opt/eclipse

It doesn’t really matter where you put eclipse.

Setup environment

The last step is to make the executable available.  If it isn’t in one of the directories that is part of you path simply add it to your path.  This could be done in a couple of different ways.

One way would be to add eclipse to your path.

export PATH=$PATH:~myuser/eclipse

Another way would be to create a symbolic link from the eclipse directory to the /usr/bin directory to eclipse.

It might not be so important when a computer is only used by one person but by placing the link in the /usr/bin directory then eclipse is available for all users.

 

Posted in programming, Setup From Scratch | Comments Off on Installing Eclipse on Linux

Installing Tomcat on Linux

Most of my needs for serving up web content can normally be summed up in a few static pages.  To serve those pages up you would usually find an Apache server in the background.  Apache is a general purpose server which serves static html data.

Tomcat on the other hand is a JSP server for serving up dynamic data.  Tomcat is a server that can interpret the commands it is given.  The server ends up running the java code and providing the output to the caller.

I wanted to test out restful web services but in order to do that I needed to get Tomcat up and running. The steps are pretty straight forward.

  • create a Tomcat user and group
  • download Tomcat
  • unpack
  • move it to install directory
  • setup directory permissions
  • create systemd control file
  • reload systemd daemon
  • restart tomcat
  • test Tomcat server

I simply followed the tutorial and within a few minutes had a working system.

https://www.digitalocean.com/community/tutorials/how-to-install-apache-tomcat-8-on-ubuntu-16-04

One interesting side note is that this tutorial also described how to setup a systemd process.

The command for starting, stopping or checking the tomcat server are quite similar.  They are not difficult commands but despite that fact I quickly put together three scripts so I don’t need to remember the details.

start-tomcat.sh

sudo systemctl start tomcat

stop-tomcat.sh

sudo systemctl stop tomcat

status-tomcat.sh

sudo systemctl status tomcat

Afterword

I had a few small difficulties with tomcat once it was installed. The problems were not Tomcat itself but rather using some of the scripts that ship with tomcat.  The script that I was specifically having problems with was version.sh which is located in the /opt/tomcat/bin directory.

In addition to the setup described in the tutorial I also did the following

cd /opt/tomcat
chmod 755 bin
cd /opt/tomcat/bin
chmod 755 *sh
chmod 754 *jar

This was only the start. I tried running the version.sh script and ended up seeing the following error.

./version.sh
Using CATALINA_BASE:   /opt/apache-tomcat-8.5.11
Using CATALINA_HOME:   /opt/apache-tomcat-8.5.11
Using CATALINA_TMPDIR: /opt/apache-tomcat-8.5.11/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /opt/apache-tomcat-8.5.11/bin/bootstrap.jar:/opt/apache-tomcat-8.5.11/bin/tomcat-juli.jar
Error: Could not find or load main class org.apache.catalina.util.ServerInfo

This last little issue was due to the jar files not being readable by a non-tomcat user (ie world file permissions)

Once I changed the permissions to 644 it was then possible to run the version script

Using CATALINA_BASE:   /opt/apache-tomcat-8.5.11
Using CATALINA_HOME:   /opt/apache-tomcat-8.5.11
Using CATALINA_TMPDIR: /opt/apache-tomcat-8.5.11/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /opt/apache-tomcat-8.5.11/bin/bootstrap.jar:/opt/apache-tomcat-8.5.11/bin/tomcat-juli.jar
Server version: Apache Tomcat/8.5.11
Server built:   Jan 10 2017 21:02:52 UTC
Server number:  8.5.11.0
OS Name:        Linux
OS Version:     4.4.0-53-generic
Architecture:   i386
JVM Version:    1.8.0_121-b13
JVM Vendor:     Oracle Corporation
Posted in programming | Comments Off on Installing Tomcat on Linux

Installing Java 8 on Linux

I hate to admit it but for some tasks on Linux are easier for some less technical people on other operating systems like windows. Simply run the executable which, most of the time, prompts for the install path and then proceeds to install to your hard disk.

Each version of Linux is a bit different so the the actual howto’s for installing software may differ slightly.  I wanted to create a small development so I took the new linux mint 18.1 version for a spin.  I rolled a virtual machine so I could do some testing.

The steps for installing the official Java is actually not much more complex than the following.

  • make sure your computer is up to date
  • uninstall existing open JDK Java
  • download official version of Java
  • unpack Java
  • copy or move to its installed location
  • make this version the default

I simply looked all of this up at the mint site.

https://community.linuxmint.com/tutorial/view/1372

My final step is no different than from the tutorial but you do need to do this for each program that you wish to use.  This is the small script that I used to set the both the java and javac program as default using this version.

sudo update-alternatives --install "/usr/bin/java" "java" "/opt/java/jdk1.8.0_121/bin/java" 1
sudo update-alternatives --set java /opt/java/jdk1.8.0_121/bin/java

sudo update-alternatives --install "/usr/bin/javac" "javac" "/opt/java/jdk1.8.0_121/bin/javac" 1
sudo update-alternatives --set javac /opt/java/jdk1.8.0_121/bin/javac
Posted in Command line, Setup From Scratch | Comments Off on Installing Java 8 on Linux

security of passwords

It was actually a very long time ago when a friend of mine contacted me about a Word Perfect data file.  It seemed that one of his employees had decided to pick up and leave the company without any notice.

Nothing was stolen, broken or sabotaged but there was a rather interesting file in the document directory that was password protected.  The internet was not quite as advanced then as it is today.  I did manage to find a company selling software that could crack the password but as the file was named resume, and she had left for a new job they didn’t really want to spend the money for this single use.

If that same event occurred today it is pretty likely that we would not be barred from viewing the file.

I did a quick check and there are a lot of people on the internet with hints/videos/software to remove form protection from Microsoft Word, sheet protection from Microsoft Excel, and passwords Adobe PDF’s just to name a few.  I would have thought that it would have been limited to file based data such as various office documents but the list also supported more versions of the Windows operating systems and even email packages.

If you don’t have physical control over your data, then you don’t control it at all.  I didn’t have the budget to purchase some of these software packages to test out their claims but I did decide to take a chance on some of the free packages that were readily available for removing passwords from some of my documents.

The only reason that I ended up dealing with password protected documents is because I just received one. It wasn’t that password protected documents was such a stimulating topic but because the password for the document I needed didn’t work.  The colleague that sent this to me was on vacation by the time I tried to access it.

Rather than finding and speaking with her coworkers as well as convincing them that I did need this info, I simply downloaded a free program of my own.  Within 10 minutes from start to finish I was able to access the contents of the document.

Rather than to over simplify the task for everyone else, I will let you find your own methods for circumventing password security on your data files.  I will simply list one of the companies which has a commercial product that does seem support removal for different software packages and operating systems.

Password removal software

http://www.openwall.com/passwords/

 

Posted in security | Comments Off on security of passwords

Governmental picking winners and losers

I didn’t follow the most US election terribly closely but I did actually hear quite a bit about some of the proposals of the two candidates. I keep thinking that “Make America Great Again” does sound like a catchy slogan and who doesn’t want to live and work someplace great.  I can also understand that having a well maintained or good infrastructure is probably a key component.

If you ask some of the Minnesotans about their collapsed bridge perhaps they would respond that they want a bit more greatness in terms of their infrastructure.  Apparently it was a bit deficient and perhaps a bit more effort on general infrastructure would be a good idea.

During the election I also heard something about coal.  I only know about coal from schoolbooks when we were studying the industrial revolution.  It was apparently a dirty way of powering those steam engines but in general humanity made a great leap forward and it was due to coal as well as iron. We literally would not be were we are today without the sacrifice of the dinosaurs (in their own way) to help produce this magic fuel.

Nobody can stop the march of time nor can they stop the march of progress.  During the first industrial revolution the steam engine was revolutionary but it wasn’t as efficient method of power generation as some of the new comers – like the internal combustion engine.  The government didn’t spend its efforts encouraging the railroads to stick with the steam engine they let the market decide and despite the romance of the steam locomotive, its time had come.

It was not that long ago when President Obama was criticized for trying to pick winners and losers in the energy game.  This was done partly by the generation of laws that favored one particular position.  If this article is to be believed the Obama administration enacted 600 major pieces of legislation that at best made business difficult.  It isn’t really a question that the government took sides in which companies should be helped and which should have to go it alone.

Should we have optimism about the new President Trump as he is from business and should not fall into the same trap?  I suspect that his greatest detractors wish, if nothing else, he doesn’t knock the apple cart over.

He has not yet taken sides as president, but he did veer pretty close to taking sides this as President Elect – with Carrier Corp in Indiana.  But my current soapbox desire has nothing to do with the newly elected President Trump but rather a few forward thinkers in Wyoming.

It seems that the Wyoming legislature wants to select a few industries as the the winners in the generation of energy while creating some pretty stiff fines for anyone who tries to use newer alternatives for generating energy.  The only acceptable options for energy creation is this list.

  • coal
  • natural gas
  • hydroelectric
  • net-metered systems (for private use)
  • nuclear
  • oil

I can understand if Wyoming wishes that all electricity is produced in state.  This does give them some level of control over power should they ever need it.  It also makes sense if there are local laws that discuss what is environmentally acceptable in the generation of that power. It does seem to be a very protectionist move, by the largest coal producing state in the United States, by legislating (if it passes) what the only alternatives are.  Creating fines against forward thinking that would make virtually impossible for a utility to test out alternative methods of energy creation.  This does seem to be a bit odd considering that the local politicians backing this legislation were all republicans.

I would have thought they would have been pro-business not pro-protectionism.  Does the protectionist policies for a specific industries really level the playing field or is it just different form of government subsidy.  How big of a leap is it from subsidy to corporate welfare?  I am certain that the fine politicians in Wyoming have this all under control as well as having an eye out for technologies for the future.

I love seeing those old steam powered trains in the movies and so perhaps a coal resurgence will bring them back.  I do look forward to a steam powered future similar to the one found in the movie “Sky captain and the world of tomorrow“.  After all, who wouldn’t want giant robots, flying robots, zeppelins and of course flying aircraft carriers.

We are lucky that the legislators in Wyoming are such visionaries.

Posted in Soapbox | Tagged | Comments Off on Governmental picking winners and losers

command line fun – finding a list of files

In Unix at the command line there is almost always an alternative method to do something. I usually remember this when I need to process a few files in the current working directory and I may not always have the most concise command at hand.

Recently, I only wanted older files and none of the contents from the subdirectories.  Of course you can simply gather up the files in a small temporary file and then process them from there.  This also gives you a chance to even manually edit the file a bit.

This doesn’t work well, when you have to ask IT to perform some task for you.  They don’t like dealing with manual steps, nor do they like dealing with more than a few files in the directory for a single request.

With this in mind, I needed to come up with a command that they could execute that would gather up all of the files without me having to create a hard coded script with hundreds of move commands.

ie.

mv  20151029_0605200032_66677701232222299.xml  manual-processed
mv  20151030_0605200032_66677701232222299.xml  manual-processed
mv  20151031_0605200032_66677701232222299.xml  manual-processed

I wouldn’t have to deal with this at all, if some my colleague’s programs cleverly dealt with cleanup after they processed their data but that is another story.  So this time I did some research on the internet and found out a elegant solution for my  problem.

The task
Move any old xml data files older than three days to the manual-processed subdirectory which exists in the current directory at the same time ignoring any other subdirectories and their contents.

The solution
Use the find command to create a list of these old files.

I like the find command because it is possible to process a directory with a virtually unlimited number of files without any “line length to long errors”.  The find command can also find files based on the last time a file has been modified.  This is the same as finding files based on a creation date if the file doesn’t get altered since creation.

First attempt

cd <data directory>
find . ! -name . -prune -name "*xml" -mtime +3 -exec mv {} manual-processed ";"

The find command on some operating systems provides maxdepth and mindepth however, as that is not an option on the Solaris I am using, I will have to use the -prune command line option.  This will not list anything underneath “pattern”, which in this case is the current working directory.

The command works just fine except it includes both the manual-processed and processed subdirectories from my working directory in the list of files.

It is possible to excludes things using the exclamation symbol.  Thus it is possible to add patterns or choices that should be excluded.

Second attempt

cd <data directory>
find . ! -name . -prune -name "*xml" ! -name processed ! -name manual-processed -mtime +3 -exec mv {} manual-processed ";"

This actually works just fine.  It filters out both the processed and manual-processed directories.  This can easily be used to solve my problem.

Note: Because I was using -name “*xml” I don’t need to worry about any subdirectories that do not contain xml in the name.  If you want to look at all files but exclude these directories then you would leave off the name (ie -name “*xml”) but add the directories (! -name processed)

Extra Credit edition

In my case, I only have two subdirectories that I need to filter out so the previous solution is just fine.  Yet, it doesn’t scale well if some unexpected directories also exist and also need to be excluded.

The key is the exclamation symbol which is used to negate the arguments that follow as part of the pattern thus excluding them.  We can use the -type argument of find command to limit what we are looking for to a certain type and then negate it.  In this case it is directories.

cd <data directory>
find . ! -name . -prune ! -type d  -mtime +3 -exec mv {} manual-processed ";"

This final command limits the find to everything in the current working directory while excluding any directories.  This will be further limited to any files that are older than 3 days.  I wasn’t really only interested in XML files I was really interested in everything.

Note: The prune command line option is pretty finicky.  In order to use it to ignore the subdirectories you pretty much need to start out your find like this.

find . ! -name . -prune (more options here)

 

Posted in Command line | Tagged , | Comments Off on command line fun – finding a list of files