Reuse a Pi – Proxy server

The Raspberry Pi was (and still is) an incredible computer. It was the size of a small deck of cards, ran Linux, could manipulate other devices via the GPIO pins, cost only 35 dollars and is just as capable as a normal personal computer.

If one was good then two must be better and so some how I ended up buying a few of them.  I did create my own little weather station and to fool around with other Arduino peripherals.  It was a lot of fun but somehow they got set aside in a shoe box.

It seems to be a shame to not use these so I am trying to find a use for these devices.  The first use was to create a Airprint server so my wife could print from her Ipad.

What is a proxy server

A proxy server is simply a server that is acting as a gateway between two networks. Typically the second network is the internet.  There are a number of different types of proxy server’s that can be installed.

Cache server The server saves the internet content locally.  This makes subsequent calls by either the same or different clients faster as the information is cached locally.

The server is defined to listen to a specific port and the client must be specifically configured to use that proxy server.

Transparent server A transparent server performs the same function as a cache server but with a single distinction.

The client does not have to be specifically configured nor know that a proxy server exists.

Reverse server A reverse proxy performs a similar function of caching results but instead of running on the client’s network for the benefit of the client’s company it runs for the benefit of the web server.  Its goal is to reduce the load from the web server.

Installing a proxy server

In the open source world there is seldom a single option for any given software solution. This is true from web servers to office suites.  There is undoubtedly a number of proxy servers as well but the one that kept coming up in my searches was Squid.

Installing squid on the Raspberry Pi is just as matter of the following apt-get command.

apt-get install squid3

This is a small and self contained program.  The program squid3 is installed into /etc/squid3 along with the squid.conf configuration file.

The default configuration for squid won’t let anything through, you will need to make a few tiny changes.  Below are a few extracts from the configuration.

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
#acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
#acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
#acl localnet src fc00::/7 # RFC 4193 local private network range
#acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl localnet src 192.168.178.0/24

In this excerpt I needed to add line 9 for my network.

 

The change above is used in conjunction with a second change to the configuration file, line 8 below to use this “localnet”.

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost 

This is actually the only configuration that is necessary in order to use the proxy server.

 

However it is possible to add a few additional lines to try and filter out advertisements a bit.  Just add them at the spot highlighted in the configuration file. This small insert allows me to define places on the internet in a text file or text files of locations that I don’t want to be accessible.

## disable ads 
acl ads dstdom_regex "/etc/squid3/ad_block.txt"
http_access deny ads
deny_info TCP_RESET ads

acl urlads url_regex "/etc/squid3/adurl_block.txt"
http_access deny urlads
#deny_info TCP_RESET urlads

For my proxy server I have setup two text files. The first one is for sites that I want filtered out.  The second text file could contain url’s for sites that deliver advertisements. This makes it easy to either reduce the advertisements for the sites you want to visit while preventing unsavory sites from getting through.

 

Blocking the URL’s is as simple as you think.  Simply enter the name of the website that you would normally enter into your browser.

adurl_block.txt
www.nakedgirls.com
www.myspace.com

The same is true for blocking the ads as well.  It is harder to give a clear list of which domains are serving advertisements.  It is probably best to find a blacklist of those domains.

There are other sites that are dedicated to keeping up-to-date lists for these advertisement domains. I don’t feel that I am adding any additional value by providing that here.  If you want such a list then you should go and find one of those sites.

https://www.calomel.org/squid_adservers.html

This site also has additional information on how to setup squid.  If you want to learn more visit calomel.org.

Setting up the client

The squid proxy server will look at port 3128 by default.

# Squid normally listens to port 3128
http_port 3128

So when setting up the browser to use the new proxy server point it to the raspberry pi and to this port.

Other notes

When updating the different ad blocking text files you could get these values reloaded by restarting squid or by running the reconfigure option which will re-process the configuration file.

squid3 -k reconfigure

 

Posted in Setup From Scratch | Leave a comment

bash – More powerful than a speeding locomotive

(flashback)

Our project was inching closer and closer to going live.  On the evening before the Easter break some clod started a massive report which tried to process all the data since the beginning of time. My colleague’s inefficient program gathered up all of the data from the database and wrote it to the data directory in a temp file.

The data directory was more than large enough for a test environment but it wasn’t a big enough for such a test.  We could have easily recovered if it had occurred while we were at work but instead the system attempted to process nightly batches over those four days without enough space and made a pretty big mess.

My boss Theodore was more upset than he should have been for a test environment and kept yammering on about what if this has been production.  He was right of course but one of the preconditions of the system is that enough resources are available.  It is our groups responsibility to write the programs but it is someone else’s to ensure that the system doesn’t run out of resources.

Anyway, we implemented the boss’ warning email feature.  Each time the program is run it checks for enough disk space when not enough space exists then send out a warning email and quits. To be on the safe side, my boss asked that I have my email address as one of the recipients.

(the present)

If I receive one more warning email from one of the test systems I am afraid I am going to kill someone.

Von: Automatically.Generated.Message@acme.com [mailto:Automatically.Generated.Message@acme.com] 
Gesendet: Dienstag, 7. Februar 2017 11:54
An: process_monitoring@acme.com; 
Betreff: Warning ... the end is near on acme-app1

An error has occurred, the copy app has not been launched 
because of insufficient disk space on following partition.

Filesystem             size   used  avail capacity  Mounted on
acme-app1_dpool/app    85G    85G     0K   100%    /appdir

Corrective action is required immediately.


The status of rest of machine is as follows.

Filesystem             size   used  avail capacity  Mounted on
/                       10G   7.7G   2.3G    77%    /
/dev                    10G   7.7G   2.3G    77%    /dev
proc                     0K     0K     0K     0%    /proc
ctfs                     0K     0K     0K     0%    /system/contract
mnttab                   0K     0K     0K     0%    /etc/mnttab
objfs                    0K     0K     0K     0%    /system/object
swap                   140G   400K   140G     1%    /etc/svc/volatile
fd                       0K     0K     0K     0%    /dev/fd
swap                   8.0G   700M   7.3G     9%    /tmp
swap                   140G    40K   140G     1%    /var/run
acme-app1_dpool/app    85G    85G     0K   100%    /appdir
acme-app1_dpool/acme_home   1.0G   353K   1.0G     1%    /appdir/home/gast
acme-app1_dpool/acme_samba   2.0G    36K   2.0G     1%    /appdir/samba
acme-app1_dpool/acme_scripts   2.0G   249M   1.8G    13%    /appdir/scripts

This is an automatically generated message for informational purposes.

The idea seemed ok; when no disk space exists then send out an email.  The underlying assumption was someone in IT would deal with the problem.

Apparently the idiot users turn off half of the system about a week back but not every process.  I came to work and found hundreds of emails clogging up my inbox.  Looking through them you could literally see the space filling up over time.

Well, hundreds of files are annoying but the general functionality is awesome.  A combination of the bash script and sendmail allows me to capture the important facts about our system and send it to someone.

Just look at the script.

#!/usr/bin/bash
SUBJECT="warning the end is near"
DF_Command=`df -h ${FILESYSTEM}`
FULL_DF=`df -h`
TO=process_monitoring@acme.com
FROM="automatically generated message"

HOSTMACHINE=acme-app1

( cat << !
To: ${TO}
From: ${FROM}
Subject: warning ... the end is near on $HOSTMACHINE

An error has occurred, the copy app has not been launched because of insufficient disk space on following partition.

${DF_Command}

Corrective action is required immediately.

The status of rest of machine is as follows.

${FULL_DF}

This is automatically generated message for informational purposes.
!
) | /usr/sbin/sendmail -t

Fill the variables with information ranging from a single word up to a lot of lines of text and then substitute them into your mail.  The bash shell will expand them before sending the mail out.

I guess that the moral of the story should be that more logic should be used because some idiot will inevitably trigger it on a non-production environment.  Well, that or just get rid of the idiots ….

 

Posted in programming | Tagged , | Leave a comment

facial scans coming to an airport near you

We want to protect you from the world.

I think that a lot of people want to do the right thing and protect others from bad things. Sometimes this takes shape in the form of a law or a rule that prevents bad or dangerous behavior.  The rule usually looks pretty innocuous – you should not put poisonous chemicals into peoples food or drink.  I think that everyone would easily agree to this simple rule.

Where exactly do you stop?  Does that mean that grilled food and french fries cannot be eaten as they may kill you?  Does that mean we need to tax food that when eaten in a large enough quantity will make you sick?  Should we tax foods with a high sugar content?  Perhaps we should ban some ingredients such as transfats for the same reason.  Some of the food that we eat is killing us so I guess that food or product should be eliminated – right?

This type of slippery slope is a classic example of mission creep where a focused task ends up morphing into something altogether different.

Biometric photoshoot at the airport

The project is called “Biometric Exit” and would match the photo of the visa holder as they are leaving.  I guess this would be an improvement.  Instead of a database with an entry that says the visa holder has left there will be a bigger database entry that also includes a departure picture.  I would guess an arrival photo would also be nice.

This might be an easy way to see who is overstaying their visit and I would guess the penalty for overstaying might be loss of future visa privileges.  This solution, even if rolled out to all of the american international airports, does not really track the people the US is most interested in – the people who have overstayed their limit.

Well, that is not entirely true.  I suspect that at all of those international airports your photo will be taken regardless of if you go from New York to Singapore or New York to Sacramento. If they do the second one then they are actively tracking people who are not tracking the country.

In addition to all this extra photo recognition there is another feature of this program. This facial recognition system will be shared with other agencies and perhaps the airlines themselves.  If this database is shared with the airlines it is just a hop skip and a jump before the shops in the airport scan your face and send you an sms trying to sell you stuff.

Just imagine what the next steps might be.  Simply connect this system up with the other government database.  Feed in the department of motor vehicles, US passport database, military roster database, former and current prisoners. That is just to get started, within a decade this type of photo check could be required before entering any Federal building in the US.

Beyond that, once all of the infrastructure is in place this system could eventually be used to track down anyone with an outstanding ticket, late child support payments or even if you owe income taxes.  This system could become a premier method of tracking people within the US and not just the visitors.

I am confident that the government doesn’t have anything like that in mind – right?

Posted in Soapbox | Tagged , | Leave a comment

From computers to microcontrollers

I kept telling Mikhail that the big advantage to using the Raspberry Pi was no matter what project I put it into I could set it on the window sill and ssh to it.  It was a fast small and affordable personal computer.  He kept telling me that the Raspberry Pi and Raspbian wasn’t a real-time solution and that I should consider something else.

He was of course right but this didn’t matter for most of the things that I was using the Raspberry Pi.  The best example of a custom solution was the led cube that I created that displayed various patterns.

The Raspberry pi did a super job for my cube but when I tried to use it on a distance sensor project using the HC SR04, I definitely had mixed results.

I wanted to use my cube more often but there was no convenient way to turn it off, well without first connecting to it via ssh and shutting it down.  It took too long to boot, too long to shutdown, and requires a proper shutdown.  What about migrating from a computer to a micro-controller – it starts faster and can simply be powered off.

Cheap as potato chips

Mikhail kept telling me that micro-controllers were cheap as potato chips and could easily be included into virtually any project. To encourage this path he decided to do this old school which meant a pile of chips, an AVR programmer, LED’s, resistors, a breadboard and a bunch of wires.

AVR Programmer

attiny85

Mikhail comes from the electrical engineering side of computers and it was him who discovered, all those years back, some of the interesting projects being done with LED’s.  Below are a few interesting projects that he showed me from the internet.

Well, I come from the software side but was willing to learn.  I no longer have photos of those old projects but I put together a rather similar example.

Instead of one of those ATTINY85’s I have used an ATMEGA328.  Granted the 328 is a much nicer chip but for the task of doing I2C the two chips are equivalent.

Hardware setup

It has been awhile since I used Fritzing or Eagle to write up a circuit diagram.  I will simply show which pins need to be connected up.

ATMEGA328 MCP 23017 Comments
A4 (pin 24) Pin 12 I2C pin SCA
A5 (pin 23) Pin 13 I2C pin SDA
5V Pin 9 (Vdd) 5 V power
GND Pin 10 (vss) Ground
Pin 15 (A0) tie to ground
Pin 16 (A1) tie to ground
Pin 17 (A2) tie to ground
Pin 18 (! reset) tie to 5v
GPA 4 connect to led
GPA 5 connect to led
GPA 6 connect to led

I2C explained

The I2C bus was created by Philips in the 1980’s.  It was actually a rather clever design decision for connecting CPU’s and peripheral chips together.  The use of a single bus is much more efficient in terms of wire used.  All chips on the bus do listen for commands that are relevant to them. This is is because each chip has its own address and thus listens for its own address.

It is possible to add any number of bytes via this bus to the receiving chip.  The bus is bi-directional, I can send messages to another chip but I can also read back responses from those chips.

The actual description of the how the low level technical details for this two line bus is actually better described elsewhere.  One explanation can be found on Wikipedia I2C.

The good news is that you don’t need to know how the bits are sent over the wire to use it.  Arduino has a very simple class Wire which makes this task almost trivial.

Wire Method Description Example
begin() initializes connect to bus Wire.begin()
beginTransmission(adr); begin command to send to device “adr” Wire.beginTransmission(0x20);
write(byte2send) sends a single byte to device from begintransmission Wire.write(0x00);
endTransmission() builds entire command and transmits it to device. Wire.endTransmission();

Lines 10 – 13 is an actual example of sending the value 0x00 to the register “port” on device 0x20.

Software

This sample bit of code will simply turn on the green led, yellow led and red led in turn with a pause between each one and then it will turn all leds on for 1500 milliseconds.  Finally it will turn off all the leds for 1000 milliseconds (one second).

This particular project is not completely interesting in itself but rather as a stepping stone to more interesting devices.  I will be replacing my raspberry pi with this Arduino in my next blog.

#include <Wire.h>

int GPA4 = 0x04;
int GPA5 = 0x08;
int GPA6 = 0x10;
int PORTA = 0x12;

void alloff(int port)
{
  Wire.beginTransmission(0x20);
  Wire.write(port);
  Wire.write(0x00); // all pins off
  Wire.endTransmission();
}

void scanner()
{
  Serial.println ("\nI2C scanner. Scanning ...");
  int found = 0;

  for (byte i = 1; i < 120; i++)
  {
    Wire.beginTransmission (i);
    if (Wire.endTransmission () == 0)
    {
      #if 0
      // this is much simpler but takes 2k more of program space.
      char buffer[128];
      sprintf(buffer,"Found address: 0x%02x",i);
      Serial.println(buffer);
      #endif
      
      Serial.print ("Found address: ");
      Serial.print (" (0x");
      Serial.print (i, HEX);
      Serial.println (")");
      found = 1;
      delay (10);
    }
  }

  if (found)
    Serial.print ("Found device(s).");
  Serial.println();
}  // end of setup

void setup()
{
  // real initialisation stuff
  Serial.begin (115200);
  Wire.begin();

  // do a bit of searching for things.
  scanner();

  // setup our 23017
  Wire.beginTransmission(0x20);
  Wire.write(0x00); // IODIRA register
  Wire.write(0x00); // set all of port A to outputs
  Wire.endTransmission();

/* we dont actually use port b
  Wire.beginTransmission(0x20);
  Wire.write(0x01); // IODIRB register
  Wire.write(0x00); // set all of port B to outputs
  Wire.endTransmission();
*/
  alloff(PORTA);
}

void expanderPinOn(int port, int pin)
{
  Wire.beginTransmission(0x20);
  Wire.write(port);
  Wire.write(pin);
  Wire.endTransmission();
}

void loop()
{
  expanderPinOn(PORTA, GPA6);
  delay(600);
  expanderPinOn(PORTA, GPA5);
  delay(600);
  expanderPinOn(PORTA, GPA4);
  delay(600);
  expanderPinOn(PORTA, GPA4 | GPA5 | GPA6);
  delay(1500);

  // off
  alloff(PORTA);
  delay(1000);
}
Posted in programming, Setup From Scratch | Tagged , | Leave a comment

enhancing the privacy we still have

Is privacy is simply too much work?

The problem is not that technology cannot help out but in my case it seems that I am constantly installing or setting up new computers or virtual machines and there is never the time to keep the digital hounds at bay.

The internet is just unrelenting in its attempts to mark us and remember everything we have ever typed.  Of course the flip side is then the various ad agencies then use this information to push their marketing directly to us but only after it has been tailored to our history.

I don’t mind the marketing, well perhaps I do a bit, but big digital shouldn’t need to know about anyone’s interest in fluffy bunny slippers or anything else.

The good news is that the Electronic Frontier Foundation apparently feels about the same as they have created a addin that helps protect against non-consensual Web tracking.  Not only that they have created a web page that can perform a test to see how protected your browser is.

 https://panopticlick.eff.org/

Simply press the big “test me” button to find out how anonymous you really are.

You can see the output when I tested my browser, not so great.  The results were much better the second time after installing privacy badger.

I don’t know if this is available for internet explorer but this is available for both Chrome and Firefox.

I can only recommend installing this on any computers that you wish to try and keep a small amount of privacy.

Posted in security | Tagged , | Leave a comment

security in obscurity

It didn’t make all that much sense to me. Why did the team write their own bug tracking application when they could have either purchased one or used an open source solution.  I guess the team felt that only they were creative enough to create a good bug tracking application.

Not created here

I never really liked the application, not because it had a tendency to slow down the process but actually because of the poor usability of the application. I am not certain where all of the textual comments were stored but the developers simply stored all attachments as files on a samba file system that was also shared. Just to keep up security or secrecy all of the files had the attribute of hidden and and read only.

I knew how the attachments were saved as did my IT colleagues also knew how the files were saved. If you wanted to retrieve the attachments it was actually more convenient to access the files directly.

During the beginning of the project there was a lot of external consultants and a number of there were also aware of this little back door.

No good deed goes unpunished

Once the project was finished everybody left but a few years later a few of them came back including John who is now an auditor. I was speaking with my colleague Kenny when I found out that John had been in touch and was asking about how we used to access these files. Kenny is a nice guy and reminded John exactly where the files were located. Kenny being an inquisitive guy asked why exactly did John need this information. The answer was too cute.

I am writing a report about weaknesses of the systems here

This is a weakness that none of the average user would ever have been able to find or to utilize. I put this story out of my memory because up until very recently Windows Explorer would display which files existed even the hidden ones. A few key presses and you could even search for the hidden attachments.

This was a nice little backdoor but you can always use the application right? Nope, some big brain wrote the application front end using MS Access about a decade ago and for some reason they couldn’t get it to work using Access 2013. I am in another group so it isn’t clear what my old group is now doing but they are still creating these hidden files.

I needed a file and could not use the application nor windows explorer.

Security in obscurity

The problem is that Windows itself is plenty willing to tell you which files exist if you ask correctly.

The simple command prompt, cmd.exe, will blab either overtly or covertly what it finds.

Name Description
dir This will do the directory listing of the normal files but when you give the /ah parameter will will display the files with the hidden attribute.
attrib I believe that this program has its roots in the DOS operating system that windows was built on top of.  This command will show all the files in the current directory and what attributes are set (archived, read only, system or hidden)

These are actually pretty direct but I was surprised that simply pressing the tab key in my command would display each file in the directory including those that were hidden.

 

I supposed that you could simply use the dir command to display all files that were hidden but it is possible to make our own command to display all the files regardless of their hidden status.  this is done using a few lines of powershell script.

All you need to do is to have the following line as a power shell script.

myfind.ps1
Get-ChildItem . -recurse -force

This will display all files in the current directory as well as all files in any subdirectories.  This is a neat little trick considering the files must be hidden for a particular reason but it is very easy to find the files anyway.

powershell -file myfind.ps1

Showing everything was actually the entire purpose of this script but it is also possible to filter this list of files down to only the hidden files.

myfind2.ps1
Get-ChildItem . -recurse -force  | where-object  {$_.mode -match "h" }

This script actually will find the hidden files that are in the current working directory as well as in any subdirectories.  This isn’t so helpful as it only lists those files but it is possible to take this to the next step and copy those files elsewhere.

myfind3.ps1
get-ChildItem c:\mysourcedir -filter *.*  -force | where {($_.extension -eq ".txt" -or $_.extension -eq ".text")} | Where-Object {$_.mode -match "h"} | copy-item -destination c:\mydestinationdir 

The only problem for this tiny script is that the files are copied but they retain their hidden attribute.  I am fairly certain that you can remove this attribute with some more clever powershell code but I couldn’t find it.

In the end I reached out to DOS history to clear this hidden attribute.  The good news is that powershell scripts can call other programs in addition to these cmdlets.  My final script copied these files and then used the DOS command “attrib” to do the clear.

myfind4.ps1
get-ChildItem c:\mysourcedir -filter *.*  -force | where {($_.extension -eq ".txt" -or $_.extension -eq ".text")} | Where-Object {$_.mode -match "h"} | copy-item -destination c:\mydestinationdir 
attrib -h c:\mydestinationdir\*.*

 

Not so secret after all

I actually realized that part of the reason I cannot see these hidden files in the windows explorer is because I have a new computer and have forgotten to change my computer settings to display hidden files.

https://www.lifewire.com/how-to-show-hide-hidden-files-and-folders-in-windows-2626085

With these changes back in place you can even search for files hidden or not using the windows explorer tool.

All of this points to the fact that you cannot use windows hidden files as any real security measure and security in obscurity may work for a while but eventually someone with time on their hands will find all your hidden gems.

Posted in security | Tagged , | Comments Off on security in obscurity

Reuse a Pi – Air printing

When the Raspberry Pi came out I went a but nuts and ended up getting a few of them.  I did give some away and did create some projects but unfortunately not all of my RasPi devices were a complete hit with the family.

Eventually they got unplugged and ended up in a small box, and yesterday that small box fell over.  I don’t have enough people interested in ancient Rasberry Pi computers – they are simple model A and B pi’s.

The raspberry pi has been out long enough that there should be a few new projects that these devices could be put into. The first project seemed to be a no-brainer.  My wife wanted to print to the printer from her iPad but was unable to.  It is possible to make this happen by using the common UNIX printing system or CUPS allows a server to act as a print server.

CUPS uses the internet printing protocol (IPP) for managing print jobs and printer queues.  Despite the how obvious this solution seems it was actually created in 1997.

Step 1 – Install operating system

The only thing that was not immediately clear was if he old pi’s needed an old raspbian distribution or if a new distribution image could be used.

There are two different ways to determine this.  The first is to download the latest distribution and give it a try.  However, apparently my SD cards have a “use by date”.  I suspect that this is the casee as my old sdcard worked for a short time and then had some odd behaviors.

Everything did work just fine on a new SD card.  This information is also available from the Raspberry Pi forums as well.

The process of actually installing the operating system to a SD card is pretty well defined.  This is especially true for raspberry pi organization.  For more information on installing the operating system check out their guide.

I used the most current Raspbian that was available from April 10th 2017.

Step 2 – Install and configure Cups

Note: It would be a good idea to ensure that your printer is supported before trying to setup this solution.  In my case, the color laserjet CM1312nfi MFP was not in the list.  I did feel lucky with this printer as I have installed cups before on LinuxMint and I did not have any problems.

Always a good idea to ensure that your repository is up to date before starting.

sudo apt-get update
sudo apt-get upgrade

Install the cups software

sudo apt-get install cups

Configure cups

The first command is to add the pi user to the printing group (lpadmin) so we do not need to use sudo.  The next command is to make cups available to the other computers on our network.

sudo usermod -a -G lpadmin pi
sudo cupsctl --remote-any
sudo /etc/init.d/cups restart

Once we do these commands we must restart the service so the changes to the configuration file are used.

Note: The cupsctl command simply modifies the /etc/cups/cupsd.conf configuration file. The changes are actually quite small, but just to make sure nothing is lost I would recommend that you use the cupsctl command.  The changes are highlighted at the end of this post.

Test Cups installation

This is just a matter of pointing your browser to the Raspberry pi that has cups installed.  Specifically both the machine and port.  When you do that, you should see a screen that looks similar to the following.

Setup a CUPS printer

The entire process from this point is really quite easy. Simply click on the link for Adding Printers and Classes.

At the beginning I checked to see if my printer was supported (https://www.openprinting.org/printers) and my exact printer was not in the list. However, I have had a lot of luck with HP printers so I went ahead anyway.

I tried the “Find New Printers” option but it did not find my network printer so I simply tried to add a new printer.  CUPS did manage to see more printer information relevant to my printer.

Step 1

 

          

Step 2                                                                                        Step 3

 

 Step 4

Step 5

In step 3, as my exact printer model was not in the list I selected “HP Color LaserJet Series PCL 6 CUPS(en)” as my model and continued.  The printer appears to have been installed for CUPS.

In order to give this better test, select “Print Test Page” from the Maintenance drop down to generate a test page.

 

Step 3 – Using the newly setup printer

Well, there is really not much to say here.  In my experiment after setting everything up printing was super easy.  I just picked up the tablet and from an application, safari in my case, I pressed the print symbol and saw that my printer was an available option.

 

Printing from Apple Phone, IPad or IPad Touch
https://support.apple.com/en-au/HT201387

When reading up on this there were a couple of things that other people said might be necessary.  In order to publish the printer name I might need to install avahi.

sudo apt-get install avahi-daemon

I did not have to install avahi in my case.

It was also mentioned that after setting everything up you might need to wait about 5 minutes for the printer name to be disseminated across the network. I suppose I did wait for a while but I cannot say how long.

Also, apple does provide pretty good possibilities for windows computers as well.  The protocol that they use to disseminate the printer information is bonjour.  They even make available a client for your windows program so you can take advantage of this as well.

https://support.apple.com/kb/DL999?locale=en_US

 

Original cupsd.conf

#
# Sample configuration file for the CUPS scheduler. See "man cupsd.conf" for a
# complete description of this file.
#

# Log general information in error_log - change "warn" to "debug"
# for troubleshooting...
LogLevel warn

# Deactivate CUPS' internal logrotating, as we provide a better one, especially
# LogLevel debug2 gets usable now
MaxLogSize 0

# Only listen for connections from the local machine.
Listen localhost:631
Listen /var/run/cups/cups.sock

# Show shared printers on the local network.
Browsing On
BrowseLocalProtocols dnssd

# Default authentication type, when authentication is required...
DefaultAuthType Basic

# Web interface setting...
WebInterface Yes

# Restrict access to the server...
<Location />
 Order allow,deny
</Location>

# Restrict access to the admin pages...
<Location /admin>
 Order allow,deny
</Location>

# Restrict access to configuration files...
<Location /admin/conf>
 AuthType Default
 Require user @SYSTEM
 Order allow,deny
</Location>

... rest of file follows ...

Changed cupds.conf 

LogLevel warn
MaxLogSize 0
# Allow remote access
Port 631
Listen /var/run/cups/cups.sock
Browsing On
BrowseLocalProtocols dnssd
DefaultAuthType Basic
WebInterface Yes
<Location />
 # Allow remote access...
 Order allow,deny
 Allow all
</Location>
<Location /admin>
</Location>
<Location /admin/conf>
 AuthType Default
 Require user @SYSTEM
</Location>

... rest of file follows ...
Posted in programming | Comments Off on Reuse a Pi – Air printing

safe computing – weak case for weakening encryption

You cannot strengthen the weak by weakening the strong.

William J. H. Boetcker

This quote has been often misattributed to Abraham Lincoln.  This is actually a very relevant thought with the respect to economics.  Indeed you cannot improve everybody’s lot in life by taking from the economically strong no matter how well intentioned your goals are.

I think that this particular quote is even more meaningful with respect to improving privacy and encryption.

Some of the forward thinking political types think that in the very near future all communications will be using encryption.  They also believe that this will destroy the ability of governments to gather intelligence of evil doings.  The “Five Eyes” are at the minimum hoping that silicon valley can find a way that will support the interests of privacy and security.

The desire of these countries sounds very positive but to some ears that sounds like creating a solution that will allows the “protectors” to read encrypted solutions in situations when it is “important”.  This last sentence when properly parsed will actually mean that some backdoor will exist to allow the governments of the world (and any really clever hackers) access to your communications.

That wouldn’t be so bad except the history of man is full of situations where people abuse their positions of power.  Who wants to send out private communications knowing that some government wonk, perhaps your ex-husband, could access your messages with the universal iPhone decryption key.

That would be worrying enough but the world is filled with malicious players.  People interested in looking at more than little text messages you sent to someone special.  They would be taking advantage of weaker communication standards to eavesdrop on our financial transactions.

Weak encryption solves problems. Right?

Every time I read about governments wanting weaker encryption I think back to a specific Dilbert cartoon.  The pointy haired boss is explaining about earnings reports and how they need to be smooth for lazy analysts – the same is true for encryption.  No encryption or easily breakable encryption makes it a lot easier for security agencies to examine messages.  Yet does this really catch more plots?

Weaker standard encryption doesn’t mean that attacks will then be caught before they take place.  People can only be caught doing bad things if they are in constant communication about their plans, someone is listening in and important details are being shared.  Wouldn’t this have helped for some of the terrible incidents that have occurred in the past such as 9/11, London bombings, or Madrid bombings?

Having no encryption at all might not have made any difference at all, assuming that any encryption was used for those incidents.  In the spy novels, action adventure books and other thrillers the bad guys don’t coordinate these things long distance.  Sure, the big boss might know that a special unit has a given mission at a certain location but that is usually it. There is no communication.

Terrorist groups, well the really successful ones, appear to plan an attack and then let that unit deal with the details of the attack.  They don’t appear to be remotely controlled by their organizers.

Recent digital access is an anomaly

It is only the last twenty or so years that the masses have been sending emails and cat pictures through the ether to a recipient.

In the distant past it was possible to listen in on the telegraphs that were being transmitted.  In the recent past it was possible to have phone conversations, and it was also possible that they could also be listened in on.  However, for most of the last 100 years it has not been possible to have a permanent exact replica of every message sent or received so that they can be replayed and stored for future use.

This relatively recent access has been most undoubtedly incredibly helpful for the security services and it is understandable why they are addicted but this is not the normal state over the last century.  The ability to retroactively look through a previously encrypted phone is probably more desirable for prosecution in a case after the fact to help prove intent.

Even if all phones were unencrypted, the issue of bad people using prepaid phones would make the argument moot.  Clever criminals and terrorists would use these phones for very small transmissions of reasonably innocuous messages.

A case of Hubris

It is pure hubris to believe that only the existing security services of the large countries are capable of creating a method or cipher that cannot be broken.

Do the security services really believe that if they weaken the encryption solutions that they are the only ones who can come up secure solution?  Of course not every code slinger can create a flawlessly secure solution but there a lot of different types of other known solutions that can easily be used.

substitution cipher

public key cryptography

one time pad

Steganography

It would obviously be much easier for less talented criminals or terrorists if their Apple phone or Android phone had built in end to end encryption but any of these other methods could be used to keep secrets.

These methods are all known by the authorities so they would only keep secrets for a certain period of time but depending on the situation the secrets might only need to be kept long enough to do bad things.

Cryptography is not a secret

The internet is full of information about encryption but it isn’t even necessary to go to those lengths.  You can purchase books from Amazon about encryption.  I started to count the but there were so many books to choose from.  They all looked pretty good to me.

Besides, people are creative when trying to find a method of keeping secrets.  Good methods can be developed by prisoners much to the chagrin of their keepers.

 

Oh, here is the original quote in context.

You cannot bring about prosperity by discouraging thrift. You cannot help small men by tearing down big men. You cannot strengthen the weak by weakening the strong. You cannot lift the wage-earner by pulling down the wage-payer. You cannot help the poor man by destroying the rich. You cannot keep out of trouble by spending

William J. H. Boetcker

 

Posted in Soapbox | Tagged , , | Comments Off on safe computing – weak case for weakening encryption

just making it – revisiting my cube

Some time back, I had a fair amount lot of free time.  I spent the time working with a friend Mikhail in the evenings to create my own LED cube from scratch.  It was actually quite a bit of fun, the electronics was not overly complicated with most of it being done by a Raspberry Pi. It was fun to see both the hardware and the software side by side.

raspberry pi powered cube

The problem with that particular cube was using the Raspberry pi.  The Raspberry pi is a great platform but it is a computer and it needs time to start up the operating system and more importantly it needs to be shutdown in an organized manner.

The Raspberry pi is an awesome little computer but the startup and shutdown times are my main problem.  I only have a few hundred lines of code that needs to be run so I don’t need a four core 1.2 gHz computer, only need a few megaherz.

Arduino to the rescue

The first release of the Raspberry pi was in 2008 at 900 mHz for 35 dollars was truly amazing but it wasn’t the first of this type of hardware for do-it-yourself electronic projects.

I actually was never involved in the DIY electronics when the Arduino came out but I think that the Arduino was pretty much the pioneer of this space.  The Arduino was not only hardware but it was open source hardware.  The fact that it was open source helped it to expand and multiply into the number of models that we see today.  Well, being open sourced helped but also due to the release of ever increasing sophisticated processors from ATMEL which powered these Arduinos.

Today it is possible to get what is essentially little computer that is not much larger than a couple of postage stamps.

This “advanced” processor is running 20 MHz and contains 32 KB of program memory. Not quite the latest gaming personal computer but it is similar to the original Apple ][ personal computer, but it is more than enough to pass the commands to the cube to switch leds on and off using the I2C protocol.

New Hardware

The Raspberry Pi supports I2C but the Atmega328 does as well.  The I2C protocol is just a matter of two lines – a data line and a clock line.  As far as the hardware was concerned, all I needed to do was to connect the data, clock, power and ground to the Arduino atmega328.

On the software side, I was pleasantly surprised at actually how easy it was using the Arduino libraries to do I2C and to convert my code over.

Sure, I did have to make a number of small changes but the process was pretty painless.  Replace my printf statements with Serial.println statements but the main changes were connected with the I2C protocol.

The libraries to do this were part of the standard development environment.  The class that you need to use is the Wire class.

Old Code

   bcm2835_i2c_setSlaveAddress(cathode);
   char cmd2[] = { IODIRA, 0x00, 0x00 };
   check_retcode(bcm2835_i2c_write(cmd2,sizeof(cmd2)));

New code

   char cmd2[] = { IODIRA, 0x00, 0x00 };
   pi_i2c_write_command(cathode, cmd2, sizeof(cmd2));

The pi_i2c_write_command is my own method but it simply does call the three methods (beginTransmission, write, endTransMision) that are required in order to send an I2C command.  Only five lines of the twenty six are required, the rest were used for monitoring the program while it was being ported.

void pi_i2c_write_command(int device_address, char cmd[], char len)
{
  int idx = 0;
  if (debug != 0)
  {
    Serial.println("begin pi_i2c_write_command\n");
    Serial.print("address ");
    Serial.println(device_address, HEX);
    Serial.print("arguments ");
    Serial.println(len, DEC);
    for (idx = 0; idx &amp;lt; len; idx++)
    {
      Serial.print(cmd[idx], DEC);
      Serial.print(" ");
    }
    Serial.println("\n");
  }

  Wire.beginTransmission(device_address);
  for (idx = 0; idx &amp;lt; len; idx++)
    Wire.write(cmd[idx]);
  Wire.endTransmission();

  if (debug != 0)
    Serial.println("end pi_i2c_write_command\n");
}

arduino powered cube

 The Problems

I did have one small problem while porting my software that was really an oversight. Before you try and transmit any data via the I2C bus you need to initialize it first with the a call to “begin”.

Wire.begin();

If you don’t make this call, none of the subsequent calls do anything.  In my haste, I did have this line but it was in the wrong spot in my initialization code.

My code does have comments but it has been a while since I was using the 23017 16 bit I/O expander.  So I did have a few questions how it worked exactly.

While doing my debugging I did read up more about the chip and I do have a few clarifications that describe the code’s behavior.

register Description
IOCON This register is used to set various configuration options on the chip. The two most interesting as far as this program goes is the BANK (bit 7) and the SEQOP option (bit 5).

The BANK bit changes the addresses of the various registers.  When BANK=1 then the address for IODIRB is 0x01 but when the BANK=0 then the address for IODIRB is 0x10.

The order of the registers is more interesting in conjunction with the SEQOP register.  The SEQOP=1 allows the chip to write multiple bytes sequentially.  When the BANK=0 then the register pairs are next to each other.

IODIRA

IODIRB

These registers are used to set the direction of the I/O. The IODIRA register controls the direction for first 8bit port (PORTA) and the IODIRB register controls the direction forthe second 8bit pot (PORTB).
OLATA

OLATB

These registers are used to set or clear the latches for the two ports.

The other important detail is that the current flows to the various LED’s (via the I/O expander) when the anode (layer) is set to on and the cathode (column) is set to off.  This sounds exactly what you would expect for lighting up a LED, power goes to anode and ground goes to cathode.

Posted in programming | Comments Off on just making it – revisiting my cube

interrogating your code – dipping your toes into java’s reflection

Perhaps the coolest programming language ability that I was never allowed to use was reflection in Java. Why? Well, thats another story.

What exactly is reflection? Well, reflection is a set of low level methods that can be used to inspect code while the program that is doing the inspecting is running.

This puts the program in a very unique situation where it can interrogate objects that it is working with. Thus the program can then make decisions based on what information it finds on the class that it is inspecting.

This makes it possible to inspect an object to see if it contains a certain method. This allows your program to make the decision which method to use if multiple valid methods exist. One, perhaps somewhat contrived, example of this would be that your program could check and see if the new 2.0 version of your method existed.

Method method = null;
try {
    method = financialCalcObj.getClass().getMethod("calcBreakEvenV2", null);
}
catch (Exceptione ex)
{
    // not really an error, use standard method
}
try {
    if (method == null)
        method = financialCalcObj.getClass().getMethod("calcBreakEvenV1", null);
}
catch (Exceptione ex)
{
    // wow, thats bad
    System.exit(1);
}
method.invoke(financialCalcObj, null);

If this doesn’t seem pretty amazing just remember that you cannot do the following unless you already have all methods defined at compile time.  (which does defeat the idea of trying to future proof your code)

try {
    financialCalcObj.calcBreakEvenV2();
}
catch (Exception ex)
{
    financialCalcObj.calcBreakEvenV1();
}

The difference between the two sets of code is that in the reflection example was compiled before version 2 of this method existed.  We just knew that it would show up eventually.  This could be extended to either use the highest version that existed or to use the version of a method based on a command line argument.

No magic just a bit of forethought required

A program that uses reflection in this manner might not make much of a difference if the same set of class files or jar files are used each time.  It does allow some flexibility but if the code base never really changes it is not necessary to go to these extremes just to change how the program behaves at runtime.

The program that I had been envisioning  was going to use this technology to dynamically support multiple language text.  The dynamic part was the program would convert numbers into text of the desired language.  The program would run every few minutes so it would be possible to exchange jar files between runs, however I think there is an even better solution.

ie.

    numberToTextGerman
    numberToTextEnglish
    numberToTextFrench

Instead of continuously replacing the language jar – each time with more languages I would take a slightly different route.  I would structure the program in such a way that each new language with its methods would exist in its own jar file.

The actual program itself would retrieve the data which needs to be processed and this would include the form language.  Most of the form text might be read from a database table or a possibly a file but the conversion between numbers and their textual descriptions would need to be some sort of program.

Because each language would exist in its own jar file and because the program would be receiving the target language in the input data, it would be possible to support new target languages whenever they were ready.

Simply copy the new jar file to the program directory and change the list of supported languages in the application.  The program would do the rest.  Well, the script or batch file that calls the program does need to ensure that it will include the new jar file.

I simply create the classpath from the list of jar files available and the script which runs the program would be called periodically from the crontab or other scheduler.  So the simple act of adding the jar file to the programs library directory automatically adds support to the program for this new language each time the script is run.

Reflection classes and methods

The Java API has the Method class which can be used to retrieve and inspect methods.  It is possible to query the name, which parameters types it expects, the return type and of course the ability to call the method.

class Method

  • String getName()
    Returns the name of the method.
  • Class[] getParameterTypes()
    Returns an array of Class objects which represent the parameters, in the order of declaration.
  • Class getReturnType()
    Returns a Class object that represents the return type of the method.
  • Object invoke(Object obj, Object... args)
    Invokes the underlying method represented by this Method object with the specified parameters.

class Field

The Method class allows you to interrogate a method to see exactly what parameters it is expecting and so forth but it is also possible to perform similar queries but on the class level for defined variables.

Once you have retrieved your variable from the class it is it can be used to then query information about the fields.  It is possible to retrieve the name, the type or even the value.

  • String getName()
    Returns the name of the field.
  • Class getType()
    Returns the type of the variable.
  • Object get(Object obj)
    Returns a Class object that represents the return type of the method.

 

Multilingual support example

For this example, it is expected that every language support class must have a certain core set of methods.  The easiest way to ensure that this happens is to create an interface of those methods.

To demonstrate this rather interesting idea using reflection, I am not going to have complete language functionality just enough to demonstrate how it works.  The language classes will each have to support two numberToText methods.

package com.acmesoft.support;

public interface languageInterface {

	public String numberToText(long value);
	public String numberToText(double value);
}

The actual reflection magic happens in the Numbers class. This class essentially is a small dispatcher by looking up the method based on the language that is passed in.

package com.acmesoft.support;

import java.lang.reflect.Method;

public class Numbers 
{
	public static String number2TextClass = "Numbers";
	public static String number2TextMethod = "numberToText";

	public String numberToText(String language, Integer val) throws Exception 
	{
		String outputtext = "";
		
		try{
			String searchObject = "com.acmesoft.support." + language ;
			Class cls = Class.forName(searchObject);
			Object obj = cls.newInstance();
			Method method = null;

			/* for methods with other parameters 
			//no paramater
			Class noparams[] = {};

			//String parameter
			Class[] paramString = new Class[1];	
			paramString[0] = String.class;

			//Double parameter
			Class[] paramDouble = new Class[1];	
			paramDouble[0] = Double.TYPE;
			*/

			//Long parameter
			Class[] paramLong = new Class[1];	
			paramLong[0] = Long.TYPE;
			
			// lets find out what the number is in words
			method = cls.getDeclaredMethod(number2TextMethod, paramLong);
			outputtext = (String)method.invoke(obj, val);
		}
		catch(Exception ex)
		{
			/*
			ex.printStackTrace();
			System.out.println(ex.toString());
			System.out.println(ex.getLocalizedMessage());
			*/
			throw ex;
		}
		
		return outputtext;
	}
}

Lines 15-17 actually lookup the class that we will need for this method.  This will be the actual language class (ie English).  Lines 34-35 prepare the type of parameter that will be used by this method as well as performing the lookup of the method with the getDeclareMethod call.  Lines 38-39 simply calls this method with our parameter.

The actual language implementations are actually rather sparse.  Having the code to convert 73626.82 into either either German or English although interesting doesn’t actually reinforce anything related to reflection so it has been simplified.

package com.acmesoft.support;

public class en implements languageInterface 
{
	public String numberToText(long val)
	{
		String retval = "";
		
		switch ((int)val)
		{
		case 1:
			retval = "one";
			break;
			
		case 2:
			retval = "two";
			break;
			
		case 3:
			retval = "three";
			break;
			
		default:
			retval = "unknown";
		}
		return retval;
	}
	
	public String numberToText(double val)
	{
		return "not yet";
	}
	
	public void testPgm(String[] args)
	{
		int val = 2;
		String valText = "";
		
		if (args.length != 0)
			val = Integer.parseInt(args[0]);
		
		try 
		{
			valText = numberToText(val);
		} 
		catch (Exception e) 
		{
			e.printStackTrace();
		}
		
		System.out.println(val + " is equal to " + valText);

	}
	public static void main(String[] args)
	{
		en pgm = new en();
		pgm.testPgm(args);
		return ;
	}
}

There is nothing in this class “en” sample code that you won’t be able to recreate in perhaps in the first few days of introduction to Java.  It is only a few lines of reflection code in our Number class that provides the flexibility.

#!/bin/bash

CP=base/final/internationalBase.jar:german/final/internationalDE.jar:english/final/internationalEN.jar:.

java -cp $CP com.acmesoft.support.TestPgm  1 en de fr
echo " "
java -cp $CP com.acmesoft.support.TestPgm  2 en de xx
echo " "
java -cp $CP com.acmesoft.support.TestPgm  3 en de

testrun

Download complete source for this language example

Download files

The darker side of reflection

The ability to get either the value or method directly is much more powerful than it appears at the first glance.  It is actually possible to retrieve values from private variables or call private methods that you normally wouldn’t have access to.

The neat thing about this is you only need to add one additional method call to enable the actual access to the value or method.

For Private methods

Method method = yourObject.getClass().getDeclaredMethod(yourMethod, argumentTypes);
method.setAccessible(true);
return method.invoke(yourObject, yourParameters);

For private fields

Field field = yourObject.getClass().getDeclaredField(yourField);
field.setAccessible(true);
field.set(yourObject, value);

Ignoring access permissions example

package payroll;

import java.lang.reflect.*;

public class Interrogate {

    public void showIncome(PayrollDept myobject)
    {
        Field privateIncomeField = null;
        
        try {
            privateIncomeField = myobject.getClass().getDeclaredField("income");
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (SecurityException e) {
            e.printStackTrace();
        }
        double income = 0;
        try {
            privateIncomeField.setAccessible(true);
            income = privateIncomeField.getDouble(myobject);
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        }
        System.out.println("Income " + income);
    }
    
    public void hackIncome(PayrollDept myobject)
    {
        Field privateIncomeField = null;
        
        try {
            privateIncomeField = myobject.getClass().getDeclaredField("income");
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (SecurityException e) {
            e.printStackTrace();
        }
        double income = 0;
        try {
            privateIncomeField.setAccessible(true);
            privateIncomeField.set(myobject, 1000000.0);
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        }

    }
    
    public Interrogate()
    {
        PayrollDept myobject = new PayrollDept();
        
        System.out.println("Name   " + myobject.firstName + " "+ myobject.lastName + ", " + myobject.title);
        System.out.println("Eyes   " + myobject.eyeColor);
        System.out.println("Height " + myobject.height);
        System.out.println("Weight " + myobject.weight);

        showIncome(myobject);
        hackIncome(myobject);
        showIncome(myobject);
        
        Method increaseIncomeMethod = null;
        Class[] paramDouble = new Class[1];
        paramDouble[0] = Double.TYPE;
        try {
            increaseIncomeMethod = myobject.getClass().getDeclaredMethod("setIncome", paramDouble);
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
        } catch (SecurityException e) {
            e.printStackTrace();
        }
        try {
            increaseIncomeMethod.setAccessible(true);
            increaseIncomeMethod.invoke(myobject,2000000.0);
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (InvocationTargetException e) {
            e.printStackTrace();
        }
    
        System.out.println("Income " + myobject.verifyIncome());
    }
    public static void main(String[] args) 
    {
        new Interrogate();
    }
}
package payroll;

public class PayrollDept extends Object
{
    public String title ;
    public int height;
    public String eyeColor;
    public double weight;
    public String firstName;
    public String lastName;
    private double income ;
    
    public PayrollDept()
    {
        title = "CEO";
        height = 204;
        eyeColor = "hazel";
        weight = 90;
        income = 1000;
        firstName = "Max";
        lastName = "Musterman";
    }
    
    private void setIncome(double val)
    {
        income = val;
    }
    private double getIncome()
    {
        return income;
    }

    public double verifyIncome()
    {
        return income;
    }
}
#!/bin/bash

CP=final/darkside.jar

java -cp $CP payroll.Interrogate

darksidetest

Download complete source for this “darkside” example

Download files
Posted in programming | Tagged | Comments Off on interrogating your code – dipping your toes into java’s reflection