Restful services in Java – 360 degree of restful client

Always try and use the most appropriate tool for the task.  The Apache HTTP server or web server is a program for serving the html web pages to the client who requests them. Tomcat is a application server that is actually used to for serving up Java Servlets or JSPs that are packaged up in a Web Archive file (war).

That isn’t to say that Apache cannot serve up Java Servlets nor does it mean that Tomcat cannot serve up static html pages.  Both programs exist because each is specialized for their particular task.

However, in order to create my web client I am going to have Tomcat serve up both the application as well as the static html web page that calls this service.  In order to do that, I will need to make a small change to my Tomcat configuration so it can also serve up these static pages.

Static Setup

Tomcat is configured by default for Java Servlets but by adding a tiny bit of extra configuration it will then serve up the static pages.

In the Tomcat configuration directory /opt/tomcat/conf is the server.xml file which contains a lot of the configuration for Tomcat.  Simply adding the following line to the Host block will define the location for the static web pages.

<Context docBase="/opt/tomcat/webapps/test" path="/test" />

The docBase attribute points to the path where the static html will be located while the path attribute is the prefix for the web page.  With the above setup, the following would be the url to access the index.html page.

http://localhost:8080/test/index.html

This block actually contains more interesting configuration that should probably be explained.

 <Host name="localhost" appBase="webapps"
             unpackWARs="true" autoDeploy="true">

    <!-- SingleSignOn valve, share authentication between web applications
         Documentation at: /docs/config/valve.html -->
    <!--
         <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
    -->

    <!-- Access log processes all example.
         Documentation at: /docs/config/valve.html
         Note: The pattern used is equivalent to using pattern="common" -->
    <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
           prefix="localhost_access_log" suffix=".txt"
           pattern="%h %l %u %t &quot;%r&quot; %s %b" />

    <Context docBase="/opt/tomcat/webapps/test" path="/test" />
 </Host>

The autoDeploy attribute gives the flexibility of deploying a war file while Tomcat is running.  This is actually pretty neat, you don’t need to stop and restart the server when installing the new war files.  Simply copy the war file to the webapp directory.

While doing some of my testing I simply would remove the war file and the directory that was unpacked from it.  After that I would copy over the new war file and wait for a few seconds.  At this point I could then call my restful service again.  Tomcat would unpack the new war file and then run the application.

Web Client

My client is really lightweight.  It simply queries the starting index and count of how many items should be retrieved.

Simply enter the values that you are interested in and click on the “fetch data” button.

The actual html for the webpage itself is trivial.  It simply makes the restful call and then parses the return list and displays each item.

The tricky part is not displaying the fields, gather the input or the button but actually the java script behind the webpage to make the AJAX call.  I am not a web developer so I will just try and point out the interesting things for the AJAX call.

360 Web Client Code

<!DOCTYPE html>
<html>
 <head>
 <meta charset="UTF-8">
 <title>JavaScript &amp; jQuery - Chapter 8: Ajax &amp; JSON - Loading HTML</title>
 <script src="js/jquery-1.12.1.js"></script> 

 <script>
 function displayname() {
 var start = document.getElementById("startidx").value;
 var count = document.getElementById("count").value;

  var xhr = new XMLHttpRequest(); // Create XMLHttpRequest object
 var myurl = "";
 myurl = 'http://localhost:8080/ThirdRestful/price/fetch?start=' + start + '&count=' + count;

  xhr.open('GET', myurl, false); // Prepare the request
  xhr.send(null); // Send the request
 
  responseObject = JSON.parse(xhr.responseText);
 
 var newContent = '';
 newContent += '<table align="left">';
 newContent += '<tr>';
 newContent += ' <th align="left">Id</th>';
 newContent += ' <th align="left">Symbol</th>';
 newContent += ' <th align="left">Bank</th>';
 newContent += ' <th align="left">Price</th>';
 newContent += ' <th align="left">Exchange</th>';
 newContent += ' <th align="left">Quote Timestamp</th>';
 newContent += '</tr>';

 for (i = 0; i < responseObject.records.length; i++)
 {
 newContent += '<tr>'; 
 newContent += '<td width="55px">'; newContent += responseObject.records[i].id; newContent += '</td>';
 newContent += '<td width="95px">'; newContent += responseObject.records[i].symbol; newContent += '</td>';
 newContent += '<td width="95px">'; newContent += responseObject.records[i].bank; newContent += '</td>';
 newContent += '<td width="95px">'; newContent += responseObject.records[i].price; newContent += '</td>';
 newContent += '<td width="105px">'; newContent += responseObject.records[i].exchange; newContent += '</td>';
 newContent += '<td width="255px">'; newContent += responseObject.records[i].quoteDateTime; newContent += '</td>';
 newContent += '</tr>'; 
 }
 newContent += '</table>'; 

 document.getElementById('content').innerHTML = newContent;
 }

 </script>

 </head>
 <body>

 <header><h1>360 WebClient </h1></header>
 <br />
 <label>Start Index: </label> <input type= text id="startidx" value="0" >
 <label>Count: </label> <input type= text id="count" value="10" >
 <input id="fetchdata" type="button" value="fetch data" onclick="displayname();" />
 <br />

 <p id="content"></p>
 <br />

 </body>
</html>

AJAX is Asynchronous JavaScript and XML which is how you can call your, or someone else’s web services.  This is neat because you can then create a fairly complicated page but by using AJAX you can pull in other information on demand to create a more interactive web experience.

I am pulling in all of this via the JavaScript library JQuery.  JQuery is not the only JavaScript library but it is one of the most popular ones.  It is possible to get either a compressed or uncompressed version of this library from the JQuery site.

https://jquery.com/download/

My web client allows the user to enter things that interest them and submit that as parameters to the restful web service.  The web service could itself go to the internet, read information from a database or perform some calculations – when it was finished that information could be returned to the calling client.

 myurl = 'http://localhost:8080/ThirdRestful/price/fetch?start=' + start + '&count=' + count;

More specifically, my example queries the values from the user and builds the URI that will call my service with the parameters in their proper locations.

This code is actually very optimistic as it assumes that the user will be smart enough to enter valid values into both fields before pressing the fetch data button.  It also assumes that data will exist.

This last assumption is a bit of a stretch as this is not a bullet proof production program but rather a proof of concept.

Creating test data

Rather than force myself to manually enter the data each time I start my server I have created a restful call to create some data.  This is done by simply calling the following URI.

http://localhost:8080/ThirdRestful/price/generate

This will generate some somewhat random data in order to better test my client. It is also possible to call this method with a parameter to generate more than the default 20 entries. It is possible to call this method any number of times.  The newly generated data will be added to the internal list.

http://localhost:8080/ThirdRestful/price/generate?count=10

The generate method will return the count of the number of items that exist in the list after adding the new day.

This is my last blog article about restful web services for the moment.  This is not the only way to create such services but it is a pretty easy way without having hundreds of additional java classes that make up some frameworks.

I think that this is a superior way of generating web services but any critics could easily point to the fact that it is not easy to know what services exist for my particular application – and they are correct.

Another way of producing web services that allow you to see what services are available is to use WSDL .  This will describe the web services and it also allows you to generate code which can be used to call each of the defined services.

I did have an opportunity to do just this for connecting with SharePoint.  I wasn’t really a big fan of the technology I was given.  There were hundreds of lines of auto-generated code and hundreds of files of odd framework code.  I suspect that some of the other frameworks out there (ie Spring) also have hundreds of new files but probably in a much better organized way.

I may end up checking out one of these other frameworks at some point, but for right now I will enjoy both the power and simplicity of the small Jersey powered restful server technology.

Posted in programming, Setup From Scratch | Tagged , , | Comments Off on Restful services in Java – 360 degree of restful client

Restful services in Java – 360 degree of restful server

This is the blog final entry for a restful service web server.  I have attempted to create more than your typical “foo” “bar” type example program. My final blog on restful services will be a fairly complete service that could be used to provide price data supplied by several “banks” on various exchanges.

The data can be  pseudo randomly generated to be later retrieved and displayed by web browser using java script to fetch, extract and display the data from a JSON formatted string.

A real production server would save server data in a more permanent manner such as a database or even perhaps files if the data lent itself to that. I didn’t feel like creating database so I have skipped over this step by adding a restful service for generating some data which is stored in memory. This is just a “temporary” server that is available for testing and not for real permanence.

This example server has the following methods.

  • generate
  • count
  • (index).xml
  • (index).json
  • (index).text
  • fetch
  • addOne
  • retrieve
  • retrieveall

The generate method is perhaps the most important of these for testing. Simply call this restful service in the beginning to generate twenty items by default or pass in a specific count to generate that many test items.

The generated data is put into a simple hash with the key to the hash being the simple index.

To look at the data that is in the hash the count, (index).xml, (index).json or (index).text.

The fetch method is actually the other main method specifically for a web client. This web service will retrieve an array of market prices in JSON format.

In a few days I will be providing a html client that will use this server and retrieve the data in the JSON format and display that to the web page.

PriceServer.java

package com.acmesoft;

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;

import java.math.BigDecimal;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.Iterator;

import javax.ws.rs.Consumes;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriInfo;
import javax.ws.rs.WebApplicationException;

@Path("/price")
public class PriceServer
{
	// should really be a database
	private static HashMap<Integer,MarketPrice> pricedatabase = new HashMap<Integer,MarketPrice>();
	private static int databasecount = 0;

	//
	// index is the actual 0 - n-1 of the map.
	//
	private String rangeAsXML(int start, int stop)
	{
		String retval = "";
		retval += "";

		for (int idx = start; idx <= stop; idx++)
		{
			MarketPrice item = null;
			item = pricedatabase.get(idx);

			if (item != null)
				retval += item.xmlString();
		}

		retval += "";

		return retval;
	}

	//
	// index is the actual 0 - n-1 of the map.
	//
	private String rangeAsJSON(int start, int stop)
	{
		String retval = "{ \"records\": [";

		for (int idx = start; idx <= stop; idx++)
		{
			MarketPrice item = null;
			item = pricedatabase.get(idx);

			if (item != null)
			{
				if (idx != start)
					retval += ", ";
				retval += item.JSONString();
			}
		}

		retval += "] }";

		return retval;
	}

	// initialize the price cache, this is pretty much only for debugging
	// 
	// this will generate 20 items by default
	// http://localhost:8080/ThirdRestful/price/generate
	//
	// this will generate 100 items instead 
	// http://localhost:8080/ThirdRestful/price/generate?count=100
	@GET
	@Path("/generate")
	@Produces( {MediaType.APPLICATION_XML})
	public Response generate(@Context UriInfo ui, @DefaultValue("20") @QueryParam("count") int debugcount) throws URISyntaxException
	{
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		MarketPrice newprice = null;
		for (int idx = 0; idx < debugcount; idx++)
		{
			switch (idx % 5)
			{
			case 0:
				newprice = new MarketPrice("EUR/USD", "DBK", new BigDecimal(1.056), "DTB");
				break;
			case 1:
				newprice = new MarketPrice("GBP/EUR", "BCY", new BigDecimal(1.179), "LIFFE");
				break;
			case 2:
				newprice = new MarketPrice("USD/MXN", "C", new BigDecimal(19.1), "CME");
				break;
			case 3:
				newprice = new MarketPrice("EUR/USD", "0UB", new BigDecimal(1.059), "SOFFEX");
				break;
			case 4:
				newprice = new MarketPrice("VEF/USD", "JPM", new BigDecimal(0.1001), "NYSE");
				break;
			}
			newprice.setId(databasecount);
			pricedatabase.put(databasecount, newprice);
			databasecount++;
		}

		//return Response.created(uri).contentLocation(uri).status(Response.Status.OK).build(); 
		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity("" + databasecount +"").build(); 
	}

	// get the count of items
	// 
	// http://localhost:8080/ThirdRestful/price/count
	@GET
	@Path("/count")
	@Produces( {MediaType.APPLICATION_XML})
	public Response count(@Context UriInfo ui) throws URISyntaxException
	{
		String retval = "" + databasecount + "";
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build();
	}


	// add a new price record
	// 
	// http://localhost:8181/ThirdRestful/price/addOne 
	@POST
	@Path("/addOne")
	@Produces( {MediaType.APPLICATION_XML})
	public Response addOne(	@Context UriInfo ui, MarketPrice item) throws URISyntaxException
	{
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);
		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).build(); 

	}

	// returns a selection of records in xml format.
	// 
	// http://localhost:8080/ThirdRestful/price/retrieve
	@GET
	@Path("/retrieve")
	@Produces( {MediaType.APPLICATION_XML})
	public Response retrieve(	@Context UriInfo ui,
			@DefaultValue("0") @QueryParam("start") int start, 
			@DefaultValue("10") @QueryParam("count") int count) throws URISyntaxException
	{
		String retval = "";
		String path = ui.getAbsolutePath().toString();
		URI uri = new URI(path);

		if (databasecount == 0)
			return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build();
		else
		{
			int end = start + count - 1;
			if (end > databasecount-1)
				return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build();
			retval = rangeAsXML(start,end);
		}
		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build(); 
	}

	// returns a selection of records in xml format.
	// 
	// this will return 10 items starting with the first
	// http://localhost:8080/ThirdRestful/price/fetch
	//
	// start with index one for three items
	// http://localhost:8080/ThirdRestful/price/fetch?start=1&count=3

	@GET
	@Path("/fetch")
	@Produces( {MediaType.APPLICATION_JSON})
	public Response fetch(	@Context UriInfo ui,
			@DefaultValue("0") @QueryParam("start") int start, 
			@DefaultValue("10") @QueryParam("count") int count) throws URISyntaxException
	{
		String retval = "";
		String path = ui.getAbsolutePath().toString();
		URI uri = new URI(path);

		if (databasecount == 0)
			return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build();
		else
		{
			int end = start + count - 1;
			if (end > databasecount-1)
				return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build();
			retval = rangeAsJSON(start,end);
		}
		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build(); 
	}


	// returns all records in xml format.
	// 
	// http://localhost:8080/ThirdRestful/price/retrieveall 
	@GET
	@Path("/retrieveall")
	@Produces( {MediaType.APPLICATION_XML})
	public Response showall(@Context UriInfo ui) throws URISyntaxException
	{
		String retval = "";
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		retval = rangeAsXML(0,databasecount-1);
		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build();
	}



	// returns single record in xml format.
	// 
	// http://localhost:8080/ThirdRestful/price/34.xml
	@GET
	@Path("{idx}.xml")
	@Produces( {MediaType.APPLICATION_XML})
	public Response getIndexXml(@Context UriInfo ui, @PathParam("idx") int idx)  throws URISyntaxException
	{
		MarketPrice retval = null;
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		if (idx >= 0 && idx < databasecount) 
                { 
                        retval = pricedatabase.get(idx); 
                        return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build(); 
                } 
                else 
                        return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build(); 
        } 

        // returns single record in json format. 
        // 
        // http://localhost:8080/ThirdRestful/price/34.json 
        @GET @Path("{idx}.json") 
        @Produces( {MediaType.APPLICATION_JSON}) 
        public Response getIndexJSON(@Context UriInfo ui, @PathParam("idx") int idx) throws URISyntaxException 
        { 
                MarketPrice item = null; 
                String retval = ""; 
                String path = ui.getAbsolutePath().toString() ; 
                URI uri = new URI(path); 
                if (idx >= 0 && idx < databasecount) 
                { 
                        item = pricedatabase.get(idx); 
                        retval = item.JSONString(); 
                        return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build(); 
                } 
                else 
                        return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build(); 
         } 

         // returns single record in json format. 
         // 
         // http://localhost:8080/ThirdRestful/price/34.text 
         @GET @Path("{idx}.text") 
         @Produces( {MediaType.TEXT_PLAIN}) 
         public Response getIndexTEXT(@Context UriInfo ui, @PathParam("idx") int idx) throws URISyntaxException 
         {
                MarketPrice item = null; 
                String retval = ""; 
                String path = ui.getAbsolutePath().toString() ; 
                URI uri = new URI(path); 
                if (idx >= 0 && idx < databasecount)
		{
			item = pricedatabase.get(idx);
			retval = item.toString();
			return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build();
		}
		else
			return Response.created(uri).contentLocation(uri).status(Response.Status.REQUESTED_RANGE_NOT_SATISFIABLE).build();
	}


}

MarketPrice.java

package com.acmesoft;

import java.io.StringWriter;
import java.math.BigDecimal;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;
import javax.xml.bind.PropertyException;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

import com.google.gson.Gson;

import javax.xml.bind.annotation.XmlAccessType;;

@XmlRootElement(name="marketPrice")
@XmlAccessorType(XmlAccessType.FIELD)
public class MarketPrice 
{
	@XmlElement
	private int id;
	@XmlElement
	private String symbol;		// EUR/USD
	@XmlElement
	private String bank;		// DBK
	@XmlElement
	private BigDecimal price;	// 1.1222
	@XmlElement
	private String exchange;	// DTB, NYSE, CBOT
	@XmlElement
	private Date quoteDateTime;	// 2015-01-01 14:32:45.032


	public MarketPrice()
	{
		id = -1;
		this.symbol= "";
		this.bank = "";
		this.price = new BigDecimal(0.0);
		this.exchange = "";
		this.quoteDateTime = new Date();
	}


	public MarketPrice(String symbol, String bank, BigDecimal price, String exchange)
	{
		DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");

		this.symbol= symbol;
		this.bank = bank;
		this.price = price;
		this.exchange = exchange;
		this.quoteDateTime = new Date();
		String debugDate = dateFormat.format(quoteDateTime);

		id = -1;
	}

	public MarketPrice(String symbol, String bank, BigDecimal price, String exchange, Date quoteDateTime)
	{
		this.symbol= symbol;
		this.bank = bank;
		this.price = price;
		this.exchange = exchange;
		this.quoteDateTime = quoteDateTime;
		id = -1;
	}

	public void setId(int value)
	{
		id = value;
	}
	public int getId()
	{
		return id;
	}

	public void setSymbol(String value)
	{
		symbol = value;
	}
	public String getSymbol()
	{
		return symbol;
	}

	public void setBank(String value)
	{
		bank = value;
	}
	public String getBank()
	{
		return bank;
	}

	public void setPrice(BigDecimal value)
	{
		price = value;
	}
	public void setPrice(double value)
	{
		price = new BigDecimal(value);
	}
	public BigDecimal getPrice()
	{
		return price;
	}

	public void setExchange(String value)
	{
		exchange = value;
	}
	public String getExchange()
	{
		return exchange;
	}

	public Date getQuoteDateTime()
	{
		return quoteDateTime;
	}

	public void setQuoteDateTime(Date value)
	{
		quoteDateTime = value;
	}


	public String toString()
	{
		String retval = getCSVData();

		return retval;
	}

	public String xmlString()
	{ 
		String retval = toXmlString();
		retval = retval.substring(retval.indexOf("\n"));

		return retval;
	}

	public String JSONString()
	{ 
		Gson gson = new Gson();
		String JSONstr = gson.toJson(this);
		return JSONstr;  
	}

	public String toXmlString()
	{
		JAXBContext ctx = null;
		Marshaller maxMarshaller = null;

		try {
			ctx = JAXBContext.newInstance(MarketPrice.class);
		} catch (JAXBException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

		try {
			maxMarshaller = ctx.createMarshaller();
		} catch (JAXBException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}

		try {
			maxMarshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
		} catch (PropertyException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}

		StringWriter writer = new StringWriter();
		try {
			maxMarshaller.marshal(this,writer);
		} catch (JAXBException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

		String xmlPrice = writer.toString();

		return xmlPrice;
	}

	private String convertToTime(long milliseconds)
	{
		Date date = new Date(milliseconds);
		DateFormat dateformat = new SimpleDateFormat("HH:mm:ss.SSS");
		return dateformat.format(date);
	}
	public String getCSVHeader()
	{
		String delim = com.acmesoft.Constants.FIELDDELIM;
		String retval;

		retval = "Id" + delim + "Symbol" + delim + "Price" + delim + "Bank" + delim + "Exchange" + delim + "QuoteDate" ;

		return retval;
	}
	public String getCSVData()
	{
		String delim = com.acmesoft.Constants.FIELDDELIM;
		String retval;

		retval = id + delim + symbol + delim + price + delim + bank + delim + exchange + delim + quoteDateTime ;

		return retval;
	}
}

PriceCount.java

package com.acmesoft;

import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement(name="marketPrice")
@XmlAccessorType(XmlAccessType.FIELD)
public class PriceCount
{
        @XmlElement
        private int count;

        public PriceCount()
        {

        }
        public PriceCount(int val)
        {
                count = val;
        }

        public int getCount()
        {
                return count;
        }

        public void setCount(int val)
        {
                count = val;
        }
}

Constants.java

package com.acmesoft;

public class Constants {
	final public static String FIELDDELIM = ",";
	final public static int OK = 200;
	final public static String DATETIMEFORMAT = "yyyy-MM-ddHH:mm:ss.SSS";
}
Posted in programming | Tagged , , | Comments Off on Restful services in Java – 360 degree of restful server

command line fun – unlimited command line length

While looking into another problem, plethora of shells, I was initially afraid that some of my fooling around was causing some odd side effects.  While investigating I ended up with a different solution that was actually overkill but it help eliminate other options as a source of the problem.

My solution was to create a file with the names of all of the files that I needed to process and use that as my input.   This allowed me to look at as many files as I wanted to without seeing anything like this.

-bash: /usr/bin/ls: Arg list too long

This isn’t the only solution, nor even the most efficient but it sure is a fun one that uses both head and tail on the same line.  A more efficient solution would probably to use xargs, find or perhaps a more efficient shell script.

Rather than to do the efficient, I am going to show off the fun solution.  This is something you could use even in a production environment as long as the number of files is reasonably small.

I decided to use both head and tail to pick out lines from a file for processing.  Simply save a list of all files to a temporary file and then process each file one at a time.

The fun example

INPUT=/home/bob/input
TMPFILE=`mktemp /tmp/XXXXXX`

pushd `pwd`
cd $INPUT
ls -1 > $TMPFILE
CNT=`cat $TMPFILE | wc -l`
IDX=1
while [ $IDX -le $CNT ]
do
   NAME=`head -n ${IDX} $TMPFILE | tail -1`

   echo processing $NAME
   # do some processing here

   IDX=$(($IDX +1)) 
done
popd
rm $TMPFILE

The actual interesting part is where the variable NAME is being assigned but before getting ahead of myself, here is a blow by blow explanation of the script.

# Line Description
1. INPUT=/home/bob/input set up our input location
2. TMPFILE=`mktemp /tmp/XXXXXX` create a unique filename in the /tmp directory
3. pushd `pwd` save our current directory
4. cd $INPUT change to input directory
5. ls -1 > $TMPFILE save a list of file names (no directory) into our temporary file.
6. CNT=`cat $TMPFILE | wc -l` get a count of the number of lines in our file.
7. IDX=1 which line of the file we are on at the moment.
8. while [ $IDX -le $CNT ] while there are lines left to process
9. do
10. NAME=`head -n ${IDX} $TMPFILE | tail -1` get the entry from the file.
11. echo processing $NAME in this case echo the filename to the screen.
12. # do some processing here nothing to see here, move along.
13. IDX=$(($IDX +1)) increase index to the next line
14. done
15. popd change back to original directory
16. rm $TMPFILE remove temporary file.

The code is pretty obvious what it is doing for most of the script.  The interesting lines might be 2 and 10.

The mktemp command simply creates a unique file that is six characters long designated by the XXXXXX format.  This is really useful as the operating system will do the hard work for you.  It is even possible to create a subdirectory with a file or just a unique subdirectory.

Line number ten is actually pretty simple when you think about it. The head command will take that many lines from our temporary file.  the first time through this will be the first line.  the tail command will then take the last line from this batch.  When the index is 1 both the head and tail will deliver a single line.

When the index is larger (ie 5) then more lines will be delivered.  The head command will deliver all of the previously processed lines along with one new one.  The tail command will pull off that one new line so it can be processed.

This method of processing data might also be interesting if there are spaces in the filenames or paths.  You would know with certainty that each line represents a single parameter or value.

Again, this is a terribly inefficient way of processing through a million lines, especially if these lines are each hundreds of characters long.

Posted in Command line | Tagged , | Comments Off on command line fun – unlimited command line length

Gentlemen’s vandal

We were watching the BBC world news a few days back and it must have been a slow news day. The reporter was speaking with someone whose face was hidden. It turned out that this particular vigilante defaced buildings and has been doing so for the past 13 years.

No, he wasn’t putting up gang graffiti letters nor faces, animals or general art. His changes were very very tiny. So tiny that the actual shop owners in some cases didn’t recognize anything was different. So what was the change?

He would either add or remove apostrophe’s as required.

Ie.

Amys Nail’s -> Amys Nails

The interviewee never said what his profession is but it could easily be that of an English teacher considering how my teacher was during English class.

It is hard to encourage vandalism but this particular type is at least a refreshing change from the norm.

A “grammar vigilante” sneaks around at night fixing an infuriatingly common error on public signs

Posted in Soapbox | Comments Off on Gentlemen’s vandal

Restful services in Java – client and server

My previous restful client example and server example were perfect, well, not exactly.  They worked but they pretty much assuming a perfect world.  That is to say that there is never a communication error or a server error.

This small example will return one of the magazine objects.  Depending how you call it, it will return either the object directly or the xml that you can marshal into the object itself.

// example 2/3 setup
ClientConfig config1 = new ClientConfig();
Client client1       = ClientBuilder.newClient(config1);
WebTarget target1    = client1.target(uribase);
		
// example 3
magazine xmlobj = target1.path("/Communicate/hello").request().accept(MediaType.APPLICATION_XML).get(magazine.class);
showMagazine("pure xml object", xmlobj);

However, it should be pretty obvious in the previous simple server code that if for any reason the server cannot find the data it cannot really communicate that fact back to the client.  Well, it cannot if you are returning magazine objects or general strings.

New and better server

The solution is to return a response record.  This record will contain both the error code as well as our object.  The code is actually not too different from from the previous example but there are some major differences.

The first is that we are no longer returning either xml(string) nor magazine objects but rather we are returning Response objects.    This allows us to set the status to one of the values that matches the result of the call.

Response.Status.OK

It is possible to use one of the predefined values such as Response.Status.OK but it is just as acceptable to put the actual value of 200.

Here is a list of the values that are defined for us to use.

Response codes Description
OK 200
CREATED 201
ACCEPTED 202
NO_CONTENT 204
MOVED_PERMANENTLY 301
SEE_OTHER 303
NOT_MODIFIED 304
TEMPORARY_REDIRECT 307
BAD_REQUEST 400
UNAUTHORIZED 401
FORBIDDEN 403
NOT_FOUND 404
NOT_ACCEPTABLE 406
CONFLICT 409
GONE 410
PRECONDITION_FAILED 412
UNSUPPORTED_MEDIA_TYPE 415
INTERNAL_SERVER_ERROR 500
SERVICE_UNAVAILABLE 503

The code in this “server code” is not much larger than the original code.

Another one of the changes is that the URI is returned as part of the response object.  This URI should point to the the data in the server.  This is especially interesting if a new object is created on your server.  The server then will return the URI to reference that object.

The final difference is that we pass back our object as part of the Response object.  In the below code this is done as a simple string but it could just as easily be a your own Java object.

package com.acmesoft;

import java.net.URI;
import java.net.URISyntaxException;

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriInfo;

@Path("/Communicate")
public class HelloWorldEx {

	/*  
	 * http://192.168.178.70:8080/SecondRestful/Communicate/mag
	 */	@GET
	@Path("/mag")
	@Produces( {MediaType.APPLICATION_XML})
	public Response sayHelloXml(@Context UriInfo ui) throws URISyntaxException
	{
		String retval =  "" +
				"2016" +
				"6" +
				"Economist" +
				"9.99" +
				"";
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build();
	}

	/*  
	 * http://192.168.178.70:8080/SecondRestful/Communicate/mag
	 */
	@GET
	@Path("/mag")
	@Produces( {MediaType.APPLICATION_JSON})
	public Response sayHelloJSON(@Context UriInfo ui) throws URISyntaxException
	{
		String retval =  "{ \"magazine\": {\"year\":2016,\"month\":6,\"title\":\"Economist\",\"price\":9.99} }";
		
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(retval).build();
	}

	/*  
	 * http://192.168.178.70:8080/SecondRestful/Communicate/oucha_bunga
	 */
	@Path("{c}")
	@GET
	@Produces( {MediaType.APPLICATION_XML})
	public Response complexURI(@Context UriInfo ui, @PathParam("c") String msg) throws URISyntaxException
	{
		String mymsg = "Thinkin about " + msg;
		String xmlStr = "" + mymsg + "";

		// this is a really clever way to get the URI but
		// if you pass in a string, it will throw an error.
		// and it all comes down to the URI that is added to the
		// response object (below)
		String path = ui.getAbsolutePath().toString() ;
		URI uri = new URI(path);

		return Response.created(uri).contentLocation(uri).status(Response.Status.OK).entity(xmlStr).build();
	}
}

The final method in this server example is is not actually called by my client but it is a clever example of a URI that can accept anything, in this case a string, and then that string can be used in some clever manner.

New Client

The new client changes is actually quite similar to the new server changes.  As a matter of fact, the client changes are actually almost transparent. There is just a few small changes to accommodate the retrieving Response objects instead of strings or other java objects.

Once the Response object has been retrieved simply call the status method to find out the status returned from the server. The range of success codes is 200-299.

	if (response.getStatus() == 200)
	{
		xmlAnswer = response.readEntity(String.class);
		System.out.println(xmlAnswer);
		magazine converted = convertXmlToMag(xmlAnswer);
		showMagazine("xml text data", converted);			
	}
	response.close();

If the call was successful then simply call the readEntity method on the Response object.  Simply pass in the class of the object that you are trying to read out.  The object is retrieved just like magic.

In order to keep control on the system resources close the Response and the Client objects, failing to do so will cause your program to fail eventually due to resource issues.

package com.acmesoft;

import java.io.StringReader;
import java.net.URI;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriBuilder;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;

import org.glassfish.jersey.client.ClientConfig;

public class TestClient 
{
	String uribase = "http://192.168.178.70:8080/SecondRestful";

	public void showMagazine(String desc, magazine value)
	{
		System.out.println("\n" + desc );
		System.out.println("year "  + value.getYear());
		System.out.println("month " + value.getMonth());
		System.out.println("price " + value.getPrice());
		System.out.println("title " + value.getTitle());
	}
	
	public magazine convertXmlToMag(String xmlstring)
	{
		magazine converted = null;
		JAXBContext jc = null;
		try {
			jc = JAXBContext.newInstance(magazine.class);
			Unmarshaller unmarshaller = jc.createUnmarshaller();
			StringReader reader = new StringReader(xmlstring);
			converted = (magazine) unmarshaller.unmarshal(reader);	
		}
		catch (JAXBException ex)
		{
			ex.printStackTrace();
		}
		return converted;
	}
	
	public WebTarget restConnect(String targetUrl)
	{
		URI uri = UriBuilder.fromUri(targetUrl).build();
		ClientConfig config = new ClientConfig();
		Client client = ClientBuilder.newClient(config);
		WebTarget target = client.target(uri);
		return target;
	}
	
	public TestClient()
	{
		ClientConfig config = new ClientConfig();
		Client client       = ClientBuilder.newClient(config);
		WebTarget target    = client.target(uribase);

		Response response  = target.path("/").path("Communicate").path("/mag").request().accept(MediaType.APPLICATION_XML).get(Response.class);
		System.out.println(response.getStatus());
		String xmlAnswer = "";
		if (response.getStatus() == 200)
		{
			xmlAnswer = response.readEntity(String.class);
			System.out.println(xmlAnswer);
			magazine converted = convertXmlToMag(xmlAnswer);
			showMagazine("xml text data", converted);			
		}
		response.close();

		response = target.path("/Communicate/mag").request().accept(MediaType.APPLICATION_XML).get(Response.class);
		if (response.getStatus() == 200)
		{
			magazine xmlobj = response.readEntity(magazine.class);
			showMagazine("pure xml object", xmlobj);
		}
		response.close();
		client.close();
	}
	
	public static void main(String[] args) 
	{
		new TestClient();
	}
}

The output is not terribly interesting, it is just a few print statements in order to prove that the client can retrieve the data.

 200
<Magazine><Year>2016</Year><Month>6</Month><Title>Economist</Title><Price>9.99</Price></Magazine>

xml text data
year 2016
month 6
price 9.99
title Economist

pure xml object
year 2016
month 6
price 9.99
title Economist

With the ability to verify the return status of the restful server calls it is possible to create much more interesting services than displayed here.

I have attempted to create more than your typical “foo” “bar” type example program. My final blog on restful services will be a fairly complete service that could be used to provide price data supplied by several “banks” on various exchanges.

The data is actually pseudo randomly generated and retrieved and displayed by web browser using java script to fetch, extract and display the data from a JSON formatted string.

Posted in programming, Setup From Scratch | Tagged , , | Comments Off on Restful services in Java – client and server

Restful services in Java – simple client example

I suppose that a restful server isn’t all that useful unless there is a way to query this information back and use it in some useful way.

It is easy enough to test this out the server from the command line using the curl command.  Simply select the return type and select the URI.

curl -H “Accept: text/plain” http://192.168.178.70:8080/FirstRestful/Communicate/hello

 

curl -H “Accept: application/xml” http://192.168.178.70:8080/FirstRestful/Communicate/hello

This is the best starting point as we can pretty much guarantee that the server is working. Should any problems occur during the development of the client we can pretty much point to the client as the culprit.

Java client

My first web services client will be essentially just a plain old java program.  Although I am using java for my client I can easily imagine that there are some systems that were programmed in some older languages that use also use restful services in this same manner.  Not only communicate with a web server but to provide an easy way to enhance those systems with abilities that were didn’t exist when they were created (I am looking at you COBOL ).

Jersey makes the process of making restful calls easy from Java.  The restConnect method groups together the necessary logic for accessing a URI that is passed in.

Once these steps are done the object is a WebTarget is initialized and it is possible to call the restful service.

In this example client example I use use three slightly different methods for setting up the URI.  The two more flexible calls (lines 68 – 78) setup a base URI and then adds to the path.  This might be the best way if quite a few similar calls were necessary.

When doing the restful calls you also pass in the datatype that you wish to be returned back.  This then does the proper call to the server to get the correct return type.

/**
 * 
 */
package com.acmesoft;

import java.io.StringReader;
import java.net.URI;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriBuilder;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;

import org.glassfish.jersey.client.ClientConfig;

public class TestClient 
{
	String uribase = "http://192.168.178.70:8080/FirstRestful";
	String urifull = uribase + "/Communicate/hello";

	public void showMagazine(String desc, magazine value)
	{
		System.out.println("\n" + desc );
		System.out.println("year "  + value.getYear());
		System.out.println("month " + value.getMonth());
		System.out.println("price " + value.getPrice());
		System.out.println("title " + value.getTitle());
	}
	
	public magazine convertXmlToMag(String xmlstring)
	{
		magazine converted = null;
		JAXBContext jc = null;
		try {
			jc = JAXBContext.newInstance(magazine.class);
			Unmarshaller unmarshaller = jc.createUnmarshaller();
			StringReader reader = new StringReader(xmlstring);
			converted = (magazine) unmarshaller.unmarshal(reader);	
		}
		catch (JAXBException ex)
		{
			ex.printStackTrace();
		}
		return converted;
	}
	
	public WebTarget restConnect(String targetUrl)
	{
		URI uri = UriBuilder.fromUri(targetUrl).build();
		ClientConfig config = new ClientConfig();
		Client client = ClientBuilder.newClient(config);
		WebTarget target = client.target(uri);
		return target;
	}
	
	public TestClient()
	{
		// example 1
		WebTarget target = restConnect(urifull);
		String plainAnswer = target.request().accept(MediaType.TEXT_PLAIN).get(String.class);
		System.out.println("raw (text)data");
		System.out.println(plainAnswer);
		System.out.println("");
		
		// example 2/3 setup
		ClientConfig config1 = new ClientConfig();
		Client client1       = ClientBuilder.newClient(config1);
		WebTarget target1    = client1.target(uribase);

		// example 2
		String xmlAnswer = target1.path("/").path("Communicate").path("/hello").request().accept(MediaType.APPLICATION_XML).get(String.class);
		magazine converted = convertXmlToMag(xmlAnswer);
		System.out.println(xmlAnswer);
		showMagazine("xml text data", converted);
		
		// example 3
		magazine xmlobj = target1.path("/Communicate/hello").request().accept(MediaType.APPLICATION_XML).get(magazine.class);
		showMagazine("pure xml object", xmlobj);
	}
	
	public static void main(String[] args) 
	{
		new TestClient();
	}
}

Below is the magazine class with the appropriate markups to easily do the JAXB conversion between plain old java objects and XML.

package com.acmesoft;

import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlAccessType;

@XmlRootElement(name="Magazine")
@XmlAccessorType(XmlAccessType.FIELD)
public class magazine {

	// Microsoft Exchange values 
	@XmlElement(name="Year")
	private  int year; 

	@XmlElement(name="Month")
	private  int month; 

	@XmlElement(name="Title")
	private   String title; 

	@XmlElement(name="Price")
	private   double price;

	public magazine()
	{
	}
	public magazine(int year, int month, String title, double price)
	{
		this.year = year;
		this.month = month;
		this.title = title;
		this.price = price;
	}

	public int getYear()
	{
		return year;
	}
	public void setYear(int value)
	{
		this.year= value;
	}

	public int getMonth()
	{
		return month;
	}
	public void setMonth(int value)
	{
		this.month= value;
	}

	public String getTitle()
	{
		return title;
	}
	public void setTitle(String value)
	{
		this.title = value;
	}

	public double getPrice()
	{
		return price;
	}
	public void setPrice(double value)
	{
		this.price = value;
	}
}

The flexibility of the restful API calls  is pretty amazing. The first example (lines 62-67) shows how easy it can be to call a restful service.  Nothing is all that surprising as the server is passing back text and that text is assigned to a string.

The second example is slightly more useful as the return data is XML formatted and it is possible to convert that into a java object using JAXB.  XML data is still text data and it shouldn’t be amazing that text data is returned and assigned to a string.  Manually parsing out the data from an XML structure is really painful so I created the method convertXmlToMag to parse the XML into a java object.

The third example is by far the most useful example.  The restful server returns XML data but when making the restful call instead of passing in a string class I pass in a magazine class.  Thus the restful call returns the magazine object directly.  This also isn’t magic as the class does need to be defined when compiling the client but that is pretty much to be expected.  If the server does have certain types of data you wish to query then the client would already have similar or identical structured objects.

raw (text)data
Hello World
Having a great time!

200
<Magazine><Year>2016</Year><Month>6</Month><Title>Economist</Title><Price>9.99</Price></Magazine>

xml text data
year 2016
month 6
price 9.99
title Economist

pure xml object
year 2016
month 6
price 9.99
title Economist
Posted in programming, Setup From Scratch | Tagged , , | Comments Off on Restful services in Java – simple client example

Restful services in Java – simple server example

There are probably a lot of ways to create web services programs but the simplest way I could think of was to create a project using Eclipse and Jersey.

It turns out that restful services are really easy to summarize but a bit more involved to describe in the way of a teaching tutorial.

I have read up on the topic in the O’Reilly published book “RESTful Java with JAX-RS 2.0” by Bill Burke.    There is a lot of information starting with how to do this the hard way from scratch all the while moving on to how JAX-RS and markup can simplify the task.

I will not summarize everything that is described in the book but I will say that it if you want to get a working knowledge of restful services this is a pretty easy way to go about it.  The book is pretty focused on the server side itself.  The nice thing about creating a server like this is that it is easy enough to test using a simple web browser.

I have gone a small bit further and my examples will include some client code that will actually use the services that I created.  This is was done both as a simple java client and also with a simple html client that also used a bit of java script.

It was this last part that actually seemed the most interesting as you can then control the entire “application” from server to client.  The book uses xml examples but I did also get do some research on both java script, JSON and html/css.  It is possible that these are also highlighted in the future but at the moment they are not of a great interest to me.

 

Restful Services Setup from Scratch

There are a number of things that need to be setup for creating a web service and there are a number of ways to get this properly setup.  The way I choose was to create a new “Dynamic web project” in eclipse.

I created a new workspace to ensure that all setup needs to be done from scratch.

In this first dialogbox you can see that the “Target runtime” is an empty list.  This is because no web server has been defined.

Simply click on new runtime to define one.

Simply select which version of Tomcat is installed on your computer. The version that I had installed was 8.5.11.

Basically select the install directory of Tomcat.

Note: The structure of the Tomcat installation should be the same regardless of what the install directory is called.  However, when I was first trying to get the permissions for Tomcat setup I had problems using any path other than /opt/tomcat.  Once the permissions were corrected any path could be used.

Once the Tomcat server is defined, it is just a matter clicking through the next three dialog boxes.

We can set the context root of our URI to anything at here.This example it is FirstRestful.

Once you press Finish the rest of the structure is created.

Step 2 – adding Jersey

What is Jersey?  It is an implementation of JAX-WS (Java API for XML Web Services) for creating web services in XML format.  This makes it really easy for creating a small server.  All that needs to be done is to move the Jersey libraries into the correct spot under the WebContent folder.  Once this has been done then the basic framework is in place.

There is both a version 1 and version 2 of Jersey available on this site, newer must be better so of course I downloaded 2.25.1.

jaxrs-ri-2.25.1.zip

This zip file contained three directories full of jar files, add all of these to the project. Simply unpack the zip and put all of the jars files into the root of the WebContent/WEB-INF/lib directory.

Now that Jersey is added to the project all that is left is the source code.

Step 3 – adding the code

The last bit is to add the restful code to the project.  This first example actually does little more than display “Hello World”.

Ok, ok, I did get a bit ahead of myself and you can also see that I have created sayHelloXml and sayHelloHtml as well.  These were actually only used from the browser, you can forget about them for right now.

package com.acmesoft;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

@Path("/Communicate")
public class HelloWorldEx {

    @GET
    @Path("/hello")
    @Produces(MediaType.TEXT_PLAIN)
    public String sayHelloTxt() {
    	return "Hello World\nHaving a great time!";
    }

    @GET
    @Path("/hello")
    @Produces( {MediaType.APPLICATION_XML})
    public String sayHelloXml() {
      	return "<Magazine>" +
		"<Year>2016</Year>" +
		"<Month>6</Month>" +
		"<Title>Economist</Title>" +
		"<Price>9.99</Price>" +
	"</Magazine>";
    }

    @GET
    @Path("/html")
    @Produces(MediaType.TEXT_PLAIN)
    public String sayHelloHtml() {
    	return "<h1>This is 'h1' heading</h1>";
    }
}

There is one other small file that needs to be setup, it is the web.xml file. This is the configuration file to setup the Jersey for this servlet.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" id="WebApp_ID" version="3.0">
   <display-name>User Management</display-name>
   <servlet>
      <servlet-name>Jersey RESTful Application</servlet-name>
      <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
         <init-param>
            <param-name>jersey.config.server.provider.packages</param-name>
            <param-value>com.acmesoft</param-value>
         </init-param>
      </servlet>
   <servlet-mapping>
   <servlet-name>Jersey RESTful Application</servlet-name>
      <url-pattern>/*</url-pattern>
   </servlet-mapping>  
</web-app>

This example creates a small hello world application using restful web services.  You can see that the URI to get the hello world is /Communicate/hello, but this is only a part.

Eclipse does a lot for you.  One of the things it does is to create the context root of the URI. We can set it to any value we wish in the last step while creating our dynamic web project.

http://192.168.178.70:8080/FirstRestful/Communicate/hello

Step 4 – creating war file

The process of creating a war file couldn’t actually be much easier.  Simply right click on the project and select export and select war file.  This will open up the following dialog.

The destination in this case is /tmp.  The file is then copied to the /opt/tomcat/webapps directory in a second step. I could have set the permissions on the webapps directory to allow my user to write to it directly, however, that is not very security minded.  This isn’t an issue on a development machine but perhaps it is a bad habit to get into in general, thus the manual extra copy step.

The three checkboxes are reasonably clear to their purpose.  It is possible to optimize for a specific version of tomcat.  Export source files simply includes the source code in the war file.  This might be interesting for an internal application for your company but probably a bad idea if you are selling your application.

Step 5 – releasing application to tomcat

Releasing the application is as simple as copying the war file to the/opt/tomcat/webapps.

  1. stop Tomcat
  2. copy application to webapps directory
  3. start Tomcat

Tomcat will unpack the war file into the webapps directory.

 

Step 6 – viewing the result

Other configuration

In my opinion, this is actually a pretty good URI and isn’t nested too deep.  However, it is possible to add another level to it.  This additional level is defined in the web.xml file.

   <servlet-name>Jersey RESTful Application</servlet-name>
      <url-pattern>/newlevel/*</url-pattern>
   </servlet-mapping>

With this change, there would be another level added to my URI but that seemed to be excessive to me.

What might really be a better choice might be to change the root context.  If you do change your mind you can easily change this to anther value.This is done by simply changing the Web Project settings.  This can be selected from the properties menu (right click on the project and select properties).

This is all the steps you would need to do in order to create a working server.  This can easily be tested out using your web browser, or you can use other tools.  I will discuss that in the next step when I create the first client.

 

Posted in programming, Setup From Scratch | Tagged , , | Comments Off on Restful services in Java – simple server example

Restful services in Java

Over the years there have been different methods of for communicating between heterogeneous systems.  Each of these methods have to deal binary messaging.  Some of the different protocols that have been used are as follows.

  • Remote procedure calls (RPC)
  • Distributed Component Object Model (DCOM)
  • Common Object Request Broker Architecture (CORBA)
  • Simple object access protocol (SOAP)
  • Windows Communication Foundation (WCF)

These protocols are of varying complexity and some of the descriptions that have been used to describe them are complexity, rigid, complex, heavyweight and complicated.

Each time the technology wheel turns a new protocol is created with the goal to create something that is powerful and lightweight.

Restful web services is an attempt at providing a lightweight but scaleable communications protocol that can be called from any platform.  It uses a simple URL instead of creating and passing complicated XML structures

This should be a very simple way of submitting information or retrieving data.  A colleague once recommended that I create my own web services for processing my weekly expenses while I was on the road.

At that time, it seemed like overkill to create such a solution in order to process anywhere from 5 to 10 expense items per week.

  • Setup a computer with web server
  • Program a web service to accept the price data
  • Create some GUI process for entering the data

I did have the interest in the project but ended up focusing my efforts on dabbling in the hardware arena by creating a LED cube.  Eventually, my interest has come back to web services.

Of course as what often happens is that you cannot start something without doing something else first – in this case all the prerequisite software.

My laptop is a bit of a mess so I choose to create a small virtual machine and do all of my work there.  This is actually a extremely interesting use of those extra cpu cycles and could become common for even basic computer users in the future.  I hope to go into that at a later point.

For working with restful web services I needed to install the following.

Once everything was setup, I did get a chance to create both a a server and a client.  I will not actually setup my own restful server setup  for remotely doing my expenses but I may end up using the technology for something else in the future.

The next few blog entries create a few simple examples to put the technology through its paces.

Difference between Soap and Rest protocols.

http://blog.smartbear.com/apis/understanding-soap-and-rest-basics/

Posted in programming, Setup From Scratch | Tagged , , | Comments Off on Restful services in Java

a plethora of shells – what is my shell interpreter

I kept seeing odd error messages from my shell script that simply made no sense.  No matter what changes I made a new error seemed to pop up.  A short while later I did notice at the top of the script that it was using a korn shell interpreter.

#!/usr/bin/ksh

This was not as helpful as you might initially guess.  Sure there are differences between the two shells but most of the scripts that I write would work under either.  Besides, I had a small script with the new code I wanted to test and it worked fine under both the bash and korn shell.

I didn’t initially consider that the the script itself wasn’t not being run in the same manner by the calling system as by me.

In the end I wanted to determine if the shell was the same as what I had selected – after all how could it not be my shell.

The most straight forward method would be to see which shell is interpreting.  It is just as easy as checking which process or processes are running.  Just to see which programs, in this case our shell, is running I ran the process status command (ps).

Nothing else is really running in my Korn shell example.  The only two processes is the korn shell and the ps command.  The Korn shell is not really all that surprising as this is a terminal window for my user and the Korn shell was the shell setup for in the /etc/passwd file.

Korn shell

> ps -f

     UID   PID  PPID   C    STIME TTY         TIME CMD
   fkadm  3819  3816   0 14:52:41 pts/2       0:00 -ksh
   fkadm  4609  3819   0 14:54:37 pts/2       0:00 ps -f

It is almost the exact same story for the Bash shell.  My user was started with a Korn shell and (process 23509) and then ran the Bash shell (process 23541).  The ps command is then run from the Bash shell reporting on anything that is running.

Bash shell

> ps -f

     UID   PID  PPID   C    STIME TTY         TIME CMD
   fkbin 23509 23507   0 10:27:31 pts/4       0:00 -ksh
   fkbin 23541 23509   0 10:27:31 pts/4       0:00 bash
   fkbin  4128 23541   0 14:53:51 pts/4       0:00 ps -f

The starting point for my finding my shell is to know which of the many processes to look at.  The good news is that I am able to get my process id using the special shell variable “$$”.

Everything from that point is pretty straight forward.  We can use the -p and -o arguments with the ps command to see specific values for a specific process.

 

process status (ps)

argument Description Description
-p process id selects process status from list where the process id matches this value.
-o format The format is one of many possible fields that can be selected from the process status.  There are quite a few possibilities, but the ones that stood out as the most immediately interesting are as follows.

  • user
  • pid
  • ppid
  • stime
  • tty
  • args
  • comm

The first six should look familiar as these are the values that are output when doing a full listing (-f).  The comm format is the actual command which in this case is the interpreter while the args is both the command and its arguments.

With this little bit of knowledge I had enough information to query what interpreter was running my script.

#!/usr/bin/bash
MYPID=$$
MYSHELL=`ps -p $MYPID -o comm | egrep -v "CMD|COMMAND"`
MYCOMMAND=`ps -p $MYPID -o args | egrep -v "CMD|COMMAND"`
echo the shell for process id $MYPID is $MYSHELL.  
echo the entire command is $MYCOMMAND

This type of shell goodness should be all that it takes to find out more about how you are being called.  I was a bit surprised to see that my script was being called with a different shell altogether.

The shell calling my program was /usr/bin/sh which is a lot less compatible with either the Korn or Bash shell.  A bit more digging and I found out that the script was being passed directly to the shell interpreter.  This explains why the Korn shell wasn’t being used.

sh /home/specialuser/scripts/myscript.sh

Well, I guess this is proof that you cannot always trust IT.

Posted in Command line | Tagged , | Comments Off on a plethora of shells – what is my shell interpreter

Windows powershell scripting

Powershell scripting is not too difficult to pick up and there is a wealth of examples on the internet to help solve any syntax related question.

Simply call the appropriate Powershell “cmdlet” to perform the required task.  The return value can be manipulated directly without being forced to assign intermediate values to a temporary variable.  It is easy enough to manipulate or format the return value.

$datestr = (get-date).tostring('yyyyMMdd')

This example uses the get-date cmdlet to get the current date and format it into reasonably friendly way.

Functions

Just like any proper language it is possible to create functions to assist in breaking up program logic.

Powershell has a number of primitives that can be used for creating functions or other program logic.  This list of supported primitive is actually fairly large but here is a smaller subset which covers the usual suspects.INT

  • BOOL
  • FLOAT
  • DOUBLE
  • STRING

Microsoft has taken a route less traveled with respect to how they define a function.  The parameter types are surrounded with square brackets. In addition, they seem to have gone back to “BASIC” and put dollar signs before each variable.

# write out log info
Function My-Add-Function([float]$firstval,[float]$secondval) 
{
	$returnval = $firstval + $secondval
	return $returnval
}

Microsoft allows you to call your user defined functions in the way you would expect.  You simply call the function with the values being passed.

ie.

	my-add-function $firstvalue $secondvalue

Yet it is also possible to actually pass in the parameters in any order if you simply give the name of each parameter as they are named in the function.

ie.

	my-add-function -secondval $secondvalue -firstval $firstvalue

I don’t know of any other languages or scripts that allow you to call your own functions in this manner.  It is interesting but I don’t see it as an important feature.

It actually took me a while to see what was missing from the function declaration – there is no return type.  This doesn’t mean you cannot return values from functions but how it is done is odd to say the least.

# function GetDateNow returns todays date
Function GetDateNow 
{
        $TimeStringNow=(get-date).tostring('yyyyMMdd')
        return $TimeStringNow
}

Basically all output that occurs in the function is returned.  Looking at this method you would think that the variable TimeStringNow is the only thing being returned and it is because of the return keyword.  This is not the case.

# function GetDateNow returns todays date
Function GetDateNow 
{
        $TimeStringNow=(get-date).tostring('yyyyMMdd')
        $TimeStringNow
}

This second function declaration is semantically identical to the first.  The return keyword simply exists so it is possible to exit a function at a specific point, it has no connection with the values being returned.

The one part about the return values that I didn’t mention is that everything that is output in the function (no matter when) will be returned to the caller.

I actually do not like how data is returned from a function but I cannot say that this is completely unique.  This is similar to how Unix shell scripting works if you call a function while capturing the output from that function to a return variable.

#!/usr/bin/bash
function unixfunction 
{
echo "here at point 1"
echo "here at point 2"
echo "here at point 3"
echo returning 6
echo 6
}

retval=`unixfunction`
echo $retval

The output looks like this.

here at point 1 here at point 2 here at point 3 returning 6 6

I don’t really like Powershell scripting but that might just be an anti-Microsoft bias. Using Powershell is actually miles better than using batch scripts from all those years ago. It is really just a matter of taste, if you are a Windows aficionado this is not only a good choice but perhaps the best choice as it comes standard with Windows.

Does it suck?  Well, I guess not.  If you look at this from the perspective of replacing the cmd.exe with Powershell you are probably trading up.  If you want more command line programs that you string together then in my opinion it isn’t really an improvement.

 

Posted in programming | Tagged , | Comments Off on Windows powershell scripting