Grids and Minis

There are two opposing forces separating the hardware aspect of the computer industry - on one hand we have the Mac Mini showing that we do not need to have a (multiple) large ugly beige box(es) around the house. At the other end we have the ubiquitous computing platform from which all we ask is for storage and compute cycles.

In the household the ease of use, no wires, and seamless connectivity with all other appliances is going to be the driver for consumer purchases. Macintoshes are just so much easier to use and integrate into a digital lifestyle and for techies we can get to the internals of Unix with no problems - think remote management of Unix versus Windows. How long before we have remote management services for the home network?

On the corporate front I have gone from managing large internal data centers, to leasing rackspace, power and bandwidth to totally managed systems, where I request boxes on a demand basis and then install application software. Here we are still dealing with unique boxes and leasing boxes as a representation of computer resources. It is cheaper in most cases to lease boxes on demand rather than own them, but the real advantage will come when we break free of the computer box and have a true grid.

It is a question of degrees of freedom - with a box (even in a 1U form factor) the degrees of freedom are limited and the amount of management required for load balancing etc is too much. Installing software on multiple boxes and then configuring load balancing etc. should not be required. This is true whether we are in a home setting or using corporate resources - our thinking is constrained by physical boxes. The ideal situation is being able to inject software into a domain and have it managed on demand.

Today we have a one to one correspondence between boxes and resources. This dependency needs to be broken to create the next wave of innovation and ease of use. The question is whether the innovation will come from the hardware vendors or whether some smart software will abstract the hardware into a virtual grid is the question. Realistically both need to happen, however the software will probably come first as the hardware makers are too entrenched in shipping boxes to change.

Once this happens and it will the question will be how to access resources in the local and global grid - this is when things will get fun as there are no longer compute or storage boundaries - we are no longer navigating static information but information that is changing as we interrogate it and follow meaning provided by the compute resources. A bit far fetched? not really, I believe we are only a 5-7 years away from this in terms of software. The hardware will follow the software as the need to own hardware is going to go the way of owning power generators and telephone switches. It is perhaps time to start thinking about the software foundations of this new platform.

Is it the message or the transport

To make integration work the discussion seems to focus on the transport rather than the message. We are trying to convey meaning the transport is largely irrelevant providing both sides speak it and HTTP is the lowest common denominator.

Phil Windley as usual makes good points in his post ? On the death of SOAP | Between the Lines | ZDNet.com but I think the issue is in the white space between the transport - HTTP and the message - XML

If I may make a bold statement - between disparate systems (even similar systems ) the transport protocol is HTTP and the message format is XML, end of story. There are other use cases and approaches but they are edge cases and very domain specific and have propriety end points such as AS/2. For the foreseeable future the common transport is HTTP(S), protocol independence is a futile exercise and (IMHO) should be abandoned.

The real issue for most applications is which end is going to do the transformation into the format necessary to process the message. Whether the message is formatted as a SOAP or REST message is largely irrelevant to the end point that needs to do the transformation, the issues that need to be addressed are:

• Is the message understandable?
• Is there sufficient fidelity for understanding?
• Can the message be converted into the receiving format?
• Can the response be returned in an understandable format to the sender?

I am not sure REST or SOAP address these issues. A well documented schema and a well written HTML page probably contribute more to message integration than any WSDL or REST URI

This is because any standard needs to both exist for at least five years and be actively deployed for five years before it makes any impact on the industry as a whole. In my current environment we are integrating systems from Windows 3.1 to the present day. This is real life, and it will always be this way, legacy systems will always be in the majority. Interesting however the information flows are pretty constant, how they are described and formatted has not changed much - they have become formal and faster but the core information is the same.

The point is therefore the transport of the information needs to be the lowest common denominator while the ability to convey meaning must the focus not the protocol.

Impedance Reduction

One of the usual starting points for me in a design is the flow of information. That usually leads me to creating a set of XML Schemas for the information flow. This works well until I need to start developing code. In the past this has been a little tiresome to create (and recreate as the design evolves) the bridge between xml and the application.

This problem seems to be rapidly going away both in the Java and .Net environments, in Java I am using XMLBeans which once you get it installed and working provides a slick and easy way to map between schema and Java. In the .Net world I have been using a tool called xsd that is part of Microsoft .Net toolkit.

On the two approaches XMLBeans seems to be the most polished and functional but it is great to have similar functionality in both environments to improve productivity and reduce the drudgery.

REST and SOAP: Fitting them into your toolbox

Two appropriately cautionary posts about SOAP and REST raise some good issues,
Jon Udell: Don't throw out the SOAP with the bathwater and
Steve Vinoski: SOAP vs REST: stop the madness.

I have always had the metaphor of a toolbox where I keep the set of tools I need to do my job. Taking it a little further I have seen the standards process (both informal and formal) as the series of events that shape and separate the tools in the toolbox. This shaping and separation ensures that the tools can work together and overlap enough to solve most problems. The struggle I am having is where do these tools fit in my toolbox?. Jon Udell asks the question:

Why must we make everything into a rivalry? Why can't we just enjoy the best of all worlds?

Today I use both for different purposes but do I want to support both for all customers - this becomes an economic issue that significantly reduces the enjoyment. If I support one over the other, what do I say to a CIO about why one versus the other should be in our mutual toolbox. Is it clear when to use SOAP and when to use REST - if it is clear then we have the best of both worlds - until then we need to keep working on shaping our tools.

RESTful coding standards

In AJAX Considered Harmful Sam Ruby starts to lay out a set of useful coding standards for REST. He addresses Encoding and Idempotency, however going a little further has helped me. I have tried to apply the KISS principal and be a minimalist i.e. only add layers of complexity when you need them

Most use cases I have come across can be reduced to a CRUD interface (and yes there are exceptions, and exceptions prove the rule). The starting point is some standard naming conventions to create some degree of consistency for users for example:

• Create POST to http://xml.domain.com/resource/Create


• Read GET from http://xml.domain.com/resource?QueryString


• Update POST to http://xml.domain.com/resource/Update


• Delete POST to http://xml.domain.com/resource/Delete

The other issue is how to pass arguments and results back and forth. Some decisions are obvious, GET always passes a query string and the return data is always XML both using UTF8. I always have issues with POST though for invocations I recommend using XML always as it is hard on a user to have to switch between POST form parameters and XML for different methods. If XML is always used it keeps everything consistent. However when a delete method only requires a single key it is very tempting to just pass form parameter or even just do a GET with a query string. So far I have stayed with consistency - I would be interested in other opinions.

The other of standardization that helps is error returns, SOAP Faults are a good idea that REST can easily adopt. With a CRUD interface there are a simple set of standard errors that can be defined that come back with a HTTP 500 status code:

• Create: Item Already Exists User has no permissions ...


• Read: Item Does not Exist User has no permissions ....


• Update: Item Does not Exist Item Locked User has no permissions ...


• Delete: Item Does not Exist Item Locked User has no permissions ....

A simple REST Fault schema can be used to always return the same format such that common utility code can be created to handle errors in a uniform way. For errors to return permissions errors there needs to be the idea of identity which requires authentication

For authentication the lowest common denominator seems to work most of the time - interested to hear about real cases where it does not. If the request is over HTTPS and the authentication is HTTP-Auth then we have pretty good security. One step further would be to PGP the message but that would only be needed where the message needed to be highly secure and it traveled over unsecured pipes before and/or beyond the HTTPS stream.

I would be interested in feedback and extensions to this as it would be nice to have a consistent pattern for programming REST interfaces. The consistent pattern would enable us all to build interfaces faster and learn to use the work of others faster and easier.

Laszlo has released 3.0b2

This release is a big milestone for OpenLaszlo (IMHO) as it provides for serverless deployment of the UI components. Obviously a server is still required if you need to interact with data but not for rendering. After working with systems such as Dreamfactory the importance of not needing a server for rendering became very clear both for simplicity of deployment and scalability. Congratulations to the OpenLazlo team for taking things in this direction

Nelson on actual implementation of SOAP and REST

Phil Windley posts from ETech: Phil Windley's Technometria Nelson Minar at the Google AdWords API. This is good summary by Nelson Minar on the issues and differences around implementing services in SOAP and REST. I particularly like Low REST and High REST definitions - a good characterization I have been struggling how to explain and rationalize in an implementation for the last few weeks.

For REST to come into its own there needs to be some more formalism along the lines that Nelson suggests and also so does David Orchard.

A good series of RESTful articles

I have not had time to read them all but ( I intend to) Carlos Perez has a great series of articles on REST - Manageability - Brainstorming a ReSTful Toolkit. If the REstful toolkit can be understood in less than a day it could be a winner.

It Ain't Easy to be Simple - but it is important to understand and communicate

I took the liberty of expanding on the title of this article It Ain't Easy to be Simple by Mike Champion. The article surprised me it was on the front page of MSDN yesterday indicating that the REST - WS debate is moving into the center of attention.

I am not sure I would argue that HTTP/REST is elegant, but rather it is a good approach to working with what we have. Many years ago circa 1997 someone said to me that HTTP is the last protocol that will be invented - the comment stuck but it was not till many years later that I realized the truth of it. Essentially the weight of the installed base of plumbing we have installed at both desktop and server virtually requires that HTTP exists everywhere for at least the next few decades. I think XML is moving into the same position as a data representation language. Therefore it is only natural that they come together to communicate meaning in machine-machine and machine-human conversations. These essence of the debate is do we take a minimalist approach to defining the conversation (REST) or do we take the solve every problem approach which is the WS-*&%$# approach?

More voices seeing the erosion of simple in web services

In this article in Jonathan Schwartz's Weblog he cites that corporate IT Architects and developers are getting worried about the erosion of simple in web services.

One reaction to this is to build complex development environments that mask the complexity and make it appear simple, but this is usually the wrong approach as good architects and developers need to understand their tools and how they work. I have never been able to make a poor developer become a great developer by giving them a tool - it takes training, monitoring and patience.

Web Services (and by web services I use the broad meaning - services that communication over the web) are important and are not going away, so we do need to solve the problem. More and more companies are moving to shared infrastructure and shared services and they need to be able to communicate, simply, transparently, reliably and securely. Simply often means solving only 80% of the problem and just getting something working in the real world, the WS-x!*&@ stack tries to solve 100% of the problem before the real world has defined the problem. Time will tell what the outcome will be, but my bet is on simple to win

Linkedin and Craigslist

Over the last several months we have been hiring in several areas. So far the best service has been CraigsList in terms of quality of applicants. However I have just started using Linkedin's new service and it is starting to show what a social network can do. The user interface is dramatically easier than CraigsList (which has completely awful search capabilities - try searching for C# or .Net) and makes use of your social network to get well matched clients.

Is it perfect - no it has a long way to go but it is the first feature in a social network that does something unique to social networking and makes business sense. To early to tell yet if the quality of applicants matches CraigsList but Linkedin has the right demographic for our type of business, technically sophisticated, early adopters and willing to push the envelope. So I imagine that it should evolve into a very good service if they can keep the signal to noise ratio low - I notice a lot of recruiters in there already. While recruiters are a necessary part of our business they are an intermediary that might not work in a social network the same way the work today. It is an area Linkedin should think about to keep the service useful.

106Miles Blog: March meeting full up

Bummer I was interested in seeing what this was all about: 106Miles Blog: March meeting full up. Maybe next time.

Leaning towards REST but.....

Living a little more in the real world these days, i.e. working at the application layer, solving customer problems I have had the chance to use Web Services (SOAP and REST) in various application senarios. From that I am leaning more towards REST style architectures because in the real world I inhabit it is unreasonable to expect all end points to have the latest technology and be able to navigate the complexities of SOAP.

The biggest problem I see with the WS-*%$#? mess is that it is quickly becoming the next CORBA and is failing one of my simple heuristics, if a single reasonalbe talented engineer cannot pick up the concepts and start being productive in a week or so then there is an issue. In this case the S has been dropped from SOAP and we just have Object Access Protocol which is another way of saying RPC and leads back to CORBA. This version is better as it not tightly locked into a synchronous RPC view of the world but the complexity still exists and is growing.

On the other hand there are things I still like about the web services stack and are missing from REST. One is the degree of formalism in defining interfaces either RPC or document style interfaces. Last week I needed to access several web services and being able to query them for their WSDL was great and made my life a lot easier. Having to go through each one and look for the documentation (we all documented everything all the time right !) would have been a major pain. This is the subject of a good article by Hugh Winkler that Mark Baker pointed to. It makes a good case for have a formal interface description for REST.

So while I lean towards REST for simplicity, the ability to talk to any end point providing they have HTTP and XML the lack of a standard machine description is a big lack - I would almost be happy with a JavaDoc or NDoc type of tool but for REST to take over the Simple part of web services I think this is required

Retail becoming more tech centric

I attended an event hosted by Sun Microsystems last week around retail technology. A part of the program was a presentation from Jeff Roster from Gartner. A significant part of his talk was devoted to how the retail industry is moving from techno-phobic to techno-centric, and he had a lot of information to back it up. A major piece of data was showing how more IT centric retailers would tend to dominate in terms of market growth

While I agree with the essence of his presentation, and I have seen the same, I think it needs to be framed in terms of results rather than just bit and bytes. What is happening is that the smart retailers are starting to use technolgoy to give them accurate numbers to run their business. It has started with Walmart and the supply chain and it is now moving to the demand (shopper facing) side of the business. While IT is great the key in the Information not the Technology part of IT. Retailers who empower IT to deliver accurate information and that the business side can act on are the ones that will succeed.

Actionable information is a critical measure of success, there is no point in knowing that store abc is losing shoppers or average basket size is declining if there is no processes in place to act on the data. A part of any information project is ensuring that there are levers to pull and knows to turn to change the information that is measured.