The Stateful Illusion

In some of my earlier postings, I’ve explained how the underlying web-application and web service architecture of both GT.M and Caché are natively stateless, by virtue of the gateways they support.  Our web application development framework, EWD, makes use of this stateless architecture.

When writing a web application, it’s normally necessary to create, for the user, an illusion that they are dealing with a stateful application.  So what do I mean by this, and what’s the technique that is used in EWD to create this illusion?

Back in the days of dumb terminal interfaces (sometimes also known as green screen or roll and scroll interfaces), applications ran on a server and the user interface (UI) was generated by writing out escape sequences: specific character sequences that would be interpreted by the visual display terminal to do things such as position the cursor, set highlights, clear areas of the screen, etc.  When a user logged onto such a system, he/she was connected to his/her own dedicated process on the server until such time as he/she logged out or was timed out (ie if they didn’t respond within a specific period of time).  During that time (known as the user’s session), the back-end process not only had access to the database, but the application logic could build up local data in the user’s process memory (commonly known as a symbol table).  In effect, the UI was in direct contact with a dedicated process for that specific user.  Once the user logged in at the start, the back-end process automatically knew implicitly which user it was dealing with throughout the rest of the user’s session.   Though we didn’t refer to them as such in those days, these processes were what we now term stateful.

In a stateless environment, it’s all very different.  A pool of back-end processes are shared amongst the user population.  The user’s interaction with the UI (typically in a browser) generates events which, in turn, create requests that ask the back-end to do some appropriate action, eg look up some information, validate and save some data, delete or change a database record, etc.  Any one of the pool of back-end processes must be able to receive a request and action it, then make itself available for the next incoming request which could come from the same or any other user.

In the case of a web application, the event-generated requests take the form of URLs that are sent to the web server and then forwarded by the gateway to a back and GT.M  or Caché process.  The only information that can be conveyed by a URL takes the form of a set of name/value pairs and/or cookies (which, in turn, are really just a special form of name/value pair).  You’ll have often seen name/value pairs as the stuff that follows a ? in a URL, eg:

In the example above, hello=world and name=rob are name/value pairs.

From this set of name/value pairs, a web application back end has to be able to figure out and do a bunch of things, including:

  • identifying the person making the request.  Is it a new user or an existing user?
  • ensuring that the request isn’t coming from someone pretending to be an existing user
  • managing to create and maintain a pool of data that is specific to the user in a way that it can be brought back into play to handle the current request, and used to support subsequent incoming requests from that user.  That pool of data must be accessible to any of the pool of stateless gateway-connected processes.

These three things are generally termed:

  • session management
  • security management
  • state management

and are the key responsibilities of a web application development framework such as EWD.

How these things are done is actually pretty simple (though, as is always the case, the devil is in the detail).

EWD has a simple rule for establishing a new user session: the developer can specify an EWD page as being a “first” page.  When a URL is received that asks to fetch such a “first page”, EWD allocates a new session Id (which is just an otherwise meaningless integer).  The session Id is known only to the back-end and not exposed to the browser directly.

When EWD generates the URLs that get sent to the user’s browser, it pre-arms them with extra name/value pairs called tokens.  Tokens are simply randomly-generated, otherwise meaningless text strings.  EWD saves these tokens into a Mumps global as a key/value pair, with the key being the token and the value being the associated session Id.

When the user clicks on some UI device or otherwise does something to generate an event, the associated pre-tokenised URL is returned to EWD (via the web server and gateway).  The first thing EWD does is to check the token:

  • was there one at all?
  • if there was, is it recognised as being in the key/value store of ones it generated?
  • has it expired? (tokens are given a limited time threshold)
  • if it’s valid, then what’s the session Id for the user making the request?  A simple look-up in the global-based key/value store against the incoming token can provide this.

If any of the tests proved invalid, then EWD will return an error page or alert, and prevent the user doing anything further.  Otherwise the token is used as a safe and secure proxy to establishing the user’s session Id.  We’ve therefore got the first essential step in creating an illusion of statefullness on top of a stateless architecture.

The second step is to have a persistent, but ephemeral, data store that can be used to hold data specific to that user in such a way that it  can be brought into use when handling any subsequent request from that user.  GT.M and Caché turn out to be ideal for this purpose, courtesy of their unique global database storage.  Not only can they store simple key/value pairs, they can just as efficiently store hierarchical persistent arrays.  By using the user’s unique session Id as the primary key,  EWD can support any complexity of session data.  Also, data stored in global storage is automatically available to any GT.M or Caché process.  So data stored into a global by one process can be picked up and used automatically by any other process.  This fits perfectly with our requirements: if the first request from a user is dealt with by one gateway-connected process and the second request is dealt with by another gateway-connected process, any data that is created by the first process and saved into global storage is automatically available to the second process!

A key requirement of session data storage for stateless web application frameworks is that it must be fast – really fast!  This is again where GT.M and Caché come into their own: their global storage engines have been highly optimised over decades to make them exceptionally fast, even on relatively modest hardware.

Session storage is, as I’ve stated above, ephemeral: ie it exists only as long as the user’s session is active.  This means either until the user explicitly invokes a “log out” event, or until the tokens associated with the user’s session expire.  Thus, another of EWD’s roles is to provide a garbage collection process which automatically deletes all the session storage keyed against any session Ids that have either logged out or expired.

In EWD, pretty much all this stuff just happens automatically for you.  Developers don’t have to worry about any of it: they just create a simple request for a particular EWD page, EWD’s compiler & run-time turn it into a full-blown tokenised URL, and EWD does all the checking, testing, session Id allocation etc automatically when the tokenised URL is sent back in a request..

In EWD, you define an application as a set of “pages”.  Each page is a set of HTML tags, XML-based EWD Custom Tags and/or Javascript logic.  EWD’s compiler turns these into a GT.M or Caché routine that, when invoked by EWD’s run-time engine, writes back the HTML and/or Javascript to the browser.

In addition, and critically, each EWD page can have a so-called onBeforeRender method, which is normally a reference to a GT.M or Caché Mumps function (it can also be a reference to a Caché class method).  The onBeforeRender method is where the developer can define back-end logic to handle the request for the specific page.  Typically that logic will:

  • run queries against the database to fetch data needed for display in the EWD page
  • validate incoming data from the request (eg from a form) and save it into the database
  • delete or modify data in the database

Additionally the developer can save data to the user’s session data store and/or retrieve data from the user’s session store.  In EWD this data store is known as the EWD Session.  EWD provides a set of built-in APIs that the developer can use to access the EWD Session, typically to:

  • save simple name/value pairs to the EWD Session
  • retrieve name/value pairs from the EWD Session
  • get hold of entire multi-dimensional arrays from the EWD Session
  • save entire multi-dimensional arrays to the EWD Session

When the developer’s onBeforeRender method is invoked by EWD, it is passed the value of its single argument which is the user’s session Id.  This is then used as an argument for the EWD Session APIs.  For example, to get the value of a simple EWD Session variable named patientId, the developer would use the API call:

  • set patientId=$$getSessionValue^%zewdAPI(“patientId”,sessid)

The variable sessid contains the user’s unique session Id.  The developer doesn’t have to  worry about how or where it came from: EWD provides it automatically at run-time.

Put these things together and EWD has everything it needs to create the illusion of a stateful environment.  EWD automatically identifies the user for the developer and presents the developer with a clean, empty symbol table in the back-end process that will handle the incoming user’s request.  The  developer can then pull information in from the user’s EWD Session data storage to create local variables in the process’s symbol table.  Conversely, at the end of each onBeforeRender method, the developer can save back into the EWD Session any symbol table data that he/she knows may be needed in order to handle some later request.

Hopefully you can now see that GT.M and Caché have turned out to have, at their disposal, everything needed to support web applications.  As I’ve said in another of my blog postings, it’s almost uncanny: it’s as if they were designed from the ground up for the web, and yet they both pre-dated it by at least a decade!  They represent a technology that has, in the past,  been little known or understood by the “mainstream”, but also a technology that has turned out to be way ahead of its time and precisely what is needed for today’s web-centric world.

In a future posting I’m going to examine an interesting challenge: how EWD can be used to breathe new life into tried and tested legacy Mumps applications that were designed as stateful, dumb-terminal-based applications.  There are a great many such applications out there, particularly in healthcare: solid, highly functional, business-critical applications whose only real downsides are that they look old-fashioned and require dumb terminals or terminal emulators in order to run them.  Persuading them to play in a stateless environment and with a modern state-of-the-art browser-based UI is simpler than might be thought and turns out not to require major intervention or rewriting.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: