Using Mumps Globals To Model a System (Part 1)

My earlier article titled A Phoenix Rises described an OO Javascript abstraction that I’ve created for Mumps Global Storage, the purpose of which is to introduce this fascinating but little-understood database technology to a new generation of modern developers.

If you’ve examined the methods and properties that my OO projection exposes for each instance of a GlobalNode object, you’ve probably concluded that this is a very low-level database technology.  You’d be right!  There’s no built-in indexing, no built-in query engine, no schema or data dictionary.  You’re probably wondering how on earth anything so apparently basic and primitive can be useful in any way.

I discussed this aspect of Mumps Global Storage in another paper about 10 years ago.  The low-level nature of Mumps Global Storage is what makes it so flexible and capable of being used as a Universal NoSQL database engine, but it’s also why a lot of people make the mistake of dismissing it as being far too basic and simple for practical and serious commercial use.  However:

  • its dominance in healthcare applications and its widespread use in business-critical financial service applications demonstrates that it is clearly fit for purpose – this is no toy database and, when correctly harnessed and tamed, it’s capable of amazing things.
  • if, like me, you’re a geek at heart and enjoy building stuff from first principles, then you can roll your sleeves up with a Mumps database and construct your own, purpose-built database abstraction, and create your own querying tools and so on.  It’s a database equivalent of a set of Lego bricks: it’s up to you to put those bricks together into something interesting, and you’ll have pretty much a free rein in how you can put those bricks together and what you can construct from them.  You’ll find that it’s not only a lot of fun, but along the way you’ll discover for yourself just what it takes to build all the bells and whistles that normally accompany a commercial DBMS.

With that latter point in mind, I want to use this blog posting to show you in detail how we can make use of these database Lego bricks and start to do something non-trivial and potentially useful and interesting.  In doing so, hopefully you’ll start to “get” how Mumps Global Storage can be harnessed and exploited and discover for yourself why it’s such a fascinating and powerful database technology.

Specifically, I want to examine one of my Node.js modules that I’ve called ewdDOM.  This module provides an implementation of the XML DOM, but with a twist: instead of modelling the DOM into in-memory structures, ewdDOM models the DOM directly into Mumps Global Storage.  It therefore implements a persistent XML DOM: the idea is that any XML documents you create will persist as DOMs in a Mumps database until you explicitly delete them, and will be potentially available to you every time you fire up a Node.js process.  This is pretty cool for two reasons:

  • the XML DOM is ideally implemented as a graph: it’s a set of nodes, each with a number of properties, and the nodes are inter-connected by pointers which represent the hierarchical relationship between those nodes.  Hence, ewdDOM demonstrates how to use a Mumps database as a graph database;
  • as the XML documents that you create using ewdDOM are persistent but in DOM format, ewdDOM is effectively demonstrating how a Mumps database can be used as a Native XML Database.

So let’s take a dive into the ewdDOM module, but in a way that you can try it out for yourself.  In doing so, you’ll discover one way in which you can use Mumps Globals Storage to model a system (in this case the XML DOM) and bring it to life.

Pre-Requisites

You can use the ewdDOM module and the examples I’m going to document here with any of the major Mumps database technologies:

  • InterSystems Caché (commercial/proprietary: Windows/Mac OS X/Linux)
  • InterSystems GlobalsDB (free/proprietary: Windows/Mac OS X/Linux)
  • GT.M (free/open source: Linux)

I’m going to base the rest of this article around GT.M.  If you want to install GT.M yourself on your own Linux server, you can follow Luis Ibanez’s instructions.  You’ll also need to install Node.js and the ewdGateway2 module before going any further.

However, to make life much simpler, I’m going to assume you’re using the free, Open Source pre-built dEWDrop Virtual Machine.  Hopefully, before long, there’s going to be an Amazon EC2 AMI for dEWDrop that will make it even easier to try things out with GT.M.

If you use your own custom GT.M configuration, or if you use a Caché or GlobalsDB system, you’ll need to adjust the instructions accordingly: primarily you’ll need to adjust the directory paths that I use in my tutorial below.

So, if you also want to use the dEWDrop VM and haven’t yet installed it, I’ve posted an earlier article on what you need to do before continuing with the next section below.

Installing ewdDOM

Log into the dEWDrop VM and switch to the /home/vista/www/node directory:

cd /home/vista/www/node

Install the ewdDOM module using npm:

  npm install ewdDOM

You should now find that a directory /home/vista/www/node/node_modules/ewdDOM has been created, containing a copy of the ewdDOM module.

We also need to install another Node.js module called sax (make sure you’re in the /home/vista/www/node directory before running this):

  npm install sax

You’ll find that this has also been installed into the /home/vista/node/node_modules directory.

The dEWDrop VM has everything else installed that we need so we can now get to work!

A Node.js Template File

The first thing we’ll do is to create a template file that we’ll use for our examples.  Copy and paste the following into a text file and save it as /home/vista/www/node/domTest.js:

// Initialisation: Get and start ewdDOM environment and OO abstraction
// for Mumps Global Storage

var dom = require('ewdDOM');
var ewd = require('/home/vista/www/node/ewdGateway2/lib/ewdGlobals');

var globals = require('/home/vista/mumps');
var db = new globals.Gtm();
db.open();

dom.init({
  db: db, 
  ewdGlobalsPath: '/home/vista/www/node/ewdGateway2/lib/ewdGlobals', 
 // store XML document DOMs in a Mumps Global named xmldom
  domGlobalName: 'xmldom'
});

ewd.init(db);
// create pointer to the Mumps Global in which XML documents will persist
var xmldom = new ewd.Global('xmldom');

// Clear down anything that might already exist in this Mumps Global, so we
// are alwways working from a clean slate:
xmldom._delete();

//
//==================================
//Our XML DOM coding will go here...
//

//
//==================================

// Finally, cleanly close down NodeM interface

db.close();

You could try running this example – it won’t actually do anything yet, but if it runs without error, then it will show that everything is installed and running correctly:

  cd /home/vista/www/node
  node domTest

What the code in this file does is:

  • loads the modules we need
  • starts up a connection with GT.M using the NodeM interface
  • tells the ewdDOM module to save any documents to a Mumps Global named xmldom
  • clears down the xmldom Global, so we’re always starting from a clean slate (normally you’d not do this, but it makes things clearer for now)
  • finally it disconnects the NodeM connection to GT.M

Creating an empty XML DOM

Now edit the template file (domTest.js) and, within the section I’ve denoted as:

  Our XML DOM coding will go here...

add the following:

  var document = dom.createDocument('myDocument');

Save it and re-run it:

  node domTest

You should be immediately returned to the command prompt as if nothing has happened, but if we look inside GT.M and inspect the xmldom Global, you’ll discover that quite a bit has happened.  To do this, we’ll need to go into the GT.M shell:

mumps -dir
MU-beta>zwr ^xmldom
^xmldom("docNameIndex","myDocument")=1
^xmldom("documentCounter")=1
^xmldom("dom",1,"creationDate")=1360419106131
^xmldom("dom",1,"docName")="myDocument"
^xmldom("dom",1,"node",1,"nodeType")=9
^xmldom("dom",1,"nodeCounter")=1

This is how a Mumps programmer would inspect a Global.   However, the Node.js OO projection provides us with another way, where we can view the Global’s contents in Javascript terms: in JSON format.  The Global’s contents will probably make more sense to you when viewed this way.

To do this,  first exit from the GT.M shell:

MU-beta>h

Now edit the domTest.js file and add the following lines:

  var document = dom.createDocument('myDocument');
  console.log("\r\nHow myDocument is physically represented in the Mumps Global:\r\n");
console.log(JSON.stringify(xmldom._getDocument(), null, 2));

Save and re-run domTest.js and you’ll now see:

{
  "docNameIndex": {
    "myDocument": 1
  },
  "documentCounter": 1,
  "dom": {
    "1": {
      "creationDate": 1360511068605,
      "docName": "myDocument",
      "node": {
        "1": {
          "nodeType": 9
        }
      },
      "nodeCounter": 1
    }
  }
}

We’ve used the _getDocument() method to retrieve the xmldom Global’s contents as a corresponding JSON object.  If you compare the view of the xmldom Global that we saw in the GT.M shell with the JSON view above, you should see how they correspond.

From now on, we’ll use this JSON view to visualise and display what we’re physically doing when we use the XML DOM APIs within the ewdDOM module (By the way, have you noticed that in using the _getDocument() method, we’re now using our Mumps database as a Native JSON database?  And this is at the same time as we’re using Mumps as a Graph Database to create a Native XML Database!).

So let’s recap what we’ve done: we ran the createDocument() API, asking it to create a new (initially empty) XML document named ‘myDocument‘.  Behind the scenes, six Global Nodes were created.  These are as follows:

First:

 
{
  "docNameIndex": {
    "myDocument": 1
  }
}

This is used as an index, allowing us to search for documents by name.  The index is pointing us to document number 1 which results from the second Global Node:

 
{
  "documentCounter": 1
}

This is a counter that is incremented whenever a new document is created, and the value that is returned is used as the identifier for the new XML document: in our case the first document we create is number 1.

Third:

 
{
  "dom": {
    "1": {
      "creationDate": 1360511068605
    }
  }
}

This defines the timestamp when the XML Document (number 1) was created.

Fourth:

 
{
  "dom": {
    "1": {
      "docName": "myDocument"
    }
  }
}

This sets the docName property for our XML DOM (number 1).  As such it’s the inverse of the docNameIndex Global Node.

Fifth:

 
{
  "dom": {
    "1": {
      "nodeCounter": 1
    }
  }
}

This is a counter that is incremented whenever a new node is created within our XML document, and the value that is returned is used as the identifier for the new Node.

And finally:

 
{
  "dom": {
    "1": {
      "node": {
        "1": {
          "nodeType": 9
        }
    }
  }
}

This defines the top-level Node in our XML document’s DOM (which the previous counter has defined as Node number 1). The Node is given a nodeType property of 9 which tells us it’s a Document_Node

To make sense of what’s happened, we now need to examine more closely how the XML DOM is defined.

The XML DOM

Let’s try to make some sense of these Global Nodes that were created when we ran the createDocument() API.  To do that, we need to understand how the XML DOM can be modelled.

The XML DOM represents any XML document as a set of Nodes that are interconnected by a set of pointers as a recursive parent/child hierarchy.

Each Node can be visualised as follows:

Screen Shot 2013-02-10 at 13.14.27

Each Node represents some component of the XML document, such as Elements (otherwise known as Tags),  Attributes,  Text Nodes and Comments.  A Node can have a number of properties that describe such things as its type, it’s name (eg if it’s an Element) or its data (eg if it’s a Text Node).

Nodes are created using APIs such as createElement(), createTextNode() etc, and are connected together using APIs such as appendChild() or insertBefore().  These latter APIs automatically put in place and/or modify the pointers that interconnect the Nodes.  A Node will only have the pointers that are appropriate to the way it’s interconnected to others.

So, what the ewdDOM module does is to represent these Nodes and pointers as Global Nodes in a Mumps database, and allows us to create and modify them via the DOM APIs that it implements.

When we ran the createDocument() API, it created a single top-level Node known as the Document_Node.  Of course, as we’ve seen, it also created other supporting data for such thing as document and node counters, and an index that allows us to search for documents by name.

Mumps Global Nodes working together

Hopefully it’s becoming clear that Mumps Global Nodes on their own don’t do a great deal for us.  It’s when they’re used in combination that they begin to allow us to represent a working system or model.

It should also be clear that it’s the application logic, in this case the logic within the ewdDOM module,  that is responsible for creating an appropriate combination of Global Nodes.  There’s nothing in the Mumps  environment itself that will do what we want for us.

This gives us an unlimited number of degrees of freedom for how we can model a particular system.   To some, this might seem to be a terrible chore, but to others (like me)  this is something very cool: it means that I can design and optimise a system exactly how I want it to work.  The Mumps database won’t get in the way and force compromises on my design.  Neither will it force me to model the system to work with the  constraints of the particular database model imposed/expected by the database (eg Relational, Document, Graph etc).  So a Mumps database is a raw, low-level database system over which I have complete control.

With this in mind, let’s take a look at exactly what happened behind the scenes when we ran ewdDOM‘s createDocument() API.

ewdDOM and createDocument() Dissected

The ewdDOM module first instantiates pointers to three key Global Nodes as follows:

 

  dom: function() {
    return new this.ewd.GlobalNode(this.global, ['dom']);
  },
  domIndex: function() {
    return new this.ewd.GlobalNode(this.global, ['docNameIndex']);
  },
  domCounter: function() {
    return new this.ewd.GlobalNode(this.global, ['documentCounter']);
  }

  ewdDOM.dom = ewdDOM.dom();
  ewdDOM.domIndex = ewdDOM.domIndex();
  ewdDOM.domCounter = ewdDOM.domCounter();

In terms of physical Mumps Global Nodes, these provide us with pointers to:

ewdDOM.dom: ^xmldom("dom")
ewdDOM.domIndex: ^xmldom("docNameIndex"
ewdDOM.domCounter: ^xmldom("documentCounter")

Now let’s look at the createDocument() function:

createDocument: function(documentName) {
    var docNo = ewdDOM.getNextDOMNo();

    var dom = ewdDOM.dom.$(docNo);
    dom.$('docName')._value = documentName;
    dom.$('creationDate')._value = new Date().getTime();
    ewdDOM.domIndex.$(documentName)._value = docNo; 
    var nodeNo = ewdDOM.incrementNodeNo(docNo);
    dom.$('node').$(nodeNo).$('nodeType')._value = 9

    return new Document(documentName);
  }

Let’s step through this. The first thing we need to do is to assign a document number to our new document. To do this we increment ewdDOM.domCounter – this is actually done by function named getNextDOMNo() which looks like this:

  getNextDOMNo: function() {
    return ewdDOM.incrementDocumentNo();
  },

  incrementDocumentNo: function() {
    return ewdDOM.domCounter._increment();
  }

So this creates:

 
{
  "documentCounter": 1
}

We can now create the docName and creationDate properties for our new document #1:

 
    var dom = ewdDOM.dom.$(docNo);
    dom.$('docName')._value = documentName;
    dom.$('creationDate')._value = new Date().getTime();

Next, we create the Document Name Index:

 
    ewdDOM.domIndex.$(documentName)._value = docNo;

Next we create the top-level Document_Node, which first requires us to increment the Node counter for our new document (which returns 1), after which we instantiate Node #1  with a nodeType property of 9 (which is the value for a Document_Node):

 

  incrementNodeNo: function(docId) {
    return ewdDOM.dom.$(docId).$('nodeCounter')._increment();

....

    var nodeNo = ewdDOM.incrementNodeNo(docNo);
    dom.$('node').$(nodeNo).$('nodeType')._value = 9

and finally we return a new instance of a Document object, representing the XML Document we’ve just created:

 
    return new Document(documentName);

We can now use any of the properties and methods available for our new XML Document.

At this point, we have a new XML document, but it doesn’t contain anything, so it’s not very interesting just yet.

In the next article, we’ll start to build out our XML document.  In doing so, we’ll explore ewdDOM’s Document properties and methods and discover how they have been modelled into a set of Mumps Global Nodes that work together in combination to create a persistent XML DOM.

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 43 other followers

%d bloggers like this: