The Use of SGML for a Police Information System		Table of contents		Indexes		An XML-Based Interchange Format for EXPRESS-Driven Data

XML Message Switching

Mike Kelly

Senior Consultant

Wang Global UK Coal Road, Seacroft
Leeds West Yorkshire United Kingdom LS14 2AP
Phone: +44 1132 734771
Fax: +44 1132 735223
Email: mike.kelly@wang.com Web: www.wang.com

Biographical notice:

Dr Mike Kelly originally trained as an Archaeologist. It was while he was struggling with paper tape and teletypes on a PDP-11 during his undergraduate training that he caught the computing bug. He went on to research computer applications in Archaeology developing graphical and statistical software to process remote sensing data. After completing his Ph.D. he dallied with the commercial software development world for several years before returning to academia to pursue research into medical IT. After a decade of running a development group, managing an IT infrastructure and providing consultancy Mike once again entered the commercial fold. Mike is now a Senior Consultant at Wang Global based in the UK. He is involved in all aspects of Internet technology design and application, with a special interest in n-tier architecture design and all the pain and glory this can bring.

ABSTRACT:

By representing all activity as XML messages, a common semantic model can be imposed across n-tier architectures. Each individual component need only concern itself with translation to and from the accepted semantic model. This model may be an established industry, application specific XML vocabulary or an ad-hoc project specific schema. This paper presents a framework based on XML message switching that attempts to provide easy build, maintain, and change of such n-tier architecture systems.

Introduction

Developing an n-tier architecture application involves lots of technology choices. First we must decide how we are going to build our components. Then we need to know how these components are going to co-operate. Current mainstream choices are COM, CORBA and Java. Once we have our technology infrastructure we need to develop a technology framework. This involves designing our component portfolio, setting out what types of components we will need in terms of functionality and how they will be assembled and, just as importantly, disassembled to produce applications.

A technology infrastructure uses a common syntax model defined for example by COM or CORBA. If we do not break any syntax rules we are guaranteed that we can plug and play components at the infrastructure level. However if we are reusing our components in different frameworks, and utilizing other people's components, then we rapidly find there is no common shared semantic model. This often leads to spending a lot of time customizing component interfaces to get semantic fit, and even rebuilding components from scratch.

To try and resolve these problems Wang Global UK have been investigating new technology solutions. One approach, which the author has been involved with, is XML Message Switching. XML provides a way of defining shared semantic models. Messaging provides a way of integrating distributed components. By combining the two concepts a powerful application level middleware solution is created.

Overview

A distributed architecture is represented as a set of components that have a common infrastructure. This could be for example COM or CORBA, or just a common transport protocol such as TCP/IP. Logical communication between components is termed activity and is modelled as messages. In practice a variety of technologies can be used to implement these messages including:

Real messaging systems such as Microsoft Message Queue Server (MSMQ) or IBM's MQSeries
Infrastructure calls, for example DCOM or CORBA
Transport protocols, for example TCP sessions or UDP datagrams

All activity is represented as XML messages. This allows a common set of semantic models to be imposed across the whole architecture. Each individual component only concerns itself with translation to and from one or more of the accepted semantic models. These models can be established industry, application specific XML vocabularies or ad-hoc project specific schema.

The XML messages pass through a set of common points that act as application switches called XML Message Switches (XMS). A component initiates communication by formatting XML messages, tagging them with the source and destination address and sending them to the local XMS. XMS will then switch the XML messages through to the destination components.

XMS Overview

Logical object based namespaces are provided. These allow all sources, destinations and activities to be declared as object classes and instances in an inheritance tree. The namespaces are declared and managed by an Enterprise Directory Service. Components can access this service to locate and address messages to the correct destination and to find the transport address of the XMS.

The XMS holds a switching table that determines the switching of XML messages between components. The switching table allows switching based on source, destination, activity or any combination of these. The switching is granular it that it allows rules based on name classes and/or instances and takes full advantage of inheritance relationships. Messages can be switched to multiple destinations. Sending the switch XML configuration messages can dynamically change the XMS's switching behaviour.

Switch Table

The XMS uses logical ports to send and receive XML messages. Lightweight pipeline applications called Portlets, can be logically chained onto XMS ports. These allow both post and pre processing of XML messages. Portlets can be used to translate propriety message formats to and from XML. This allows existing system components to talk natively to XMS. There is no need to visit these components to make them XML aware. Portlets can also be used to transform XML messages from one DTD to another. This allows communication to external systems that may hold a different semantic model. Propriety message formats can even be tunnelled through the XMS if required.

Message Structure

All messages are defined as XML documents. These must be well formed and should also be valid. The responsibility of ensuring the correctness of the XML lies with both the sending and receiving components. XMS will not attempt to parse messages it receives, as this would introduce considerable latency in the switching process.

XML messages will characteristically be small. For example a typical retail banking transaction message might look like:

<?xml version="1.0"?>

<Deposit Type="cash">

	<Account Type="90DayPlus">
		MAKELLY-123456
	</Account>

	<Amount Currency="sterling">
		500.30
	</Amount>

</Deposit>

An XML message will have two parts, the head and the body. The body contains the actual message. The head contains addressing information.

Addressing Messages

The message head contains:

SRC - source, the address of the component sending the message
DST - destination, the address of the component the message is being sent to
MSG - message, the type of the message being sent

These addresses are defined within a set of object based namespaces. Inheritance trees of object name classes are declared for SRC, DST and MSG. For example:

SRC://WangBank
SRC://WangBank/BankBranch
SRC://WangBank/BankBranch/Teller

DST://WangBank
DST://WangBank/BankHQ
DST://WangBank/BankHQ/CustomerAccounts

MSG://WangBank
MSG://WangBank/CustomerAccountTX
MSG://WangBank/CustomerAccountTX/Deposit

The name classes therefore declare the types of SRC, DST and MSG allowed. Instances of SRC and DST can also be declared. The syntax of an instance name is dependent on the class. For example:

SRC://WangBank/BankBranch?Leeds.CoalRoad

DST://WangBank/BankHQ?London.Putney

MSG name classes do not have instance names as the message body defines the actual message.

These namespaces are declared and managed by an Enterprise Directory Service, such as Microsoft's Active Directory. This allows all components and XMS access to a common set of addressing information. Although all namespaces are declared as inheritance trees of objects, the actual entities the names refer to do not need to be aware of these relationships. The Directory Service also holds the transport addresses of instance names that represent actual system components.

Message addressing can now be more precisely defined as:

SRC - source, the identity of the source class and instance that sent the message
DST - destination, the identity of the destination class and instance that is to receive the message
MSG - message, the identity of the message class

The choice of how to represent the message head has important implications for the design and functioning of XMS. The message head could be represented as XML within the message itself. XMS would then need to parse the message to determine SRC, DST and MSG. This would introduce significant latency in the switch. To remove this, message sources add a precompiled message head that contains SRC, DST and MSG. The message head is represented as an encoded byte string, defined in an XML Processing Instruction (PI) called switch.

<?xml version="1.0"?>
<?switch FE3423E3120012C4120900001245FFFF23 ?>

<Deposit Type="cash">

	<Account Type="90DayPlus">
		MAKELLY-123456
	</Account>

	<Amount Currency="sterling">
		500.30
	</Amount>

</Deposit>

The PI removes latency in the message switching. It is transparent to other XML applications. As all the information required for switching is held in the PI, XMS does not need to check the message to be valid or well formed XML. In fact the rest of the message need not be XML. This could allow proprietary message formats to be tunneled through XMS.

Sending Messages

A component first constructs an XML message. It will locate and contact the Directory Service to retrieve the correct SRC, DST and MSG addresses. A shared code component is used to encode these addresses into the message head. The message head is added as a PI to the XML message. The transport address and port of a XMS is found. This could be via the Directory Service or even a DNS service lookup. The message is sent to the XMS.

XMS relies on the underlying transport mechanism to ensure reliable delivery of information. XMS does not provide any sort of message delivery acknowledgement. If this is required, the destination components should return suitable XML acknowledgement messages.

Receiving Messages

A component can receive an XML message from an XMS. The component parses the XML and extracts the message head PI. If required a shared code component is then used to decode the message head into SRC, DST and MSG addresses. The Directory Service can then be located and contacted to check these addresses.

The component actions the contents of the XML message. This may result in the component sending its own XML messages.

Switch Behaviour

The core concept that XMS implements is intelligent switching. A component sends an XML message to another component via XMS. When XMS receives the message, it can identify who sent the message, who is supposed to receive the message and what type of message it contains. XMS can be configured to modify this behaviour, by switching through to alternate or multiple destinations. The sending and receiving components operate independently of this switching behaviour. This means switching patterns can be changed, even dynamically, without having to alter the functionality or communication interface of components.

Logically XMS receives an XML message. This maybe preprocessed. The SRC, DST and MSG addresses of the message are extracted. These are used to map the message onto none to many DSTs. For each DST the XML message maybe postprocessed. The XML messages are then sent to the appropriate DSTs, using the Directory Service to find the transport addresses.

XMS contains a number of definable PORTS. A port is either an InPort through which XML messages are received, or an OutPort through which XML messages are sent. The significance of PORTS is that they provide logical addresses within XMS where both pre and post processing of XML messages can be defined. In the current design of XMS, sending components must be aware of these port addresses so that messages are sent to the correct port.

XMS contains a SWITCH TABLE that determines which DSTs and OutPorts a message received on an InPort is sent to. There are three types of switching defined for an InPort:

Source switching - SRC is used for DSTs and OutPorts lookup
Destination switching - DST is used for DSTs and OutPorts lookup
Action switching - MSG is used for DSTs and OutPorts lookup

Any combination of these three types of switching can be used on an InPort.

A SWITCH ALGORITHM is used for lookup in the switch table. The switch table contains name class and instance patterns with associated DSTs and OutPorts. Given a target SRC, DST or MSG; none, one or many patterns may match. The order of pattern matching is defined as:

Class and instance
Class
Recursive parent class

Once a match has been found, all further searching is stopped.

The switch algorithm allows XMS to display intelligent switching behaviour. For example XMS might be configured to switch all XML messages sent by all bank branches through to their default destination:

<Switch>

	<LookFor>
		SRC://WangBank/BankBranch
	</LookFor>

	<SendTo OutPort="BankHQ">
		DST://DEFAULT
	</SendTo>

</Switch>

On a monthly basis a special audit is carried out on all activity associated with a selected branch. An audit component needs to see all that branch's activity. XMS can be configured to switch all XML messages sent by that particular branch through to both their default destinations and also the audit component.

<Switch>

	<LookFor>
		SRC://WangBank/BankBranch?Leeds.CoalRoad
	</LookFor>

	<SendTo OutPort="BankHQ">
		DST://DEFAULT
	</SendTo>

	<SendTo OutPort="BankHQ">
		DST://WangBank/BankHQ/SpecialAudit?London.Putney.SA2
	</SendTo>

</Switch>

The switching algorithm ensures that messages from the SRC class and instance pattern SRC://WangBank/BankBranch?Leeds.CoalRoad has a different switching behaviour then generic SRC class pattern SRC://WangBank/BankBranch.

Switch Management

XMS holds its switching table in memory to facilitate fast lookup. This can be persisted to permanent storage via the Directory Service or a Relational Database Management System. XMS is configured and queried by sending management messages. For example:

<Switch Action="Add">

	<LookFor>
		SRC://WangBank/BankBranch
	</LookFor>

	<SendTo OutPort="BankHQ">
		DST://DEFAULT
	</SendTo>

</Switch>

Portlets

Portlets are lightweight pipeline applications that process XML messages within XMS. This processing can be carried out between receiving and switching the message (pre-processing), or between switching and sending the message (post-processing). Portlets can be logically chained onto XMS ports.

Portlets can be used to alter the contents or format of messages. As this will usually involve parsing the message body, this can introduce significant delay. Therefore Portlets should be lightweight in their processing overhead. Portlets are installed as components within XMS. Several instances of the same Portlet can be used on different ports.

Portlets can be used to translate propriety message formats to and from XML. This allows existing components to send a non-XML based message to XMS. An appropriate Portlet can translate the received message into XML before passing it through to the switching table. Switched XML messages can also pass through a Portlet that translates them into non-XML format, before they are sent to their destination

Portlets can also be used to transform XML messages from one DTD to another. This allows message exchange to external systems that may hold different semantic models. XSL provides a generic mechanism to transform XML.

Scalability and Availability

As XMS provides a common point of switching for all activity, it will effectively halt all system activity if it should fail. Therefore XMS needs to be highly scalable and constantly available.

Multiple instances of XMS provide a scalable solution. Persistence of the namespaces and XMS configuration in the global available Directory Service ensures easy management of multiple distributed XMSs. By load balancing across several XMSs, message switching can be evenly shared. New XMSs can be simply added to cope with increasing demand. Load balancing also ensures high availability. If a particular XMS should fail, then message switching will simply continue to be balanced across the remaining active XMSs.

Scalability

Security

The current XMS design assumes a complete trust model. It is assumed all messages sent are authentic, i.e. that the stated SRC address in the message header is the actual source address of the component that sent the message. XMS could be made to validate the transport address of the component that sent the message against its recorded transport address in the Directory Service. However, transport addresses can be spoofed.

Existing security functionality in the underlying transport mechanisms can be utalised. For example if IP is being used as the transport protocol, Secure Sockets Layer (SSL) can be used to provide encryption and some level of authentication.

Transactional Integrity

The current XMS design does not provide any inherent transactional integrity. For example component A sends a message to component B via XMS as part of a transaction. XMS's configured switching behaviour actually switches this message to components B and C. For some reason B cannot commit on the processing resulting from receiving this message, so aborts the transaction. XMS has no awareness that a transaction is in progress. It cannot notify component A of the fact that component B has aborted. It cannot tell component C to rollback the message it has just been sent.

Transactional support can however be built on top of XMS, at the application level. In the previous scenario, XML messages representing BEGINTX, ENDTX, COMMIT, ABORT and ROLLBACK could be defined. An application level transactional protocol could then be implemented.

Example Application

As XMS can switch messages based on their source, destination and content, it can be used for example to renew legacy products with Plug and Play migration.

Problem: the Wang Bank needs to replace its existing legacy Savings Account product with a new system.

Solution: build the replacement Savings Account product system. XMS is configured to switch account messages to both the new product and the legacy product. The new product enters learning mode. It is monitoring all legacy product activity and using it to populate its own data stores. It may even prompt the legacy system by spoofing account messages to elicit specific account data from the legacy product. Migration becomes a side effect of natural account activity. Once customers have been migrated from the legacy product system to the new product system, XMS can be configured to selectively switch individual customer account activity to the new system. Account activity can be gradually quenched from the legacy product system.

Learning mode

Active mode

Alternative migration policies can be easily implemented. For example, only new accounts are to be hosted on the replacement product system, existing accounts will be migrated over time. XMS is configured to switch all Open Account messages to the replacement product system. This will then dynamically configure XMS to switch all subsequent account activity for this customer to only the replacement product system. Existing customer account activity continues to be switched through to the legacy product system.

Summary

XMS provides an application level middleware switching system. Distributed architectures are modeled as discrete components communicating via XML messages. These XML messages pass through XMS which can intelligently switch the messages to none, one or more destination components. This switching behaviour of XMS can be dynamically configured.

This hub and spoke delivery of messages essentially allows the separation of function and activity flow. Activity flow is now soft wired and can be changed without having to alter components. This opens up interesting ways to change and evolve large distributed component-based systems. The XMS model provides an easy way to build, maintain and change such n-tier architectural systems.

Acknowledgments

The author would like to thank fellow developers and consultants at Wang Global UK, whose experience and conversations have contributed to the development of the ideas in this paper.


The Use of SGML for a Police Information System		Table of contents		Indexes		An XML-Based Interchange Format for EXPRESS-Driven Data