XML Programming Models for Electronic Commerce Systems   Table of contents   Indexes   Smarting up your legacy data
Kurt, Christopher
 Redmond 
The Open Applications Group
 USA 
 		Christopher Kurt
 XML Team Leader
The Open Applications Group
 16541 Redmond Way PMB 290 Redmond (Washington)  USA (98052) Web site:"http://www.openapplications.org"
 Biography
 Christopher Kurt is the XML project team leader for the Open Applications Group, and is formerly a Director of Supply Chain Systems Integration for PricewaterhouseCoopers. As the XML project team leader for the Open Applications Group, he is primarily responsible for the design and development of the XML resources, both DTDs and schemas, supporting the specification.
 

Session Capsule

 With the sudden advent of XML and the relative lack of supporting business information, except at a technical level, many organizations have found their business plans thrown into doubt. Both business and technical management now have serious questions to ask about where XML fits into an overall application integration architecture, how XML can be used to integrate business applications, what advantages in reliability, implementation speed, and cost savings XML provides, and the challenges and issues that need to be addressed when using XML for application integration. Non-technical, business-oriented answers to these and other questions are answered in this session.
 

Introduction

 XML is an important technology, and it is having a significant impact on the way we approach all types of information management challenges. Among other applications, XML can be leveraged for Supply Chain integration, building e-commerce solutions, and sharing business information between systems and organizations.
 XML can be leveraged as a key component of an Enterprise Application Integration (EAI) solution. This session is decidedly non-technical, focusing more on the pragmatic realities around incorporating the technology into a broader solution, and the issues that must be directly addressed as we establish an infrastructure to support EAI, and incorporating XML.
 Before we dig into the details of how we might use XML for application to application integration, we'll take a look at some traditional approaches to integration, and some of the reasons for today's integration challenges. Next, we will look at the current generation of solutions, and some of the issues that these products address. Finally, we will discuss where XML technologies fit into the overall integration architecture, as well as one of the industry efforts to built open content standards around the technology.
 While XML provides some exciting opportunities, there is still more work to do. Near the end of our time, we'll review each of the surrounding design considerations, the remaining challenges related to the technology, and the longer-term opportunities that an integration architecture based on XML provides.
 

Integration Architecture

 Before we start, let's clarify what we mean by EAI. It seems that everyone has his or her own definitions. For the purposes of this presentation, we'll borrow from Rick Adam from Neon. Enterprise Application Integration may be summarized as:
 
  •  Integrating existing applications
  •  Installing application packages
  •  Loading data marts
  •  Integrating acquired companies
  •  Electronic Data Interchange (EDI)
  •  Internet electronic commerce
  •  Decreased, or zero information latency
 If these are the major components of enterprise integration, historically, we have done a poor job of meeting the integration needs of our businesses.
 

Point-Point Integration

 IT organizations have been under incredible pressure to deliver increasingly complex solutions, under tighter and tighter schedules. As a result, we have been primarily focused on the problem at hand. Our solutions have become increasingly tactical, with little investment in more strategic efforts.
 
  •  Our solutions have become focused on today's information needs and requirements.
  •  As a result, are designs are tactical, and do not address longer term infrastructure needs.
  •  We typically approach integration in terms of the pairs of applications we need to build interfaces for. Our interfaces are described as 'Product X to Product Y', or even 'a module of Product X to a module of Product Y'. This is certainly a short-term mindset.
  •  Additionally, we have implemented multiple technologies and solution designs. Among other designs, these include:
     
    •  File transfers
    •  Database updates or replication
    •  Low-level application programming
    •  Message queuing
    •  Request brokers
    •  Screen scraping
    •  and transaction captures
 Worse yet, we typically implement many of these approaches within the same architecture.
 To make matters more complicated, organizations may have implemented many of these different approached within the same environment. The end result is painfully clear.
 While this diagram shows what may result when no consistent integration strategy is present, it actually represents a significant step forward for some organizations. Simply having a diagram showing the integration points within the application portfolio is an improvement. Many large organizations couldn't even begin to document how information is passed between each of their systems.
 

Portfolio Evolution

 Integration challenges have not developed overnight. They are a natural result of a fairly long evolutionary process. If we take a look at how our application portfolios have developed, it's not surprising that we have created some challenges.
 Many large organizations started with a single, monolithic system to support their businesses. Many of these were custom developed. As business needs evolved, additional system were developed and pasted into the environment. Integration strategies were not a primary consideration, so any number of approaches were taken to make information move between the systems.
 During the 90's, there was a strong move toward package ERP systems. Some of this movement was fueled by year 2000 issues, or a desire to migrate away from custom solutions. While these packages were implemented, significant pieces of the business remained within the legacy environment. Now, in addition to the links between our custom applications, we needed to integrate the package applications as well.
 More recently, the limitations of ERP implementations have become understood, and organizations have started to incorporate best of breed solutions specifically aligned with their requirements, and more advanced business solutions.
 Further complicating matters, business and industry requirements have driven additional maintenance and development requirements into the solution portfolio.
 
  •  Vendor upgrades and expanding sets of features prevent the portfolio from stabilizing.
  •  Mergers and acquisitions require us to continue to incorporate and remove major applications from our portfolios.
  •  Businesses are becoming more decentralized, with disparate systems deployed throughout the organization.
 One of the results of this evolution is a continuing onslaught of integration requirements.
 Information Technology organizations have an incredible workload. The volume of changes required by the business, and the pace of change in available technologies forces many IT organizations to focus on near-term deliverables.
 This short-term focus extends into our integration efforts. Without a long-term strategy, we make matters even more difficult for ourselves.
 
  •  For each new integration requirement, we treat it as a new project. "One new requirement, one new interface.
  •  Our disparate systems are often treated as islands - with complete staffs dedicated to specific platforms or products. The limits reuse and consistency across the portfolio.
 

Spaghetti Integration

 When we look at the results, we see that we have links going every direction within our application portfolio. This diagram is far simpler than what we experience in the real world.
 So what happens when business requirements force us to replace or significantly change a major system? Integration points throughout the portfolio are broken.
 When I look at this, I think of the bricks of firecrackers I would play with when I was growing up. The fuse would be lit at just one point, but the ties between all of the firecrackers were interwoven. One small flame resulted in hundreds of explosions going off at the same time.
 Each of us has experienced the results. We burn significant time and money for every change we make. We can't avoid it…
 
  •  "…enterprises today and tomorrow will require a far higher degree of integration… to shift gears more rapidly, to change competitive formulas, and to reorganize more quickly than ever before." -Forbes
  •  The magnitude of investment is significant:
  •  "Many companies run a tangle of incompatible software systems, which cost industry some $100 Billion a year to create, update, and maintain." -Business Week Nov. 3, 1997
  •  $40 Billion a year is spent on application integration - IDC, 1997
  •  40% of corporate IT budgets are spent on integration - Forrester, 1997
  •  20% to 80% of application implementation costs are related to integration - Forrester, 1997
 

EAI Backbones

 Developing an overall EAI strategy acts to decrease these costs. However, developing this overall strategy requires us to take a step back from the requirements of today, and look into the future.
 Our design must take into consideration future requirements and must be as flexible as possible.
 
  •  It must be based on organizational and industry standards, and not on proprietary requirements or specific to individual applications.
  •  It must be both technology and vendor independent.
  •  Changes are inevitable; the approach should provide as much flexibility as possible.
 While we are investing significant time and effort into our current projects, we must acknowledge a harsh reality.
 "The products we are implementing today, with all the corresponding effort and expense, will be replaced over time. Plan today to throw them away. The requirement to do this will come from compelling business requirements, not the IT organization. The organizations who adopt this mindset will become significantly more agile than those who believe that their application portfolio will stabilize, or standardize around a single set of products."
 So, what might an open integration framework look like?
 Instead of integrating two applications, we integrate to a common backbone. As you can see by this diagram, the solution is much simpler than the spaghetti we saw just a few moments ago. When we deploy a new solution, we integrate with the infrastructure, and not the other application.
 As a result, changes to the application portfolio are isolated. They do not cascade throughout the rest of the infrastructure. Rework and additional development is limited.
 

Impact on Cost and Maintainability

 When we take this type of framework-based approach, we begin to realize cost and work effort reductions almost immediately.
 The number of integration points in the portfolio is greatly reduced. The work effort required for each is also decreased.
 The graph on this slide depicts the overall effect on both costs and effort. As we introduce more and more changes to the portfolio, or increase the number of applications we support, the associated costs go up dramatically. The framework-based approach limits the expansion of these costs to something much more manageable.
 

Architecture Components

 So, let's take a look at the components of the framework, and finally talk about where XML fits into the overall solution. There are three major components to the solution - the applications, their agents, and the supporting middleware infrastructure.
 Of course, the applications provide the business functionality. If we're lucky, they will have a fairly structured set of interface calls that we can take advantage of.
 The middleware infrastructure, without being specific about technology choices, provides delivery, directory, workflow, security, and management services. In short, it ensures that the business information reliably moves throughout the environment.
 The agents are bridges between the applications and the middleware. They take care of mapping, formatting, and error handling. The mapping function translates between the application requirements and the information requirements of our standard messages. The composition function actually builds the message, or parses it into pieces the receiving application can understand. Once it is sent, the error handlers take care of anything that may have gone wrong.
 Now, regardless of what the sales and marketing reps may say, there is not one single production available today that will provide the entire solution. This architecture requires a number of focused solutions, from middleware to development or management tools. In building this framework, each component of the solution should be selected for its ability to position your organizations for future changes and requirements.
 

Where XML Fits

 Where does XML fit in this environment? It provides the structure for each set of information that is sent between applications. Within the application agents, we can incorporate composition and parsing tools to support message formatting. This is all transparent to both the middleware and the applications.
 Many of the EAI solutions in the marketplace are beginning to provide each of these functions. Additionally, we will need to provide the supporting middleware infrastructure, a set of development tools to support our custom solutions, and an overall systems management framework. XML is but one important component of the overall architecture.
 

EAI Today

 Before we look specifically at XML, we'll review a few of the other requirements of the architecture, and where application integration frameworks may be applied.
 

Current Requirements

 Of course, integrating application is the primary objective. However, there is a host of other considerations that must be addressed as a part of the initial solution design. Many of these are unrelated to XML.
 We need performance and reliability. Will the information be received quickly? Can we rely on the infrastructure not to lose any information?
 The integrity of the information must be maintained. It can not be changed in transit. In some cases, we need to be able to incorporate additional security and/or encryption facilities.
 The solution needs to be extendible. Other systems should be able to be incorporated without significant impact.
 We will rely on the infrastructure for moving information throughout our environment. A robust framework for monitoring and managing the environment is absolutely essential. Implementing an integration backbone without a supporting systems management infrastructure would be a huge mistake - putting the business at significant risk.
 Should something go wrong, either in transmitting information, or in processing it when it is received, we need to have facilities to identify and address the problem.
 While we are addressing each of these issues, we need to maintain platform independence as much as possible. Large organizations will implement solutions based on all platforms available at a given time. Our solution should work across operating systems, database engines, network protocols, and development tools.
 Lastly, we need to be as flexible as possible, while limiting the effects of changes on other systems. Just as our systems portfolio is guaranteed to change over time, so is the information, and associated rules, we will be moving between them.
 In order to meet all of these requirements, a foundation of standards in message content, technologies, and management is essential. We'll dig into the details of this in just a few minutes.
 

Types of Applications

 While most people consider EAI solutions within the context of package applications, there is significant opportunity to extend them throughout the rest of your environments.
 Of course, an integration framework can be used to integration both ERP and best of breed packages. Since these applications typically include integration facilities, they are the easiest (relatively speaking) to integrate.
 Organizations stand to benefit the most from applying the solution to their custom applications. While these often represent a much bigger integration challenge, the rewards are significant as well.
 An interesting byproduct of the architecture is related to reporting and decision support systems. As a larger amount of our business information is transmitted through the infrastructure, in a standard format, we gain the ability to mine this information. Translation need not happen as information is added to data warehouses or reporting systems. We have already done this as we move information between our business systems.
 Finally, we can extend this infrastructure throughout our supply chains; allowing close business partners to leverage the architecture as a part of information sharing strategies.
 Generally, integration for any of these types of applications may be approached from a number of different levels. The traditional middleware vendors focused on the EAI space may concentrate on lower level data movement, or transaction and business event level integration. Many of the newer EAI solutions build upon this foundation and add process-level or inter-business collaboration solutions.
 XML enhances the capabilities of any of these approaches.
 

Business Information Structure

 One of the most valuable characteristics of XML for application integration is its ability to retain context or relationships when transmitting business information.
 Without XML, we are often left with flat files or structures for sharing business information. This forces us to 'flatten' and denormalize our data. The message files are not normalized, and often contain redundant information.
 This directly conflicts with the structure of the information we use to run our businesses. It is decidedly hierarchical. Take a purchase order for example. Each order contains a set of header information, and one or more lines. Each line, possibly items on the order, could contain a set of sublines. These sublines may contain detailed charge or delivery schedules.
 These relationships are clearly important to maintaining the integrity of the information. Through XML, they can be clearly communicated without introducing unnecessary redundancy.
 

XML Solution Components

 How do we build an integration solution based on XML? In the next few minutes we'll review the steps we need to take to build the framework, as well as additional design considerations.
 

Business Object Modeling

 The first step is to standardize, as much as possible, the information we transmit between our business systems. An inventory of our existing interfaces can provide a jumpstart in this process.
 To start, we can break the types of information we need to model into broad categories. These may include Master Information, Reference Resources, 3rd Party or Supply Chain transactions, Planning, and Execution. For each category, we may establish different rules for how information is transmitted, error handling, and service quality.
 These standards provide the foundation for development and reuse. There are a growing number of resources available to use as a starting point for the design. When your project is started, refer to the broad set of Internet sites dedicated to XML solutions. Leveraging industry standards improves the accuracy of the solution, and reduces the amount of effort required to complete the design activities.
 Developing standard information models is a challenge, and a potentially painful process, especially if multiple areas of your organization are involved. However, it is a key to decoupling the applications in your portfolio.
 When the process is complete, we remove pairwise integration requirements, and can develop a more open framework. In short, we no longer view an integration requirement as an interface between Application A to Application B. Instead, Application A is integrated with the backbone framework, as is Application B. When one of the applications is changed or replaced, we need not do any rework for the other, as long as our standards are maintained.
 

Flexibility and Extensibility

 As we start to develop our standard content models, we should bear two things in mind - flexibility and extensibility.
 We will be required to change the set of objects, or their content, over time. In order for our integration framework to be successful in the long term, it should be structured to readily adapt to these changes.
 Both business and technical drivers should be considered.
 
  •  Application portfolios will change - our design should be independent of each.
  •  Our organizational structure may change
  •  We will want to integrate more closely with our business partners. Appropriate auditing and security information should be included.
     From a technical perspective, our solution design should allow for:
  •  changing or evolving standards and capabilities
  •  managing performance and the overall infrastructure
  •  an ever-widening set of supporting tools
 While it is important to incorporate these considerations and capabilities into our design, it is just as important to remain independent of any underlying products or tools. Each will change over time; the impact of these changes should be isolated from the rest of the solution.
 Additionally, we should look to leverage the work of other organizations and standards bodies, especially those working on solutions based on XML technology. We'll look briefly at one of these consortiums in just a few minutes.
 One of the distinct advantages XML provides in this process is the set of 3rd party tools that are available. From freeware to commercial products, these tools further speed the design and development process. They are a huge improvement over building our own parsers and utilities.
 

Composition

 When the first cut at standard content models is complete, we can drill a bit deeper into additional technical requirements that need to be incorporated into the solution. The way we address each of these considerations may differ for each implementation, based on business process or system requirements.
 
  •  How do we trigger information to move between systems?
  •  How do we handle more than one source of information, or multiple masters?
  •  What controls or audit information do we need? How is this information logged and monitored?
  •  How is the integrity of the message maintained. For XML, where is validation performed, and what overhead is acceptable in the process? How do we handle validation errors?
  •  How are cross-platform issues handled?
  •  Is transaction management a requirement, and how is it supported?
  •  Routing and security information can be handled at a number of different levels. Which is best for your environment?
  •  And of course, how is the bridge between your applications and the middleware environment going to be developed?
 As we develop or acquire our integration agents, we will be required to understand and address each of these issues. While putting the business information into an XML message may be straightforward, support for these other facilities must be included as a part of each message we send.
 

Parsing and Mapping

 When receiving a message, the process is reversed. Again, parsing the business information from an XML message may be straightforward as well, but there are a number of additional issues that are raised.
 These include:
 
  •  Target system mapping
  •  Application facilities for handling the information - transaction triggers, published APIs, or direct database updates.
  •  How do we log activities and handle processing or data integrity errors?
  •  How do we support confirmations at a business processing level?
 There is a significant difference between 'yes I have received the message', and 'yes I have received the message, understood it, and processed it successfully'.
 

Implementation Notes

 As we have worked through a number of integration projects, we have learned some valuable lessons, and where some key technology and design decisions need to be made. Some of the lessons are related the overall design realities, and others specific to the state of XML today.
 

Solution Design

 The first lesson learned from XML-based EAI implementations goes directly against my personal desire to be a minimalist and make every message as small as possible.
 XML is verbose, and standard information model exacerbate this be including information that some systems do not require, or can not process.
 Fortunately, XML compresses well, and the benefits far outweigh the costs of carrying some information along.
 To address this, compression technologies should be considered as a part of the solution. The parsing tools we select should be optimized for us to find the information we need as efficiently as possible.
 A positive lesson learned from establishing XML based content standards is the ability to initiate parallel efforts early in the implementation.
 Dependencies between the projects are limited. Development can be performed independently (to a great extent) of the other systems. Of course, each project needs to remain synchronized with the others, and communication between the teams is still essential.
 

Supporting Tools

 Selecting XML tools is an important decision point that needs to be made early in the design process.
 Parsers, of course, are one of the keys. First, we need to decide whether to build our own, or use a 3rd party tool. If we use choose to use tools and utilities from 3rd parties, we also need to determine whether we are comfortable with those that are freely available, or whether commercial products are required.
 Each implementation may drive a different set of decisions, depending on the project requirements.
 Also, a number of the components of in the XML specification are not required to be enforced by validating parsers. These loopholes are either an asset or a liability, depending on the project.
 Specifications related to XML continue to develop. While new releases of supporting tools become available soon following, they will be first releases, and not established for production use.
 In terms of development and testing tools, they simply aren't any of suitable quality readily available. Simple hacks can be downloaded from the Internet, but aren't all that useful, except for looking at other people's work. Commercial SGML tools could be used for some projects, but they tend to be extremely complex and expensive.
 The most useful tools still seem to be command line parsers and text editors. Hopefully, this will change as XML solutions become more established in the mainstream marketplace.
 

Enabling Applications

 XML solutions go far beyond these low-level parsing and development tools, and into the enabling applications. Many of the EAI solution vendors are releasing products that are XML aware.
 These vendors seem to have developed from two different perspectives. One constituency has evolved from traditional integration and tool solutions, some even from EDI. Others have essentially been created from the introduction of XML.
 This space is getting increasingly crowded some with few differentiators or experience. Many of the low-level tools we are using today will migrate into these more sophisticated solutions over time.
 While these products are somewhat focused on the integration, messaging, and toolkit components of the solution, many are missing key management capabilities. The more established middleware vendors have incorporated management and monitoring, but newer organizations are still building the core of their product. As the integration framework becomes a critical component of the infrastructure, traditional systems management disciplines still apply.
 While we choose to leverage these tools as they become available, we need to be cautious as well. Over time, many of today's solution providers will either be acquired, merged, or just 'go away'. Today's partners may simply not be around tomorrow.
 

Performance and Infrastructure

 In terms of performance and integration, the results are certainly mixed. As we select technologies to incorporate into the solution, each may have an impact on the overall results. The specific combination of four selections may drastically impact performance.
 These include:
 
  •  Hardware and Operating Systems
  •  JVM and/or other compilers or interpreters
  •  XML tools and libraries, such as parsers and validation agents
  •  Overall middleware and processware tools
     Similar to the migration from host-based to client/server designs, this type of solution has more moving parts than have previously been a part of our infrastructure. Each one can become a bottleneck, and troubleshooting is still a bit of a black art.
  •  Testing tools are limited or complex
  •  Diagnostic and troubleshooting tools are also difficult to find
  •  Handoffs often cross architecture and/or platform borders
  •  Integration design can impact overall application performance
  •  XML is verbose - compression may make the messages smaller, but introduces a bit more overhead
 The only way to identify each of the challenges is to test, test, and test again. As time runs short on project schedules, we are often tempted to decrease the amount of testing that is performed. Performance and volume testing is essential before migrating the solution to production.
 

The Open Applications Group

 The Open Applications Group is an industry consortium that has invested a significant amount of effort in adopting XML. It has become a cornerstone of the integration specification, and may represent the most comprehensive set of XML business resources that are available today. Here, we'll take a brief look at the design considerations and potential future directions.
 

Integration Specifications

 The Open Applications Group is comprised of application software vendors, EAI solution providers, systems integrators, and customers. The focus of the group is simple. It is focused on providing specifications to enable interoperability between business applications.
 The result is an integration specification defining:
 
  •  Interactions between business application components
  •  Transaction definitions for each interaction
  •  Content models for each transaction codified in XML DTDs
 The specification covers interactions in the domains of traditional Enterprise Resource Planning, Supply Chain Integration, and Manufacturing Execution. Additional functional areas are currently under development within the consortium as well. In total, the specification represents the largest repository of XML business resources generally available today.
 

XML Solution

 The Open Applications Group XML solution was developed to meet a number of specific design requirements.
 
  •  The contents of the specification are to be formally defined. Ambiguity is removed
  •  The reference material should be suitable for both business analysts and application developers
  •  Available standards and tools should be leveraged wherever possible
  •  The solution must be platform and architecture neutral
     No specific ties may be established to operating system, database, integration approach, or any other religions
  •  A framework for ongoing enhancements must be established as a part of the solution
 

Design Considerations and Assumptions

 As a part of the XML solution design, we had to establish a set of constraints, and make a number of assumptions. Most of these have proven out over the past few months.
 The baseline considerations had a significant impact on the overall design:
 
  •  As much specification information as possible should be included in the DTDs
  •  Extensibility and other implementation requirements, as outlined above, should be readily supported
  •  Only formally released standards or recommendations will be leveraged. Notes and other in progress initiatives will be monitored, but not allowed to cause the design to thrash around as progress is made within the W3C
  •  Deviation from the released standards should be avoided in order to leverage 3rd party tools as much as possible. Should some deviations be made, they should be well understood, with a clear migration path to the core standards.
  •  The design is also based on a set of assumptions:
  •  Application vendors and customer organizations can (and will) leverage 3rd party parsing and mapping tools in their solutions
  •  3rd party solutions will be robust and reliable enough for production use
  •  Performance of available libraries (Java and C) will be acceptable
  •  Validation will be used sparingly in production
 

Solution Components

 The Open Applications Group XML solution is comprised of a large set of DTDs and XML sample files. These include:
 
  •  Reference DTDs for resources that are shared across the entire specification.
     
    •  Low-level fields and data domains
    •  Common information structures - date/time, financial amounts
  •  A DTD for each transaction in the specification
  •  Sample XML files for testing and showing the general structure
  •  Most importantly, the XML DTDs are provided within the context of business events in the integration specification
 

Future Solution Directions

 While a baseline of XML resources have been established, there is still quite a bit of work left to do. Some of the potential enhancements on the horizon include:
 
  •  Stronger data typing
  •  Incorporation of additional W3C specifications and initiatives
     
    •  Namespaces
    •  Schema projects, including XML-Data
  •  Adoption of additional design and implementation standards as they are established
  •  Performance testing and related enhancements
 

Application to Implementations

 These types of architectures are often associated with package software products. While they certainly apply here, there are other applications for the architecture as well. The integration framework supports:
 
  •  Both ERP and best of breed approaches
  •  Legacy and custom applications
  •  Workflow and process-based integration
  •  Electronic commerce and value chain integration
  •  End-user reporting
  •  Decision support
  •  Information modeling
 As we design and implement our infrastructure, we should consider each of these solution types and incorporate them into our design.
 

Challenges and Opportunities

 Now, these implementations have exposed a number of challenges and opportunities related to using XML for application integration.
 

Multiple XML Initiatives and Projects

 In some respects, XML has been a victim of its own success. Throughout the industry, initiatives around XML content and standards have been created. Many of these have overlapping (or unclear) scope. These initiatives include:
 
  •  Vertical or business specific models
  •  Traditional standards groups
  •  Industry, application vendor, and EAI vendor efforts
     Even with the W3C, multiple projects are underway.
  •  We have the core XML 1.0 recommendation as a foundation
  •  Namespaces
  •  Data, link, etc
  •  Numerous other domain or industry-specific topics
  •  ...as well as some great debates around Schema and Style Sheets
 As these initiatives progress, and additional capabilities evolve, our current implementations will undergo fundamental changes. While we stand to reap some significant rewards by developing solutions based on today's XML, there is a bit of risk involved.
 

Lack of Industry Experience

 An additional challenge is simply the fact that a broad base of experience has not been established yet. We continue to search for more and more information on what approaches work, and opportunities to share experiences. A number of major topics come immediately to mind:
 First, the long-term performance implications of our designs are not well understood. Design and implementation decisions may have a significant impact.
 
  •  What development languages are to be used - Java, C, or others?
  •  Do we develop parsers specifically for our needs, or leverage those that are available from others?
  •  What structures within DTDs or schemas should be avoided to make processing as efficient as possible?
  •  JVM performance varies widely across vendors. Which are the best for specific platforms or types of applications?
 The next area where we need more experience is in internationalization. Traditional integration approaches have typically been homogeneous. XML allows us the additional flexibility to support multiple languages, code pages, and character sets. What are the best practices for this?
 As I mentioned before, there are a number of standards efforts in overlapping areas. Time and experience will determine which work better than others. Until then, the foundations we select for our implementation designs may be chosen arbitrarily, or based on a well timed magazine article or press release, instead of more focused selection criteria.
 Media hype has further obscured the situation be leaving the impression that XML will magically solve all the information sharing needs for the world. I have received phone calls from clients wondering where they can get XML, so they don't have to do any more development.
 Misinformation and groundless hype around the technology may increase awareness, but does a huge disservice to the very organizations that stand to benefit from implementing the technology. Real world experience, supported by education throughout the industry, as well as honest conversations about the challenges that still remain is sorely lacking in many areas.
 

Tool Availability

 The next area where challenges remain is in the area of tool availability. While more and more tools are released every week, the vast majority is still primitive. Of course, we need to walk before we run. Many of the 'freeware' tools will soon evolve into commercial products. Even the commercially released solutions are still at version 1.0.
 Unfortunately, we need them today. Implementation projects are making due without them, and accepting unnecessary risks as a result.
 Freeware or shareware tools are still the most prevalent. Developers are using these for commercial applications without the typical support or service level structures that would be a part of a commercial product.
 Where do you go when you need support? Can production solutions for large organizations be based on tools or libraries that are not formally backed by a support organization? Unsupported products should be avoided for mission-critical solutions. Unfortunately, companies are pressing forward anyway, usually violating whatever usage agreements came with the tools.
 Of course, the pressures that are being placed on the application and tool vendors are tremendous. The pace of evolution and feature development within the XML space far outpaces a commercial product vendor's ability to react within a release cycle. How can formal development priorities be set when the standards and customer requirements are evolving faster than code can be cut and tested?
 

Lack of Design Standards and Best Practices

 The final set of challenges is in the area of design standards and best practices. How do we start a project? Where do we start, what resources should we leverage, and what are the design tradeoffs?
 Fortunately, this need is starting to be addressed by some visionary organizations. Just over the past few weeks, a number of repositories and clearinghouses have been announced on the web to help focus design and general information sharing. Open forums for exchanging information and experiences will certainly help this space to mature more quickly. We should applaud and support these efforts by making sure each of our successes and failures are available for others to learn from.
 

Advanced Solutions - The Next Steps

 While the list of challenges is daunting, it is nothing compared to where we would be without XML. Each can (and will) be addressed over time. They certainly should not preclude us from using XML technologies as a backbone for all of our application to application integration needs. Once in place, we can take advantage of some more interesting solutions. Next, we'll review some of the other opportunities that an XML framework provides.
 

Leveraging the Framework

 When the standards-based XML framework is in place, different types of solutions can be developed on top of it. For some of these to work well, the initial design assumptions of loosely coupled applications and standard content models need to be well established.
 These solutions include
 
  •  Information Subscriptions
  •  Self-service forms
  •  Closer integration with business partners of all types
  •  Reporting and decision support
 Each of these could provide significant value to your organizations beyond the core of application integration.
 

Information Subscriptions

 Deploying a publish-subscribe architecture changes how we approach the overall challenge to information sharing. While it adds complexity when we begin to consider confirmations and overall control, a small change in mindset expands the solution beyond just business applications.
 Our subscribers could be decision support applications, data warehouses, or even groupware or end-user applications. As we expand the types of solutions that we incorporate into the framework, we transition to a model of 'vertical integration'.
 

Self Service Forms

 A combination of XML and web browsers sets the stage for self-service forms. In addition to business applications generating messages, simple forms can be developed to allow end users or some business partners to generate transactions.
 Some of the applications of this may be:
 
  •  Infrequent use of general business systems
  •  Personal information updates in HR systems
  •  Event notification -
     
    •  Orders or Deliveries
    •  Trouble Tickets
    •  Catalog or Pricing Changes
  •  Editing in-progress transaction information to handle errors, etc.
 

Value Chain Integration & E-Commerce

 Solutions similar to those deployed for self-service forms can easily be extended to business partners or customers.
 These may include
 
  •  Catalog information
  •  Order or Shipment status
  •  Invoicing and Reporting
  •  Other Inquiry and Notification facilities
     Standard XML tools may be used to update, review or edit information shared with partners, including
  •  Synchronization of master files
     
    •  Products, contracts, pricing, etc.
  •  Integration with EDI or other related transaction systems
 

Reporting and Decision Support

 A final extension to our XML infrastructure is in the areas of reporting or decision support systems. Our integration framework processes each transaction moving through our business in a standards-based format. With no additional translation or post-processing, we can mine this as a data source unto itself.
 The information is in a standard format, regardless of the source or destination systems.
 Each of the key business transactions will be passed through the framework.
 Reporting and decision support needs are easily handled. These include:
 
  •  Information granularity
  •  Time sequencing
  •  Aggregation and translation across systems
  •  Consolidation across locations or business units
 

Conclusion

 

A Silver Bullet?

 Is XML a 'Silver Bullet' for application integration? The results are mixed at best.
 We can realize significant benefits through establishing an integration framework based on XML:
 
  •  Information sharing across systems is facilitated through standard business information models - with decreased latency of each update.
  •  Both support and implementation costs can be drastically reduced.
  •  Tools to support all parts of the solution are becoming widely available, along with the skills to support them.
     However, more work is still required
  •  Application support for XML is only now emerging. More work is required to develop robust feature sets and required reliability.
  •  More tools, and fewer overlapping initiatives are still needed.
  •  Additional experience is required to establish best practices and design standards.
  •  XML is one part of a broader solution. The rest of the infrastructure is essential.
  •  XML is simply an enabler.
  •  Organizational change, information standards, and the overall architecture are still required.
XML Programming Models for Electronic Commerce Systems   Table of contents   Indexes   Smarting up your legacy data