XML and Topic Maps   Table of contents   Indexes   XMLTools: The tools, the categories, the leading players
 

XML Conformance

Boland, Tim
 Gaithersburg 
 Maryland 
NIST
 USA 
 		Tim  Boland
Computer Scientist,  NIST 
 Building 820, Room 562
Gaithersburg  (Maryland)  (USA) 20899 
Email: frederick.boland@nist.gov

Biographical notice

Mr. Boland is a Computer Scientist in the Software Diagnostics and Conformance Testing Division, Information Technology Laboratory, National Institute of Standards and Technology, U.S. Department of Commerce, Gaithersburg, MD. He holds a MS Degree in Computer Science from the University of Maryland, and has been active for over 10 years in the development of testing and measurement services for forward-looking information technology standards.

 OASIS 
XML Conformance
 

OASIS, The Organization for the Advancement of Structured Information Standards

 The Organization for the Advancement of Structured Information Standards:
 
  • international non-profit consortium of users and vendors
    •  
  • formerly named SGML Open, founded in 1993
    •  
  • dedicated to accelerating the further adoption, application and implementation of structured information standards
    •  
  • marketing, education and technical programmes supported by volunteer member
    •  

    OASIS Mission

      This nonprofit organization is an international consortium dedicated to accelerating the adoption of product-independent formats based on public standards. These standards include SGML, XML, HTML as well as others that are related to structured information processing.
     Members of OASIS are providers, users and specialists of the technologies that make these standards work in practice. OASIS provides an open forum where its members discuss market needs and directions, and recommend guidelines for product interoperability. The consortium creates, receives, coordinates, and disseminates information describing methodologies, technologies and implementations of the standards. The continued work of the consortium complements that of standards bodies, focusing on making these standards easy to adopt, and the products practical to use in real-world, open system applications.
     

    Technical Programme

     
  • chaired by Norbert Mikula (DataChannel)
    •  
  • formerly chaired by Bill Smith (Sun Microsystems), and Paul Grosso (ArborText)
    •  
  • committee work performed by member companies as a benefit of membership
    •  
  • subcommittees produce technical reports and recommendations that are published to the industry for open adoption
    •  
  • notable SGML accomplishments to date include catalog and table interoperability between vendors, resulting in easier application deployment by users
    •  

    XML Conformance Subcommittee

     The Conformance Subcommittee was chartered in August 1997 in the week before the first XML Developers' Day. As of October 1998:
     
  • chairman: G. Ken Holman, Crane Softwrights Ltd.
    •  
  • technical lead: Trevor Veary, Software AG
    •  
  • editing lead: Tim Boland, NIST
    •  
  • input from 15 member companies
    •  The Subcommittee gratefully acknowledged input from outside OASIS:
     
  • from NIST (National Institute of Standards and Technology) for having supplied the lead editor
    •  
  • from resources already developed that would be incorporated into the report
    •   The objective of the Subcommittee was:
     
  • identify and then create a useful freely available resource for the XML Community to help promote a better
    •  
  • understanding of XML Conformance, thereby promoting interoperability of XML-related technologies, tools and products
    •  

    What Does the XML Standard Say About Conformance?

     Section 2.8 - Prolog and Document Type Declaration: The version number "1.0" should be used to indicate conformance to this version of this specification; it is an error for a document to use the value "1.0" if it does not conform to this version of this specification.
     Section 5. - Conformance
     5.1 Validating and Non-Validating Processors
     5.2 Using XML Processors
     Note that the final recommendation never mentions the word "parser", so the successful act of parsing an XML file for fatal and non-fatal errors is only one aspect of conformance testing.
     

    Committee History

     The Subcommittee formed during OASIS 1997 Summer Workshops. A brief announcement was made two weeks later in Montréal at XML Developers' Day 1997 about existence of committee, and an invitation made for input from the community regarding requirements.
     To date, zero requirements have arrived from the XML community at large. Volunteers were unwilling to put in a lot of time until XML finalized. A meeting in Washington DC at SGML/XML'97 resulted in re-affirmation of the committee effort; NIST representative interested in conformance is introduced to the committee. Although XML still not finalized, members are thinking about requirements. A brief presentation was made in Seattle at XML Conference in March of 1998. An invitation was made for input from the community regarding requirements. Again zero requirements arrived from the XML community at large.
     Following the March conference, committee volunteers begin to apply some time to the issue of conformance. Concerns were expressed privately about maturity of necessary accompanying standards (e.g.: object model). Then at the GCA meeting in Paris at SGML/XML Europe'98 different opinions about what conformance means to different members of the committee became apparent. Interest began to grow and quite a busy meeting was held in San Diego at OASIS Summer Workshops 1998. Therepossible activities for the subcommittee were discussed, a direction is decided, and editing and technical leads volunteered. brief announcement made three weeks later in Montréal at XML Developers' Day 1998 about existence of the conformance efforts.
     

    What Does Conformance Mean?

      Committee members were divided regarding what conformance meant to each of them. Two predominant themes were identified:
     
  • confirm a processor returns information to an application about a putative XML instance as defined by the standard
    •  
  • confirm an XML instance produced by an application conforms to the productions of the standard
    •  

    Duplication of Effort

     Everyone agreed it was not interesting, nor useful to the community, to duplicate any effort already underway. The following was identified as relevant to the effort:
     
  • James Clark's test suite recognized to be a valuable asset already available to the community
    •  
  • Tim Bray's Annotated XML recognized to be a valuable asset of interpretations and explanatory notes
    •  

    Potential Offerings

     After some discussion, the following possible offerings from OASIS were then summarized:
     1) - provide an instance validation service
     2) - provide some bases for processor performance measurement (names to be refined if better suggestions are put forward for consideration):
     2A) - a set of binary tests
     2B) - a set of processor tests
     2C) - a set of property tests
     3) - provide an annotation to the specification regarding areas of interpretation related to conformance and the possibly ambiguous uses of the term "may"
     

    Validation Services

     The consortium may at some point be in a position to offer as yet undefined validation services:
     
  • reporting on the conformance of XML documents to the specification
    •  
  • reporting on the conformance of XML processors to the specification
    •   It was recognized this was a gray area where we may choose not to enter.
     

    Binary Tests

     These tests indicate if an XML processor correctly accepts or rejects XML syntax as defined by the recommendation:
     
  • correct documents are accepted by the processor without errors
    •  
  • incorrect documents are rejected by the processor with appropriate errors
    •  It was recognized a valuable resource already exists:
     
  • James Clark's suite of hundreds of test files ... but ...
    •  
  • there exists little or no documentation in which these tests are placed in context of the specification
    •  
  • nuances of some of the tests are difficult to recognize without supporting documentation
    •  
  • represents the opinions of one person (albeit it is James)
    •  

    Processor Tests

     These tests indicate if an XML processor correctly passes information to an application as defined by the recommendation:
     
  • example from 2.10 White Space Handling: An XML processor must always pass all characters in a
    •  
  • document that are not markup through to the application.
    •  

    Property Tests

     These tests indicate if an XML processor correctly passes all information described by an XML document to the application:
     
  • identify, or define if one does not exist, a property set or object model for all possible XML documents
    •  Conformance and left the meeting with a different set of expectations.
     

    Conformance Expectations

     A straw poll of members present revealed:
     
  • a majority preference for addressing 2A immediately (the binary tests)
    •  
  • a minority preference for addressing 2C soon (the property tests)
    •  
  • no preference for addressing 1 (the instance validation service) or 2B (the processor tests)
    •   Subsequent (lively!) discussion reached a consensus to do:
     
  • binary tests and related documentation
    •  
  • annotations related to conformance
    •  

    Progress

     Progress has been made in:
     
  • defining environment for representation of test report
    •  
  • creation of a test report document structure
    •  
  • generation of tests to be inserted into the test report document structure
    •  

    First Subcommittee Deliverable

     The consensus was reached in July 1998 to:
     
  • document categories of tests for the testable statements identified in the recommendation
    •  
  • enumerate and categorize test files according to the selected categories of tests
    •  
  • identify aspects of the recommendation for which a test does not yet exist and create test files to add to the suite
    •  
  • document the purpose of each test to help new constituents of our community
    •   The goal is to publish as the first deliverable:
     
  • a report discussing conformance and the use and interpretation of each of the test files
    •  
  • a suite of associated test files
    •   To our collective knowledge
     
  • there is no similar initiative in progress within any OASIS member organization
    •  
  • we believe there are no similar initiatives under way anywhere
    •  

    How Can You Help and Contribute?

     
  • ask yourself what XML Conformance means to you and what is important to you
    •  
  • find an OASIS member organization you know (or want to know):
    •  
  • talk to their technical committee representative
    •  
  • tell them what is important to you for XML conformance
    •  
  • talk to your XML software vendor:
    •  
  • encourage their membership in OASIS
    •  
  • tell them what is important to you for XML conformance
    •  
  • ask them what is important to them for XML conformance
    •  
  • find Norbert Mikula, Trevor Veary, Tim Boland, Ken Holman or any other member of the subcommittee walking around XML conferences:
    •  
  • tell us what is important to you for XML conformance
    •  
  • (if necessary) contact Ken Holman directly: gkholman@CraneSoftwrights.com
    • XML and Topic Maps   Table of contents   Indexes   XMLTools: The tools, the categories, the leading players