Towards an SGML Diff Using Temporal Documents   Table of contents   Indexes   Authoring and Translation for the International Market
  Moffatt  Godfrey 
 

Introducing SGML into the RAF Flight Manuals World or Throttle to Bottle in Two Extraordinary Years

 

INTRODUCTION

 

BACKGROUND

  RAFHS (ROYAL AIR FORCE HANDLING SQUADRON) produce theAM (Aircrew Manuals) andFRC (Flight Reference Cards) required by the aircrew of all three United Kingdom services -- Army, Navy and Air Force. Members ofRAFHS are specialists in the aircraft types and roles flown by the forces. The mix is approximately 50/50 serving officers and retired officers. The retired officers, in general, are over 55 and only just computer literate having moved in 1989, reluctantly, from a cut and paste editorial system. A system that owed its genesis to advanced technological change much earlier in the century; essentially coloured pencils, scissors and glue.
  AM are substantial documents covering the handling of the aircraft under normal and abnormal flight conditions, the description of the aircraft structure and systems and their operation under normal and failure modes and the limitations that apply to all phases of flight. Extensive use is made of A3 and A4 illustrations in both black and white and colour.FRC are, for most modern aircraft, large documents in A5 format and can be as large as 160 sheets. The contents are checks and drills to be carried out by the crew during the period they occupy the flight deck or cockpit and include normal and emergency procedures.
  In the early 1990s the UK MOD took the decision that all new projects would be acquired using anILS (Integrated Logistic Support) process; part of that process would be the acquisition of documentation tagged to ISO 8879 using theAECMA (Association Européenne des Constructeurs de Matériel Aèrospatial) 1000D DTD and output either as anIETM (Interactive Electronic Technical Manual) or as paper as required by the Publications Policy. The decision to adoptILS was driven by the need to apply a rigorous discipline to project costs and, in particular, to whole-life costs. The Logistic Support Analysis process allows a more accurate assessment of the cost impact of design or other changes over the life of the project.
  The Maintenance policy organisation seized upon this and moved very quickly into defining standards and setting the way ahead using working groups to agree national and international standards . Operational publications policy lagged well behind and only whenRAFHS was invited in 1994 to join theAECMA 1000D Technical Publications Working Group as a member and six months later asked to chair the Aircrew Publications Working Group did we realise that we had a major task ahead. A major problem was thatAECMA 1000D did not, among its marked sections, include aircrew DTD. Early on it became apparent that the generation of an agreed international standard aircrew DTD would take longer than the time available. We, therefore, had to put in place an interim standard to meet the project's timescales which would be to publish by 1997.
 
 

Past Practices

  RAFHS producedCRC (Camera Ready Copy) using a commercialDTP (Desktop Publishing) application. Information was received from a variety of sources in several different formats including paper text and graphics as well as proprietary word processing formats. Graphics were always provided on paper and were either scanned-in by the authors and saved electronically or pasted at theCRC stage. Because of the limitations in storage and the speed of the equipment most graphics were pasted. Any changes to the graphics meant them being returned to the originators for amendment and the whole process started again. The management of all the documents was a manual paper file system, as was the audit trail of the reasons for changes to the documents.
 
 

Timescales

 In 1992 we started to put in place a proving system which would explore the problems and allow work to start with delivery of draft for the first SGML project in 1994. However the aircraft project was delayed significantly. IT funding was withdrawn leaving only a UNIX file server with a trial UNIX pagination engine capable of producing layed-out text to the test DTD and a network of the old DOS machines in place. In 1994 the project was revived with confirmation that two new aircraft projects would come onstream in 1996 and we would require to have an SGML system and output capability in place by mid 1996.
 I was tasked with the programme implementation. Planning started immediately for the definition of our requirements and acquisition of a system. Funding was already in place having been identified in 1992 and rolled over and indexed for each financial year. What is now very obvious but not so at the time, because we were all newcomers, was that we were essentially SGML greenhorns. We were given the task based on the simple management philosophy that in the land of the blind the one eyed man is king.
 

THE PROJECT

 The initial project milestones were:
 
 
  • Document analysis.
    •  
  • Produce DTD.
    •  
  • Select and train development team.
    •  
  • Define System hardware and software requirements.
    •  
  • Market survey for cost effective hardware and software solutions.
    •  
  • Output specification for chosen system.
    •  
  • Train users.
    •  This rather abbreviated list covers a multitude of crises both major and minor which hopefully will serve to illustrate the many and varied pitfalls set in the path of aspiring SGML system project managers.
     What rapidly became very apparent was that two important threads had to run throughout the programme:<
     
     
  • Plan for change.
    •  
  • Education.
    •  Planning for change became a way of life. As items of the programme were resolved it was essential to evaluate their impact on all of the stages of the project including those that were already notionally complete.
     Education of all those involved, both management and user, was also essential. The tendency of the human race to fight change must not be underestimated particularly when the technology and reasons for change are poorly understood. The education process can be time consuming and during periods of high project activity may even feel like another obstacle being placed in the way of progress. However, it is essential that all involved are kept up-to-date with developments in the project as well as under going planned formal training at appropriate milestones in the programme.
     
     

    Document Analysis

     Proper document analysis is an absolutely fundamental activity. It isn't cheap and it is very difficult to explain to budget managers, who only listen to the music of the bottom line, the criticality of completing a full and proper analysis.
      Logic, however, prevailed and after talking to several different companies we went out to contract. Aircrew documents are currently produced to a layout and formatting specification, AvP70, introduced in the 1970s. The minutiae of layout and presentation whilst superficially similar were left to the author/editors compliance with the standard. Indeed, at first glance, all documents were recognisably AvP70 but particularly withFRC s, differing aircraft types and roles demanded additional structures to allow for differences in cockpit layout and crew composition. For example a multi-seat aircraftFRC needs some means of identifying the individual crew member specific checks whilst fast jet single seat pilots need no such structures.
      We decided to present to the contractor the most up-to-date examples of aircrew documents for the analysis: a fast jetFRC and a multi-seatAM . This was a serious mistake based on ignorance which compounded itself all the way through the project and caused several headfulls of hair to be lost over the next 18 months. A full document analysis must cover the range of published documents.
     Having received the analysis we thoroughly reviewed it, proposed amendments and eventually agreed that it reflected the structure of the documents accurately. The next stage was to produce a DTD from the analysis.
     
     

    Type of DTD

      We were aware that, in general terms, there were 2 types of DTD: modular and a book or contiguous DTD. Both had their advantages and disadvantages.AECMA 1000D used a modular DTD but did not contain an aircrew marked section. Furthermore, inAECMA the preferred output specification was anIETM with paper as a secondary option. Aircrew publications policy still required paper output and the only standard available was AVP70.
     
     

    Modular DTD

      The modular type, in general, dealt with small packets of information,DM (Data Module) which could be strung together at the output stage by a 'pick-list' to create a paper document but which, in general, were more suited to the construction of anIETM . Any paper document constructed from manyDM loses some of the more powerful capabilities inherent in SGML:<
     
     
  • The capability to x-reference throughout a document.
    •  
  • Auto numbering of structures and entities except within aDM .
    •  
  • Inability to construct complete TOC, and other auto-generated Front Matter except by reference to theDM number.
    •  
     

    Book DTD

      A book DTD reflects the complete structure of the document. We required two DTD: one for theAM , the other for theFRC . Both documents can be large: theAM can comprise over 30 chapters averaging 8 A4 pages a chapter plus front matter.FRC , whilst not as large, are complex in structure terms and to be usable require careful construction in the output phase. The disadvantage of a book DTD was the system power required to load complete documents for editing and production. The document analysis concluded that if paper output was the preferred option a contiguous DTD would ensure that the greatest use could made of the capabilities inherent in SGML. It was also noted that modularisation of the information could be achieved at structures below chapter level which could serve during the construction of theAECMA DTD Aircrew marked section . The book or contiguous DTD to AvP70 standards was selected.
     
     

    DTD Production

     We used the same company that carried out the review to write the DTD as it was considered that they would understand the nuances of the structure better than a writer working solely from the analysis.
     The process of DTD generation proceeded to plan. We also arranged for the company to provide a training programme for new users and management to coincide with the delivery date . The course lasted four days and covered an introduction to SGML and finished with a day working with the new DTD and the documentation.
     The introduction of the practical aspects of the move to SGML, to both user and management, early as part of planning for change philosophy was considered essential; it was also a double edged sword. Management were happy to comply with the MOD policy initiative but to set aside four days was deemed to be too difficult for most although those that stayed the course have since been the most supportive during the inevitable small crises that arise. The future users, though, were singularly unimpressed. SGML introduced new disciplines and furthermore removed from their grasp the most satisfying part of the production process; the construction, layout and presentation of the document. However, the function of an author/editor is to provide the clearest, most unambiguous text for the subject involved and the move to SGML ensures that the individual's energies are restricted strictly to that function. About a fifth of the users, however, displayed both an interest and an aptitude, not surprisingly, for working with DTD and SGML. It was from this group that the in-house team for the implementation of the system was chosen.
      As part of our early work we decided that for testing purposes, both for software and hardware, we required a test SGML file constructed to our DTD. It had to include all the structures that were in theAM DTD and specifically differing tables and graphics. This was built and used as the template during the testing and selection for all but the development of the output specifications when we used the selected software to convert existing documents to SGML.
      Our first problems arose when we distributed the DTD to the future users and the designer of the output specification. They to a man came back with proposals for change under the guise of improvement. Some were patently a much improved way of achieving the desired outcome; others were certainly driven by the limitations of the system they were using. What we had not done, however, was to put in place a system for maintaining the DTD. For a brief period we attempted to manage change and version control the DTD ourselves, but it became obvious very rapidly that the sensible option was to place a contract with the originators of the DTD and allow them to manage change based on the original philosophy. The development of the DTD was rapid and we quickly reached moved from V1.0 through V1.5 to V2.0 for both theAM andFRC . The construction of the output specifications led to a major re-write of the DTD over about 10 hours as we worked a Saturday away in San Diego. The reasons for change were the need to accommodate the pagination engine output requirements, particularly in specific graphic areas, the fact thatFRC vary across aircraft types and new structures had to be accommodated and the intrusion of output into the DTD structure. After 7 months both DTD were at V2.1 and remain so after more than a year.
     
     

    System Configuration

     During the early phases of the move to SGML we instituted a critical review of our IT system as part of the planning for change. We concluded that the fundamentals of the system were in place but that power/sizing requirements would depend critically on the software selected.
     
     

    Software

     After the SGML training course the newly created in-house development team, consisting of the project manager, two authors and the system integrator/manager, compiled a shopping list of the essential needs to be met by the SGML software. These were:
     
     
  • A CALS compliant receipt and delivery system.
    •  
  • A CALS compliant graphics viewing package.
    •  
  • An intuitive editor.
    •  
  • Ability to load and access DTDs easily.
    •  
  • DTD structure visible.
    •  
  • Ability to create and view tables.
    •  
  • The ability to identify reasons for change.
    •  
  • The ability to identify dates of changes.
    •  
  • The ability to identify the author.
    •  
  • An integrated package allowing the author to work with the same interface for authoring, editing and output.
    •  
  • A means of controlling access to individual publications.
    •  
  • The ability to output to the house style specifications.
    •  A review of the market place indicated that there were a considerable number of editors, fewer integrated systems and little in the way of output systems that allowed the simple creation of output specifications.
     Testing of the proposed systems gave us a problem. We used an instance tagged to our DTD with all the problem areas, specifically, graphics and tables included. Having this available early gave us an understanding of the process and proved extremely efficient as tool for assessing the pros and cons of differing applications. Even though the instance was quite small it became apparent during the testing of different applications that working with a full document would become quite tedious. Loading and unloading a full document in order to change a word or add several paragraphs in one area was sufficiently slow to alienate authors totally. Problems with tables and the handling of the various CALS standards graphics were also revealed.
     Work flow control also created problems . We needed a system that would provide more than the ability to progress documents through the creation process. Whilst generally an author would look after his own manuals we needed to identify each change, the reason, the date, and the author for air worthiness audit purposes . We also needed to deny editing rights to others whilst retaining the ability to allow more than one individual to work on a document.
     These requirements led us to search for a database solution; one preferably object based that would work with all the objects in the DTD below document level, ideally it should already be integrated or capable of integration with a choice of editors and output systems.
      The output system requirements were difficult. We needed to produce relatively simpleAM but in presentation requirements quite complexFRC ; the latter posed an SGML-to-paper problem not yet, to our knowledge, adequately overcome in the aircraft industry. We had had a structure-based pagination engine delivered as part of our test system in 1992. It was decided that we would retain the application, upgrade it and have new output designs written to meet our needs.
     The upgrading of our existing output software meant that whilst we retained an application which used and understood structure as the basis of its operation, intervention at programming level was needed both to load DTD and output structured compliant documents. However, we found no application that could improve, in overall terms, our existing software although some could manage either input or output more effectively but not both.
     
     

    Hardware

     We had now reached a point where we had to review the impact of the work done in the software area upon our existing and planned hardware configuration. It was immediately obvious that the choice of output system required the UNIX side to be upgraded and either the DOS machines upgraded or replaced. The pagination engine required a UNIX platform but as all authors would also have DOS legacy projects into the forseeable future we decided that the output application would run in an X-window on the DOS machine. The natural sequel to this decision was to replace the existing DOS machines with top-of-the-range machines as new projects came on stream. We initially used Pentium 166Mhz with 32 Mb. The file server was also upgraded with more memory and processing power.
     

    THE SOLUTION

     Our solution was based on the availability of Astoria, an SGML database product which incorporates a significant number of the features we required plus, significantly, the ability to access individual elements within a document and to extract that single element to an SGML editor integrated with the database. This feature markedly reduced the loading time of our large documents and has made the working regime more than acceptable. We, as one of the first users, had a choice of one integrated editor, InContext, but we understand that several others are now available. However, the available editor was intuitive and one which our authors found easy to use; albeit with some limitations in the scope of table work, which, thus far, has not proved significantly limiting. We are also able to access all illustrations as we work using a CALS standards viewer-CALSVIEW.
     Of prime importance from the management point of view is that we are able to identify in the database, at element level, the changes to the document, the date, the reason and the author, thus providing tight version and quality control. As several aircraft projects use common items of equipment we have also been able to create a common library of illustrations and text which can be drawn upon by the different projects. Create once use many times in practice.
      The CAPS pagination engine we used for output worked with structures in a very similar manner to SGML but still required intervention in the loading of the DTDs and the creation of output specifications. We contracted for two output designs which reflected our DTDs. These work well and create the large bulk of the front matter and provide all of the automatic benefits that flow from an SGML system. Layout, section, chapter, para, table and graphics numbering, cross references are continually updated as well as a standard house output style.FRC are a major step forward and we must, at all costs, retain the output capability gained thus far. We have yet to complete work on theAECMA DTD and whilst we are hopeful thatDM DTD can be built using a tag-mapping procedure from our existing structures we know that further work will be require in the output area to keep our current capability. CAPS as a black box pagination engine has worked well and even with the demanding layout requirements ofFRC has required less than 5% of operator intervention to achieve the desired result. However, whilst very powerful, the interactive editing capability is rather less than user friendly and any text amendments cannot be reflected in the source SGML document.
     Syntegra, the systems integration business of British Telecom put together an integrated hardware and software package which provided a seamless workflow for the authors allowing them to operate with a single interface from their workstation through all phases of the process. The major benefit is in operator confidence when presented with a user friendly work process.
     As part of our document creation work flow we take and comment upon source information and draft from the aircraft design authorities. Axiomatic to a successful process is the correct marking-up of information by our suppliers. We have found that there is a serious learning curve to be overcome at the start of each contract frequently requiring several exchanges of tagged text to ensure that the output engine formats the text correctly. 1840A tape is the present transfer medium. This works well enough although we do need normally to work through one or two iterations to ensure that all is well. The commenting process is carried out with our main supplier by means of a small SGML file generated using a separate DTD available in the Editor which can be returned to the originators on tape allowing them a means of receiving comment without their database files being changed.
     Training was an integrated element of the solution and the authors were trained during the period that the software and hardware were delivered and installed. Our training philosophy was to reinforce the earlier SGML training briefly at each applications training phase to ensure that SGML fundamentals were well grounded. The sequence of application training was planned to output authors in the month before they started working with their new projects. Our timing was immaculate but all the aircraft projects slipped to the right so we now have a continuing user familiarity problem with the software. We installed a training container with instances in Astoria and when they find time authors can run through a refresher training cycle. As a result we find that, in general, as authors start work they quickly adapt to the new work processes.
     Finally, a proper acceptance test schedule covering all aspects of the system must be put together and used to test out all the areas of integration and known problem areas within the applications to ensure that where fixes have been made they stand up to a rigorous examination.
     

    THE FUTURE

     To date we have been primarily concerned with getting our data secure, that is, getting it converted into SGML and getting it stored in a manageable way. Now the question of output is exercising our minds. The options seem inexhaustable. Options under consideration:
     
     
  • SGML transformed into HTML/XML for output on the Web;
    •  
  • SGML composed and paginated to paper; and
    •  
  • SGML used as source for delivery in anIETM application (using HyTime constructs).
    •  When we started we viewed SGML as a means to an end; now we need to decide what some of those ends will be.
     

    CONCLUSION

     Moving to SGML provides considerable benefits, many well documented which need not be rehearsed again here, others less well known like the focusing, within our organisation, of the authors' energies. To achieve these benefits cost effectively and efficiently many lessons can be drawn from our, initially quite naive, but ultimately very successful change to the standard. The previous paragraphs have attempted to outline the process we put in place and the pratfalls we suffered. We now have the basis of a system which will protect our information in the long term. The list below should provide an adequate checklist for those about to embark on the same invigorating, worthwhile and challenging task.
     I have assumed that risk and cost/benefit analyses have been undertaken and that adequate funding has been made available; however if this is not the case then they are essentials. If the expertise is not available within the business then consultants should be used. We were driven by MOD policy and despite employing consultants at what appeared to be the critical phases of the process still managed to create problems along the way, so take heed and what you need from the following:
     
     
  • Know the principles of SGML and understand your document structures and graphics requirements.
    •  
  • Educate all concerned because a little background goes a long way.
    •  
  • Assemble a small in-house team who are aware of your business processes and are forward thinking.
    •  
  • Start planning for change.
    •  
  • Define what do you require from your system and therefore your hardware and software.
    •  
  • Work closely with your consultants to ensure they understand your requirements.
    •  
  • Ensure that a thorough, comprehensive document review is completed which covers all your requirements.
    •  
  • Define what type of output you require, understand the limitations and select a DTD which will enable you to achieve the best solution.
    •  
  • Plan for change.
    •  
  • Put in place a timely and thorough training programme.
    •  
  • Facilitate the process; have examples of your documentation prepared for evaluation of applications; make sure that problem areas are included.
    •  
  • Plan for change. Ensure that as decisions are taken their implications on hardware, software and peopleware are considered.
    •  
  • Test the DTD early against examples of your documentation, check that you have all the necessary attributes in place.
    •  
  • Put in place a version control process for the DTD. Ensure that the DTD documentation is up-to-date, and that all concerned are working with the correct version.
    •  
  • Ensure that your software, and particularly your pagination engine/viewer have a migration path planned to encompass open standards.
    •  
  • Put in place a proper acceptance test schedule to test all aspects of the system before delivery.
    •  
  • Ensure that proper support arrangements are in place with the suppliers to ensure teething problems are handled timely and efficiently.
    • Towards an SGML Diff Using Temporal Documents   Table of contents   Indexes   Authoring and Translation for the International Market