Digital Archiving Pilot Project: Excavation Records

Archaeology Data Service


Introduction

Aims and objectives

Digital archives

Internal documentation

The focus group

The report

Related initiatives

Methods statement: Archive preparation

Dataset documentation

Focus group

Preparation of project report

Location Map


Introduction

The Archaeology Data Service (ADS) aims to collect, describe, catalogue, preserve and provide user support for all digital resources that are created as a product of archaeological research. As one of five service providers in the Arts and Humanities Data Service (AHDS), the ADS also has a responsibility for promoting standards and guidelines for best practice in the creation, description, preservation and use of spatial information in the humanities. For those classes of archaeological data where there are existing archival bodies, the role of the ADS is to collaborate with the appropriate national and local agencies and to promote greater use of existing services.

A meeting held between the ADS, Association of Local Government Archaeological Officers, English Heritage and Royal Commission for the Historical Monuments of England (RCHME) in May 1997 confirmed the need for pilot studies to examine the digital archiving of primary fieldwork data. A joint co-operation statement signed in July 1997 between the ADS and RCHME confirmed that both organisations recognize the importance of developing costed models for digital archiving and specifying optimum digital media and storage conditions.

Subsequent meetings with English Heritage suggested that a strong area for collaboration was development of digital excavation archiving pilot projects. This reflects English Heritage’s position as a key funding agency for English archaeology and its ability to encourage grant recipients to follow best practice in preparing archives for post-excavation deposition.

Exploration of projects suitable for use as pilot projects was undertaken with a variety of units. Two excavations were identified as most suitable: the Eynsham Abbey excavation by the Oxford Archaeological Unit (OAU) and the Royal Opera House excavation by the Museum of London Archaeology Service (MOLAS).

Key factors in identifying appropriate projects included:

Eynsham Abbey - Oxford Archaeological Unit

The Eynsham Abbey excavation was undertaken by OAU from 1989 - 1992. The project was never designed as an exercise in digital data collection or management, but rapid changes in computer technology over the last five years have meant that the project has developed in this direction. One major goal in involving the OAU in this pilot project is to assist them as they develop digital data guidelines and policies for future projects. An important spin-off of digital archiving may be the opportunity to remove unwieldy and technical data from publications. There is a desire to shift this information into a digital archive, but in order to finalise publication contents the digital archive needs to be planned soon. The existence of a digital archive can then be advertised in the publication itself.

Royal Opera House - Museum of London Archaeology Service

Excavation under the Royal Opera House scenery store in Covent Garden, London took place in 1996 and all data from the project was collected in computer-based formats. This excavation was the first for which MOLAS used its state-of-the-art integrated database and GIS recording system. As this system will be their standard for all future projects, this is an excellent time to ensure that appropriate provision for digital archiving has been made and that appropriate standards are used. MOLAS is keen to develop internal guidelines for preparation of digital archives. Furthermore, the Museum of London is rethinking its future archive options and is keen to participate in an archiving pilot project with ADS and MOLAS.

Aims and objectives

The goal of this digital excavation archiving pilot project is to:

  1. evaluate current practices with respect to data collection, management and preservation in archaeological units
  2. identify commercially sensitive data and copyright concerns
  3. preserve digital data from the Eynsham Abbey and Royal Opera House projects, including full documentation and metadata
  4. quantify the effort and costs to both units and archives in preservation of computer-based data
  5. evaluate user reactions to digital archives made accessible over the Internet
  6. recommend improvements in unit standards and practices
  7. inform development of digital archiving guidelines

In order to meet the above aims, two pilot studies in digital excavation archiving are to be conducted followed by a focus group to evaluate the resulting digital collections.

Digital archives

Digital archives for each project will be deposited with the Archaeology Data Service to be distributed freely via the Internet to interested users. Location of the digital archives will be advertised in project publications, where possible. The process of preparing and depositing these archives will fulfil project aims 1, 2, 3 and 4.

Eynsham Digital Archive

Evidence dating to the Late Bronze Age, Saxon and Medieval periods was recovered from the Eynsham Abbey excavation and the digital files include information about bone, coins, glass, ivory, lead, metal, pottery, stonework and stratigraphy. It is anticipated that the digital data archived as a result of the excavation will be of particular interest to finds specialists, archaeologists who study monasticism and Late Anglo-Saxon period specialists.

Data to be made available for the Eynsham project digital archive includes a selection of the most appropriate site plans, combined structure plans, phase plans and three-dimensional reconstructions as AutoCAD 14 files. The project database, held in Microsoft Access 97 software, contains a site table with context information (soil type, structure, period, phase, rephase, potential, artefacts present or absent) linked to a structure table and from there to AutoCAD drawings. Linked Access data tables contain the finds information. Text-based reports are held in Microsoft Word 97.

Royal Opera House Digital Archive

Considerable interest from the public and the professional community was generated by the discovery of Saxon settlement beneath the Royal Opera House. Evidence gathered dates to the Saxon and Medieval periods. Post-excavation analysis is almost complete and preparation of a draft text for publication is scheduled for late 1998. It is anticipated that the digital data archived as a result of the Royal Opera House excavation will be of particular interests to those interested in the history of London, finds specialists, computing archaeologists and Saxon / Medieval specialists.

Digital data is primarily held in a fully integrated Oracle database and ArcView GIS system for both finds and stratigraphy. The exceptions to this are EDM data which are held in Penmap software, all plans and sections which are held as AutoCAD files (including 1:100 site plan) and texts which are held as Microsoft Word files. Standard MOLAS terminology lists have been used to control entries in the project database (e.g. for ceramic fabric types).

Internal documentation

Each unit participating in this pilot project will have the opportunity to develop and refine their internal documentation along the lines of an example from CAS presented as two appendices to this report (Appendices 1 and 2). This process will fulfil project aims 1, 3 and 7.

Museum of London Archaeology Service

MOLAS already has internal documentation detailing the procedures used to collect and manage the digital data that they create. This pilot project will allow them to refine this documentation to make it more easily understood by external contractors and consultants. Documentation will also be expanded to include information about digital data storage and archiving procedures.

Oxford Archaeological Unit

OAU will be able to standardise and document its procedures for collecting, managing and storing digital data.

The focus group

The ADS will host a focus group in which a cross-section of archaeologists (including students) and a few non-archaeologists are brought together to discuss the resulting archives and the archive documentation. The aim of this focus group will be to determine how useful these digital archives are to different kinds of people, how the documentation or presentation of archives might be improved, how the information in the archives might be reused and what types of support would be necessary to encourage the reuse of information in the digital archive. This will be the forum for obtaining user feedback necessary for fulfilling project aims 5, 6 and 7.

The report

A report will be produced summarizing development of the pilot project, results with respect to each of the project aims, the nature of documentation required to create an orderly digital archive, the amount of time required by units to produce appropriate documentation, the costs to both units and the ADS in creating and managing the digital archive and the nature of user feedback.

In order to ensure that these results are widely disseminated, a copy will be made freely accessible on the ADS web site (after approval of the project steering committee). Additionally, the results of the project will be disseminated via conference presentations and traditional printed newsletter and journal articles.

Related initiatives

There are several projects related to this pilot project currently underway.

Excavation and Fieldwork Digital Archiving Guide to Good Practice

A working party convened by the ADS is producing a Guide to Good Practice in digital archiving datasets from excavation and fieldwork. There are obvious synergies between the development of this Guide to Good Practice and the digital excavation pilot projects. First, the Guide to Good Practice is currently available in draft and the pilot project will allow some of the recommendations made therein to be tested. Second, the pilot project will allow detailed time and cost estimates to be developed that can be included in future revisions of the text of the Guide to Good Practice.

Digital data: Survey of user needs

As an initial step to developing a national archiving policy for digital data, a user survey has been undertaken by the Archaeology Data Service on behalf of Arts and Humanities Data Service, Cadw, Built Heritage Division of the Department of the Environment for Northern Ireland, English Heritage, Heritage Council of the Republic of Ireland, Historic Scotland, RCAHMS, RCAHMW and RCHME in order to:

The report from this survey is scheduled for publication in the spring of 1999. The survey clearly demonstrates a desire for access to digital information in large sectors of the archaeological community and a preference for digital data as opposed to paper-based information in a growing number of sectors.

Archiving guidelines review by MGC/SMA

A committee has been organized by the Museums and Galleries Commissions (MGC) and the Society of Museum Archaeologists (SMA) to review archiving guidelines in the wake of Hedley Swain’s survey of archives. Informal conversation with other members of the SMA suggests that there is also activity toward convening a group to develop archiving guidelines for archaeological units that include the transfer of digital information collected by units to museum collections management systems in digital format.

These two initiatives are clearly related to the Digital Archiving from Excavation and Fieldwork Guide to Good Practice and the Survey of User Needs. The project described here would allow the needs and practices of archaeological units to be considered in the development of national archiving policy and would ensure that the costs to them in terms of staff time and infrastructure are adequately considered.

Portable antiquities recording scheme

Sponsored by the Department of Culture, Media and Sport, the portable antiquities recording scheme project is examining strategies for promoting the voluntary recording of archaeological finds. This project includes development of standards for recording finds, the development of a database for storing this information and strategies for making this information available via the Internet.

Methods statement: Archive preparation

Eynsham project digital archive

Items for inclusion in the digital archive include:

Database files

Information about contexts, structures, pottery, small finds, window glass, vessel glass, lead objects and tile are held in Access format and will be included in the digital archive. This includes full information about context character, context potential, preservation, period, phasing, links to image files, artefact character, specialist comments, indices of x-rays and illustrations and conservation notes.

Numerous disparate databases exist each of which has been created by individual specialists over the last two years. These data will be evaluated, specialists will be asked to provide their data keys and terminology standards, data will be normalised and finally data will be incorporated into a single database structure. All data tables will be checked for consistency and errors.

As it will be difficult for some archaeologists to use Access database files, but for those who can use Access files it will be easier, it will be necessary to preserve two versions of the Eynsham database: one is Access format and one in either DBF or ASCII delimited format. Documentation accompanying the archive will need to detail the relationships between the tables so users of the DBF/ASCII version can rebuild these. Also important to document will be any specialized OAU terms or concepts used to structure the data that will be unfamiliar to users.

Image files

Images will be selected to accompany the background text about the project archive. These will include scanned slides, phase plans, structure plans, a scanned reconstruction drawing and a 3-D model of the kitchen courtyard area.

Four hundred plans are to be converted from FastCAD to AutoCAD. Photographs necessary for archive use will be selected for scanning. A 3-D reconstruction is well underway, but needs to be completed to finish the archive. (Note that this 3-D reconstruction was not envisaged when the project was undertaken, but has proven extremely useful during analysis phases. Ensuring that this reconstruction is adequate for public dissemination is thus required to complete an orderly archive.)

For archiving purposes each AutoCAD file selected for inclusion will also need to be exported in DXF. Both the native file format and DXF file will be preserved. For other digital images such as scanned photographs, two formats will be required: TIFF for archiving and JPEG for dissemination. TIFF images should be uncompressed and scanned at high resolution and full size. JPEG images can be reduced in size. For the 3-D reconstruction a VRML 2.0 file will be constructed if possible to accompany an image of the final reconstruction.

Text files

Textual information about the site held in Word 97 format will be included in the digital archive. This includes an introduction to the project, summary of results, phase summary and structure summary.

Specialist contributors to the archive will be asked to approve a summary of the main content of their assemblages for inclusion. Redundant files (e.g. interim reports that have been replaced with final versions) will be weeded out and commercially sensitive information will be filtered. Commercially sensitive information includes correspondence, research designs, post-excavation assessment, specialist reports and personnel lists. Most of these records (except the correspondence) would normally accompany a paper archive, but there could be a time lag before this information was made available to others. It might be possible to include this information in the digital archive with a similar time lag prior to dissemination. Obviously, the full text and figures planned for publication will not be included with the digital archive initially as they do not want to undermine publication sales, but OAU is prepared to consider deposition of the project DTP files for circulation when the publication goes out of print. The format of the DTP files can not currently be specified and will be agreed between the ADS, OAU, the publisher and the printer.

Royal Opera House project digital archive

Items for inclusion in the digital archive include:

Database files

Information about the project, finds and environmental remains from the MOLAS Oracle relational database will be included in the digital archive. This includes a table with basic site archive information, a table with basic interpretative information, a finds table with dating evidence by context, a registered finds table containing everything but the internal MOLAS comments field, an animal bone table and a plant remains table.

As it will be impossible for many archaeologists to use Oracle database files, it will be necessary to export these data into DBF or ASCII delimited formats for archiving and dissemination. With these formats only the data are preserved, not the relationships between the tables. Careful documentation of the relations and how to rebuild them will be required to accompany the database files in the digital archive. Also important to document will be any specialized MOLAS terms or concepts used to structure the data (e.g. groupings, land use concepts).

Image files

Graphical information about the site held in ArcView and AutoCAD formats will be included in the digital archive. This includes site location plans, trench location plans, selected group plans with some attribute information, selected period plans with some attribute information and selected photographs.

At present there are only about 130 graphical files containing approximately 11 Mb of data, but this quantity is expected to increase by a factor of four or five during the publication phase of the project. We will not digitally archive the c. 600 image files that will result. Instead a careful selection of images necessary to understand the site, the project, the key results and the archive will be made.

ArcView files will need to be exported as ArcView shape files. For archiving purposes, each AutoCAD file selected for inclusion will also need to be exported in DXF format. Both the native file format and DXF file will be preserved. For other digital images, such as scanned photographs, two formats will be required: TIFF for archiving and JPEG for dissemination. TIFF images should be uncompressed and scanned at high resolution and full size. JPEG images can be reduced in size.

Text files

Textual information in Word 6 and Excel formats will also be included in the digital archive. This will initially include texts summarizing the project and its key finds. When the publication text is complete, it will be possible to access more text data for the archive, but there may be a delay before this information can be made accessible to the public so as to avoid undermining publication sales. As there are currently more than 500 text files containing nearly 60mb of information in the project directories, it will be important to carefully assess each for inclusion in the digital archive. Redundant files (e.g. interim reports that have been replaced with final versions) will be weeded out and commercially sensitive information will be filtered.

Dataset documentation

Preparation of full archive documentation by unit staff according to ADS Guidelines for Depositors. The units will keep records of how they document their data for digital archiving and how much time this takes.

For the Eynsham Project, specialists who worked on the project but who are not employed by OAU will be asked to provide details of the recording and analytical systems used and a key to the fields in their database.

Additionally, each dataset will need to be accompanied by information for inclusion in the ADS catalogue. The ADS will prepare a ‘cross-walk’ between database fields and ADS catalogue fields to enable automatic generation of catalogue records for databases. For text and images, cataloguing will be compiled by unit staff according to ADS guidelines.

Focus group

When the datasets are mounted in the ADS catalogue, a one day focus group will be held in York. A cross-section of archaeologists (including students) and a few non-archaeologists will be brought together to discuss the resulting archives, the archive documentation and the accessibility of the digital archive. The aim of this focus group will be to determine how useful these digital archives are to different kinds of people, how the documentation or presentation of archives might be improved, how the information in the archives might be reused and what types of support would be necessary to encourage the reuse of information in the digital archive.

Preparation of project report

The ADS Project Manager will prepare a final project report summarizing development of the pilot project, the nature of documentation required to create an orderly digital archive, the amount of time required by units to produce appropriate documentation, the costs to both units and the ADS in creating and managing the digital archive and the nature of user feedback. This report will address all seven of the project aims.


Top of
Document
Arch Division
Homepage
List of
Summaries