A brief guide to Digitisation Guidelines

Brief-guide-digitisation-guidelines-header-Matt-Nix-TownsWeb-ArchivingOver the past 10 years the world of cultural heritage digitisation has seen a considerable and increasingly collective effort to establish best practices and set digitisation guidelines; and if you are planning a digitisation project one of your first ports of call may be to look at the range of guidelines available.

As Melissa Terras, Director at UCL Centre for Digital Humanities, points out:
“Digitisation guidelines, whether practical, managerial, or technical in nature, are the first point of reference for those about to undertake a digitisation project, and there is no need, nowadays, for a project to establish every detail themselves.”

(Digitisation and Digital Resources in Humanities, 2012)

Guidelines are available from a range of sources but the setting of guidelines has predominantly been driven at the level of national institutions, with guidelines being available from many different national libraries and archives. However guidelines have also been published by academic institutions as well as professional bodies.

For instance The National Archives provide good practical guidance for the handling of material during the digitisation process. The International Federation of Library Associations and Institutions (IFLA) has published IFLA guidelines for those planning projects involving the digitisation of special collections. With more technical guidelines are available from the Federal Agencies Digital Guidelines Initiative (FADGI) as well as the Metamorfoze project.

The aim of guidelines is to establish a level of consistency and accuracy across the digitisation process; this accuracy and uniformity can help to ensure the long-term viability of digitised content. As a result guidelines can be used as a benchmark to help define projects; goals, requirements or specification. Most of the current guidelines available today have been through several iterations and have come a long way.

Matt-Nix-Digitisation-Technician-at-work-station

Above: Matt Nix is an Imaging Technician at TownsWeb Archiving and specialises in managing colour profile capture.

Background to Digitisation Guidelines – where have we come from?

Many early approaches to digitisation focused solely on the goal of preserving an object at the informational level, for instance aiming only to duplicate the text on the page. With the extreme of this idea being the use of destructive scanning processes by some mass digitisation projects (with books being unbound, and newspapers disposed of after scanning) electing to only preserve the information in a digital form.

This philosophy is reflected in the guidelines available at the time. For example, the Library of Congress in its 2007 recommendations advised that greyscale images be produced where colour content “does not exist or is not deemed significant.” With the American Library Association stating that “it was common to capture only in black and white, otherwise known as bi-tonal, for textual documents or line drawings” – totally discarding all colour and tonal information.

Digital-capture-options-colour-greyscale-bitonal

Above: Click to enlarge. Images show the same page scanned to capture colour (left), greyscale (centre), and bi-tonal images.

Whilst this approach can be attributed in part to the technological limitations and the cost of storing large amounts of digital data at the time; it can be argued that such curatorial approaches to digitisation were inherited from Microfilming (microfilm itself being a monochromatic medium). Early guidance also borrowed heavily from the approaches of microfilm.

For instance many early digitisation guidelines based assumptions on the QI formula; a method of determining the required resolution needed to capture typed text, based on the character size of the font. The aim being to ensure the legibility of text on microfilm which has been duplicated more than once (duplication being a source of degradation).

You can trace how these ideas have propagated through guidelines for digitisation, for example the 2010 FADGI guidelines use the QI formula for recommending resolutions for capturing clear printed text (page 59), these recommendations are then referenced as a more general rule of thumb in guidance published by DFG in Germany in 2013 (Page 16). Steven Puglia and Erin Rhodes argue that while this approach is appropriate for capturing text with high-contrast characteristics, it ignores other possible characteristics the original may have, and that these approaches were simply aiming to be the “digital equivalent of microfilm”.

These early approaches and guidelines have also been criticised by some, like Paul Conway and Don Williams, who have argued that fifteen years of guidelines have helped fuel ubiquitous digitisation practices that lack adequate attention to imaging science. However modern guidelines have gone some way to address this concern…

Current Guidelines – where are we now?

Digitisation practices have moved on from viewing digitisation as a way to preserve items at just an informational level. Instead favouring to focus on better capturing and representing more of the characteristics of the original physical item, greatly increasing the artefactual value of the digitised content. With this idea being reflected in current guidelines, the two guidelines which have seen the widest adoption and been most influential are the FADGI technical guidelines and the Metamorfoze Preservation imaging guidelines.

The Metamorfoze guidelines stemmed from the Metamorfoze program. The wider program is focused on the preservation of paper archives, and is co-ordinated jointly by the Koninklijke Bibliotheek [National Library of the Netherlands] and the Nationaal Archief [National Archives of the Netherlands]. The imaging guidelines were first published as a draft copy in 2007, and last updated in 2012.

The FADGI technical guidelines for digitising cultural heritage materials are a result of the Federal Agencies Digital Guidelines Initiative. The initiative was set up in 2007 as a collaborative effort involving various US federal agencies and other governmental bodies as the Federal Agencies Digitization Guidelines Initiative, with the explicit aim of setting digitisation guidelines. However at the start of this year the initiative chose to rename itself as to better reflect its broader work. The initiative includes both the National Archives and Records Administration and the Library of Congress, who had both previously published their own guidance.

The FADGI guidelines were first published in 2010, then updated in 2015 and again more recently in September 2016. The 2010 guidelines drew substantially on guidelines already published by the National Archives and Records Administration in 2004. The updates in 2015 were made in order to make the guidelines recommendations compatible with the Metamorfoze guidelines. As a result both make similar recommendations. However the FADGI guidelines also offer more practical guidance and are perhaps more accessible in the way in which the document is presented, breaking recommendations for different materials into clear individual sections.

Both guidelines base their approach on already existing and well established imaging science to define clear metrics for evaluating image quality, in both cases these take the form of performance scales with increasing levels of precision. The FADGI guideline adopts a star ranking system, whereas the Metamorfoze guideline divides itself into three quality levels.

Both guidelines also use the same ISO standard methods to measure image quality. However, as Paul Conway and Don Williams highlight, while the “FADGI and Metamorfoze documents represent a triumph of imaging science over curatorial judgement in the digitization workflow, and yet the guidelines themselves are noticeably silent on the cumulative impact of many small technical decisions on the shape of the resulting digital product.”

If you are to compare the recommendations of both of these guidelines to those of The National Archives, you will find both FADGI and Metamorfoze to be much more detailed. Though perhaps one key difference to be aware of is the National Archives recommendation to use the sRGB colour space for all colour material. sRGB has a relatively small colour gamut compared to other larger colour spaces; while it suitable for material with a limited colour gamut like newspaper print, material such as photographs or works of art may contain colours outside the space of sRGB.

As a result the Metamorfoze guidelines recommend the use of a larger gamut colour space, and the FADGI guidelines which till only recently had only recommended using sRGB in certain cases. However As Don Williams and Peter Burns point out the “difficulty of using one of the larger colour spaces is the need for increased bit depth in order to avoid posterization or [colour] banding”, and adopting a wide gamut may also, “actually serve to diminish colour fidelity by increasing the encoded quantization interval between neighbouring colours. Larger quantization intervals can reduce colour and tone discrimination, which makes it difficult to render subtle near neutral tones in slowly varying image areas.“

quanitifying-effects-using-large-colour-space-digitising-diagram

Above: The above image illustrates the quantizing effects of using a large colour space (in this case ProPhoto RGB) at different bit-depths. The charts plot the position of each pixel from the smaller section in the CIE xyY colour space, for both a 8 and 16 per channel version of the image.

It can also be argued that the benefit of the sRGB colour space is its widespread ubiquity, and the inability of some software to correctly handle ICC profiles. However recently in September 2016 FADGI guidelines changed its guidance allowing for use of the sRGB colour space in more applications.

Future Guidelines – where are we heading?

The reasons for the latest updates to the FADGI guidelines were laid out in a recent blog update by the Library of Congress, which states:
“The Guidelines have also been reviewed by ISO TC/42 JWG26, which is formulating ISO standards related to cultural heritage imaging. As a result of this ISO review the 2016 Guidelines will be fully compliant with the emerging ISO standards relating to cultural heritage imaging.”

The new ISO standard titled: “Archiving systems — Part 1: Best practices for digital image capture of cultural heritage material”, is expected to be published in 2017 and is a collaborative effort involving many who have previously been responsible for publishing past guidelines. It has been described by Roy Bern’s as “Combining FADGI, Metamorfoze, and practical experiences.”

The new standard is not the only thing being worked on by ISO TC/42 JWG26; last year they published ISO 19262 – A vocabulary defining terms used in the imaging of archival material – and are also working on a standard for image quality analysis for digitisation.
Separately there has also been an effort to create an ISO standard Tiff image file format for archival purposes by the TI/A group, which was submitted to the ISO technical committee at the end of October 2016.

Further guidance for Digitising specific media types

Specific digitisation guidelines for different media types – including bound books, photographic collections, 35mm slides, newspapers, oversize items and more – can be found in our How to Digitise guides. Each guide also contains practical advice on ideal image capture resolutions, digital image formats, and unique challenges posed by each medium.

Or if you have a specific query please feel free to ask us and we’ll be happy to help.

This entry was posted in Digitisation, Digitisation guidelines, FADGI, Guest post, IFLA, Metamorfoze | Comments are off for this post