Abstract

The DAVID project¹ studies how to keep digital audio-visual content usable in the face of adversity: obsolescence, media degradation, and indeed failures in the very people, processes, and systems designed to keep digital content safe. For file- and tape-based digital video DAVID analyses the origin of potential damage and its consequences on the usability of content, detects and restores damage that has already happened, and develops strategies for avoiding future damage in a way that balances long-term costs, risks of loss, and content quality.

The presentation gives an overview of the major findings of the two-and-a-half-year research project, it will present results on audiovisual preservation metadata standardisation (MPEG MP-AF) and will especially focus on the detection and restoration of damage in digitised and born-digital audiovisual content. We present solutions to detect and repair errors of file-based audiovisual objects in the baseband/essence/content level; that is, we will show how digital tape dropouts (Digital Betacam, IMX and others), field issues, and other baseband errors can be detected and restored.

Keywords

archive, video, damage, detect, repair.

Introduction

DAVID is a European Commission Framework Program 7–funded research project that aims at keeping audiovisual content (video) usable in the face of adversity: format obsolescence, media degradation, and failures in the very people, processes, and systems designed to keep digital video content safe. The aim of the DAVID project is to address following questions:

•  How does damage occur in digital file and digital video tape-based systems used for preservation and access? What are the consequences of this damage on the ability to make use of audiovisual content?

•  How can this damage be efficiently monitored and detected?

•  Accepting that damage will occur, how can content be repaired to enable reuse?

•  Are there better ways to structure media files that make them more resilient?

•  How can the technical quality of content be improved beyond the original state to satisfy requirements of new use channels and to ensure efficient storage utilisation?

•  How can all this be done at scale and at speed for large audiovisual collections?

•  How can effective risk management and quality assurance techniques be built into preservation systems so that the systems themselves become more robust and resilient?

•  How can better preservation techniques be incorporated directly into the devices and systems that produce born digital content?

The project consortium includes the broadcaster Österreichischer Rundfunk (ORF) and the national archive Institut National de l’Audiovisuel (INA), both with experience of research and development in digital preservation; the two industrials HS-ART Digital Service GmbH (HSA) and Cube-Tec International GmbH (CTI) on the supply-side providing media migration, digital restoration, and quality control; and the two research partners Joanneum Research (JRS) and University of Southampton—IT Innovation Centre (ITInnov) with long histories in digital audiovisual content analysis and restoration, risk management, and storage technologies and services.

DAVID has produced following main project results:

•  Report on Data Damage and its Consequences on Usability (Deliverable D2.1)

•  Report on Analysis of Loss Modes in Preservation Systems (Deliverable D2.2)

•  Digital BETACAM Dropout Detection Tool

•  Block Dropout Detection Tool

•  Advanced Field Error Repair Tool

•  Noise Repair Tool

•  Video Echo and Overshoot Correction Tool

•  MXF D10 File Repair Tools

•  BPRisk: Risk-Aware Preservation Workflow Planning and Optimisation Tools

•  Preservation Metadata Model and Library

•  Report on Recommendations for Born Robust AV Content (Deliverable D3.5)

More information on 0the tools developed can be found at the website spotlight section,² and public reports (including those mentioned above) are available at the website deliverable section.³

Selected DAVID Project Results

Selected results are provided for the project research areas damage detection, damage repair and damage prevention.

Damage Detection

DIGITAL BETACAM DROPOUT DETECTION

The Digital Betacam tape format is a major digital, high-quality archiving format. First mass migration projects (from tape to file) started recently. Due to the high market acceptance, the migration of all Digital Betacam tapes is expected to be continued for many years. A major class of errors which may occur is digital tape dropout. They need to be found during migration of content into IT-based storage systems, as later detection and repair causes higher costs due to the need of re-ingesting tapes or is even impossible when tapes are further degraded.

Within the DAVID project, JRS developed a content-based (essence-based) algorithm for the detection and repair of Digital Betacam tape dropouts. We have developed initial detector prototypes for two specific dropout classes, as there are luminance dropouts and chrominance dropouts, first, by utilizing specific spatial properties of single blocks affected and second, by using also the spatiotemporal distribution of multiple affected blocks within the frames.

The detector supports two modes. The basic mode provides a decision about which frames within a video are damaged to what degree, and the detailed mode provides the exact position of all damaged blocks within a single frame. The basic mode prototype of the Digital Betacam tape dropout detector can be incorporated in essence QC systems like VidiCert; the detailed mode prototype can be utilized within AV media repair systems like DIAMANT.

BLOCK DROPOUT DETECTION

For detecting a broad range of digital tape dropouts, originated from tape formats different to Digital Betacam, a more general detection algorithm was needed . Similarly to the previously mentioned type, these dropouts may also occur during the migration process, but have a different appearance. Instead of a chessboard structure, these dropouts appear as blocks or horizontal stripes within a constant block size. They need to be found during migration of content into IT-based storage systems, as later detection and repair causes higher costs due to the need of re-ingesting tapes or is even impossible when tapes are further degraded.

Within the DAVID project, Joanneum Research developed a Block Dropout detection algorithm which is able to find block dropouts in one or multiple frames of the video. An automatic grid detection algorithm allows for the detection of the block size and its corresponding regular block grid location.

ADVANCED FIELD ANALYSER

The Advanced Field Analyser (AFA) is a tool which is able to automatically detect various field issues which may be found in digital AV files. The AFA was designed as a core library and can easily be integrated into different workflows.

Damage Repair

ADVANCED FIELD ERROR REPAIR

The Advanced Field Processor (AFP) developed by HS-ART Digital Service uses the results of the Advanced Field Analyser and is able to correct various field issues. Some corrections can be done quite simply; for example, if the fields are only in the wrong order, we just need to exchange upper and lower field. A key functionality is to remove a pull-down generated by the telecine process. After the pull-down removal (reverse telecine) we should have progressive images again. The AFP is able to handle scene changes and pull-down pattern changes automatically if the AFA is delivering the right patterns.

But there are a lot more field issues out there which cannot be handled automatically by any software. And there is no solution on the market which allows an interactive fix by a user using “manual” methods in an easy way. So the other strength of the AFP is that it will allow to manually fix a large variety of field problems.

NOISE REPAIR

The occurrence of noise, digital sensor noise as well as film grain noise, reduces the perceived quality of video and film significantly and lowers the viewing experience of a consumer. In times where Full HD TV (1920 × 1080) is mainstream and Ultra HD TV (3840 × 2160) is upcoming, this becomes even more evident. Furthermore, noise has an adverse influence on efficient storage utilization for long-term preservation as the high-frequency noise components require encoding with a higher bitrate, or when the bitrate is limited (e.g., in a Blu-Ray production or for satellite delivery) noise causes encoding defects (e.g., blocking).

JRS developed within the DAVID project a highly automated, highly scalable noise repair algorithm which allows archives to exploit their content properly—for example, for TV or DVD/Blu-Ray release. Due to the high degree of automation and the good scalability, the method is expected to work for a broad range of archive content and the costs for content exploitation are expected to decrease. We plan to achieve a high degree of automation by adapting the noise repair algorithm automatically to the noise characteristics (noise magnitude, coarseness, signal dependency) occurring in the content.

An initial noise repair algorithm prototype was developed within the first year of DAVID; it utilizes the information from the current frame (or field) and its temporally neighboring frames in order to reduce noise in the current frame. Novel methods have been researched which are applied within the initial noise repair algorithm in order to keep the introduction of artifacts at an absolute minimum. A sample noise repair video is available at the DAVID website.

MXF D10 FILE REPAIR

In 2013 ORF detected in its archives a defective collection of MXF files, which originated from an earlier mass migration project of SD video tapes. In total, about eight thousand hours of video material were affected to a degree that no further usage was possible.

Re-ingestion of the original video tapes was not an option because of cost and time constraints. ORF was looking for a software solution that is capable to repair the damaged files. Because there was none available it was decided to solve that problem within the DAVID project.

ORF sent a representative subset of damaged files to DAVID partner Cube-Tec International for pre-investigation. Cube-Tecs engineers were able to analyse the origin and symptoms of the occurring errors. As a result, a customized repair tool has been developed. This version can now fully automatically detect the occurring errors in different combinations, in the MXF container as well as in the MPEG bitstream. With this new procedure, errors are fixed selectively, and at the same time the bitstream is wrapped into a fully standard-compliant MXF file. Neither reencoding nor rewrapping would have solved these problems. Transcoding was no option; because of the quality reasons, re-ingest would have been the only alternative.

Additionally, an MD5-checksum is created to allow easy independent checks of the file integrity at any time. The progress of the repair process as well as results of the file checks are automatically monitored and reported monthly in consolidated form directly to the ORF. The Cube-Tec team provide the operation and maintenance of this repair service via remote supervision.

Damage Prevention

PRESERVATION METADATA MODEL AND LIBRARY

DAVID has developed a preservation metadata model, which has been contributed to standardisation of the MPEG Multimedia Preservation Application Format (MP-AF).

The scope of the preservation metadata model is to document the history of creation and processing steps applied, as well as their parameters. The model serves as interface between various components that produce preservation-related metadata in different (often proprietary) formats and consuming applications, such as risk management application.

The model represents the preservation actions that were actually applied. The model supports a set of specific types of activities in the model (e.g., digitisation, with possible further specialisations, such as film scan), in order to improve interoperability between preservation systems. It also describes the parameters of these activities.

The model is designed around three main groups of entities: content entities (DigitalItems, their components, and related resources), activities, and operators (agent, tool) and their properties. The content entities are created, used or modified in an activity, which involves operators that contribute to performing the activity. The basic entities of the model and their relations are shown the figure below.

The DAVID preservation metadata model has been contributed to the standardisation process for the MPEG Multimedia Preservation Application Format (MP-AF), expected to be finalised in 2016. More details about the data model can be found in this publication.

In order to handle documents conforming to MP-AF, JRS has implemented a C++ library. With this library application developers are able to create preservation metadata descriptions, manipulate them, serialize to XML, and deserialize—with validation—from XML. Target operating systems are Windows, Linux/Unix, and Mac OS X systems, supporting the 32- and 64-bit versions of the systems respectively. The library has been published under the open-source license LGPL v3, and is available at https://bitbucket.org/wbailer/mpaflib.

PETER SCHALLAUER has worked with Joanneum Research since 1995 as scientific and development coordinator for creating numerous digital video/movie technologies and productive systems. Systems for high-quality digital film restoration (DIAMANT-Film), automatic movie and video content analysis, content description, information mining and content-based retrieval, efficient digitisation and documentation of audiovisual archives, semantic analysis of video, traffic video analysis, and efficient human computer interaction. In recent years he has focused his research on signal-based video and movie quality assessment tools for improving the efficiency of archive digitisation and production processes (VidiCert). He is actively involved in relevant standardisation activities (EBU QC, EBU/AMWA FIMS QA). He coordinated the EC FP7 project DAVID—Digital AV Media Damage Prevention and Repair.

FRANZ HOELLER is the managing director of HS-ART Digital, and the product manager for the DIAMANT-Film Restoration Software. He works as trainer and consultant in the fields of digital film restoration. As project manager he was involved in several international research projects in the digital media area. He has a master’s degree in telematics from the technical university in Graz and has worked as R & D software engineer in the fields of image restoration and processing at Joanneum Research in Austria and Pandora-International in the UK.

Notes

1.  Project website address: www.david-preservation.eu.

2.  http://david-preservation.eu/spotlights/.

3.  http://david-preservation.eu/publications/public-deliverables/.