Is Medieval Distant Viewing Possible? : Extending and Enriching Annotation of Legacy Image Collections using Visual Analytics

Overview

• The paper describes a method for working with historical image datasets, such as medieval manuscript images, that have conflicting or overlapping metadata.
• The researchers aimed to (1) create a more uniform set of descriptive labels to bridge the combined dataset, and (2) establish a high-quality hierarchical classification that can be used as input for supervised machine learning.
• To achieve these goals, the researchers developed visualization and interaction mechanisms that enable experts to combine, regularize, and extend the vocabulary used to describe these image datasets.

Plain English Explanation

The researchers were working with historical images, such as from medieval manuscripts, that had confusing or contradictory information about what was in the images. Since manually fixing this metadata would be very time-consuming, they wanted to find a way to create a more uniform set of labels to describe the images and organize those labels into a clear hierarchy.

To do this, they developed some interactive visual tools that allowed experts to look at the relationships between the different labels and metadata. This helped the experts combine, organize and expand the vocabulary used to describe the images. The goal was to end up with a high-quality set of labels and categories that could be used to train machine learning models on the historical image data.

Technical Explanation

The paper describes a method for working with historical image datasets, such as medieval manuscript images, that have conflicting or overlapping metadata. The researchers aimed to (1) create a more uniform set of descriptive labels to bridge the combined dataset, and (2) establish a high-quality hierarchical classification that can be used as input for supervised machine learning.

To achieve these goals, the researchers developed visualization and interaction mechanisms that enable experts to combine, regularize, and extend the vocabulary used to describe these image datasets. The visual interfaces provide experts an overview of relationships in the data going beyond the sum total of the metadata. Word and image embeddings as well as co-occurrences of labels across the datasets enable batch re-annotation of images, recommendation of label candidates and support composing a hierarchical classification of labels.

Critical Analysis

The paper addresses an important challenge in working with historical image datasets, where the quality and consistency of metadata can be a significant obstacle. The researchers' approach of leveraging expert knowledge and interactive visualization tools to regularize and extend the vocabulary is a reasonable solution, though the effectiveness would depend on the specific datasets and experts involved.

One potential limitation is the scalability of the approach, as manually reconciling and curating metadata can become prohibitively time-consuming for very large datasets. The researchers do not provide details on the size or complexity of the datasets they worked with, so it's unclear how well the method would scale.

Additionally, the paper does not discuss potential biases or blindspots that may be introduced by relying on expert knowledge. The experts' perspectives and interpretations could influence the final hierarchical classification, which could then impact the performance of any subsequent machine learning models.

Overall, the research presents a thoughtful approach to a challenging problem, but further investigation into the scalability and potential biases of the method would be valuable.

Conclusion

This paper describes a novel approach to working with historical image datasets that have inconsistent or conflicting metadata. By developing interactive visualization tools, the researchers enabled experts to combine, organize, and expand the vocabulary used to describe the images. This resulted in a high-quality hierarchical classification that can be used as input for supervised machine learning models.

While the approach has some limitations in terms of scalability and potential biases, it represents an important step forward in making historical image collections more accessible and useful for research and applications. The techniques developed in this paper could be broadly applicable to other domains grappling with the challenges of working with legacy data.