Panel 4: Metadata Creation, Discovery, and AI Applications

This report examines knowledge organization involved in the process of digital humanities research, using " The Secret Societies in Qing China: Archival Studies and Digital Humanities" project as a case study. The study will focus on the following aspects: (1) Transform the preliminary research questions of historians into competency questions that are machine-processible and analyzable. Further develop the domain ontology, as well as develop a SPARQL query template for use in Linked Open Data. (2) Define Qing dynasty official documents as a specific genre and deconstructing important features of this type of textual content from various perspectives such as form, content, and function. This serves as the design foundation for metadata format, authority files and ontology. (3) Explore ways for human-machine collaboration by using tools such as the text analysis system and ChatGPT, to assist in the NER (Named Entity Recognition) of related subjects, properties and objects within the archival content, as well as the extraction of roles of semantic relationships between entities. This can also help to strengthen the task of automatic metadata construction to be able to identify and integrate entity information from different archival sources.

In AI/ML, data serves as the raw material from which models learn and make predictions. Training AI/ML models demands substantial quantities of high-quality data. Regardless of the algorithm's sophistication, realizing its full potential hinges on the presence of a robust data infrastructure. This data infrastructure is composed of AI/ML datasets. An AI/ML dataset is a collection of data used to train and evaluate artificial intelligence (AI) and machine learning (ML) algorithms. For AI/ML projects, datasets may be built from scratch, derived from large datasets, such as library collections, or reused from existing pre-packaged datasets. Effectively managing them is crucial to ensure that AI and ML models are trained on high-quality, representative data, which is essential for achieving accurate and reliable results in AI/ML applications.

An AI/ML-friendly metadata architecture is critical for effective dataset management. An AI/ML friendly metadata architecture needs to meet the following basic technical requirements: 1) Furnishing structured information about data content, sources, and context to enhance dataset discovery; 2) Precisely specifying the dataset's size; 3) Enumerating and defining labels comprehensively; 4) Clearly identifying data types; 5) Offering comprehensive insights into data preprocessing, cleaning, and transformation procedures; 6) Delivering in-depth details about data scope, including contents, timeframes, and geographic coverage.

A practical use case will be presented for developing a metadata solution for a dataset that is used in an AI/ML-powered project for detecting illustration objects within Chinese rare books.

As powerful and useful as AI and ML tools, there are risks in applying them if they are not sufficiently explainable, robust, transparent, or not stick to the fairness and privacy principles, which can seriously jeopardize the trustworthiness of AI. Metadata play an important role in capturing and documenting the data (input & output), parameters, models, environment configurations, and versions of data and models. All of these contribute to the trustworthiness of AI. This presentation reviews the requirements for trustworthy AI as well as metadata categories that have been used in representing trustworthy AI. Through examples, issues and challenges will be discussed for AI applications in metadata generation/creation.