Search

Although data reusers request information about how research data was created and curated, this information is often non-existent or only briefly covered in data descriptions. The need for such contextual information is particularly critical in fields like archaeology, where old legacy data created during different time periods and through varying methodological framings and fieldwork documentation practices retains its value as an important information source. This article explores the presence of contextual information in archaeological data with a specific focus on data provenance and processing information, i.e., paradata. The purpose of the article is to identify and explicate types of paradata in field observation documentation. The method used is an explorative close reading of field data from an archaeological excavation enriched with geographical metadata. The analysis covers technical and epistemological challenges and opportunities in paradata identification, and discusses the possibility of using identified paradata in data descriptions and for data reliability assessments. Results show that it is possible to identify both knowledge organisation paradata (KOP) relating to data structuring and knowledge-making paradata (KMP) relating to fieldwork methods and interpretative processes. However, while the data contains many traces of the research process, there is an uneven and, in some categories, low level of structure and systematicity that complicates automated metadata and paradata identification and extraction. The results show a need to broaden the understanding of how structure and systematicity are used and how they impact research data in archaeology and in comparable field sciences. The insights into how a dataset’s KOP and KMP can be read is also a methodological contribution to data literacy research and practice development. On a repository level, the results underline the need to include paradata about dataset creation, purpose, terminology, d, URDAR Riksbankens Jubileumsfond (In19-0135:1)

Metadata extraction is the process of describing extrinsic and intrinsic qualities of the resource such as document, image, video, including getting data from references. References form an essential part of electronic scholarly publications. A reference is the way of giving acknowledgment to individuals for their creative and intellectual works that one utilized in his or her research work. It can also be used to locate particular sources and combat plagiarism. A reference style dictates the information necessary for a reference and how the information is ordered. Accurate and automatic reference metadata generation provides scalability, interoperability and usability for digital libraries of both public and private institution and their collections. Accurate reference metadata extraction becomes an intriguing task to researchers who want to collect data of scientific publications; therefore, this research work proposes a metadata extraction from references of different styles with the use of regular expression. This work accurately extract metadata such as author, title of article, volume, year of publication and institution from references of different styles limiting it to six referencing style.

Electronic Theses and Dissertations (ETDs) contain domain knowledge that can be used for many digital library tasks, such as analyzing citation networks and predicting research trends. Automatic metadata extraction is important to build scalable digital library search engines. Most existing methods are designed for born-digital documents such as GROBID, CERMINE, and ParsCit, so they often fail to extract metadata from scanned documents such as for ETDs. Traditional sequence tagging methods mainly rely on text-based features. In this paper, we propose a conditional random field (CRF) model that combines text-based and visual features. To verify the robustness of our model, we extended an existing corpus and created a new ground truth corpus consisting of 500 ETD cover pages with human validated metadata. Our experiments show that CRF with visual features outperformed both a heuristic baseline and a CRF model with only text-based features. The proposed model achieved 81.3%-96% F1 measure on seven metadata fields. The data and source code are publicly available on Google Drive1 and a GitHub repository2.

The class ``CS 5604: Information Storage and Retrieval'' in the fall of 2019 is divided into six teams to enhance the usability of the corpus of electronic theses and dissertations maintained by Virginia Tech University Libraries. The ETD corpus consists of 14,055 doctoral dissertations and 19,246 masters theses from Virginia Tech University Libraries’ VTechWorks system. Our study explored document collection and processing, application of Elasticsearch to the collection to facilitate searching, testing a custom front-end, Kibana, integration, implementation, text analytics, and machine learning. The result of our work would help future researchers study the natural language processed data using deep learning technologies, address the challenges of extracting information from ETDs, etc. The Collection Management of Electronic Theses and Dissertations (CME) team was responsible for processing all PDF files from the ETD corpus and extracting well-formatted text files from them. We also used advanced deep learning and other tools like GROBID to process metadata, obtain text documents, and generate chapter-wise data. In this project, the CME team completed the following steps: comparing different parsers; doing document segmentation; preprocessing the data; and specifying, extracting, and preparing metadata and auxiliary information for indexing. We finally developed a system that automates all the above-mentioned tasks. The system also validates the output metadata, thereby ensuring the correctness of the data that flows through the entire system developed by the class. This system, in turn, helps to ingest new documents into Elasticsearch.

Consumption of tobacco causes health issues, both mental and physical. Despite this widely known fact, tobacco companies had sustained their huge presence in the market over the past century owing to a variety of successful marketing strategies. This report documents the work of the Collection Management Tobacco Settlement Documents (CMT) team, the data ingestion team for the tobacco documents. We deal with an archive of tobacco documents that were produced during litigation between the United States and seven major tobacco industry organizations. Our aim is to process these documents and assist Dr. David M. Townsend, an assistant professor at Virginia Polytechnic Institute and State University (Virginia Tech) Pamplin College of Business, in his research towards understanding the marketing strategies of the tobacco companies. The team is part of a larger initiative: to build a state-of-the-art information retrieval and analysis system. We handle over 14 million tobacco settlement documents as part of this project. Our tasks include extracting the data as well as metadata from these documents. We cater to the needs of the ElasticSearch (ELS) team and the Text Analytics and Machine Learning (TML) team. We provide tobacco settlement data in suitable formats to enable them to process and feed the data into the information retrieval system. We have successfully processed both the metadata and the document texts into a usable format. For metadata, this involved collaborating with the above-mentioned teams to come up with a suitable format. We retrieved the metadata from a MySQL database and converted it into a JSON for Elasticsearch ingestion. For the data, this involved lemmatization, tokenization, and text cleaning. We have supplied the entire dataset to the ELS and TML teams. Data, as well as metadata of these documents, were cleaned and provided. Python scripts were used to query the database and output the results in the required format. We also closely interacted wi

This project deals with the implementation of a metadata visualisation tool for the neural network developed by Ugiat Technologies. As it is a video processing software, the implemented tool is a player with extra features that allows to check if the metadata provided by the neural network is right. It has been done using web technologies for better portability, mainly in JavaScript. This way almost any device can use it, but it is created for desktop use, no matter the operating system, since it runs in a web browser, but the screen resolution should be the one of a laptop or a desktop computer, no mobile usage is covered by the design. For a better user experience, it allows voice commands to control some basic functions like playing, pausing and stopping the video as well as for enabling and disabling some of the extra features added to the video player. All the implementation is completely open source and it allows to continue adding functionalities. The product obtained is good not just for helping the development and tests of the processing software, but also for making demos of its performance.

With the increase of online education, a good description of learning resources has become vital for educational resource sharing and reuse. Resource description has been under the spotlight in recent years. Educational platforms can benefit from good resource organisation and description, thereby providing a higher quality of services and attracting more learners to use their systems. Furthermore, well-described resources with metadata, promote content sharing and re-use. This work starts with an extensive literature research on metadata generation techniques and breaks the findings down to metadata types. A detailed taxonomy of metadata types, based on this research, is provided. The taxonomy takes into account properties common to these types. Second, this work analyzes the state-of-the-art metadata collection techniques in literature and real-world educational content repositories including a showcase with the TUDelft library, in order to estimate the gap of metadata employment in the field of education. Following the results of this research and based on the observation that similar steps are often performed together, a set of easy-to-follow and generic enough design patters for generating metadata was identified. These design patterns aim at assisting content authors or data professionals with filling in metadata and thereafter, allowing for feature development or improvement in the respective platforms. The patterns for metadata extraction are based on the identified taxonomy of metadata. Finally, semantic metadata is extracted as proof of concept for two of the proposed patterns. A satisfactory to a high-quality result was achieved, showing that the patterns are intuitive and the data extracted with them, can be potentially used to describe the respective Educational Resource (ER) by adding the extracted information to its metadata., Computer Science | Web Information Systems

This quarter, we re-designed the Shape-DNA based rule line cleaning algorithm to minimize the degradation of the shape of text characters. Recall that in the Shape-DNA based cleaning approach, the projection onto the Shape-DNA space produces a rule line distance image that is used to clean the rule lines. However, this cleaning process can and does remove portions of legitimate text characters that resemble rule lines. Therefore, instead of using the rule line distance images for directly cleaning rule lines, we now use this image to model the rule lines present in the document. Specifically, by applying Hough transform to the rule line distance image, we compute a set of model parameters. In addition, we estimate the average thickness of the rule lines using the original input image. Finally, we use both the rule line model parameters and the rule line thickness information with a sliding window to clean the rule lines. Figure 2 shows an example where the performance of the new rule line cleaning algorithm is compared with the performance of the previous version of the shape-DNA cleaning. This reporting period, we also improved the restoration algorithm for removing the artifacts introduced by rule line cleaning. Similar to rule line cleaning algorithm, Shape-DNA based restoration algorithm also includes an off-line training process, where text characters shapes are learned off-line by training about 100 handwritten text images (with no rule lines) and a Shape-DNA database is computed from the shape patterns. These shape blocks from the input image onto the database and by searching for the closest shape pattern in the database. Unlike our previous version, where shape-DNA restoration was applied to entire image, we now use the estimated rule line model parameters to constrain the restoration into the local proximity of detected rule lines., Program Code 7M30. The original document contains color images. All DTIC reproductions will be in black and white.

The objective of MADCAT is to produce a robust, highly accurate transcription engine that ingests documents of multiple types and produces English transcriptions of their content. For addressing the technical challenges implicit in that goal, the BBN-led team proposed a system that embodies integration of five major operations: (1) pre-processing and image enhancement, (2) page segmentation, (3) text recognition, and (4) metadata extraction. In Phase 1 of the MADCAT effort, we made significant improvements in all the above areas. In addition, we developed an end-to-end system for processing the Phase 1 evaluation data. The evaluation system exceeded the Phase 1 program goal of 40% accuracy on 70% of the documents. Below, we summarize the work performed by the BBN-led team in Phase 1 of the MADCAT effort. We highlight our accomplishments by each technical area and also indicate the performers in that area., Program Code 7M30. The original document contains color images. All DTIC reproductions will be in black and white.

Metadata is data that describes data. There are many computer forensic uses of metadata and being able to extract metadata automatically provides positive forensic implications. This thesis presents a new technique for batch processing disk images and automatically extracting metadata from files and file contents. The technique is embodied in a program called fiwalk that has a plug-in architecture allowing new metadata extractors to be readily incorporated. Output from fiwalk can be provided in multiple formats such as ARFF and text. The plug-ins created for this thesis include one created by Simson Garfinkel for extracting metadata from .jpeg files, two for Microsoft Office documents (one for prior to Office 2007 release and one for Office 2007 release), and a default plug-in for extracting metadata from .gif, .pdf, and .mp3 files. To better understand the metadata available in common file formats such as .doc, .docx, .odt, .pdf, .mp3, .mp4, .jpeg, .tiff, and .gif, an examination of these formats is provided.

This paper is concerned with 'intranet search'. By intranet search, we mean searching for information on an intranet within an organization. We have found that search needs on an intranet can be categorized into types, through an analysis of survey results and an analysis of search log data. The types include searching for definitions, persons, experts, and homepages. Traditional information retrieval only focuses on search of relevant documents, but not on search of special types of information. We propose a new approach to intranet search in which we search for information in each of the special types, in addition to the traditional relevance search. Information extraction technologies can play key roles in such kind of 'search by type' approach, because we must first extract from the documents the necessary information in each type. We have developed an intranet search system called 'Information Desk'. In the system, we try to address the most important types of search first - finding term definitions, homepages of groups or topics, employees' personal information and experts on topics. For each type of search, we use information extraction technologies to extract, fuse, and summarize information in advance. The system is in operation on the intranet of Microsoft and receives accesses from about 500 employees per month. Feedbacks from users and system logs show that users consider the approach useful and the system can really help people to find information. This paper describes the architecture, features, component technologies, and evaluation results of the system. Copyright 2005 ACM.

Práce porovnává dostupné vědecké vyhledávače se skriptem pro extrakci metadat z vědeckých prací vyvinutým Tomášem Lokajem na FIT VUT v Brně. Výsledky potvrzují nedostatky v extrakci metadat. Tato bakalářská práce zároveň představuje ucelený návod na porovnání různých informací., Work compares accessible scientific locater and program for extraction metadata from scientific papers created by Tomáš Lokaj on FIT BUT. Results affirm imperfections in extraction metadata. This bachelor thesis introduce integral manual for comparing various informations.

Práce porovnává dostupné vědecké vyhledávače se skriptem pro extrakci metadat z vědeckých prací vyvinutým Tomášem Lokajem na FIT VUT v Brně. Výsledky potvrzují nedostatky v extrakci metadat. Tato bakalářská práce zároveň představuje ucelený návod na porovnání různých informací., Work compares accessible scientific locater and program for extraction metadata from scientific papers created by Tomáš Lokaj on FIT BUT. Results affirm imperfections in extraction metadata. This bachelor thesis introduce integral manual for comparing various informations.

This paper presents a system of services for the automated processing of collections of scientific documents. These services provide verification of document compliance to the accepted rules of formation of collections and their conversion to the established formats; structural analysis of documents and extraction of metadata, as well as their integration into the scientific information space. The system allows to automatically perform a set of operations that cannot be realized for acceptable time with the traditional manual processing of electronic content. It is designed for the large collections of scientific documents.

© Springer International Publishing Switzerland 2014. The paper discusses the approach to automate the processing of electronic mathematical documents and their transformation into semantic documents. Structuring electronic storage of periodical issues in mathematics and multi-volume works conferences was performed.

© Springer International Publishing Switzerland 2014. The paper discusses the approach to automate the processing of electronic mathematical documents and their transformation into semantic documents. Structuring electronic storage of periodical issues in mathematics and multi-volume works conferences was performed.

Práce porovnává dostupné vědecké vyhledávače se skriptem pro extrakci metadat z vědeckých prací vyvinutým Tomášem Lokajem na FIT VUT v Brně. Výsledky potvrzují nedostatky v extrakci metadat. Tato bakalářská práce zároveň představuje ucelený návod na porovnání různých informací., Work compares accessible scientific locater and program for extraction metadata from scientific papers created by Tomáš Lokaj on FIT BUT. Results affirm imperfections in extraction metadata. This bachelor thesis introduce integral manual for comparing various informations.

Práce porovnává dostupné vědecké vyhledávače se skriptem pro extrakci metadat z vědeckých prací vyvinutým Tomášem Lokajem na FIT VUT v Brně. Výsledky potvrzují nedostatky v extrakci metadat. Tato bakalářská práce zároveň představuje ucelený návod na porovnání různých informací., Work compares accessible scientific locater and program for extraction metadata from scientific papers created by Tomáš Lokaj on FIT BUT. Results affirm imperfections in extraction metadata. This bachelor thesis introduce integral manual for comparing various informations.

Práce porovnává dostupné vědecké vyhledávače se skriptem pro extrakci metadat z vědeckých prací vyvinutým Tomášem Lokajem na FIT VUT v Brně. Výsledky potvrzují nedostatky v extrakci metadat. Tato bakalářská práce zároveň představuje ucelený návod na porovnání různých informací., Work compares accessible scientific locater and program for extraction metadata from scientific papers created by Tomáš Lokaj on FIT BUT. Results affirm imperfections in extraction metadata. This bachelor thesis introduce integral manual for comparing various informations.