Back to Search
Start Over
Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies
- Publication Year :
- 2020
-
Abstract
- Consider the situation where a data analyst wishes to carry out an analysis on a given dataset. It is widely recognized that most of the analyst's time will be taken up with \emph{data engineering} tasks such as acquiring, understanding, cleaning and preparing the data. In this paper we provide a description and classification of such tasks into high-levels groups, namely data organization, data quality and feature engineering. We also make available four datasets and example analyses that exhibit a wide variety of these problems, to help encourage the development of tools and techniques to help reduce this burden and push forward research towards the automation or semi-automation of the data engineering process.<br />Comment: 24 pages, 1 figure, submitted to IEEE Transactions on Knowledge and Data Engineering
- Subjects :
- Computer Science - Databases
Subjects
Details
- Database :
- arXiv
- Publication Type :
- Report
- Accession number :
- edsarx.2004.12929
- Document Type :
- Working Paper