BioKleisli: a digital library for biomedical researchers.

Data of interest to biomedical researchers associated with the Human Genome Project (HGP) is stored all over the world in a number of different electronic data formats and accessible through a variety of interfaces and retrieval languages. These data sources include conventional relational databases with SQL interfaces, formatted text files on top of which indexing is provided for efficient retrieval (ASN.1-Entrez), and binary files that can be interpreted textually or graphically via special purpose interfaces (ACeDB). Researchers within the HGP want to combine data from these different data sources, add value through sophisticated data analysis techniques (such as the biosequence comparison software BLAST and FASTA), and view it using special purpose scientific visualization tools. However, currently there are no commercial tools for enabling such an integrated digital library, and a fundamental barrier to developing such tools appears to be one of language design and optimization: The data formats and software packages found throughout the HGP contain a number of data types not available in conventional databases, such as lists, variants and arrays; furthermore, these types may be deeply nested. We present in this paper a framework for providing read access to multiple data sources with complex structured data, and illustrate its use in an application called BioKleisli which accesses data sources critical to the HGP. The three primary components of this framework are: (1) a powerful language for querying and transforming complex structured data; (2) an extensible architecture for implementing the query primitives; and (3) optimization techniques that extend many known techniques to these more complex data types. [ABSTRACT FROM AUTHOR]