Search

We describe the first practical algorithms for finding minimal perfect hash functions that have been used to access very large databases (i.e., having over 1 million keys). This method extends earlier work wherein an 0(n-cubed) algorithm was devised, building upon prior work by Sager that described an 0(n-to the fourth) algorithm. Our first linear expected time algorithm makes use of three key insights: applying randomness whereever possible, ordering our search for hash functions based on the degree of the vertices in a graph that represents word dependencies, and viewing hash value assignment in terms of adding circular patterns of related words to a partially filled disk. Our second algorithm builds functions that are slightly more complex, but does not build a word dependency graph and so approaches the theoretical lower bound on function specification size. While ultimately applicable to a wide variety of data and file access needs, these algorithms have already proven useful in aiding our work in improving the performance of CD-ROM systems and our construction of a Large External Network Database (LEND) for semantic networks and hypertext/hypermedia collections. Virginia Disc One includes a demonstration of a minimal perfect hash function running on a PC to access a 130,198 word list on that CD-ROM. Several other microcomputer, minicomputer, and parallel processor versions and applications of our algorithm have also been developed. Tests including those wiht a French word list of 420,878 entries and a library catalog key set with over 3.8 million keys have shown that our methods work with very large databases.

For hash tables in which a strict physical separation exists between primary storage and storage for overflow records, with bucket capacity at least three, a complete probability model is described. A measure of hash table efficiency is introduced, called the table sufficiency index (TSI), and defined as the probability that the overflow space is sufficient assuming that the set of hashed keys has a uniform distribution. The constructed probability model may be used to compute the TSI for hash tables with parameters chosen from a restricted domain. The TSI is advocated as a tool for making decisions about the parameters of small hash tables.

This paper presents pseudochaining as a new collision-resolution method. Pseudochaining is half way between open addressing and chaining. It owes its name to the fact that link fields are present in each cell of the hash table which permits “chaining” of the first overflow items in the table. The efficiency of the method is derived and a tradeoff analysis is given.

A new method for entering and retrieving information in a hash table is described. The method is intended to be efficient if most entries are looked up several times. The expected number of probes to look up an entry, predicted theoretically and verified by Monte Carlo experiments, is considerably less than for other comparable methods if the table is nearly full. An example of a possible Fortran implementation is given.

A method is presented for computing machine independent, minimal perfect hash functions of the form: hash value ← key length + the associated value of the key's first character + the associated value of the key's last character. Such functions allow single probe retrieval from minimally sized tables of identifier lists. Application areas include table lookup for reserved words in compilers and filtering high frequency words in natural language processing. Functions for Pascal's reserved words, Pascal's predefined identifiers, frequently occurring English words, and month abbreviations are presented as examples.

Introduced is a hash coding method based on fixed-point division rather than multiplication or logical operations. This new method allows the hash table to have almost any length. Also a new method of handling collisions is discussed. Known as quadratic search, this method is faster than random search and free from the “clusters” that build up with a linear search.

A new linear search for hash tables whose increment step is a function of the key being addressed is presented. Comparisons with known methods are given, in terms of efficiency and computation complexity. In particular, the new method applies to tables of size n = 2 r . It allows full table searching, and practically eliminates primary clustering at a very low cost.

Secondary clustering as a cause of hash code inefficiency is discussed, and a new hashing method based on its eliminiation is presented. Comparisons with previous methods are made both analytically and empirically.

Previous work on quadratic hash methods is limited mainly to the case where the table size is a prime number. Here, certain results are derived for composite numbers. It is shown that all composite numbers containing at least the square of one of the component primes have full-period integer-coefficient quadratic hash functions.

I would like to comment on an article by Fabrizio Luccio [1] which discusses weighted increment linear search for scatter tables.

The quadratic residue search method for hash tables avoids much of the clustering experienced with a linear search method. The simple quadratic search only accesses half the table. It has been shown that when the length of the table is a prime of the form 4 n + 3, where n is an integer, the whole table may be accessed by two quadratic searches plus a separate access for the original entry point. A search method is presented which is computationally simple, has all the advantages of the quadratic search, and yet accesses all the table in one sweep.

A new method of hash coding is presented and is shown to possess desirable attributes. Specifically, the algorithm is simple, efficient, and exhaustive, while needing little time per probe and using few probes per lookup. Performance data and implementation hints are also given.