Search

Pattern matching for network intrusion/prevention detection requires extremely high throughput with frequent updates to support new attack patterns. Most of current hardware implementations have outstanding performance over software implementations. However, the requirement for dynamic update pattern set is still challenging for hardware researchers. This paper describes a novel FPGA-based pattern matching architecture using a recent hashing algorithm called Cuckoo Hashing. The proposed architecture features on-the-fly pattern updates without reconfiguration, more efficient hardware utilization, and higher performance. Through various algorithmic changes of Cuckoo Hashing, we can implement parallel pattern matching on SRAM-based FPGA. Our system can accommodate the latest Snort rule-set, an open source network intrusion detection/prevention system, and achieve the highest utilization in terms of SRAM per character and logic cells per character at 17 bits/character and 0.043 logic cells/character, respectively on major Xilinx Virtex architectures. Compared to others, ours is much more efficient than any other Xilinx FPGA architectures.

This article addresses the problem of solving the collisions through chaining in the simple uniform hashing. The personal contribution consists of the improvement of the algorithm of dispersion through linking, by modifying the linking schema of the elements of the hash table. A contribution presents solving the collisions by creating combined linked ordered lists for storing the elements that have the same hash value. The code returned by the second hash function applied to the key of each element, is used for ordering the elements of the combined linked list. The proof of the performance of the proposed algorithm consists in the computing of the time needed for the search with and without success and of the comparative analysis of the time of execution of the algorithms.

Bucketized versions of Cuckoo hashing can achieve 95-99% occupancy, without any space overhead for pointers or other structures. However, such methods typically need to consult multiple hash buckets per probe, and have therefore been seen as having worse probe performance than conventional techniques for large tables. We consider workloads typical of database and stream processing, in which keys and payloads are small, and in which a large number of probes are processed in bulk. We show how to improve probe performance by (a) eliminating branch instructions from the probe code, enabling better scheduling and latency-hiding by modern processors, and (b) using SIMD instructions to process multiple keys/payloads in parallel. We show that on modern architectures, probes to a bucketized Cuckoo hash table can be processed much faster than conventional hash table probes, for both small and large memory-resident tables. On a Pentium 4, a probe is two to four times faster, while on the Cell SPE processor a probe is ten times faster.

A Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support membership queries. In recent years, Bloom filters have increased in popularity in database and networking applications. A Bloom filter has two steps that called programming and membership query. In this paper, we introduce a new approach to integrate a hash table with Bloom filter to decrease the hash table access time. This means that when a Bloom filter for an incoming item is programmed, the incoming item simultaneously is stored in a hash table. In addition in the membership query step, if the query is successful, simultaneously the address of item in the hash table is generated. Furthermore, we analyze the average bucket size, maximum search length and number of collisions for the proposed approach and compare to the fast hash table (FHT) approach. We implemented our approach in a software packet classifier based on tuple space search with the $H3$ class of universal hashing functions. Our results show that our approach is able to reduce the average bucket size, maximum search length and number of collisions when compared to a FHT.

Locality Sensitive Hash (LSH) is widely used in peerto- peer (P2P) systems. Although it can support range or similarity queries, it breaks the load balance mechanism of traditional Distributed Hash Table (DHT) based system by replacing consistent hash with LSH. To solve the imbalance problem, current systems either weaken the locality preserve ability from similarity preserved to order preserved or adopt load aware peer join mechanism. The first method does not support similarity query as it loses the similarity information and the second method is greatly affected by the dynamic nature of P2P networks. In this paper, we propose a novel system, cuckoo ring, which can preserve similarity information while load balanced. It does not guide the newly joining peer to the hot areas but move the items in the hot areas to cold areas so that the short life time peers are distributed uniformly across the network instead of being guided to the hot areas. Compared to traditional DHT systems, cuckoo ring only maintains a little more information about the global light load peers and the moved indexed items.

A standard technique from the hashing literature is to use two hash functions h1(x) and h2(x) to simulate additional hash functions of the form gi(x) = h1(x) + i h2(x). We demonstrate that this technique can be usefully applied to Bloom filters and related data structures. Specifically, only two hash functions are necessary to effectively implement a Bloom filter without any loss in the asymptotic false positive probability. This leads to less computation and potentially less need for randomness in practice.

In typical applications of hashing algorithms the amount of data to be stored is often too large to fit into internal memory. In this case it is desirable to find the data with as few as possible non-consecutive or at least non-oblivious probes into external memory. Extending a static scheme of Pagh [11] we obtain new randomized algorithms for maintaining hash tables, where a hash function can be evaluated in constant time and by probing only one external memory cell or O(1) consecutive external memory cells. We describe a dynamic version of Pagh's hashing scheme achieving 100% table utilization but requiring (2+e)nlogn space for the hash function encoding as well as (3+e)nlogn space for the auxiliary data structure. Update operations are possible in expected constant amortized time. Then we show how to reduce the space for the hash function encoding and the auxiliary data structure to O(nloglogn). We achieve 100% utilization in the static version (and thus a minimal perfect hash function) and 1–e utilization in the dynamic case.

Hashing is one of the fundamental techniques used to implement query processing operators such as grouping, aggregation and join. This paper studies the interaction between modern computer architecture and hash-based query processing techniques. First, we focus on extracting maximum hashing performance from super-scalar CPUs. In particular, we discuss fast hash functions, ways to efficiently handle multi-column keys and propose the use of a recently introduced hashing scheme called Cuckoo Hashing over the commonly used bucket-chained hashing. In the second part of the paper, we focus on the CPU cache usage, by dynamically partitioning data streams such that the partial hash tables fit in the CPU cache. Conventional partitioning works as a separate preparatory phase, forcing materialization, which may require I/O if the stream does not fit in RAM. We introduce best-effort partitioning, a technique that interleaves partitioning with execution of hash-based query processing operators and avoids I/O. In the process, we show how to prevent issues in partitioning with cacheline alignment, that can strongly decrease throughput. We also demonstrate overall query processing performance when both CPU-efficient hashing and best-effort partitioning are combined.

For applications that rely on large databases as the core data structure, the need for a fast search process is essential. Hashing algorithms have widely been adopted as the search algorithm of choice for fast lookups. Hashing algorithms involve the creation of hash values from the target database entries. A hashing algorithm that transforms the database to hash values with a distribution as uniform as possible would lead to a better search performance. When a database is already value-wise uniformly distributed, any regular hashing algorithm, such as bit-extraction, group-XOR, etc., will lead to a statistically perfect hashing result. In almost all known practical applications, the target database rarely demonstrates uniformly distributed characteristic. The use of any known regular hashing algorithm can lead to a performance far less than desirable. This paper aims at designing a hashing algorithm that can deliver a better performance for all practical databases. An analytical preprocess is performed on the original database to extract critical information that would significantly benefit the design of a better hashing algorithm. The process includes sorting database hash bits to provide a priority that would facilitate the decision-making on which bits and how these bits should be combined to generate better hash values. The algorithm follows an ad hoc design that is critical to adapting to real-time situation when there exists a changing database with an irregular non-uniform distribution. The proposed technique clearly outperforms all known regular hashing algorithms by a significant margin.

The aim of this paper is to extend the analysis of Cuckoo Hashing of Devroye and Morin in 2003. In particular we make several asymptotic results much more precise. We show, that the probability that the construction of a hash table succeeds, is asymptotically $1-c(\varepsilon)/m+O(1/m^2)$ for some explicit $c(\varepsilon)$, where $m$ denotes the size of each of the two tables, $n=m(1- \varepsilon)$ is the number of keys and $\varepsilon \in (0,1)$. The analysis rests on a generating function approach to the so called Cuckoo Graph, a random bipartite graph. We apply a double saddle point method to obtain asymptotic results covering tree sizes, the number of cycles and the probability that no complex component occurs.

Flow state tables are an essential component for improving the performance of packet classification in network security and traffic management. Generally, a hash table is used to store the state of each flow due to its fast lookup speed. However, hash table collisions can severely reduce the effectiveness of packet classification using a flow state table. In this paper, we propose three schemes to reduce hash collisions by exploiting the locality in traffic. Our experiments show that all our proposed schemes perform better than the standard practice of hashing with overflow chains. More importantly, our move and insert to front scheme is insensitive to the hash table size.

Hashing is an important tool in randomized algorithms, with applications in such diverse fields including information retrieval, data mining, cryptology and parallel algorithms. However, the worst case behavior of a regular hash-based searching is O(n). Perfect hashing is a solution to this problem that offers a worst case performance of O(1) only for the static key set. In this paper we have proposed an adaptive hashing scheme that works on dynamic key sets and still enables keys to be searched in constant time. It has been further established that, if the hash functions are carefully chosen, then the space requirement of the hash structure is O(n).

We describe a simple randomized construction for generating pairs of hash functions h_1,h_2 from a universe U to ranges V=[m]={0,1,...,m-1} and W=[m] so that for every key set S\subseteq U with n=|S

The author presents an exact probability model for finite hash tables and applies the model to solve a few problems in the analysis of hashing techniques. The model enables exact computation of table sufficiency index, a parameter useful in the design of small hash tables. The author also presents an exact analysis of the expected length of the longest probe sequence in hashing with separate chaining, and successful search length in infinite uniform hashing giving explicit expressions. It appears that the model can be extended to analyze other hashing schemes such as bounded disorder index method, and to problems in robust data structures, etc. >

We generalize Cuckoo Hashing [16] to d-ary Cuckoo Hashing and show how this yields a simple hash table data structure that stores n elements in (1 + ?) n memory cells, for any constant ? > 0. Assuming uniform hashing, accessing or deleting table entries takes at most d = O(ln 1/?) probes and the expected amortized insertion time is constant. This is the first dictionary that has worst case constant access time and expected constant update time, works with (1+?) n space, and supports satellite information. Experiments indicate that d = 4 choices suffice for ? ? 0.03. We also describe a hash table data structure using explicit constant time hash functions, using at most d = O(ln2 1/?) probes in the worst case.A corollary is an expected linear time algorithm for finding maximum cardinality matchings in a rather natural model of sparse random bipartite graphs.

Perfect hashing functions are determined that are suitable for hardware implementations. A trial-and-error method of finding perfect hashing functions is proposed using a simple universal/sub 2/ class (H/sub 3/) of hashing functions. The results show that the relative frequency of perfect hashing functions within the class H/sub 3/ is the same as predicted by the analysis for the set of all functions. Extensions of the basic scheme can handle dynamic key sets and large key sets. Perfect hashing functions can be found using software, and then loaded into the hardware hash address generator. Inexpensive associative memory can be used as a general memory construct offered by the system services of high-performance (super) computers. It has a potential application for storing operating system tables or internal tables for software development tools, such as compilers, assemblers and linkers. Perfect hashing in hardware may find a number of other applications, such as high speed event counting and text searching. >

We present a simple dictionary with worst case constant lookup time, equaling the theoretical performance of the classic dynamic perfect hashing scheme of Dietzfelbinger et al. [SIAM J. Comput. 23 (4) (1994) 738-761]. The space usage is similar to that of binary search trees. Besides being conceptually much simpler than previous dynamic dictionaries with worst case constant lookup time, our data structure is interesting in that it does not use perfect hashing, but rather a variant of open addressing where keys can be moved back in their probe sequences. An implementation inspired by our algorithm, but using weaker hash functions, is found to be quite practical. It is competitive with the best known dictionaries having an average case (but no nontrivial worst case) guarantee on lookup time.

The performance of hash tables is analyzed in a parallel context. Assuming that a hash table of fixed size is allocated in the shared memory of a PRAM with n processors, a Ph-step is defined as a PRAM computation, in which each processor searches or inserts a key in the table. It is shown that, in the case of uniform hashing, the average duration of a Ph-step for all insertions (worst case) in Ω(log1αn) and O(log1α′n), where α and α′ are the load factors of the table before and after the execution of the Ph-step.

This paper compares the parameters sizes and software performance of several recent constructions for universal hash functions: bucket hashing, polynomial hashing, Toeplitz hashing, division hashing, evaluation hashing, and MMH hashing. An objective comparison between these widely varying approaches is achieved by defining constructions that offer a comparable security level. It is also demonstrated how the security of these constructions compares favorably to existing MAC algorithms, the security of which is less understood. ispartof: pages:24-41 ispartof: Lecture Notes in Computer Science vol:1592 pages:24-41 ispartof: EUROCRYPT 1999 location:CZECH REPUBLIC, PRAGUE date:2 May - 6 May 1999 status: published

In this talk we will study the optimal solution to a class of random assignment problems with dependent weights. The result is then used to show that, in double hashing, the expected retrieval cost with respect to an optimal static hash table is O(1) even if the table is full. This confirms a conjecture of Gonnet and Munro (SIAM J. on Computing 8 (1979), 463–478).

Linear hashing and spiral storage are two dynamic hashing schemes originally designed for external files. This paper shows how to adapt these two methods for hash tables stored in main memory. The necessary data structures and algorithms are described, the expected performance is analyzed mathematically, and actual execution times are obtained and compared with alternative techniques. Linear hashing is found to be both faster and easier to implement than spiral storage. Two alternative techniques are considered: a simple unbalanced binary tree and double hashing with periodic rehashing into a larger table. The retrieval time of linear hashing is similar to double hashing and substantially faster than a binary tree, except for very small trees. The loading times of double hashing (with periodic reorganization), a binary tree, and linear hashing are similar. Overall, linear hashing is a simple and efficient technique for applications where the cardinality of the key set is not known in advance.

We discuss the problem of hashing in a full or nearly full table using open addressing. A scheme for reordering the table as new elements are added is presented. Under the assumption of having a reasonable hash function sequence, it is shown that, even with a full table, only about 2.13 probes will be required, on the average, to access an element. This scheme has the advantage that the expected time for adding a new element is proportional to that required to determine that an element is not in the table. Attention is then turned to the optimal reordering scheme and the minimax problem of ordering the table so as to minimize the length of the longest probe sequence to find any element. Both arranging problems can be translated to assignment problems. A unified algorithm is presented for these, together with the first method suggested. A number of simulation results are reported, the most interesting being an indication that the optimal reordering scheme will lead to an average of about 1.83 probes per se...

Two parallel hashing algorithms based on open and chained hashing are discussed. The parallel chained hashing algorithm is efficient even in the environment where deletion and insertion frequently occur. The chained hashing memory with fewer memory banks has a performance equivalent to that of the open hashing memory.

When the space allocation for a hash-coded table is altered, the table entries must be rescattered over the new space. A technique for accomplishing this rescattering is presented. The technique is independent of both the length of the table and the hashing function used, and can be utilized in conjunction with a linear reallocation of the table being rescattered. Moreover, it can be used to eliminate previously flagged deletions from any hash-coded table, or to change from one hashing method to another. The efficiency of the technique is discussed and theoretical statistics are given.

In a recent paper [5] de Villiers and Wilson discuss hashing functions in connection with chained hash storage of sparse matrices. This short communication expands upon some points raised in the cited paper and gives a brief comparison of the storage requirements of open and chained hash tables in this context.

We discuss the problem of hashing in a full or nearly full table using open addressing. A scheme for reordering the table as new elements are added is presented. Under the assumption of having a reasonable hash function sequence, it is shown that even with a full table only about 2.13 probes will be required, on the average, to access an element. This scheme has the advantage that the expected time for adding a new element is proportional to that required to determine that an element is not in the table. Attention is then turned to the optimal reordering scheme (which is a maximum flow problem) and the minimax problem of arranging the table so as to minimize the length of the longest probe sequence to find any element. A unified algorithm is presented for both of these as well as the first method suggested. A number of simulation results are reported, the most interesting being an indication that the optimal reordering scheme will lead to an average of about 1.83 probes per search in a full table.

In this paper procedures are studied for storing, accessing, updating, and reorganizing data in large files whose organization is direct, an organization used when a fast response time is required. "Distribution-dependent" hashing functions and the division method are compared as methods of indirect addressing."Distribution-dependent" hashing functions are characterized. These hashing functions generate addresses from a set of keys by using knowledge of the distribution of that key set within the key space or range of keys. A study of the performance measures obtained during tests of these functions on several key sets indicates that in certain cases, distribution-dependent methods perform better than the division method. This result is extended by a demonstration that distribution-dependent hashing functions can accommodate a change in the distribution of keys without being redefined. A number of insertions to and deletions from the key set can be made before a distribution-dependent hashing function gives poorer performance than the division method under identical circumstances.If many additions are made to a set of keys, it becomes necessary to reorganize, in a larger storage area, the direct file of records identified by that key set. Although processor time must be sacrificed in order to redefine a distribution-dependent hashing function, the division method requires substantially greater access time in a reorganizational situation.

Recent technological developments in cloud computing and the ensuing commercial appeal have encouraged companies and individuals to outsource their storage and computations to powerful cloud servers. However, the challenge when outsourcing data and computation is to ensure that the cloud servers comply with their advertised policies. In this paper, we focus in particular on the scenario where a data owner wishes to (i) outsource its public database to a cloud server; (ii) enable anyone to submit multi-keyword search queries to the outsourced database; and (iii) ensure that anyone can verify the correctness of the server's responses. To meet these requirements, we propose a solution that builds upon the well-established techniques of Cuckoo hashing, polynomial-based accumulators and Merkle trees. The key idea is to (i) build an efficient index for the keywords in the database using Cuckoo hashing; (ii) authenticate the resulting index using polynomial-based accumulators and Merkle tree; (iii) and finally, use the root of the Merkle tree to verify the correctness of the server's responses. Thus, the proposed solution yields efficient search and verification and incurs a constant storage at the data owner. Furthermore, we show that it is sound under the strong bilinear Diffie-Hellman assumption and the security of Merkle trees.

Applications such as large-scale sparse linear algebra and graph analytics are challenging to accelerate on FPGAs due to the short irregular memory accesses, resulting in low cache hit rates. Nonblocking caches reduce the bandwidth required by misses by requesting each cache line only once, even when there are multiple misses corresponding to it. However, such reuse mechanism is traditionally implemented using an associative lookup. This limits the number of misses that are considered for reuse to a few tens, at most. In this article, we present an efficient pipeline that can process and store thousands of outstanding misses in cuckoo hash tables in on-chip SRAM with minimal stalls. This brings the same bandwidth advantage as a larger cache for a fraction of the area budget, because outstanding misses do not need a data array, which can significantly speed up irregular memory-bound latency-insensitive applications. In addition, we extend nonblocking caches to generate variable-length bursts to memory, which increases the bandwidth delivered by DRAMs and their controllers. The resulting miss-optimized memory system provides up to 25% speedup with 24× area reduction on 15 large sparse matrix-vector multiplication benchmarks evaluated on an embedded and a datacenter FPGA system.

Searchable symmetric encryption (SSE) has been studied extensively for its full potential in enabling exact-match queries on encrypted records. Yet, situations for similarity queries remain to be fully explored. In this paper, we design privacy-assured similarity search schemes over millions of encrypted high-dimensional records. Our design employs locality-sensitive hashing (LSH) and SSE, where the LSH hash values of records are treated as keywords fed into the framework of SSE. As direct combination of the two does not facilitate a scalable solution for large datasets, we then leverage a set of advanced hash-based algorithms including multiple-choice hashing, open addressing, and cuckoo hashing, and craft a high performance encrypted index from the ground up. It is not only space efficient, but supports secure and sufficiently accurate similarity search with constant time. Our designs are proved to be secure against adaptive adversaries. The experiment on 10 million encrypted records demonstrates that our designs function in a practical manner.

Secondary clustering as a cause of hash code inefficiency is discussed, and a new hashing method based on its eliminiation is presented. Comparisons with previous methods are made both analytically and empirically.

A new method of hash coding is presented and is shown to possess desirable attributes. Specifically, the algorithm is simple, efficient, and exhaustive, while needing little time per probe and using few probes per lookup. Performance data and implementation hints are also given.