Stanford InfoLab Publication Server

Index Structures for Information Filtering Under the Vector Space ModelC

Yan, T. and Garcia-Molina, H. (1993) Index Structures for Information Filtering Under the Vector Space ModelC. In: Tenth International Conference on Data Engineering (ICDE 1994), February 14-18, 1994, Houston, Texas.




With the ever increasing volumes of electronic information generation, users of information systems are facing an information overload. It is desirable to support information filtering as a complement to traditional retrieval mechanism. The number of users, and thus profiles (representing users' long-term interhandled by an information filtering system is potentially huge, and the system has to process a constant stream of incoming information in a timely fashion. The effciency of the filtering process is thus an important issue. In this paper, we study what data structures and algorithms can be used to effciently perform large-scale information filtering under the vector space model, a retrieval model established as being effective. We apply the idea of the standard inverted index to index user profiles. We devise an alternative to the standard inverted index, in which we, instead of indexing every term in a profile, select the significant ones to index. We evaluate their performance and show that the indexing methods require orders of magnitude fewer I/Os to process a document than when no index is used. We also show that the proposed alternative performs better in terms of I/O and CPU processing time in many cases.

Item Type:Conference or Workshop Item (Paper)
Subjects:Computer Science
Related URLs:Project Homepage
ID Code:19
Deposited By:Import Account
Deposited On:25 Feb 2000 16:00
Last Modified:05 Feb 2009 16:12

Download statistics

Repository Staff Only: item control page