Dynamically updating relevance judgements in probabilistic information systems via users’ feedback

A decision maker’s performance relies on the availability of relevant information. In many environments, the relation between the decision maker’s information needs and the information base is complex and uncertain. A fundamental concept of information systems, such as decision support and document retrieval, is the probability that the retrieved information is useful to the decision maker’s query. This paper presents a sequential, Bayesian, probabilistic indexing model that explicitly combines expert opinion with data about the system’s performance. The expert opinion is encoded into probability statements. These statements are modified by the users’ feedback about the relevance of the retrieved information to their queries. The predictive probability that a datum in the information base is applicable to the current query is a logistic function of the expert opinion and the feedback. This feedback enters the computation through a measure of association between the current query-datum pair with previous, relevant query-datum pairs. When this measure is based on the proportional matching of multiple attributes, the predictive probabilities have a recursive formula that makes the model computationally feasible for large information bases.