Web business intelligence: Mining the web for actionable knowledge

It is estimated that over seven billion static pages exist in the Web today, and backend databases can potentially produce at least three times as many dynamic pages. However, the best search engines index only approximately 20% of the static pages. So the real question is: While the Web is certainly the most amazing and comprehensive information source ever created, are you really getting all the information you need for your specific purpose? The answer to this question is mostly ‘yes’ for the individual user, who uses the Web as an information source for casual purposes. However, for an individual who uses the Web as an essential and comprehensive source of information – for business or research – the answer is quite the opposite. Even a sophisticated Web user requires a significant amount of time and effort to find all of the information needed for a given task. In this paper the concept of Web Business Intelligence (WBI) is introduced, an emerging class of software that leverages the unprecedented content on the Web to extact actionable knowledge in an organizational setting. The contributions include an architecture for WBI, a survey of technologies relevant to the various components of the architecture, and illustration of the value of WBI by means of a detailed example from the e-finance domain. This article concludes with a discussion on the future of WBI.