Essay on Assessing The Effectiveness Of Ir Systems

928 Words Jun 30th, 2016 4 Pages
The evaluation methodologies present a benchmark of test collection to assess the effectiveness of IR systems. The effectiveness measure can be estimated using several metrics to quantify the ability of the retrieval system in providing more relevant documents at cut-off value $k$ of retrieved documents. The evaluation metrics categorised into two classes, which are set-based metrics and position-based metrics. The set-based metrics concern the number of relevant documents whilst position-based approach depends on the position of relevant documents in the evaluation. subsubsection{Set-based Evaluation Metrics} label{setbasedmetrics} The set-based metrics is considered one of the most basic metrics with situations that do not account the relevant documents ' positions into evaluation in the ranked result list. There are three prominent set-based metrics, which are Precision, Recall, and F-measure. The Precision refers to the number of relevant documents to the retrieved documents in the result list. While the Recall denotes to the number of relevant documents in the result list to the whole relevant documents of a specific query. Given the ranked list $R_{q}$ of a query $q$ of retrieved documents and $Rel_{q}$ of the whole relevant documents of the query $q$, the metrics Precision and Recall are defined as follow:

Precision@k = frac{|Rel_{q} cap Ret_{q}^{k}|}{|Ret_{q}^{k}|} qquad Recall@k = frac{|Rel_{q} cap Ret_{q}^{k}|}{|Rel_{q}|} end{equation} where…

Related Documents