Automatic Evaluation of Extract Summaries Using Fuzzy F-Score Measure

Ravindra, G and Balakrishnan, N and Ramakrishnan, KR (2004) Automatic Evaluation of Extract Summaries Using Fuzzy F-Score Measure. In: Fifth International Conference on Knowledge Based Computer Systems (KBCS 2004), Dec 19-22, 2004, Hyderabad, India, pp. 487-497.

Preview	Postscript paperid-40.ps Download (210kB)
Preview	PDF Automatic.pdf Download (234kB)

Abstract

This paper describes a fuzzy union based approach for automatically evaluating machine generated extract summaries. The proposed method represents every sentence within a machine generated summary as a fuzzy set. Sentences in the reference summary are assigned membership grades in each of these fuzzy sets using cosine distance measure. Finally Fuzzy union (s-norm operation) is used to compute an F-score measure. Max s-norm and Frank's s-norm operators are discussed and the advantages of using a modified Frank's s-norm operator for evaluating summaries is explained. The proposed evaluation method is compared to ROUGE evaluation system with some test cases.

Item Type:	Conference Paper
Keywords:	summary evaluation;Fuzzy f-score measure;LCS match;Frank's s-norm operator
Department/Centre:	Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre
Date Deposited:	03 Jun 2005
Last Modified:	19 Sep 2010 04:19
URI:	http://eprints.iisc.ac.in/id/eprint/3265

Actions (login required)

View Item