Ravindra, G and Balakrishnan, N and Ramakrishnan, KR (2004) Automatic Evaluation of Extract Summaries Using Fuzzy F-Score Measure. In: Fifth International Conference on Knowledge Based Computer Systems (KBCS 2004), Dec 19-22, 2004, Hyderabad, India, pp. 487-497.
|
Postscript
paperid-40.ps Download (210kB) |
|
|
PDF
Automatic.pdf Download (234kB) |
Abstract
This paper describes a fuzzy union based approach for automatically evaluating machine generated extract summaries. The proposed method represents every sentence within a machine generated summary as a fuzzy set. Sentences in the reference summary are assigned membership grades in each of these fuzzy sets using cosine distance measure. Finally Fuzzy union (s-norm operation) is used to compute an F-score measure. Max s-norm and Frank's s-norm operators are discussed and the advantages of using a modified Frank's s-norm operator for evaluating summaries is explained. The proposed evaluation method is compared to ROUGE evaluation system with some test cases.
Item Type: | Conference Paper |
---|---|
Keywords: | summary evaluation;Fuzzy f-score measure;LCS match;Frank's s-norm operator |
Department/Centre: | Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre |
Date Deposited: | 03 Jun 2005 |
Last Modified: | 19 Sep 2010 04:19 |
URI: | http://eprints.iisc.ac.in/id/eprint/3265 |
Actions (login required)
View Item |