Pawagi, M and Kumar, V (2023) GuardRails: Automated Suggestions for Clarifying Ambiguous Purpose Statements. In: 16th Annual Compute Conference, COMPUTE 2023, 9 December 2023 through 11 December 2023, Hyderabad, pp. 55-60.
|
PDF
acm_int_con_pro_ser_2023.pdf - Published Version Download (1MB) | Preview |
Abstract
Before implementing a function, programmers are encouraged to write a purpose statement i.e., a short, natural-language explanation of what the function computes. A purpose statement may be ambiguous i.e., it may fail to specify the intended behaviour when two or more inequivalent computations are plausible on certain inputs. Our paper makes four contributions. First, we propose a novel heuristic that suggests such inputs using Large Language Models (LLMs). Using these suggestions, the programmer may choose to clarify the purpose statement (e.g., by providing a functional example that specifies the intended behaviour on such an input). Second, to assess the quality of inputs suggested by our heuristic, and to facilitate future research, we create an open dataset of purpose statements with known ambiguities. Third, we compare our heuristic against GitHub Copilot's Chat feature, which can suggest similar inputs when prompted to generate unit tests. Fourth, we provide an open-source implementation of our heuristic as an extension to Visual Studio Code for the Python programming language, where purpose statements and functional examples are specified as docstrings and doctests respectively. We believe that this tool will be particularly helpful to novice programmers and instructors. © 2023 ACM.
| Item Type: | Conference Paper |
|---|---|
| Publication: | ACM International Conference Proceeding Series |
| Publisher: | Association for Computing Machinery |
| Additional Information: | The copyright for this article belongs to authors. |
| Keywords: | Functional programming; Heuristic methods; Open source software, Docstrings; Function designs; Language model; Natural language explanations; Novice programmer; Open source implementation; Purpose statement; Python programming language; Unit tests; Visual studios, Visual languages |
| Department/Centre: | Division of Electrical Sciences > Computer Science & Automation UG Programme |
| Date Deposited: | 16 Nov 2024 15:09 |
| Last Modified: | 16 Nov 2024 15:09 |
| URI: | http://eprints.iisc.ac.in/id/eprint/85309 |
Actions (login required)
 |
View Item |
