Pawagi, M and Kumar, V (2024) Probeable Problems for Beginner-level Programming-with-AI Contests. In: 20th Annual ACM Conference on International Computing Education Research, ICER 2024, 13 August 2024through 15 August 2024, Melbourne, pp. 166-176.
|
PDF
ACM_Con_ Int_ Com_ Edu_ Res_1_2024.pdf - Published Version Download (1MB) | Preview |
Abstract
To broaden participation, competitive programming contests may include beginner-level problems that do not require knowledge of advanced Computer Science concepts (e.g., algorithms and data structures). However, since most participants have easy access to AI code-generation tools, these problems often become trivial to solve. For beginner-friendly programming contests that do not prohibit the use of AI tools, we propose Probeable Problems: code writing tasks that provide (1) a problem specification that deliberately omits certain details, and (2) a mechanism to probe for these details by asking clarifying questions and receiving immediate feedback. To evaluate our proposal, we conducted a 2-hour programming contest for undergraduate Computer Science students from multiple institutions, where each student was an active member of their institution's ACM student chapter. The contest comprised of six Probeable Problems for which a popular code-generation tools (e.g., GitHub Copilot) were unable to generate accurate solutions due to the absence of details. Students were permitted to work individually or in groups, and were free to use AI tools. We obtained consent from 26 groups (67 students) to use their submissions for research. To determine whether Probeable Problems are suitable for such contests, we analyze the extent to which the code submitted by these groups identifies missing details. © 2024 ACM.
| Item Type: | Conference Paper |
|---|---|
| Publication: | ICER 2024 - ACM Conference on International Computing Education Research |
| Publisher: | Association for Computing Machinery, Inc |
| Additional Information: | The copyright for this article belongs to the authors. |
| Keywords: | Program debugging, Active member; Algorithms and data structures; Ambiguity; Code generation tools; Code specification; Code-writing; Computer science students; Immediate feedbacks; Problem specification; Programming contests, Students |
| Department/Centre: | Division of Mechanical Sciences > Divecha Centre for Climate Change Division of Electrical Sciences > Computer Science & Automation |
| Date Deposited: | 28 Sep 2024 06:04 |
| Last Modified: | 28 Sep 2024 06:04 |
| URI: | http://eprints.iisc.ac.in/id/eprint/86267 |
Actions (login required)
 |
View Item |
