  Large Language Models (LLMs) perform well on basic programming problems.
However, they encounter challenges when dealing with complex tasks involving
the use of diverse algorithmic and data structure skills, particularly
programming competition-level problems. Notably, ChatGPT exhibits proficient
performance on problems it has encountered during its pre-training phase, but
this performance deteriorates when faced with novel problems. Consequently,
enhancing the ability of LLMs to address unfamiliar problems has emerged as a
pivotal research focus. The problem-solving process of LLMs mirrors human
programmers' approach to a certain extent. When confronted with new programming
tasks, human programmers engage in task planning and code writing with the
previously acquired knowledge about algorithms and data structures. Despite
having learned such knowledge, LLMs struggle to effectively apply it when faced
with specific new problems. To address this issue, we constructed a novel
dataset, CodeF, which contains a portion of programming problems that ChatGPT
has not previously encountered. Furthermore, we developed a Knowledge Library
tailored for Python programming contest problems and introduced the concept of
Knowledge-Aware Code Generation (KareCoder). KareCoder bolsters the models'
understanding and problem-solving capabilities by integrating prompt and
knowledge from the library into the LLMs' code generation reasoning process,
especially on Pass@1 metrics. Upon testing on the CodeF and APPS datasets,
KareCoder demonstrated outstanding performance in handling novel problems
previously unencountered by LLMs. In contrast with the code directly generated
by ChatGPT, KareCoder achieved a relative improvement of 23.3% on the Pass@1
metric on the CodeF post2021-9 dataset. Additionally, it performs well compared
to other methods when dealing with problems that LLMs have previously
encountered.
