Meta has launched Code Llama, an advanced tool built upon its Llama 2 large language model. This new offering is designed to autonomously generate code and debug existing code, promising to augment the capabilities of developers across the board.
Code Llama is available under the same community license as Llama 2 and will be free for both research and commercial use. Beyond its foundational code generation capabilities, Code Llama can craft detailed code strings based on user prompts, offer debugging solutions, and complete existing code snippets.
To cater to a broader spectrum of developer needs, Meta has rolled out two specialized (fine-tuned) versions: Code Llama-Python, optimized for Python programming, and Code Llama-Instruct, designed to understand instructions given in natural language.
However, Meta has emphasized that these versions have specific applications, advising against using the main Code Llama or Code Llama-Python for tasks involving natural language instructions.
In a detailed blog post, Meta highlighted the potential of such tools, noting, “Programmers are increasingly using LLMs for a diverse array of tasks. With Code Llama, we aim to streamline developer operations, allowing them to focus on the more nuanced aspects of their roles.”
The major highlight is that Code Llama has a 100k token context window, “The Code Llama models provide stable generations with up to 100,000 tokens of context. All models are trained on sequences of 16,000 tokens and show improvements on inputs with up to 100,000 tokens.”.
Preliminary tests suggest that Code Llama has a competitive edge, outperforming other publicly available LLMs. While the specific models it was compared against were not disclosed, Meta did reveal that Code Llama secured a score of 53.7% on the HumanEval code benchmark.
Meta has announced that Code Llama will be available in three sizes, with the smallest variant optimized for single GPU operations, catering to tasks that prioritize speed.
The coding world is already familiar with AI-assisted tools. Earlier this year, GitHub merged Copilot with OpenAI’s GPT-4, which assists developers in writing and reviewing code. Similarly, Amazon’s AWS introduced CodeWhisperer, while Google hinted at the forthcoming AlphaCode.
However, it’s worth noting that not all ventures in this domain have been controversy-free. Microsoft and OpenAI are currently facing legal challenges over Copilot, which is alleged to replicate copyrighted code.
