list of open-source publicly-available llms models for code

[andre15silva]

	Name	Publication Date	Model Type	Sizes	URL
-	CodeGen	03/22	Decoder	350M, 2B, 6B, 16B	https://huggingface.co/Salesforce/codegen-16B-mono
-	InCoder	04/22	Decoder	1.3B, 6.7B	https://huggingface.co/facebook/incoder-6B
-	CodeGeeX	09/22	Decoder	13B	https://huggingface.co/spaces/THUDM/CodeGeeX
-	santacoder				https://huggingface.co/bigcode/santacoder
-	replit				https://huggingface.co/replit/replit-code-v1_5-3b
-	codet5				https://huggingface.co/Salesforce/codet5-large
-	plbart				https://huggingface.co/models?other=plbart

[monperrus]

diff-codegen-350m
diff-codegen-2b
diff-codegen-6b

all fine-tuned from Salesforce’s CodeGen code synthesis models

ref: https://carper.ai/diff-models-a-new-way-to-edit-code/

[monperrus]

GPT-J / GPT-J-6B

demo at https://6b.eleuther.ai/
https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b

used by https://arxiv.org/pdf/2208.08289.pdf

[andre15silva]

to merge: https://github.com/eugeneyan/open-llms (section "Open LLMs for code")

[andre15silva]

codegen2 (also supports infilling)

https://github.com/salesforce/CodeGen2

[monperrus]

https://github.com/bigcode-project/starcoder
15.5B parameter model supports code generation and infilling

[monperrus]

StableCode by StableAI
https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b

[monperrus]

WizardCoder
https://huggingface.co/WizardLM/WizardCoder-15B-V1.0

[monperrus]

code-llama / CodeLlama

code-llama by Meta
https://about.fb.com/news/2023/08/code-llama-ai-for-coding/

Code Llama: Open Foundation Models for Code
https://arxiv.org/pdf/2308.12950

[monperrus]

The Mistral models https://mistral.ai/

@martinezmatias says they are good.

Mistral 7B
https://arxiv.org/pdf/2310.06825

[monperrus]

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model
https://arxiv.org/pdf/2310.06266

[monperrus]

Qwen

CODE-QWEN and CODE-QWEN-CHAT 通义千问 (Alibaba)

QWEN TECHNICAL REPORT
https://arxiv.org/pdf/2309.16609.pdf
https://github.com/QwenLM/Qwen

Qwen2. 5-Coder Technical Report
https://arxiv.org/pdf/2409.12186

Nov 2024:

Qwen 2.5-Coder-32B-Instruct Performance: @Alibaba_Qwen announced Qwen 2.5-Coder-32B-Instruct, which matches or surpasses GPT-4o on multiple coding benchmarks. Early testers reported it as "indistinguishable from o1-preview results" (@hrishioa) and noted its competitive performance in code generation and reasoning.

updated @andre15silva

[monperrus]

For the record: CodeTrans

• paper CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing
• model https://github.com/agemagician/CodeTrans

[monperrus]

DeepSeek Coder: Let the Code Write Itself

• https://deepseekcoder.github.io/
• https://github.com/deepseek-ai/DeepSeek-Coder