GPT1到Deepseek R1所有公开论文 The 2025 AI Engineer Reading List

GPT1到Deepseek R1所有公开论文 The 2025 AI Engineer Reading List GPT1到Deepseek R1所有公开论文 The 2025 AI Engineer Reading List Modified February 13, 2025 readinglist.rar 743.67MB 4. Voyager paper Nvidia’s take on 3 cognitive architecture components (curriculum, skill library, sandbox) to improve performance. More abstractly, skill library/curriculum can be abstracted as a form of Agent Workflow Memory Voyager 论文 英伟达利用 3 个认知架构组件（课程、技能库、沙箱）来提高性能。更抽象地说，技能库/课程可以抽象为一种代理工作流记忆形式 Voyager An Open Ended Embodied Agent with Large Language Models.pdf 5. Anthropic on Building Effective Agents just a great end 2024 recap that focuses on the importance of chaining, routing, parallelization, orchestration, evaluation, and optimization. See also OpenAI Swarm. Anthropic 的《 构建有效的代理》——只是一篇关于 2024 年的精彩回顾，重点关注 连锁、 路由、并行化、协调、评估和优化的 重要性 。 另请参阅 Lilian Weng 的《Agents》（前 OpenAI）、 Shunyu Yao的《LLMAgents》（现 OpenAI）和 Chip Huyen 的《Agents》。 Building effective agents Anthropic.pdf We covered many of the 2024 SOTA agent designs at NeurIPS, and you can find more readings in the UC Berkeley LLM Agents MOOC. Note that we skipped bikeshedding agent definitions, but if you really need one, you could use mine. 我们在NeurIPS上介绍了许多2024SOTA代理设计， 你还可以在加州大学伯克利分校LLM代理的慕课中找到更多资料。需要注意的是，我们跳过了单车甩尾代理的定义，但如果你真的需要，可以参考我的定义 。 第 6 节：代码生成 Section 6: Code Generation 1. The Stack paper the original open dataset twin of The Pile focused on code, starting a great lineage of open codegen work from The Stack v2 to StarCoder 堆栈论文——《堆栈》最初的开放数据集孪生兄弟专注于代码，开启了从 The Stack v2 到 StarCoder 的开放代码生成工作的伟大历程 The Stack 3 TB of permissively licensed source code.pdf StarCoder 2 and The Stack v The Next Generation.pdf 2. Open Code Model papers choose from DeepSeek Coder, Qwen2.5 Coder, or CodeLlama. Many regard 3.5 Sonnet as the best code model but it has no paper. 开放代码模型论文——从 DeepSeek Coder、 Qwen2.5 Coder 或 CodeLlama 中选择 。许多人认为3.5 Sonnet 是最好的代码模型 ，但它没有论文。 DeepSeek Coder When the Large Language Model Meets Programming The Rise of Code Intelligence.pdf Qwen2.5 Coder Technical Report.pdf Code Llama Open Foundation Models for Code.pdf 3. HumanEval/Codex paper This is a saturated benchmark, but is required knowledge for the code domain. SWE Bench is more famous for coding now, but is expensive/evals agents rather than models. Modern replacements include Aider, Codeforces, BigCodeBench, LiveCodeBench and SciCode. HumanEval/Codex 论文 这是一个饱和的基准但却是代码领域的必备知识。SWE Bench 现在在编码方面更有名，但价格昂贵/评估代理而非模型。现代的替代品包括：Aider、Codeforces、BigCodeBench、LiveCodeBench 和 SciCode。 HumanEvalCodex Evaluating Large Language Models Trained on Code.pdf Codeforces Competition Level Problems are Effective LLM Evaluators.pdf 4. AlphaCodeium paper Google published AlphaCode and AlphaCode2 which did very well on programming problems, but here is one way Flow Engineering can add a lot more performance to any given base model. AlphaCodeium 论文 谷歌发布的 AlphaCode 和 AlphaCode2，在编程问题上表现优异，但这里有一种方法，即Flow Engineering可以为任何给定的基础模型增加更多性能。 AlphaCodeium Code Generation with AlphaCodium From Prompt Engineering to Flow Engineering.pdf 5. CriticGPT paper LLMs are known to generate code that can have security issues. OpenAI trained CriticGPT to spot them, and Anthropic uses SAEs to identify LLM features that cause this, but it is a problem you should be aware of. CriticGPT 论文——众所周知，LLMs 生成的代码可能存在安全问题。OpenAI训练CriticGPT来发现这些问题，Anthropic则使用SAE来识别导致安全问题的LLM特性，但这是一个你应该注意的问题。 Large Language Models and Code Security: A Systematic Literature Review Large Language Models and Code Security A Systematic Literature Review.pdf known Large Language Models and Code Security A Systematic Literature Review.pdf CriticGPT llm critics help catch llm bugs.pdf CodeGen is another field where much of the frontier has moved from research to industry and practical engineering advice on codegen and code agents like Devin are only found in industry blogposts and talks rather than research papers. 代码生成是另一个领域，该领域的许多前沿问题已从研究转向工业，关于代码生成和代码代理（如 Devin）的实用工程建议只出现在工业博文和会谈中，而不是研究论文中。 第 7 节：视觉 Section 7: Vision 1. Non LLM Vision work is still important: e.g. the YOLO paper (now up to v11, but mind the lineage), but increasingly transformers like DETRs Beat YOLOs too. 非LLMVision 的工作仍然很重要：例如 YOLO论文 （现在已经到了第 11 版，但要注意脉络），像 DETRs Beat YOLOs 这样的转换器也越来越多 。 YOYO You Only Look Once Unified, Real Time Object Detection.pdf DETRs Beat YOLOs on Real time Object Detection.pdf 2. CLIP paper the first successful ViT from Alec Radford. These days, superceded by BLIP/BLIP2 or SigLIP/PaliGemma, but still required to know. CLIP 论文——Alec Radford 首次成功提出的ViT。如今，它已被 BLIP/BLIP2 或 SigLIP/PaliGemma 所取代 ，但仍需要了解。 CLIP Learning Transferable Visual Models From Natural Language Supervision.pdf VIT An Image is Worth 16x16 Words Transformers for Image Recognition at Scale.pdf BLIP Bootstrapping Language Image Pre training for Unified Vision Language Understanding and Generation.pdf BLIP 2 Bootstrapping Language Image Pre training with Frozen Image Encoders and Large Language Models.pdf 3. MMVP benchmark (LS Live) quantifies important issues with CLIP. Multimodal versions of MMLU (MMMU) and SWE Bench do exist. MMVP基准测试（LS Live）——量化 CLIP 的重要问题。MMLU (MMMU) 和 SWE Bench 的多模式版本 确实存在。 MMVP benchmark Eyes Wide Shut Exploring the Visual Shortcomings of Multimodal LLMs.pdf MMMU A Massive Multi discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI.pdf SWE bench Multimodal Do AI Systems Generalize to Visual Software Domains.pdf 4. Segment Anything Model and SAM 2 paper (our pod) the very successful image and video segmentation foundation model. Pair with GroundingDINO. Segment Anything Model 和 SAM 2 论文——非常成功的图像和视频分割基础模型。与 GroundingDINO 配对。 Segment Anything.pdf SAM 2 Segment Anything in Images and Videos.pdf 5. Early fusion research: Contra the cheap “late fusion” work like LLaVA (our pod), early fusion covers Meta’s Flamingo, Chameleon, Apple’s AIMv2, Reka Core, et al. In reality there are at least 4 streams of visual LM work. 早期融合研究——与 LLaVA（播客）等廉价的 "后期融合"工作不同 ，早期融合包括Meta的lamingo、 Chameleon、Apple 的 AIMv2、Reka Core 等 。 Chameleon Mixed Modal Early Fusion Foundation Models.pdf LLaVA Visual Instruction Tuning.pdf AIMv2 Multimodal Autoregressive Pre training of Large Vision Encoders.pdf Reka Core Flash and Edge A Series of Powerful.pdf Generalized Visual Language Models Lil'Log.pdf Much frontier VLM work these days is no longer published (the last we really got was GPT4V system card and derivative papers). We recommend having working experience with vision capabilities of 4o (including finetuning 4o vision), Claude 3.5 Sonnet/Haiku, Gemini 2.0 Flash, and o1. Others: Pixtral, Llama 3.2, Moondream, QVQ 如今，许多前沿的 VLM 工作已不再发表（我们最后得到的是 GPT4V 系统卡及其衍生论文）。我们建议对 4o（包括微调4o视觉）、Claude 3.5 Sonnet/Haiku、Gemini 2.0 Flash 和 o1 的视觉功能有工作经验。其他：Pixtral、Llama 3.2、Moondream、QVQ。 GPT 4V(ision) System Card.pdf The Dawn of LMMs Preliminary Explorations with GPT 4V(ision).pdf 第 8 节：声音 Section 8: Voice 1. Whisper paper the successful ASR model from Alec Radford. Whisper v2, v3 and distil whisper and v3 Turbo are open weights but have no paper. Whisper 论文——Alec Radford开发的成功的自动语音识别模型。Whisperv2、v3、distil whisper和 v3 Turbo 的权重是开源的，但没有论文。 2. AudioPaLM paper our last look at Google’s voice thoughts before PaLM became Gemini. See also: Meta’s Llama 3 explorations into speech. AudioPaLM 论文 在 PaLM 成为 Gemini 之前，我们对谷歌语音思考的最后一瞥。另请参见：Meta 的 Llama 3 对语音的探索。 The Llama 3 Herd of Models.pdf 3. NaturalSpeech paper one of a few leading TTS approaches. Recently v3. 自然语音论文 几种领先的语音合成方法之一。最近的 v3。 a. NaturalSpeech End to End Text to Speech Synthesis with Human Level Quality.pdf NaturalSpeech 3 Zero Shot Speech Synthesis with Factorized Codec and Diffusion Models.pdf 4. Kyutai Moshi paper an impressive full duplex speech text open weights model with high profile demo. See also Hume OCTAVE. Kyutai Moshi 论文——一款令人印象深刻的全双工语音 文本开源模型，并提供了高规格的演示。另请参阅 Hume OCTAVE。 Moshi a speech text foundation model for real time dialogue.pdf 5. OpenAI Realtime API: The Missing Manual Again, frontier omnimodel work is not published, but we did our best to document the Realtime API. OpenAI 实时 API：缺失手册——同样，前沿的全模型工作并未公开发表，但我们尽了最大努力记录实时 API。 We do recommend diversifying from the big labs here for now try Daily, Livekit, Vapi, Assembly, Deepgram, Fireworks, Cartesia, Elevenlabs etc. See the State of Voice 2024. While NotebookLM’s voice model is not public, we got the deepest description of the modeling process that we know of. 我们建议您暂时不要选择大型实验室，可以尝试 Daily, Livekit, Vapi, Assembly, Deepgram, Fireworks, Cartesia, Elevenlabs 等。请参阅《2024 年语音现状》。虽然 NotebookLM 的语音模型并未公开，但我们获得了目前所知的最深入的建模过程描述。 With Gemini 2.0 also being natively voice and vision multimodal, the Voice and Vision modalities are on a clear path to merging in 2025 and beyond. 由于Gemini 2.0 原生具有语音和视觉多模态功能，因此语音和视觉模态在2025年及以后的融合前景十分明朗。 第 9 节：图像/视频扩散 Section 9: Image/Video Diffusion 1. Latent Diffusion paper effectively the Stable Diffusion paper. See also SD2, SDXL, SD3 papers. These days the team is working on BFL Flux [schnell|dev|pro]. 潜扩散论文 实际上是稳定扩散论文。另见 SD2、SDXL、SD3 论文。如今，该团队正在开发 BFL Flux [schnell|dev|pro]。 Latent Diffusion High Resolution Image Synthesis with Latent Diffusion Models.pdf SDXL Improving Latent Diffusion Models for High Resolution Image Synthesis.pdf SD3 Scaling Rectified Flow Transformers for High Resolution Image Synthesis.pdf 2. DALL E / DALL E 2 / DALL E 3 paper OpenAI’s image generation. DALL E / DALL E 2 / DALL E 3 论文 OpenAI 的图像生成。 DALL E Zero Shot Text to Image Generation.pdf DALL E 2 Hierarchical Text Conditional Image Generation with CLIP Latents.pdf DALL E 3 Improving Image Captioning with Better Use of Captions.pdf 3. Imagen / Imagen 2 / Imagen 3 paper Google’s image gen. See also Ideogram. Imagen / Imagen 2 / Imagen 3 论文 谷歌的图像生成。另见：表意文字。 Imagen Photorealistic Text to Image Diffusion Models with Deep Language Understanding.pdf Imagen 3.pdf 4. Consistency Models paper this distillation work with LCMs spawned the quick draw viral moment of Dec 2023. These days, updated with sCMs. 一致性模型论文 这项与 LCMs 的蒸馏工作引发了 2023 年 12 月的快速绘图病毒式传播。这些天，又更新了sCMs。 LCMs Latent Consistency Models Synthesizing High Resolution Images with Few Step Inference.pdf sCMs Simplifying, Stabilizing and Scaling Continuous Time Consistency Models.pdf Voyager cognitive architecture Agent Workflow Memory Building Effective Agents OpenAI Swarm the 2024 SOTA agent designs at NeurIPS the UC Berkeley LLM Agents MOOC use mine The Stack paper The Stack v2 StarCoder DeepSeek Coder Qwen2.5 Coder 3.5 Sonnet as the best code model HumanEval/Codex paper Aider Codeforces BigCodeBench LiveCodeBench SciCode AlphaCodeium paper AlphaCode AlphaCode2 CriticGPT known uses SAEs to identify LLM features Large Language Models and Code Security: A Systematic Literature Review practical engineering advice on codegen code agents like Devin YOLO up to v11 mind the lineage DETRs Beat YOLOs CLIP ViT BLIP BLIP2 SigLIP/PaliGemma MMVP benchmark LS Live MMMU SWE Bench Segment Anything Model https://ar5iv.labs.arxiv.org/html/2304.02643? immersive translate auto translate=1 SAM 2 our pod GroundingDINO LLaVA our pod Flamingo Chameleon AIMv2 Core are at least 4 streams of visual LM work GPT4V system card derivative papers finetuning 4o vision Pixtral Llama 3.2 Moondream QVQ Whisper v2 v3 distil whisper v3 Turbo AudioPaLM Llama 3 explorations into speech NaturalSpeech v3 Kyutai Moshi high profile demo Hume OCTAVE OpenAI Realtime API: The Missing Manual the State of Voice 2024 we got the deepest description of the modeling process Latent Diffusion SD2 SDXL SD3 BFL Flux DALL E DALL E 2 DALL E 3 Imagen Imagen 2 Imagen 3 Ideogram Consistency Models LCMs quick draw viral moment of Dec 2023 sCMs readinglist.rar 743.67MB readinglist.rar 743.67MB 4. Voyager paper Nvidia’s take on 3 cognitive architecture components (curriculum, skill library, sandbox) to improve performance. More abstractly, skill library/curriculum can be abstracted as a form of Agent Workflow Memory Voyager 论文 英伟达利用 3 个认知架构组件（课程、技能库、沙箱）来提高性能。更抽象地说，技能库/课程可以抽象为一种代理工作流记忆形式 Voyager An Open Ended Embodied Agent wi