Blowing in the wind

A Field Guide to LLM and AI Agent Benchmarks

Sat, 26 Sep 2026 23:00:00 +0200

Model reports often list ARC-E, ARC-C, MMLU, GPQA, GSM8K, HumanEval, SWE-bench, GAIA, WebArena, and OSWorld side by side. Their scores are not interchangeable: each benchmark uses different tasks, tools, environments, inference budgets, and scoring rules.

This guide maps common evaluations from static question answering to agents completing real tasks, and explains what each benchmark can—and cannot—tell us.

Reading nanochat Source: From Configuration to a Training Step

Sat, 26 Sep 2026 21:00:00 +0200

This article follows the execution order of scripts/base_train.py: command-line arguments, random seeds, DDP setup, model construction, weight initialization, scaling laws, optimizers, data loading, gradient accumulation, and finally one complete training step.

documents → tokenizer and sequence packing → inputs/targets
          → GPT forward → cross-entropy loss
          → backward and gradient accumulation → optimizer step
          → evaluation, sampling, checkpointing, and resume

The organization and code-reading path follow my nanochat Notion notes. The prose has been edited for clarity, while the source snippets and their original reading sequence are preserved.

Prerequisite: If token embeddings, causal self-attention, MLPs, residual connections, or next-token loss are still unfamiliar, start with Transformer Architecture: From Token Embedding to the Training Loop and then return to this source-level walkthrough.

About

Sat, 26 Sep 2026 00:00:00 +0000

For more than five years, I worked on the core engineering and technical leadership of real-time collaborative documents at Kingsoft Office. My work spanned system and application architecture, core implementation, code review, performance engineering, and team delivery. I care about more than making a feature run: I want systems to remain understandable, reliable, and evolvable under high concurrency, long-lived connections, and failure. Since September 2025, I have been pursuing an MSc in Computer Science at Leiden University, with a focus on Advanced Computing and Systems and Artificial Intelligence.

Transformer Architecture: From Token Embeddings to the Training Loop

Mon, 14 Sep 2026 10:25:02 +0200

Transformer appears to be made up of many components, but a GPT-style forward path can be summarized as:

Token IDs
   ↓
Token Embedding + Position Embedding
   ↓
[LayerNorm → Self-Attention → Residual]
   ↓
[LayerNorm → FFN → Residual]
   ↓
Repeat for multiple Transformer blocks
   ↓
Final LayerNorm → LM Head
   ↓
Next-token logits → Cross-Entropy Loss

This article starts from a simple GPT implementation and explains the role and tensor shape of each step along the data flow. The complete source code is placed at the end of the article. After reading the previous principles, you can compare and understand it with the code.

Anthropic Agent Evolution I: From Workflows to Agents—and Why Harnesses Matter

Sat, 12 Sep 2026 21:08:45 +0200

“Agent” is often used as a broad label for any system that calls tools or takes multiple steps. Anthropic’s engineering work offers a more useful distinction: a workflow follows code paths defined in advance; an agent lets the model dynamically choose its process from environmental feedback.

This first article in the “Anthropic Agent Evolution” series establishes the vocabulary: augmented LLMs, fixed workflows, autonomous agents, and the harness that surrounds the model.

Anthropic Agent Evolution II: Context Reset and Structured Handoff

Sat, 12 Sep 2026 21:07:45 +0200

The central tension in long-running work is not whether a model can write code. It is that a task may last hours or days while every context window is finite. Replaying the entire history eventually creates both capacity and noise problems.

Anthropic’s first long-running agent harness answered with a simple principle: reset conversational context, but externalize project state into structured artifacts that the next agent can recover.

Anthropic Agent Evolution III: The Planner–Generator–Evaluator Quality Loop

Sat, 12 Sep 2026 21:06:45 +0200

The first harness let an agent continue across context windows, but continuity is not quality. Generators routinely mistake “mostly runs” for “good enough,” especially in subjective product and design work.

Anthropic’s next step separated planning, generation, and evaluation so that an external evaluator became a source of evidence the generator had to respond to.

Anthropic Agent Evolution IV: Managed Agent Runtime—Session, Harness, and Sandbox

Sat, 12 Sep 2026 21:05:45 +0200

Earlier harnesses focused on how a model could continue work and improve quality. A managed service introduces system questions: how is a session persisted, how does work recover after a harness crash, how can sandboxes be replaced, how do customer VPCs connect, and how are credentials kept away from model-generated code?

Anthropic Managed Agents answers by decoupling the brain, hands, and session behind stable interfaces.

Stanford CS329A: A Complete Framework for Self-Improving AI Agents

Tue, 11 Aug 2026 06:27:01 +0200

Self-Improving AI Agent is not a mysterious system that recursively modifies itself indefinitely. A more practical understanding is: Agent forms a closed loop between generation, action, observation and verification, and uses the feedback obtained during reasoning to improve current answers, subsequent decisions, and even the next round of training.

This article follows the main line of the Stanford CS329A course and strings together test-time calculations, verifiers, tool feedback, planning search, reinforcement learning, in-depth research and long-term evaluation into a complete framework.

Stanford CS230: Career Advice for the AI Era

Mon, 23 Mar 2026 12:22:13 +0100

AI is rapidly changing software engineering: the cost of writing code continues to fall, the tasks engineers can complete are becoming more and more complex, and the bottleneck of product development is also shifting. This article summarizes five suggestions on AI career development shared in CS230 - understand changes, choose peers, measure efforts with output, lay a solid technical foundation, and pay attention to Small AI.

Translation: Linux I/O Multiplexing - select vs. poll vs. epoll

Thu, 11 Feb 2021 09:45:47 +0800

Introduction Before reading this article, you can first understand the five I/O models of Liunx References: I/O Models –Section 6.2 of “UNIX® Network Programming Volume 1, Third Edition: The Sockets Networking” by Richard Stevens The following is the text of the article One of the basic concepts of Linux (actually Unix) is that everything in Unix/Linux is a file. Each process has a file descriptor table that points to files, sockets, devices, and other operating system objects related to the process.

Introduction to CSRF and CORS

Thu, 04 Feb 2021 17:32:47 +0800

Introduction Before introducing CRSF(Cross-site request forgery)&CORS(Cross-origin resource sharing), we first introduce HTTP. If readers are familiar with HTTP, they can skip this content directly. HTTP request An HTTP request consists of four parts: request line, request header, blank line and request body. 1 2 3 4 5 6 7 GET /mix/76.html?name=kelvin&password=123456 HTTP/1.1 Host: www.fishbay.cn Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.

Translation: Domain-Driven Design - Everything You Always Wanted to Know (1)

Tue, 02 Feb 2021 12:06:47 +0800

Domain-Driven Design original address As a personal code base grows, its complexity inevitably increases. As this happens, it becomes very difficult to maintain the organization and structure of the code as originally intended, which is known as “software entropy”. After a series of iterations, maintaining good focus points and correctly decoupling classes and modules becomes more challenging if strict architectural guidelines are not enforced. In a traditional Model-View-Controller (MVC) structure, the “M” layer would retain all business logic but provide no clear guidelines on how to properly delineate responsibilities.

A Practical Introduction to Go Assembly (2)

Fri, 01 Jan 2021 12:34:14 +0800

Function call from assembly perspective Let’s start with a simple example: 1 2 3 4 5 6 7 8 9 10 package main func main() { add(1, 1) } func add(a, b int64) (c int64) { c = a + b return } After executing the go tool compile -S compliation_add.go command, the assembly code is as follows 1 2 3 4 5 6 7 .

A Practical Introduction to Go Assembly (1)

Wed, 30 Dec 2020 11:25:47 +0800

Program storage space layout A process is an executing program instance. Each instance has its own address space and execution state. A program becomes a process when the operating system adds the appropriate information to the kernel data structures and allocates the necessary resources to run the program code. A thread is an abstract data type that represents a thread of execution within a process. A thread has its own execution stack, program counter value, register set and state.

CAP and a Comparison of Service Discovery Systems

Sat, 29 Feb 2020 09:41:00 +0800

Overview of CAP theory **A distributed system can only satisfy at most two of the three requirements of consistency (consistency), availability (Availability) and partition tolerance (Partition tolerance) at the same time. ** Introduction Overview of CAP theory: CAP theory of distributed systems Consistency model Consistency refers to all nodes see the same data at the same time, that is, after the update operation is successful and returned to the client, the data of all nodes at the same time is completely consistent.

Understanding I/O: Random and Sequential Access

Thu, 06 Feb 2020 21:06:10 +0800

Transfer: https://blog.csdn.net/BaiWfg2/article/details/52885287 Storage for DBAs: Ever been to one of those sushi restaurants where the food comes round in dishes on a conveyor belt? As each dish travels around the loop you eye it up and, as long as you can make your mind up in time, grab it. However, if you are as indecisive as me, there’s a chance it will be out of range before you come to your senses – in which case you have to wait for it to complete a further full revolution before getting another chance.

Use a Mutex or a Channel?

Thu, 16 Jan 2020 11:27:00 +0800

Use a sync.Mutex or a channel? Gives us 3 channel usage scenarios respectively: 1. Transfer ownership of data 2. Assign work units 3. Convey asynchronous results Usage scenarios of 2 mutex: 1. Cache 2. Status Chapter 2 “Go’s Philosophy on Concurrency” in the book “Concurrency In Go” also has a decision tree and detailed discussion Summary Go implements the CSP communication model through channels, which are mainly used for message passing and event notification between goroutines.

HTTP/1.1 vs. HTTP/2

Thu, 16 Jan 2020 10:25:00 +0800

HTTP 1.1 In January 1997, HTTP/1.1 version was released, only half a year later than version 1.0. It further improved the HTTP protocol and has been used to this day 20 years later, and is still the most popular version. Main optimization points: Support long connection by default HTTP/1.1 supports persistent connections and request pipelining. Persistent connections reuse established TCP connections and avoid repeating the RTT cost of the three-way handshake.

Inside Go's Channel Runtime

Sun, 05 Jan 2020 11:19:10 +0800

Channel is a very important type in the Go language and is the first object in Go. Through channels, Go implements memory sharing through communication. Channel is an important means of transferring data and synchronizing between multiple goroutines. Note: All source code analysis in this article is based on Go1.13.3. Different versions may have different implementations. channel syntax The syntax for declaring a channel is as follows:

Implementing the Raft Protocol in Go (1)

Sun, 22 Dec 2019 17:13:00 +0800

Introduction The implementation idea comes from: MIT 6.824 course Lab 2: Raft Before doing the experiment, you should read the raft paper carefully, here it is: raft paper raft translation Recommended reading: Students’ Guide to Raft Raft Understandable Distributed Consensus lab content In this lab, you will implement most of the Raft design described in Extended Paper, including saving persistent state and reading it after a node fails and then restarts.

Translation: Contiguous Stacks

Sat, 30 Nov 2019 13:14:00 +0800

Contiguous stacks original address Allocate a continuous stack memory space for each go coroutine. When the memory is used up, it will be reallocated/copied and grown. Why? The current stack splitting mechanism has a “hot split” problem - if the stack space is almost full, calling the function will trigger the forced allocation of a new stack block. When the calling function returns, the new stack block will be released.

Inside Go's Main Goroutine Scheduler (2)

Wed, 13 Nov 2019 22:20:10 +0800

9. Enter runtime/proc.go and scheduler initialization 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 func schedinit() { // raceinit must be the first call to race detector.

Why Is Quicksort Used More Often Than Merge Sort?

Wed, 13 Nov 2019 20:32:10 +0800

What is honey to you is arsenic to another. Quick sort: 1 2 3 * Worst-case time complexity: O(n^2) * Best-case time complexity: O(n log n) * Average time complexity: O(n log n) Merge sort: 1 * Worst-, best-, and average-case time complexity: O(n log n) Since the time complexity of merge sort is >= the time complexity of quick sort in different situations, why are the sorting algorithms we use in actual programming quick sort instead of merge sort?

Inside Go's Main Goroutine Scheduler (1)

Sun, 10 Nov 2019 14:40:10 +0800

Under the compilation, all traces are revealed. 1.go program entry_rt0_amd64_linux: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 // The TEXT instruction defines the symbol _rt0_amd64_linux, the global data symbol is declared with DATA, and GLOBL defines the data as global. // SB SB virtual register: saves the starting address of the program address space; the value saved by this SB register is the starting address of the TEXT segment, which is mainly used to locate global symbols.

Notes on Critical Thinking

Mon, 01 Jan 0001 00:00:00 +0000

Introduction The author’s own definition of critical thinking: thinking about ways of thinking The benefits of being a good thinker How to become a good thinker–>Develop a critical thinking style–>The importance of critical thinking Detailed explanation of the concept of critical thinking–>Analysis and evaluation Re-emphasized the importance of critical thinking and briefly introduced the measurement criteria of critical thinking 1 2 3 4 The essence of learning is improving and controlling how you think.