Functions
Small, one level of abstraction, results instead of side effects.
Functions return results. They don't mutate their arguments.
A caller reading x = f(a, b) assumes a and b are unchanged. When they aren't, every call site becomes a trap.
One level of abstraction per function, and at most four parameters.
A function that mixes "decide what to do" with "flip these bits" makes the reader zoom in and out on every line. More than four parameters usually means a missing object.
Hidden mutation
A free function that takes the caller's containers and fills them in is the most common way readable code turns unreadable.
def _admit(request, tokenizer, prompts, generated, budgets):
ids = tokenizer.encode(request["prompt"])
if len(ids) > MAX_PROMPT_TOKENS:
log.warning("prompt too long: %s", request["id"])
return
prompts[request["id"]] = ids # mutates the caller's dict
generated[request["id"]] = [] # and this one
budgets[request["id"]] = request.get("max_tokens", 256) # and this oneclass Scheduler:
def admit(self, request: Request) -> bool:
if len(request.prompt) > self.max_prompt_tokens:
log.warning("rejected %s: prompt has %d tokens", request.id, len(request.prompt))
return False
self.running[request.id] = Sequence(request.id, request.prompt, request.max_new_tokens)
return TrueThe method changes state too, but it changes its own state, which is what a reader expects from self.admit(...). The rule is about surprise: mutate yourself, never your arguments. Tokenizing also moved out, to the boundary where the request is parsed, so admit works at one level of abstraction.
Too many parameters
def sample(logits, temperature, top_k, top_p, repetition_penalty, seed, history):
...@dataclass(frozen=True)
class SamplingParams:
temperature: float = 1.0
top_k: int = 0
top_p: float = 1.0
repetition_penalty: float = 1.0
seed: int | None = None
def sample(logits: Tensor, history: list[int], params: SamplingParams) -> int:
...The parameters that travel together became one object with defaults, which also gives every request a single place to carry its sampling settings.
Guidelines
Command or query, not both
A function either answers a question (is_finished(), has_free_slot()) or does something (admit(), cancel()). Avoid functions that do something and return an unrelated answer. Returning whether the action succeeded is fine.
Return early, keep the happy path flat
def next_request(self) -> Request | None:
if not self.waiting:
return None
if len(self.running) >= self.max_batch:
return None
return self.waiting.popleft()
Keyword-only for anything ambiguous
pad(batch, 512, 0, True) tells the reader nothing. pad(batch, length=512, value=0, left=True) does. Use * to force keywords for booleans and anything after the first two arguments.
No closures defined inside long functions
A helper defined halfway through a 100-line function captures variables from around it, and the reader has to work out which. Make it a method or a module-level function with explicit parameters.
Type-annotate signatures
Annotate every public function's parameters and return value. It's the cheapest documentation there is, and it stays correct because the type checker reads it too. Inside function bodies, annotate only where it helps the reader.