Comments and docstrings

Four kinds of comment to delete

# Increment the retry counter.
retries += 1

# NOTE: previously the batch was capped at 32, but that starved long
# prompts (see PR #1234 and section 3.2 of the design doc, which also
# discusses the alternative we rejected).
max_batch = config.max_batch

# 2. Build the batch
batch = [s for s in self.running.values() if not s.paused]

# def old_decode(self, batch):
#     return self.model(batch)

retries += 1

max_batch = config.max_batch

batch = self.next_batch()

1. Narration restates the code. Delete it.
2. History ("previously…", "we used to…", "changed in PR…") belongs in the commit message. Delete it.
3. Section markers ("# 2. Build the batch") are a sign the function should be split into named steps. Extract a method instead (chapter 2).
4. Commented-out code is in git. Delete it.

Comments worth writing

# Sort by prompt length so padding stays small: the attention kernel has no
# support for ragged batches, and padding costs real compute.
batch.sort(key=lambda s: len(s.prompt))

# decode() returns a view into the tokenizer's reusable buffer, so copy it
# before the next call overwrites it.
text = str(tokenizer.decode(seq.generated))

# Must run on the thread that created the CUDA context (driver restriction).
self.graph.replay()

Each one explains a reason or a constraint that the next person would otherwise rediscover the hard way.

Docstrings

Write them for contracts, not for every function

A public function or class whose behaviour isn't obvious from its name and signature gets a docstring. A private helper called next_batch() that returns list[Sequence] doesn't need one.

Lead with one sentence

The first line says what it does, in the imperative or as a noun phrase. Then, only if needed: what can go wrong, what the caller must guarantee, what's surprising.

def submit(self, request: Request) -> bool:
    """Queue a request for generation. Returns False if the queue is full.

    Submitting an id that is already queued or running is a no-op and returns True.
    """

Plain text, not documents

No markdown bold, no headings, no tables, no links to design documents, no section references. A docstring is read in an editor tooltip; if it needs a table, the design belongs in a doc and the docstring should be one line.

TODOs carry an owner and a ticket

# TODO(ticket-123): free KV blocks eagerly once preemption lands. A bare # TODO is a wish nobody owns.