Performance Work That Starts Without Deciding What to Fix

When I joined the annotation platform, the brief handed to me was “fetching a Task sometimes means waiting 2 seconds.” The initial hypotheses were prefetching and indexes. But the existing speedup proposals written around that direction hadn’t actually made anything faster.

So I chose to begin without deciding where to fix.

From finding to re-measurement

log / span → 異常を finding 化 → 原因を仮説化 → code / query plan → 改善して再計測

From logs and spans, harvest abnormal counts, unexplained durations, and dominant operations into findings. From a finding, form a causal hypothesis, then inspect the code, the query plans, and external I/O. Once fixed, re-measure under identical conditions.

The ordering that mattered was refusing to lead with a hypothesis. Entering from prefetch-and-indexes, none of the three cases described below would ever have surfaced.

The GET /api/tasks/next trace

We ship OpenTelemetry spans from the frontend and backend Cloud Run services to Cloud Trace, so within one trace ID we can follow service boundaries and processing times.

Opening the trace for GET /api/tasks/next showed 2.46 seconds end to end. The breakdown:

span	Time
SELECT	3–16 ms
transaction	377 ms
BEGIN	177–211 ms

SELECT is fast. The waiting turned out to be the transaction opening that sets the RLS context.

What we hunt for here shifts from “heavy queries” to “transaction boundaries.” What the tracing investment bought us was narrowing down to a boundary worth fixing.

After the fix, the same endpoint’s trace read 309 milliseconds for the full request, 31 milliseconds for the transaction, and 2–3 milliseconds for BEGIN. BEGIN didn’t vanish entirely; the waiting that occurred on paths applying RLS simply shrank. Fix the boundary you found, then confirm the effect at that same granularity—that’s the loop.

Three cases, three layers

The causes this method turned up sat at different layers.

Case one: the same mapping-table load firing over and over. The doorway was the same message stacking up at an unnatural rate in Cloud Logging; the cause was Node.js’s timer cap.

Case two: a 3.6-millisecond query wrapped in a 304-millisecond API. An outbound POST right under the controller owned 217 milliseconds, and that POST was authentication.

Case three: the RLS policy function being evaluated per row. The more rows a listing read, the more it repeated the same authorization decision.

A runtime timer, I/O against an external service, a database plan. None of these emerge from a “find the slow query” approach. The only thread tying them together: anomalies were spotted in logs and spans first, then we descended into the layer beneath.

Whole-suite measurement with benchmarks

Even the localized measurements already showed sizeable gains.

Target	Before	After
Task list	19–130 ms	0.159 ms
Annotation history	33.146 ms	9.263 ms
Work history UUID	4,689.9 ms	0.054 ms

But that alone can’t tell us whether screens got faster overall. Localized measurement only sees what we touched.

So we ran each endpoint for 20 rounds to extract p50 and p95, then took the median across the 21 endpoints measurable both before and after the improvements.

• p50 median: 16,310 ms → 127 ms
• p95 median: 22,329 ms → 367 ms
• Largest win: Task label summary retrieval fell 31,467 ms → 97 ms (322.9x)

Averaging the same 21 endpoints restricted to successful responses: p50 dropped from 20,381 ms to 176 ms, and p95 from 27,078 ms to 441 ms.

Benchmark conditions vs. production data

The benchmark dataset is 1 Project, 100 Requests, 10,000 Tasks. By volume it’s built deliberately larger than a typical single Project. Larger than typical, yes—but it does not reproduce peak production scale.

Mapping the numbers back onto user behavior, representative operations landed here:

User	Representative operation
Annotator	Task detail 25,721 → 205 ms, Annotation fetch 15,813 → 154 ms
Requester	Statistics 22,514 → 224 ms, Completion status 9,107 → 105 ms
Project operator	Preset list 17,580 → 88 ms, Member list 27,555 → 118 ms

Every one of them now comes in under a second. An upfront caveat: this is not field-measured productivity from user research—it’s an operational delta explainable by a reproducible benchmark.

---

Based on material presented at the results presentation on July 31, 2026.