perf(session): cache messages across prompt loop to preserve prompt cache byte-identity

OpenCode updates tool part states in-place (pending → completed + output)
between consecutive API calls in the tool-execution loop. When the next
API call serializes the conversation, the previous assistant message has
different bytes (completed state + output vs pending/error placeholder),
breaking Anthropic's prompt cache from that point forward.

On real sessions this causes ~20% of turns to re-write the entire context
at the cache-write price (12.5× cache-read). On April 21st alone, this
cost $2,264 in cache writes vs $1,234 in cache reads.

Fix: cache the conversation array across prompt loop iterations. On tool-
call continuation steps, only append genuinely NEW messages instead of
reloading all messages from the DB. Existing messages retain their
original part states (as the API last saw them), preserving byte-identity
for the prompt cache.

Full reloads still happen after compaction, subtask handling, and overflow
recovery — these operations structurally change the conversation.

Author: BYK

packages/opencode/src/session/prompt.ts

@@ -1279,11 +1279,21 @@ NOTE: At any point in time through this workflow you should feel free to ask the
         let step = 0
         const session = yield* sessions.get(sessionID)
 
+        // Cache conversation across prompt loop iterations to preserve prompt
+        // cache byte-identity. Full reload only on first iteration and after
+        // compaction/subtask. Tool-call continuation appends only NEW messages
+        // to avoid re-reading updated tool parts that break the cache prefix.
+        let msgs: MessageV2.WithParts[] | undefined
+        let needsFullReload = true
+
         while (true) {
           yield* status.set(sessionID, { type: "busy" })
           yield* slog.info("loop", { step })
 
-          let msgs = yield* MessageV2.filterCompactedEffect(sessionID)
+          if (needsFullReload || !msgs) {
+            msgs = yield* MessageV2.filterCompactedEffect(sessionID)
+            needsFullReload = false
+          }
 
           let lastUser: MessageV2.User | undefined
           let lastAssistant: MessageV2.Assistant | undefined
@@ -1335,6 +1345,7 @@ NOTE: At any point in time through this workflow you should feel free to ask the
 
           if (task?.type === "subtask") {
             yield* handleSubtask({ task, model, lastUser, sessionID, session, msgs })
+            needsFullReload = true
             continue
           }
 
@@ -1347,6 +1358,7 @@ NOTE: At any point in time through this workflow you should feel free to ask the
               overflow: task.overflow,
             })
             if (result === "stop") break
+            needsFullReload = true
             continue
           }
 
@@ -1356,6 +1368,7 @@ NOTE: At any point in time through this workflow you should feel free to ask the
             (yield* compaction.isOverflow({ tokens: lastFinished.tokens, model }))
           ) {
             yield* compaction.create({ sessionID, agent: lastUser.agent, model: lastUser.model, auto: true })
+            needsFullReload = true
             continue
           }
 
@@ -1489,6 +1502,20 @@ NOTE: At any point in time through this workflow you should feel free to ask the
                 auto: true,
                 overflow: !handle.message.finish,
               })
+              needsFullReload = true
+            } else {
+              // Tool-call continuation: append only the new assistant message
+              // instead of reloading all messages from DB. This preserves the
+              // byte-identity of prior messages (especially tool parts that
+              // transitioned pending→completed after the previous API call)
+              // for Anthropic's prompt cache.
+              const fresh = yield* MessageV2.filterCompactedEffect(sessionID)
+              const existingIds = new Set(msgs!.map((m) => m.info.id))
+              for (const msg of fresh) {
+                if (!existingIds.has(msg.info.id)) {
+                  msgs!.push(msg)
+                }
+              }
             }
             return "continue" as const
           }).pipe(Effect.ensuring(instruction.clear(handle.message.id)))