Discovered that chunking strategy matters more than embedding model choice. Switching from fixed 512-token chunks to semantic paragraph splitting improved retrieval accuracy by 34% on our legal doc corpus.

LoRA rank 16 was the sweet spot for our use case — rank 32 overfit on our 8k sample dataset. Training on diff-format examples outperformed full-file examples by a wide margin for catching logic bugs.

Supervisor-worker pattern with explicit handoff protocols reduced hallucination in complex research tasks by 60%. Key insight: agents need structured output schemas at every handoff point.

Implementing semantic caching with a similarity threshold of 0.92 cut our API costs by 45% without noticeable quality degradation. Redis + pgvector hybrid approach works best for mixed workloads.

Built a deterministic eval harness that runs 200 scenarios in under 3 minutes. The trick is mocking tool outputs with recorded traces rather than hitting real APIs. Regression detection caught 12 bugs pre-deployment.

Token-by-token streaming feels janky for structured outputs. We switched to chunk-based streaming (sentence boundaries) with skeleton loaders and perceived latency dropped by 70% in user testing.