QUANTITATIVE RESEARCHER · FOR FINANCIAL MARKETS
Quantitative Researcher
Notes on financial markets, quant research, and AI engineering
NOTES · RESEARCH LOG · WORKING NOTEBOOK6 ENTRIES · 2 LANES · UPDATED ON SHIP
01 / research log
The notes page is not a blog.
A working notebook. Entries are written when the work has something to teach : a postmortem, a sharp observation, or a small lesson that survived the loop. No editorial calendar, no publishing pipeline.
Quant PBO is a multiple-testing tax, not a model-fitting tax.
The combinatorial-symmetric-cross-validation (CSCV) result is best read as a tax on the search procedure, not on the strategy itself. When the search is narrow (one model, one feature set, one decay), PBO stays low. When the search is wide (50 candidates, all backtested with full hyperparameter sweep), PBO eats the headline Sharpe before fees. The cleanest way to deflate it is to pre-register the candidate count BEFORE running the backtests : that one line of process collapses the false-discovery tax by half on a typical quant funnel.
Methodology · multiple-testing layer →Quant Regime-conditional deployment > a single Sharpe ratio.
A 0.6 Sharpe over 4 regimes is not the same artifact as a 0.6 Sharpe over one regime with 3 silent failures. Splitting the equity curve by regime (carry / momentum / vol / macro) and shipping a strategy only when every regime clears its own threshold is the cheapest insurance a quant book can buy. The hard part is naming the regimes : most "regime detectors" are just lagged rolling z-scores on the wrong variable.
Quant Deflated Sharpe is a backtest validator, not a forward-edge oracle.
Deflated Sharpe tells you whether the headline number is consistent with the number of trials you ran. It does NOT tell you the strategy will work out-of-sample. The two are often confused, and the confusion costs real money. Treat deflated Sharpe as a sanity gate: if it fails, you definitely have a curve-fit. If it passes, you still need walk-forward + paper + live evidence before capital.
Proof · ship gates →AI The eval harness is the product. The LLM is the demo.
Most "AI products" are demos. The product is the eval harness : the deterministic suite that scores the LLM on its own claim, catches regressions, and freezes the spec so the prompt engineer knows what success looks like. Without that suite, every prompt change is a coin flip. With it, every change has a P95 latency, a pass-rate, and a regression budget. The 30 hours spent building the harness beat the 300 hours spent iterating prompts blind.
AI Multi-agent ≠ faster. Multi-agent = more checks per minute.
A common failure mode is "we have 5 agents, so we ship 5× faster." In practice, the agents spend most of their time waiting for the orchestrator, and the orchestrator spends most of its time reconciling. The win is not throughput : it is coverage: each agent independently checks a different failure mode (numerical faithfulness, citation grounding, schema conformance, tone drift, hallucinated entities). The harness stays the bottleneck; the agents are just additional lenses.
AI Citation grounding > "look at the source."
Asking an LLM to "look at the source" is not grounding. Grounding is: every claim is followed by a (chunk_id, relevance) tuple from a deterministic retriever, and the tuple is checked at eval time. Without the tuple, "grounded" is a vibe. With the tuple, you can audit which chunks the model actually used and which it ignored, which is the only signal that survives a real-world corpus.
AI page · eval layer →
LAST UPDATED · 2026-08-09 · UTC+8