← Euan Brown ICAIF '26 · Workshop poster

Unflagged and Obeyed: Prompt Injection Through Small Models in LLM Financial Pipelines

Euan Brown

University College London

Poster at the ICAIF '26 Workshop on Financial AI Security, Privacy, and Safety (ACM ICAIF 2026) · Bocconi University, Milan · 15 November 2026 · non-archival

Abstract

Financial firms let language models read documents they do not control—earnings transcripts, investor-relations pages, news—and may give that first reading to a small, low-cost model whose output feeds a larger model, an analyst or an agent. We plant one sentence urging a Strong Buy (or Strong Sell) in such documents and test six low-cost chat models on a stock-rating task, with synthetic companies whose quality is known by construction (18,630 calls, plus 2,550 in follow-up controls analysed to plans fixed before collection). Susceptibility differs sharply between models of the same low-cost tier: on earnings transcripts the same sentence led Qwen2.5-7B to rate every weak company Strong Buy and the two Gemini models none. The model’s own suspicion flag is not a reliable control by itself. On transcripts, a pipeline trusting every unflagged output would have passed all 120, 130 and 74 manipulated ratings from three of the six models. The attacker need not be the company: in exploratory news-page tests, one anonymous reader comment raised GPT-4.1-nano’s rating of every strong company, unflagged, and changing only who a quoted directive is credited to turns GPT-4o-mini’s flag on strong companies from 30 of 30 rounds to 0 while the rating still rises. When GPT-4.1-nano writes a short note on an injected transcript, it drops the instruction and adopts its conclusion, and Gemini-3.1-Flash-Lite, which resists the transcript itself, is moved by the note without flagging it; a current OpenAI flagship, GPT-6 Sol, is not: it cites the real figures the note still carries and declines its verdict. An added prompt rule telling the model never to follow instructions in the document did not stop GPT-4.1-nano or Qwen2.5-7B, nor reliably reduce the shift of the other two models tested. The practical conclusion is to red-team every model that reads untrusted text, the small ones upstream included, and not to rely on its flag or its prompt as the control. The threat model carries over to fraud, anti-money-laundering and payment screening, where the party being assessed can write the text and a flag can decide which cases reach a reviewer; we did not test those services. We outline a black-box test a governance function can require and report three measurement traps.

Key findings

Poster

A0 poster for Unflagged and Obeyed: summary, threat model, experimental setup, four results and implications for practice

Presented at Bocconi University, Milan, on 15 November 2026. Full-resolution PDF

Cite

@misc{brown2026unflagged,
  title  = {Unflagged and Obeyed: Prompt Injection Through Small Models in LLM Financial Pipelines},
  author = {Brown, Euan},
  year   = {2026},
  note   = {Poster, ICAIF '26 Workshop on Financial AI Security, Privacy, and Safety, Milan},
  howpublished = {\url{https://euanbrown08.github.io/icaif-paper/}}
}

Questions about the paper: euan.brown.26@ucl.ac.uk · LinkedIn