← Back to full report
What This Report Is About
This report explains how large language models work, from tokens and embeddings through the transformer architecture and training pipeline. It is written for accounting and finance academics who have used a web-based chat interface but have not been exposed to the underlying mechanisms. No computer science background is assumed.
The report covers the full LLM lifecycle: the foundations, training methods, capabilities and limitations, applications in accounting research and practice, implications for academic workflows including teaching and publishing, AI safety and governance, and a structured research agenda.
Key Findings
LLMs are prediction engines, not reasoning agents. At their core, these models predict the next word fragment in a sequence. The transformer architecture, introduced by Vaswani et al. (2017), makes training at massive scale possible by processing all words at once rather than one at a time. A simple way to think about it: an old model reads a sentence left to right; a transformer reads the whole sentence and, for each word, asks "which other words matter most right now?" Training compute has grown roughly 4.5-fold per year, and frontier model parameter counts (where not publicly confirmed) are estimated in the range of 1.5 to 3 trillion.
Hallucination is improving but remains a structural problem. On the Vectara HHEM document summarisation benchmark, hallucination rates have fallen from 6 to 12 per cent in early 2024 to 1 to 3 per cent for the best mid-2026 models. On open-ended generation, rates stay higher. The training objective, predicting the most probable next token rather than checking truth, means no current approach can eliminate hallucination. Every AI-generated factual claim should be independently verified.
AI adoption in accounting is accelerating faster than expected. AI adoption among accounting firms jumped from 9 per cent in 2024 to 41 per cent in 2025 (Wolters Kluwer). The Big Four have each announced major AI investments totalling USD 4.4 billion. Junior accounting hiring fell by 16 per cent over roughly two years as routine tasks were automated (Stanford AI Index, 2025). The profession is shifting from computation toward judgment.
Key Statistics
4.5x per year: growth rate of training compute for notable AI models since 2010 (Epoch AI, 2026)
100 million users in two months: ChatGPT adoption, fastest consumer application on record (Reuters/UBS, Feb 2023)
1.5 per cent: hallucination rate of GPT-4o on document summarisation (Q4 2024), down from 6.2 per cent for GPT-3.5 (Q1 2024) (Vectara HHEM leaderboard)
41 per cent of accounting firms used GenAI in 2025, up from 9 per cent in 2024 (Wolters Kluwer, Future Ready Accountant)
68 per cent of tax and accounting professionals feel excited or hopeful about GenAI (Thomson Reuters, 2025)
16 per cent decline in junior accounting hiring over roughly two years, attributed to automation (Stanford AI Index, 2025)
USD 4.4 billion: combined announced AI investment commitments by the Big Four (firm announcements, 2023-2025)
EUR 35 million or 7 per cent of global turnover: maximum penalty under the EU AI Act (Regulation 2024/1689)
300 trillion tokens: estimated stock of quality human-generated text for AI training; may be exhausted by 2026-2032 (Epoch AI, 2024)
112 AI-related publications in top accounting journals projected for 2026, up from 8 in 2017 (author keyword search; 2026 projected)
The Transformer at a Glance
The architecture that enables every modern large language model was described in a single 2017 paper: "Attention Is All You Need" by Vaswani and colleagues at Google. Its key innovation was replacing sequential processing with parallel attention.
Before transformers: models read sentences left to right, one word at a time, like reading a novel. By the time they reached the end of a long sentence, they had often forgotten the beginning. And because each step depended on the previous one, the work could not be split across computer chips.
With transformers: the model reads the entire sentence at once and, for each word, works out which other words matter most. Think of a detective scanning a room of 10 people and asking, for each person, "who here is most relevant to this person?" Words that relate to each other get high attention scores; unrelated words get low scores. In the sentence "The animal did not cross the street because it was too tired," the word "it" pays high attention to "animal" (what it refers to) and "tired" (why), linking them directly even though they sit several words apart.
Because every word is processed in parallel, transformers can use modern GPU hardware efficiently. This is what made training models with hundreds of billions of parameters practical. A modern transformer stacks 32 to 128 of these attention layers, with multiple attention "heads" running in parallel to track grammar, numbers, time references, and entity relationships.
Why It Matters
Large language models are not just a new tool. They change what accounting and finance researchers can measure, how research is done, how students are taught and assessed, what professional services look like, and what governance frameworks are needed. The scholars who engage with these questions now will shape the research agenda for years to come.
The full report includes eight interactive data visualisations, eight downloadable datasets with methodology documentation, and a Python replication script that regenerates every chart and statistic.