At a small firm, AI assistant seats tend to arrive in a batch and go out by request. The partners get them because they asked first, or because seniority seems the natural order. A few weeks later someone asks whether the seats are doing anything, and nobody can say, because nobody wrote down what the work looked like before.
The operating decision is who gets the assistant first, what the firm measures once they have it, and how it watches the quality of work from the people whose work nobody reviews. I went looking for evidence before writing a rollout plan, and found the largest field study of an AI assistant at work that I know of. It is not about investing, and I say where that matters.
Source
What the study measured
Erik Brynjolfsson (Stanford), Danielle Li (MIT), and Lindsey Raymond (MIT) published "Generative AI at Work" in The Quarterly Journal of Economics (volume 140, issue 2, pages 889 to 942, 2025). The article is open access. I read the published text on Oxford Academic and checked every figure below against it. The publisher's site would not serve me its PDF, so the page numbers refer to the authors' version on arXiv, dated November 2024, which reports the same figures. Its figure numbers differ from the journal's.
The setting is customer support at one company: a Fortune 500 firm that sells business-process software to small and medium-sized businesses in the United States. Its agents answer technical questions by chat, sessions average 40 minutes, and most of the agents work in the Philippines (page 10). In all, 89 percent were outside the United States (page 13).
The assistant was built on a model from OpenAI's GPT family and trained further on the firm's own support conversations, with particular weight on those of its top performers. It suggested replies and links to internal documentation, shown only to the agent, who was free to ignore them (pages 11 and 12). It was rolled out mainly in fall 2020 and winter 2021, at a pace set by the number of trainers and licenses available, so agents on the same team got access at different times (pages 12 and 13).
The data cover 3 million chats by 5,172 agents (page 13, Table 1) between September 2019 and June 2021 (page 59). The authors compare each agent before and after access with colleagues who did not yet have it, controlling for the agent and for months of tenure (page 15). The main measure is resolutions per hour: chats successfully resolved. They also track handle time, the share of chats resolved, and a customer satisfaction score from post-chat surveys (pages 13 and 14). An agent's skill is ranked from speed, resolution rate, and satisfaction together, in the month before access (page 19).
Source
What it found
On average, the assistant raised resolutions per hour by 0.30, or 15 percent (page 16, Table 2). Chats got 3.7 minutes shorter, an 8.5 percent drop from a baseline of 43 minutes. The share of chats resolved rose 1.3 percentage points from a baseline of 82 percent, a change too small to be statistically significant, and customer satisfaction did not change in any meaningful way, moving 0.12 points on a mean of 80 (page 16, Table 3).
The gains depended on who the agent was. Agents in the lowest fifth by skill gained 0.5 resolutions per hour, or 36 percent. The most skilled agents saw no productivity increase (page 19, Figure 3). By experience, agents with less than a month of tenure gained 0.7 resolutions per hour, and agents with more than a year gained nothing (page 20, Figure 3). Across less skilled and less experienced workers, the authors put the gain at about 30 percent (page 2).
The most skilled lost a little quality. For them, the effects on resolution rate and customer satisfaction were small but statistically significant declines (page 19, Figure A.6). The most experienced agents showed the same pattern (page 20, Figure A.7). The authors suggest the suggestions may distract top performers, or lead them to take the faster option rather than compose their own reply (page 19). They also report that top workers follow the assistant more over time, even though its suggestions marginally lower the quality of their conversations (page 4).
New agents climbed the experience curve faster. Agents with two months of tenure and the assistant performed as well as or better than agents with more than six months and no assistant (page 21).
Customers behaved better. Customer sentiment, measured on a scale from minus 1 to 1, rose 0.18 points, about half a standard deviation (page 32, Table 4). Requests to speak to a manager fell by almost 25 percent from a baseline of about 6 percent of chats (page 33). Here too the most skilled and most experienced agents saw the smallest benefit (page 33).
Describe this figure
Two parts. The first is a horizontal bar chart of resolutions per hour gained with the AI assistant, on a linear scale from 0 to 0.8. All agents gained 0.30, or 15 percent. Agents in the lowest fifth by skill gained 0.5, or 36 percent. The most skilled agents, the highlighted row, saw no productivity increase, drawn as a labelled zero, and small, statistically significant declines in resolution rate and customer satisfaction, for which the text gives no size. Agents with under a month of tenure gained 0.7. Agents with over a year of tenure gained nothing, also a labelled zero. The second part lists four quality and customer measures for all agents, each in its own unit and without bars: the share of chats resolved rose 1.3 percentage points on a baseline of 82 percent, too small to be statistically significant; customer satisfaction moved 0.12 points on a mean of 80, no meaningful change; customer sentiment rose 0.18 on a scale from -1 to 1, about half a standard deviation; requests for a manager fell almost 25 percent from about 6 percent of chats. The most skilled and most experienced agents saw the smallest benefit to customers. The study is of customer support at one software firm, 5,172 agents, 2019 to 2021, not an investment firm; Brynjolfsson, Li, and Raymond, QJE 2025.
Agents did not follow the assistant blindly. They adopted 38 percent of its suggestions on average, with an interquartile range of 23 to 50 percent (page 23). Exposure seemed to teach: during outages, when the assistant was unavailable, agents who had used it still did better than before they had it, more so the longer and more closely they had used it (pages 22, 25 and 26). The gains were largest on moderately rare problems, and the time saved was greatest on problems an agent had met least often (pages 27 and 28).
The authors are plain about the limits: one tool, one firm, one occupation, and not to be generalized across all occupations and AI systems (page 34).
Synthesis
What the study does not tell an investment firm
The setting is far from ours, and the gap is wide. The support operation handled millions of similar chats and recorded the time and outcome of each one. A small investment firm writes a few dozen memos, letters, and questionnaire answers a quarter, each different, and measures none of them. The study's assistant was trained on that firm's own best agents. A small firm's assistant is a general model that has never seen its best memo. The support product was relatively stable (page 34). A firm's questions change with each company and each fund.
The quality measures differ too. In the study, quality meant a resolved problem and a satisfied customer. At a firm, quality means a correct figure in a letter to limited partners, an accurate answer in a due diligence questionnaire, and sound judgment in a memo. A small decline in those is a different kind of cost.
The model is also old. The published article names it as GPT-3, well behind what firms use now. That cuts both ways, and nothing here measures current models.
Synthesis
What carries across anyway
The mechanism carries. The assistant helped most where it could carry what the best people already did to people who did not yet know it. A small firm has that situation wherever a newer person does repeatable written work that someone senior has done well before: a first-draft investment memo, a quarterly portfolio update, the answers to a questionnaire from a prospective investor. The experience-curve finding matters more at a small firm than a large one, because each new hire is a larger share of the firm's capacity.
The finding about rare problems carries in a useful way. An experienced person doing a task they meet once a year is, for that task, closer to a new hire. That is a second place to give the assistant early.
The warning about the most skilled carries most of all. The people whose work was best gained the least, lost a little quality, and followed the assistant more over time. At a small firm, those people are the partners and the senior operations and compliance staff, and their drafts are the ones nobody reviews. There is also a longer effect. In the study, top performers' conversations trained the assistant (page 11). At a firm, the senior people's memos and letters are the examples newer staff learn from and the material a future assistant will be given. If those drift toward the assistant's defaults, the firm loses its best examples without noticing.
Proposal
Who gets it first, and what the firm measures
I would give the assistant first to the newest people, for the repeatable written work they already do, and to experienced people for the tasks they do rarely. I would give it to senior people for their core work last, with the quality watch below in place from the first day. Seats are not a reward for seniority.
Before anyone starts, I would settle what the assistant may see: which documents, which accounts, and whose access it uses, as I set out in before an AI assistant can access firm data.
For each task in the rollout, I would pick two measures the firm can count without new tools: turnaround, the days from request to a reviewed draft, and review findings, the corrections a reviewer makes. I would record both for six to eight weeks before access, then for twelve weeks after. With a handful of people, this is a structured look, not a statistic. The point is to have a before.
Proposal
How I would watch the most experienced people
Each month, a peer reads two pieces of a senior person's assisted work against the same checklist the firm used before, and records what they find. Each quarter, each senior person writes one of each recurring piece without the assistant, which gives the firm its own version of the study's outages. When a draft goes out largely as the assistant wrote it, the file says so, because following the assistant more while quality slips is the pattern the study found. And the firm's library of example work keeps the senior person's own versions, not the assisted ones.
At 90 days, the firm decides for each task whether to widen access, hold, or withdraw it, from the measures and the review findings.
Open question
Will newer staff learn the work, or only the output?
The study found evidence that support agents learned from the assistant and kept the gains when it was off. At an investment firm, a junior person learns judgment by drafting and being corrected. If the first draft is the model's, the correction teaches something different. I do not know whether that is better or worse. Asking newer staff to write one piece each quarter without the assistant, as the senior people do, is the cheapest way I know to find out.