Notes on LLMs - July 2026

28 July 2026

The last few months have been a transformative time in LLM research. In this post I look at some research directions I’ve been interested in, and propose how they might be advanced further.

The assistant persona isn’t perfect

If you’ve used any LLM product at all, you’ll have been engaging with the “assistant” persona. This is a vocabulary, style of speech, and set of behaviours that is trained into the model to turn it from a really good next-word predictor into a helpful Q&A-style task completer. It’s this process that effectively makes the model usable for real tasks. However, there are issues with the way these assistant-style models function.

Overly Apologetic Language

Sometimes when working on a large task involving lots of back-and-forth with an LLM agent, you’ll see language like this:

“You’re absolutely right! I’m so sorry I missed that. Now I have the full picture!”

As HumanLayer founder Dex Horthy says in a talk, if you see this as a developer, it’s probably time to wipe the slate and start over. This occurs when the model realises it’s made a mistake, and needs to be corrected by the human user. This means that either you’ve been barking up the wrong tree, or the model has. There are two issues here:

• 1: Models are anecdotally very bad at pushing back against bad advice. If a model does some correct work, and you provide moderately incorrect feedback, it’s quite easy to get the model to agree with you and implement the bad advice. This is terrible for software engineering and other domains where precision is paramount
• 2: Language models are pattern-continuers: everything in the context window acts as evidence about what should come next, so each occurrence of a pattern makes its recurrence more likely. Liquid AI measured exactly this at the token level, with repeated phrases climbing towards probability 1. My worry is the semantic version: every “You’re absolutely right, I missed that!” in the history is evidence that this is an assistant that gets things wrong and concedes, making future outputs more likely to be self-corrective, even when they shouldn’t be. Push far enough and the model gets stuck in a “doom loop”, or “self-corrects” away from output that was actually right

Liquid AI’s anti-doom-loop work is an interesting existing study in this area. In their work, a doom loop is a repetition spiral inside a single model output. Often the repeated phrase is a self-corrective one such as “Wait, let me reconsider…” etc. They managed to largely eliminate them with a small preference-training run. I want to highlight two of their findings:

• With each repetition of a phrase, the probability of that phrase being repeated again climbs closer to 1
• Small, targeted post-training runs can heavily mitigate this kind of failure mode without harming benchmark performance

What they don’t test is a conversational version: Whether apologetic, self-corrective language in a multi-turn history makes a model more likely to “correct” work that was already right. I think this is worth investigating. A possible setup could be:

• 1: Select a set of open weight models
• 2: Give each model a task it solves correctly, then continue the conversation with injected histories containing varying numbers of turns of apologetic / corrective language
• 3: Measure the logit-probability of the self-corrective openers (“Wait”, “Actually”, “You’re right”, etc.) as the history grows and the rate at which the model revises a correct answer into an incorrect one

If this effect is measurable, then compare the effectiveness of two mitigation strategies:

• Context engineering to remove the apology turns from the history
• Liquid-AI-style targeted training fix aimed at semantic self-correction rather than verbatim repetition

Impossible Semantic Density

If you work on abstract, difficult, or long planning tasks, you’ll start seeing language like this:

Source: @mykola on X - https://x.com/mykola/status/2081450468178579594

What does this even mean? It reads like a riddle you’re given at the start of a quest in a medieval fantasy game, not like technical feedback from a colleague.

Whether the example above is real or not, I’ve seen far worse in my real coding agent sessions. I’ve spent 15 minutes trying to read 10 words so dense I expected them to collapse into a singularity.

This behaviour is very frustrating because it forces you to waste time trying to understand unnecessarily complicated sentences that could be expressed far more simply.

So, what causes this? Part of the training process of large language models involves changing the language / vocabulary / style of the model. Some of this measurably shifts models towards denser, more compacted prose. Additional possible effects on increased semantic density are:

• When training reasoning models, if we only incentivise the correctness of the final answer of the model, the opaqueness of the language used internally by the model seems to increase. Under these conditions, models may naturally drift towards an “internal” vocabulary that is useful for improving final answer correctness at the cost of human legibility
• Part of the training process of modern language models known as RLHF (Reinforcement Learning from Human Feedback) appears to significantly reduce the diversity of outputs from the model while increasing the generalised performance of the model to unseen, real-world scenarios. This may help to explain why so many of the same aphorisms, sayings, and words appear again and again when working on difficult tasks
• Training modern language models sometimes involves using other AI models such as reward models or LLM-as-a-judge systems to “judge” the performance of a model in training. These models used in the training loop have been found to favour prose style over factuality

The origin of excessive semantic density is worth investigating. Some ideas are:

• Training-stage ablations - Take one base model and compare SFT-only, RLHF, and reasoning-RL variants using the same density metrics to measure the effect of each stage
• Scratchpad leakage - Measure the stylistic similarity between a model’s chain of thought and its user-facing prose, and whether user-facing semantic density scales with the amount of reasoning-RL a model has had
• Aphorism selection - Audit reward models directly: Score matched pairs of responses (same content, aphoristic vs. plain phrasing) and quantify how much reward the aphoristic version collects for style alone

And one idea for reducing semantic density rather than explaining it: When an LLM judge scores a response today, it usually sees the entire conversation that produced it, so a final message full of private shorthand still scores well, because the judge already knows what every phrase refers to. A human doesn’t read this way: They usually read the final message only. If the judge were shown only the final message, and asked whether every referent is introduced and every claim understandable on its own, it would experience exactly what the human experiences. That could help to make legibility measurable, and anything measurable can be trained.

Removal of Voice

Nathan Lambert, previously the post-training lead at Ai2, has written about the issues of AI-generated writing, and probable contributors to these problems that arise in the training process. The most frustrating one from my perspective is the “voice” or “personality” that all AI-generated writing is devoid of. This is even observable when asking a model to take voice-rich human-authored text and summarise or rephrase it. While the models will do a good job retaining the details & structure, much of the meaning of the writing evaporates away as you watch the output generated in real time before you.

This property of the models probably results from the training process aggregating style preferences of the average user for use as a training signal, which smooths out rough edges, leaving a bland, inoffensive, safe kind of persona. Great or interesting writing tends not to be written by such characters.

As a side note I suspect that this property is at least part of the reason why many creative industries (writing, media, music) seem vehemently anti-AI, and I understand their perspective. Although intangible, AI-generated writing and artistic content is largely empty and soulless.

I think it would be interesting to measure whether “voice” survives being passed through another model, and what kind of data carries it. A possible plan would be:

• 1: Take a teacher model with a strong, measurable persona. The open character-training models, released with LoRA adapters, are ideal: fully reproducible, with no dependence on a closed model’s outputs
• 2: Use the teacher to generate one fixed corpus of conversations
• 3: Prepare that same corpus in three forms, so the content is identical and only the transformation differs: the raw transcripts, LLM-written summaries of them, and LLM paraphrases of them
• 4: Fine-tune the same fresh base model on each form, producing three “student” models
• 5: Measure how much of the teacher’s persona each student inherited, not by asking the students who they are, but with behavioural probes scored by a judge, which the Personascope harness was built for

A recent study asked whether models distilled on Claude outputs inherit Claude’s persona, but it only tested prompting existing models to claim the identity (“You are Claude…”); no model was actually trained. If summarisation strips voice the way it feels like it does, the student trained on summaries should inherit measurably less persona than the student trained on raw transcripts. That gap is a number I could not find in any published research.

Long-horizon Performance Decay

LLMs are primarily evaluated on their performance at “one-shot” tasks. Even multi-step agentic benchmarks evaluate a single static goal with one final verification. This means benchmarks often consist of a set of single prompts which are sent to a model. Each output is then evaluated independently and the scores are aggregated. This is mostly how model training companies advertise the power of their products. The problem with this is that it doesn’t reflect how most white-collar work is actually done - iteratively, and with changing requirements.

Using software as an example: when developing a piece of long-term successful software, you will start with an initial plan. At numerous points in the software’s lifecycle, large changes will be required that couldn’t have been anticipated when starting out due to business requirements, economic factors, unseen technical problems etc. This means key skills of any good software developer include:

• A: Writing software that effectively meets current requirements, but which may be easily extended in the future if necessary
• B: When starting out, choosing technologies and architectures that are unlikely to result in strict constraints or painful re-writes later on

It seems to me that currently, models largely aren’t trained with skills A & B in mind. This results in a situation where when starting out with a coding agent, many developers are extremely impressed and begin to rely on them more. Over time, however, they notice a degradation in the output that is difficult to explain. If that degradation is left unchallenged, many codebases will devolve into what has become known as “slop” - code that might functionally work, but that is extraordinarily difficult to understand and thus maintain going forward.

Presently, although LLMs are excellent at writing code, any developer using AI heavily will tell you that they struggle with this iterative requirements process. This could plausibly be because of how models are primarily trained: Take a single task, solve it, and judge the result. This is not representative of how software developers work week to week. To become representative, models would need to be trained and evaluated somewhat like this:

• 1: Take a single task, solve it, and judge the results
• 2: Introduce a new constraint or design requirement that punishes poor skill at A or B and ask for a rewrite. Judge the result
• 3: Repeat step 2 many times and judge the overall results at the end
• 4: Repeat steps 1 - 3 over many long-horizon tasks

This is much more difficult to measure than single-shot tasks, but I want to highlight SlopCodeBench as a recent piece of work investigating this in detail. They present a benchmark consisting of 20 problems with evolving requirements checkpoints over time. They introduce metrics to attempt to quantify “slop” and find that on average, these metrics get worse step-to-step inside each problem. Using the same metrics, albeit on different tasks, they find that human work degrades less checkpoint-to-checkpoint:

Figure 4 from Orlanski et al., SlopCodeBench

The authors also test a couple of common context-engineering tactics used by developers to mitigate this issue. These are:

• 1: Having the agent plan the exact set of steps it will take to solve the task at hand before making any changes at all
• 2: Explicitly instructing the agent to avoid unnecessary complexity, verbose code and other code properties associated with “slop”

They find that although these tactics raise the initial level of quality for each multi-step task, they have little impact on the iterative degradation. This could suggest that this problem is best mitigated in the training process, not with context engineering. Although the authors don’t do any model training in this work, I think a natural extension of this would be to run post-training experiments on coding agents using this style of iterative data.

I think training on refinement trajectories would be really interesting. This could involve:

• 1: Extract git histories for chains of sequential feature addition commits
• 2: Build multi-step refinement training data
• 3: Fine-tune a small open coding model on it
• 4: Compare quality degradation on held-out problems against the base model

Where I’m taking this

I hope to run the experiments I’ve outlined above in the coming weeks and months. When I do I’ll write about them here. If you find any of these topics compelling or have thoughts of your own on the subjects, reach out to me at john1091w@gmail.com - I’m keen to collaborate.