Anthropic already ships a warning about this in the system prompt. It loses to a note the model wrote about me last month, and the reason is not authority.
I came across this too, some memories in Claude were making my evaluations fail, it took me a while to find out that those memories were making prod succeed at the same task while evals where failing
That’s a great example of why hidden memory is so difficult to debug. The same task can behave differently in prod and evals because the agent is carrying context you didn’t realize was part of the system.
I came across this too, some memories in Claude were making my evaluations fail, it took me a while to find out that those memories were making prod succeed at the same task while evals where failing
That’s a great example of why hidden memory is so difficult to debug. The same task can behave differently in prod and evals because the agent is carrying context you didn’t realize was part of the system.