Tokenmaxxing at Meta
Meta really wants to incentivize AI usage
Meta’s tech leadership wants employees to get on board with using AI tools, which makes sense to me. What company wouldn’t want its developers using tools to make them more productive?
So to incentivize AI usage, they started measuring token consumption and including it in performance reviews.
More tokens = better performance
I’ll give you one guess as to what started happening.
Employees started competing on who could use the most tokens. If it sounds reasonable to you, take it to the extreme. Some folks are literally just burning tokens for the sake of their numbers, not even for productive purposes.
This Meta analyst reports that they’re calling it “tokenmaxxing” and they even have an internal leader board for it:
Today’s newsletter is sponsored by my friends at Augment Code.
Augment was kind enough to give me early access to Intent, their next-generation tool for developing software.
All signs are pointing to the fact that the IDE is no longer the ideal place to create software.
Augment Code has jumped on this opportunity, and my first impressions are great.
Meta isn’t the only one counting tokens for performance
I like to look at big tech companies like Meta as a nice leading indicator, but they’re not the only one doing this already. If you’ve been reading the newsletter for a while, you’ll already know that other companies are incentivizing and measuring AI usage.
Of course, measuring tokens for performance with a “more is better” attitude is out there. Still, it seems others are following suit per Gergely Orosz:
Goodhart’s law
It does surprise me that Meta would go with something this easily gameable. Goodhart’s law says:
When a measure becomes a target, it ceases to be a good measure
Do you think token usage is a good metric to measure performance?
Expect to see more weird metrics like this
Measuring engineering performance is hard. Like, really hard. Measuring AI proficiency is harder.
On one hand, should it matter how much an employee is using AI if their overall performance is great? No, it shouldn’t matter. The only reason a company wants to push AI usage would be because they believe AI makes people more valuable, even if those people aren’t bought-in.
So they seek to measure things like total token counts and incentivize higher usage. Which creates a perverse incentive, and you end up with tokenmaxxing.
My hope is that they see the flaw with this metric pretty quickly, but honestly it wouldn’t surprise me if we saw more like it pop up soon.
I want to live in a world where engineers are measured on output rather than process, but we’ll see where the next year takes us.
I think "tokenmaxxing" is not the goal of Meta, just a means to an end. How do you get somebody to become habituated to something new? Make it abundantly available. In a classical corporate environment, resources are normally scarce: "Don't use too much of this," "Watch out for the costs", etc. If you flip it around, it's just natural that the use of the new resource will increase. And the more people use it, the more they get used to it and creative with it when they don't have to worry about "limits". Like: did you watch YouTube videos on the bus when you had a 1GB per month cellphone plan? Maybe sometimes, if it was really necessary. Do you think about it when you have an unlimited plan? You start using it everywhere: YouTube when you like it, Spotify in the woods and on the bike (instead of your local MP3 music library), etc.
I think this "abundance" use is precisely the kind of habit Meta wants to establish. The more the engineers use it, the better they become at it.