Multiple status ladders are better than a monoculture
Reflections on my love-hate relationship with bay area culture.
I just got back from a week in the bay area for rationalist summer camp, which was a great time all around1. It reminded me of all the irreplaceable things I love about the bay area. And also of why I ended up leaving.
I should start with all the things I love2. You’re constantly surrounded by your kind of people. People who love talking about ideas they enjoy, whether it’s because they read a blog post about them and think they were cool or they’re professionals working in the field for twenty years. San Francisco is the global center of startup culture, not just because of economic agglomeration, but also because it’s the place where the culture exists to push people from the first category to the second, to give the guy who just read a cool blog post on cryonics social permission to just go ahead and start his own cryonics startup3. It’s the place where you can aimlessly hang out in the park and meet one group of people practicing juggling, another couple reading HPMOR, and a third guy with a backyard hydroponics garden or a rocket startup4.
And it’s not just that. People have dreams. The whole “making the world a better place” trope is real, people do actually want to do things that matter. In other places people talk about getting through the day, but in San Francisco they talk about solving humanity’s biggest problems or building things no one’s ever built before. You don’t get social status for working at the best hedge fund or getting cast in the best broadway show5, you get it for building a cool thing, and people talk about their big ambitious plans. To quote UNSONG:
California had never entirely been a real place. It was impossible to live there for any amount of time and think it was just another state, like Nevada or Ohio or Vermont. It was a state like joy, or exaltation, the ultimate west, part of the world only by a technicality. Named for an Amazon queen in the terrestrial paradise. Colonized by fortune-seekers who were told the rivers were strewn with gold nuggets the size of your fist.
The beach bums and the wannabe actresses and the hippies and the venture capitalists, all alike in that they had one foot on that little patch and the other in some fantasy of their own imagination. From that tiny winding wire of precious flat ground had come John Steinbeck, hippies, gay rights, the computer revolution, Ronald Reagan, every Hollywood movie, blue jeans, Barbie dolls, Joe DiMaggio, fortune cookies, popsicles, lap dances, hula hoops, the Beach Boys, Disneyland, an entire continent’s worth of positive affect scrunched up into a coastline and paved over with Mission architecture.
And this is, oddly enough, just as true for the bay area tech culture as it is for LA.
Having praised California to this extent - and I really do love it - the time has come to explain why I left it, by describing all the ways in which this kind of culture breaks down.
The problem with monocultures, explained via combinatorics
Let’s start by doing some math.
The stable matching problem is a classic introductory combinatorics problem. It’s defined this way: Assume there are N men and N women who all want to get married6. Each person has ranked everyone of the opposite sex from in order of marriageability, and we would like to pair them up. A pairing is unstable if there exist a man and a woman who are not married, but both prefer each other to their current partner. The problem asks if we can always find at least one stable pairing.
It turns out that we can do this by following the obvious algorithm: Each man hits up his favourite woman, and at each step each woman rejects all the men who hit on her except for her top choice among them. The rejected men then go on to hit on the next woman down their list, and so on until everyone has a single partner. It can be shown that this eventually converges to a stable matching7.
The interesting thing about this problem is that the results are worse the more homogenous people’s preferences are.
In a world where everyone has their one true love who they prefer above all others because they’re deeply compatible, and everyone’s soulmate is personal and different from everyone else, we get the best possible outcome: Everyone meets their ideal partner on the first step and lives happily ever after.
Conversely, in a world where everyone judges everyone else in terms of attractiveness on the same objective scale, everyone starts off hitting on the single hottest woman (who has to fend off all the guys except the single most attractive one), upon which they hit on the second most attractive woman, and so on. This is a very unpleasant process for everyone involved that ends up with pretty much everyone (except for the one alpha couple) wishing they could be with someone else8.
This brings us to the first problem with monocultures: They have a pretty linear status hierarchy. Not completely - whether someone looks up to startup founders, high-level bigtech employees, or famous bloggers varies somewhat - but the status ladder is pretty universally correlated. This isn’t only a problem with dating (although it is a problem there9). It hits employment to some degree (both in the sense that a lot of people want the same high status jobs, making people in those jobs very competitive and people who couldn’t get them feeling bad)10.
It incentivises people to “do a startup” - not necessarily a startup about something, just going through the motions of creating a startup about whatever is hot at the time11. It incentivises a lot of empty hype. It means that if you’re a 90th percentile engineer, you’re doomed to boring social mediocrity, because everyone around you wants to talk to the 99th percentile engineer at the party (if you’re not an engineer or someone with status on the tech ladder, you’re probably not even there).
You see this dynamic pretty blatantly in bay area social events. There’s a few big name people12, and eventually conversation circles start around them and expand as more people tag on until someone moves away.
I don’t want to push this too far. There’s also plenty of other more normal conversations around, and plenty of just good normal socializing. The status ladder is correlated but not nearly to the 100% “everyone has objective hotness” scenario levels, and you still can make a lot of friends based on personal compatibility. The toxic dynamic is present, it makes things generally worse, but society still goes on despite the dysfunction13. I still love the bay area enough to enjoy visiting, in the end. But it’s enough that I don’t want to live there anymore.
Status ladders and preferences
I’ve talked so far about status ladders as the things people get judged on, but people have preferences aside from status ladders. Society is nicer when people have a wide variety of status ladders because it lets people choose which status ladder they want to climb in a way that matches their actual talents and preferences.
Consider a hypothetical healthy society14 where people have wildly varying status ladders and preferences: Alice can feel successful running a coffee shop, Bob can feel good about himself being a 90th percentile engineer (after all, he’s a brilliant engineer with a lot of interesting knowledge compared to his friend Alice), Charlie feels good about himself despite having an unfulfilling job as a janitor because it’s low-pressure work that lets him get a beer with his friends after work (and, because there’s such a heterogenous status ladder, his friends don’t judge him too harshly for not having an impressive job), and so on. And it also lets society be more functional in terms of what gets done, because we actually do need a wide range of jobs to make society work.
The big issue here is fertility. Having kids - especially for mothers - is a huge resource cost that requires sacrifice along one-dimensional status ladders. Societies need to either give high status to mothers (especially mothers with many children), or have a heterogenous enough status ladder that women don’t feel they’re sacrificing something important by having children, if they want to retain replacement-level fertility15.
The most uniform societies aren’t San Francisco. They’re places like South Korea and China where there’s a single national status ladder with exams and everyone’s constantly working at peak intensity to optimize for it. Noncoincidentally, these places are also pretty unhappy, low-fertility countries despite their amazing economies and infrastructure16.
Applying this to AI
We started off talking about the bay area, so like all bay area conversations, we need to tie this back in to AI. Specifically, we’ll use transformers to explain why this is bad for society, beyond the effect of making individuals unhappy. Consider the humble transformer.
An important piece in making transformers work is the “Add & Norm” layer after each attention head17. The effect of this is to scale the vector to avoid exploding gradients, or any one element of the vector dominating the computation: The effect of each compute should be a direction (and one of many contributed directions at that), not a magnitude.
I think of one-dimensional status ladders the same way: Knowing where someone is on a status ladder is a magnitude rather than a direction. But to navigate (or guide) society, the direction of the vector is more useful than the magnitude, and putting too much weight on the vector magnitude risks tunnel vision and blindness. It’s useful to have a lot of different status ladders so that we can focus more on what kind of person someone is, not just how statusy they are.
Thanks again to everyone who came up to me and mentioned this blog.
Not mentioned here: The views (beautiful misty hills rising over the bay), and the BART, which is a rare legitimately good American transit system (except for their surprisingly buggy payment system).
Or a clean energy battery tech startup, or one making B2B meeting scheduling software for businesses with 200-500 employees, or “putting AI in ice cream”.
Yes, I’ve had all three happen to me on the same day.
I have zero idea how broadway street cred works.
This is a poor analogy to the bay area, because everyone involved in this hypothetical is straight and monogamous.
The proof of this is left as an exercise to the reader.
Another interesting feature of this matching algorithm is that if men are the ones hitting on women, the men all end up with the best possible (stably-matched) wife, while the women end up with their worst possible (stably-matched) husbands. From this we learn two things:
First, that it’s often better to be proactive in going after what you want (you’ll have to put in more work, but you’ll end up with better results).
Second, since the metaphor of men approaching women isn’t entirely coincidental, we learn that any theory of feminism that doesn’t put demands on women can’t fully succeed at achieving optimal results for women, since part of the way men get ahead is by being more proactive (how, whether and to what degree this is an issue feminists should be addressing is a bit too culture war-y for me here, but if you want to understand gender relations this is an effect worth keeping in mind).
I’m not going into the dynamics of the bay area dating market, but the skewed gender ratio among single young people makes these issues even worse, in pretty much the ways you’d expect.
To some degree, there’s an identical problem in finance jobs (only worse, because there’s much fewer of those to go around). In another way it makes tech jobs kind of toxic though - since people derive status from “smart engineering”, there’s a lot of pointless complexity and makework in big tech companies, purely as a form of status game.
This reached peak stupidity in the crypto/NFT era, when people were doing funded startups literally devoid of meaning. In a weird way AI has been good for Silicon Valley’s soul, because at least now the hot startup culture is actually about something real that’s worth taking seriously.
I’m going to avoid naming names here because despite being the focal points of a dysfunctional society, all the bigname bay area people I’ve met at these events were incredibly nice and good about not having a big ego about it, and handled the dynamic as well as anyone reasonably can.
It’s worth noting that it’s worse the more you scale up - I’ve found that getting invited to and making friends at bay area social events is doable, but organizing events and having people show up if you’re not famous or well-connected is much harder there than in New York.
Broadly I think Israel does this better than the US, which in turn does it better than East Asia. Of places I haven’t lived in but have an impression of I think Eastern Europe and Latin America are pretty good about this, but I don’t know them well enough to be confident about it.
This is true both in the sense that people respond to incentives so we should set up incentive design to encourage healthy outcomes, and in the sense that mothers do important prosocial difficult work and deserve to be rewarded for it.
But see also this explanation for it.
I know normalization isn’t specific to transformers, but the image is a classic so I co-opted it into my metaphor.