Thanks for sharing your fears, Abraham! I'd love to see you try to flesh out a few suggestions on what to look out for, what to improve, how to keep what makes EA EA (in another article), if you can make the time.
It's very hard to build a brand that can't be coopted by assholes and EA failed catastrophically at doing that in ways that harmed all associated efforts.
> Outside of EA, this pressure isn’t surprising — but that this pressure now feels like it is coming from many in EA, or affiliated with the communities I care about, is what confuses me.
This is sort of self-explanatory, unfortunately. Consider that EA, like all prestigious subcultures, may not be as we hoped it could have been. There is nothing protective about Berkeley "rationality". There is nothing self-corrective in noticing you're confused. There is nothing special about gathering together a bunch of mildly autistic mildly bright kids, aside from the mild autism - they are susceptible to the same incentives as anybody else.
I learned about EA through SF AI safety. I was a CS major first, disappointed in hyperscalars and skeptical of accelerationists second, exposed to the AI safety industry third (my calling it an "industry" has been contentious, but I stand by it), and then brought into the logics of EA that should govern AIS last.
I'm youngish, so I wasn't online enough to discover the infamies of EA when they were circulating. My entire sense of EA was developed through SF AIS people explaining the concept to me. Some people there felt that "EA was dying," and that it was more important to focus on the truly impactful work of "solving AI," letting the moral codes of AIS guide people rather than EA's. I suspect you may be scared about that. Other people (generally older, but who in AI safety is old?) felt that it was important to acknowledge AI safety's EA roots.
I have a lot of feelings about AI safety, and a few thoughts on EA. Mostly I'm just not well-versed in its doctrines. But it seemed to me that the difference between AI safety and AI socioethics is that AI safety cares about the future, infinitely bad threat of superintelligence over nearly everything else. I've been told how this logic leads to Anthropic-style development of AI---if the US gets there first, surely that's a better future? I'm not American, and the US has colonized all the countries I'm from. This take does not come intuitively to me. I've been told how it's more effective to go secret-Congress style with policy efforts. Think tanks, influencing senators, even direct donations and political intervensions. In one startling case, someone mentioned gerrymandering. Power- and money- seeking, especially in AI safety policy circles, seems not only incidental but inherent to its doctrines of doing everything possible to stop the infinite suffering of the end of the world.
Data centers being built in legally underrepresented communities already experiencing environmental risks, the possible epistemic and agency issues of accelerating and overemphasizing the possibilities of frontier AI, the harms of creating systems that funnel political power without democratic input, concentrating US power by directly maintaining its control over the supply chain: these, I have been told, are necessary sacrifices for stopping the possible takeover of AI. Infinite future good over small current harms. Don't be scope sensitive.
For all my privileges, I am not a wealthy, American white man, so I wouldn't be able to immediatly assume the power that seems to come to that archetype. Maybe it is in my sensitivities to being powerless that I am able to criticize those that exercise that kind of paternalistic political power over their country and other countries, regardless of the outcomes they seek.
I guess I'm surprised that EA-first (as opposed to AIS-first) people are also alarmed by it. I attributed a lot of AI safety's utilitarian quirks to its roots in EA. I respect the doctrine of rationalism for how it pushes people to act in ways they would otherwise dismiss. I do think the ontological certainty is wild, and that the lack of pluralism in the AI safety space will rot its touch with humanity.
Anyway, thank you for posting! This has inspired me to learn more about EA. I've received counsel from other members representing the movement, but this is the first one that has convinced me that EA is compassionate as well as demanding.
Thanks for sharing your fears, Abraham! I'd love to see you try to flesh out a few suggestions on what to look out for, what to improve, how to keep what makes EA EA (in another article), if you can make the time.
It's very hard to build a brand that can't be coopted by assholes and EA failed catastrophically at doing that in ways that harmed all associated efforts.
> Outside of EA, this pressure isn’t surprising — but that this pressure now feels like it is coming from many in EA, or affiliated with the communities I care about, is what confuses me.
This is sort of self-explanatory, unfortunately. Consider that EA, like all prestigious subcultures, may not be as we hoped it could have been. There is nothing protective about Berkeley "rationality". There is nothing self-corrective in noticing you're confused. There is nothing special about gathering together a bunch of mildly autistic mildly bright kids, aside from the mild autism - they are susceptible to the same incentives as anybody else.
I learned about EA through SF AI safety. I was a CS major first, disappointed in hyperscalars and skeptical of accelerationists second, exposed to the AI safety industry third (my calling it an "industry" has been contentious, but I stand by it), and then brought into the logics of EA that should govern AIS last.
I'm youngish, so I wasn't online enough to discover the infamies of EA when they were circulating. My entire sense of EA was developed through SF AIS people explaining the concept to me. Some people there felt that "EA was dying," and that it was more important to focus on the truly impactful work of "solving AI," letting the moral codes of AIS guide people rather than EA's. I suspect you may be scared about that. Other people (generally older, but who in AI safety is old?) felt that it was important to acknowledge AI safety's EA roots.
I have a lot of feelings about AI safety, and a few thoughts on EA. Mostly I'm just not well-versed in its doctrines. But it seemed to me that the difference between AI safety and AI socioethics is that AI safety cares about the future, infinitely bad threat of superintelligence over nearly everything else. I've been told how this logic leads to Anthropic-style development of AI---if the US gets there first, surely that's a better future? I'm not American, and the US has colonized all the countries I'm from. This take does not come intuitively to me. I've been told how it's more effective to go secret-Congress style with policy efforts. Think tanks, influencing senators, even direct donations and political intervensions. In one startling case, someone mentioned gerrymandering. Power- and money- seeking, especially in AI safety policy circles, seems not only incidental but inherent to its doctrines of doing everything possible to stop the infinite suffering of the end of the world.
Data centers being built in legally underrepresented communities already experiencing environmental risks, the possible epistemic and agency issues of accelerating and overemphasizing the possibilities of frontier AI, the harms of creating systems that funnel political power without democratic input, concentrating US power by directly maintaining its control over the supply chain: these, I have been told, are necessary sacrifices for stopping the possible takeover of AI. Infinite future good over small current harms. Don't be scope sensitive.
For all my privileges, I am not a wealthy, American white man, so I wouldn't be able to immediatly assume the power that seems to come to that archetype. Maybe it is in my sensitivities to being powerless that I am able to criticize those that exercise that kind of paternalistic political power over their country and other countries, regardless of the outcomes they seek.
I guess I'm surprised that EA-first (as opposed to AIS-first) people are also alarmed by it. I attributed a lot of AI safety's utilitarian quirks to its roots in EA. I respect the doctrine of rationalism for how it pushes people to act in ways they would otherwise dismiss. I do think the ontological certainty is wild, and that the lack of pluralism in the AI safety space will rot its touch with humanity.
Anyway, thank you for posting! This has inspired me to learn more about EA. I've received counsel from other members representing the movement, but this is the first one that has convinced me that EA is compassionate as well as demanding.