if Hermione Granger can be a general in one of Quirrell’s armies I can send a message asking some guys if there’s been any progress on the thing
Posts tagged #r a t i o n a l i t y
a maybe interesting question to ask is, “what’s your biggest constraint?”
three common ones I know of are time, energy, and money. idk what the other big ones are
I think people often have a hard time understanding each other when they’re constrained on different things
also moving from one to another is weird. I switched from “energy” to “time” a couple years ago and it took a long time to actually internalize that time was expensive
I think I’ve identified a fourth one: someone I know calls it “brainspace.” (these days I feel sometimes more time-constrained and sometimes more brainspace-constrained I think)
it feels a bit hard to describe, but some symptoms:
- feeling like you have so many things on your to-do list that half of your time is just spent remembering what all the things on your to-do list are
- feeling like you’ve hit a limit of “decisions per day” that takes time to reset
- feeling like time sleeping or away from work has a cost > 1, because of the additional time it takes to reset and load everything back into working memory when you start working again
I sort of feel like I don’t have that much responsibility, in the grand scheme of things, and am curious how other people handle it. Like what do CEOs of real companies do? Or presidents?
Say your partner cheated on you, years ago. It was only one time, and they felt really bad about it, and it has no implications for how they’ll behave toward you going forward; but if you knew, it would really hurt you.
Is it better to know or not?
One school of thought says, obviously, yes. (I think this is probably most people’s first intuition?)
Another school of thought says, wait, no. If you don’t know, you can continue to live your life happy; if you do know, you’ll experience a lot of pain. From a basic hedonic utilitarian standpoint, knowing seems bad.
I think I’m in favor of knowing; or at least, being the sort of person who wants to know. It’s not that I don’t buy the utilitarian argument: I’m all for experience machines. But real life isn’t a perfect experience machine, and I think if you’re committed to flinching away from truths that might hurt you, you’ll end up living in fear that they’ll pop up anyway. Seeking out the worst and most painful truths, and finding that you’re able to deal with them, lets you be free.
Not to mention, being in the habit of flinching away from the truth on things that don’t matter probably sets you up to flinch away from the truth on the things that really do matter.
Say your partner cheated on you, years ago. It was only one time, and they felt really bad about it, and it has no implications for how they’ll behave toward you going forward; but if you knew, it would really hurt you.
Is it better to know or not?
One school of thought says, obviously, yes. (I think this is probably most people’s first intuition?)
Another school of thought says, wait, no. If you don’t know, you can continue to live your life happy; if you do know, you’ll experience a lot of pain. From a basic hedonic utilitarian standpoint, knowing seems bad.
I think I’m in favor of knowing; or at least, being the sort of person who wants to know. It’s not that I don’t buy the utilitarian argument: I’m all for experience machines. But real life isn’t a perfect experience machine, and I think if you’re committed to flinching away from truths that might hurt you, you’ll end up living in fear that they’ll pop up anyway. Seeking out the worst and most painful truths, and finding that you’re able to deal with them, lets you be free.
Try questioning whether your epistemic humility is just fear by another name.
Imagine no one else knew the answer, and it was your job to figure it out. How does that feel?
If it feels scary, that’s a good sign: stay with that, and keep imagining it, and start trying to figure it out as if it were your only option.
Because, actually, all the progress and decisions in the world are made by people not that much smarter than you, if at all, and either you can at least sorta try to figure it out or we’re really fucked.
I think I’ve mentioned being raised by economists meant already growing up with a lot of rationalist/EA values–the biggest one that was missing was what I guess people call “taking ideas seriously.” I’m higher on this dimension than the rest of my family, which was enough to get me into EA in the first place, but I’m still lower than average for EA I think.
I was at a dinner party the other day with some (unrelated to me) academic economists. One brought up a recent study about CO2′s effects on cognition and said something about how in forty years atmospheric CO2 would be high enough that we’d start seeing noticeable effects. Another said “well, I guess by then maybe the robots will have taken over anyway.” The tone was like … kind of flippant but not really joking, and in response people like, nodded and shrugged.
It’s interesting to see ideas about AI risk getting more accepted among certain groups, and how they just don’t get taken seriously, but I guess taking ideas seriously is pretty rare.
The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World by Tom ChiversMy rating: 3 of 5 stars
Man, if you had told me a couple years ago that a Buzzfeed writer would write a popular book on AI risk and the rationalist movement, and my main complaint would be that it wasn’t sensationalist enough, I would have been very confused. But here we are.
The author seems like a really nice and reasonable person, who thinks AI risk might be important and wants to understand and explain it. But mostly it seems like most rationalists were reasonably pretty wary of talking to a journalist, so he didn’t get much unfiltered content from them. So the book ends up being largely a summary of the Sequences and some other rationalist ideas, with a few interviews thrown in. Which is cool, the ideas are the important part, but if you really want the ideas you can just read the Sequences. (Though this is a nicely digestible version of the highlights.) I think the book just needed him to go undercover to about three really crazy rationalist parties.
I did like the chapter at the end where he does internal double crux with Anna Salamon; it felt very true.
View all my reviews
For the record, during my CFAR workshop I was kind of unhappy about all the hippie bullshit and since then it’s just hung around in my mind and infiltrated more and more of my thoughts.
My speech patterns have become more NVC-like. I think about relationships in terms of attachment theory. I try to guess people’s enneagram types. I might be into crystal healing.
Not sure if any of this is right or if it’s all dumb but it feels like I’m having a lot of personal growth idk??
I kinda want my new tumblr aesthetic to be “obscure rationalist woo” but every time I try this I get nice people responding to my posts with “what does this mean? can you give an example?” and I feel bad. how do people deal with this
I can’t remember any of the details, but #oathgate vaguely reminded me of some rat controversy a while ago about people not being reliable, i.e. if someone said they would do a thing there was less than 100% probability they would do it?
Anyway, I think promises of the form “I will absolutely 100% do X, no matter what” are not very useful, hence the general preference for normal contracts over smart contracts, the paucity of Unbreakable Vows in the wizarding world, etc. When I say “I will do X” I generally mean something like “I will try hard to do X in situations where a reasonable person would think that I should, conditional on my having promised to do X.”
As I recall, that was about people failing to meet a “reasonable person” level of reliability, and not considering it fair to hold them to that standard.
Oh okay, maybe you’re right. I am generally in favor of people meeting a “reasonable person” standard of reliability, for the record.
I can’t remember any of the details, but #oathgate vaguely reminded me of some rat controversy a while ago about people not being reliable, i.e. if someone said they would do a thing there was less than 100% probability they would do it?
Anyway, I think promises of the form “I will absolutely 100% do X, no matter what” are not very useful, hence the general preference for normal contracts over smart contracts, the paucity of Unbreakable Vows in the wizarding world, etc. When I say “I will do X” I generally mean something like “I will try hard to do X in situations where a reasonable person would think that I should, conditional on my having promised to do X.”
I used to think that the main choice you made was between good and evil, and choosing evil was tempting but–it was signposted. It was evil. There were skulls.
Now I think maybe the real choice, the hard choice, is between madness and reason. Red pill and blue pill. Faerie kingdoms and the realms of men.
And every kernel of understanding you gain is a point closer to
Hiiiii, future mes! So I decided I’d be the me who found out what would happen if she just raised ERO as high as she could. And, oh my god, I can’t tell you anything because you are so not ready to hear it, but take it from me, you were all worried about nothing. This is great. I don’t have a name any more though.
and the understanding and the madness aren’t just related they’re two sides of the same coin
Magic shall be written upon the sky by the rain but they shall not be able to read it;
Magic shall be written on the faces of the stony hills but their minds shall not be able to contain it;
In winter the barren trees shall be a black writing but they shall not understand it.
I don’t actually read Overcoming Bias regularly so I don’t think I’m great at interpreting Robin Hanson/discerning what level of trolling he means any particular thing at …
but is the “everything is about signaling” guy really arguing that good writing should be free of signals and just state facts/make logical arguments? I’m confused
I also am not a dedicated reader of Robin Hanson, but my basic understanding is that he thinks everything is about signalling, and that this is bad and that everything should instead be about actually important, non-signalling things.
So him arguing that good writing should be free of signals and just state facts + make logical arguments is exactly what I would expect.
Isn’t it just the difference between descriptive and prescriptive claims? Hanson thinks that everything is full of signalling, but it (or at least good writing) *shouldn’t* be.
Makes sense. I guess “everything is about signaling, therefore try to be really good at it” just seemed like the obvious conclusion to me.
(I guess the blog is called Overcoming Bias, not Pointing Out Bias and Throwing Up Our Hands)
But okay, if this is a sincere argument, then
- you can’t just avoid signaling by refusing to pad your statements with explicit value affirmations and presenting logical arguments! everything you say is signaling something!
- to conclude from “people say in a twitter poll that they don’t need a writer to affirm their shared values in order to engage with them” anything besides “people [who read Robin Hanson’s twitter] want to signal that they are rational” seems like giving a bit too much credit to people’s honesty and self-awareness here
- engagement is a limited resource. developing memetic immune reactions to things is not inherently bad. developing signaling mechanisms to subvert these reactions is the rational response.
occasionally mentioning that you think rape is bad is really not that costly
I don’t actually read Overcoming Bias regularly so I don’t think I’m great at interpreting Robin Hanson/discerning what level of trolling he means any particular thing at …
but is the “everything is about signaling” guy really arguing that good writing should be free of signals and just state facts/make logical arguments? I’m confused
[This character] is doomed to be eternally tortured by an artificial intelligence, but she’s also kind of like Marie Antoinette
me irl
Inadequate Equilibria: Where and How Civilizations Get Stuck by Eliezer YudkowskyMy rating: 4 of 5 stars
I ended up liking this book a lot, and more than I was expecting at first.
The first three chapters didn’t really say anything I don’t already believe and hadn’t already thought about a reasonable amount, but they were a clear and entertaining presentation of the material. The last four chapters were basically a thoughtful examination of why people hate Eliezer and why he thinks that those people are wrong. I’m not sure if I agree with everything in them–I think that I think it’s harder to choose between disagreeing experts in fields in which you are not expert than he thinks it is. (I agree that if you can just try, and make bets, and see if you’re right, and update, that’s good. I think there are a lot of cases where it’s just hard to do this.) But I think being skeptical of reasoning motivated by status-regulating emotions is an important and good thing (for me personally, at least) to do.
If you like Eliezer, you’ve probably read this book already. If you hate Eliezer, I think you should read this book, and if you still hate him at the end then you can give up. If you don’t know who Eliezer is, probably don’t read this book and read Slate Star Codex instead.
View all my reviews
tfw you get locked out of your office and try to nonsuspiciously follow the next guy who walks into the building, only to realize
- he is Nate Soares and
- you need your key card to get off on the correct floor
so you awkwardly ride the elevator all the way to his floor while resisting the urge to reassure him that you are not in fact the agent of a UFAI trying very incompetently to break into MIRI
there is this thing I keep encountering where like … rationalists value being Well Calibrated which is great, but it turns out even in a group that values appropriate levels of confidence sounding confident is the best way to be listened to and get status
me, yesterday: [in an assertive tone] oh that definitely might not be true
guy: … that is such a rationalist thing to say
Anonymous asked:
Thanks for the clarification! And apologies everyone for my spurious etymological inferences :(
I.
To understand “beep-boopism” as I will describe it here you first must understand rationalist tumblr. To describe rationalist tumblr, otherwise known as the Tumblr Academic Freedom Club, would be far outside the scope of this account. If you are unfamiliar with it, I would encourage you to investigate the literature for yourself.
As people have pointed out, the practice of criticizing the overly analytical by comparing them to robots certainly has a long history, and in fact the term “beep boop” has likely been used many times before in this context. Here my aim is merely to give an account of the history of the term as it intersects with rationalist tumblr. The entire controversy certainly has relevance within a wider cultural context, but it is also worth noting that rationalist tumblr is quite idiosyncratic.
II.
As far as I know, the first time “beep boop” entered The Discourse was in a post by perversesheaf, a member of the hallowed tradition of rationalist-critical denizens of rationalist tumblr. (It is a much commented-upon phenomenon that the surest way to be accepted into rationalist tumblr’s ranks is to get a tumblr and begin criticizing it, a mark of the high value placed on contrarianism.) Unfortunately perversesheaf has since deactivated their blog, making it more difficult to investigate the origins of the beep-boopism debate. However, if I recall correctly it began with a post claiming that jealousy is weird by comparing it to bowling. It would be odd to be jealous that your partner had gone bowling with someone else, the logic went, and it is similarly odd to be jealous if your partner has sex with other people.
The discussion continued when thesublemon posted an essay by former tumblr user moteinthedark titled “Aesthetics are moral judgments.” This drew criticism from perversesheaf, who wrote “The author is very intelligent, writes well, yet fundamentally misconceives a basic aspect of the human experience. It sets off the exact same alarm bells the “How is sex different from bowling?” conversation did.” Tumblr user ogingat, an ally of perversesheaf, agreed with the criticism and added to it by criticizing moteinthedark’s use of mathematical metaphors.
This prompted a reaction from ozymandias271, who replied in an evenhanded tone defending the use of metaphor. However, their next post on the subject was less evenhanded: they wrote “beep boop rationalists are robots! It’s funny because a quarter of rationalists are autistic” and tagged it #vagueblogging. This post was criticized by tumblr users such as prophecyformula, who responded “beep boop highly autistic communities might have some systematic epistemological problems related to the high prevalence of autism.” Most notably, it began an argument with perversesheaf.
In this case, “beep boop” is used to mean something adjacent to “autistic.” Or maybe “people who have a hard time understanding things that normal people grasp intuitively.” The specialness of sex is certainly something that most people believe in intuitively, but many rationalists find odd. Aesthetic judgments are another example of this, as most people have an intuitive sense of aesthetics that is different from what is described in moteinthedark’s post.
(Not long after the argument began, ozymandias271 posted that their strongly negative reaction was due in part to their “~~tragic backstory”, in part to defensiveness of their friend moteinthedark, and in part to a depressive episode. For their part, perversesheaf and ogingat both deactivated the next day, finding tumblr an suboptimal place to conduct The Discourse.)
III.
Two months later, on September 24, “beep boop” made its return, in a response severnayazemlya wrote to an ask. An anon asked him for empirical evidence of his claims of the harms likely to result from immigration. He responded that the visible effects of immigration, particularly the societal decay it causes, are hard to quantify. He wrote in a clear criticism of tendencies in the rationalist community, “Number fetishism won’t get you very far here … There’s plenty of empirical evidence … but it’s not the sort of thing a beep boop can process, which means beep-boopism is epistemically harmful.”
The meaning of “beep boop” here seems slightly different than in the previous case. It refers less to a lack of understanding of neurotypical experiences and social norms, and more to an overreliance on statistics, the quantifiable, and the scientific establishment in forming opinions.
On September 26, nostalgebraist wrote “So excited for “beep-boopism” to become the next hot undefined-except-by-extension-and-connotation mystery term on my dash”. It received a smattering of likes and proved to be quite prescient, as nostalgebraist acknowledged two days later with his reblog, “the prophecy … has been fulfilled …”.
The discourse on beep-boopism took off around 9 am PST on September 28, when veronicastraszh made a post with the title “Beepy Boopy Veronica Style.” What she wrote seems to have little connection to the severnayazemlya post and is instead more reminiscent of the original argument between perversesheaf and ozymandias271. She describes beep-boopism as “hyper logic and hyper detachment … refusing to get caught up in the cult of social stigma.” Her main criticism of beep-boopism is that it is important not to dismiss social subtleties, as this can lead to things like people refusing to acknowledge sexism.
Tumblr user davidsevera, best known for his frequent jokes but increasingly considered a valuable contributor to the political Discourse as well, wrote another popular post on the issue three hours later, around 12 pm PST. His post focused on the issue brought up by severnayazemlya. He criticized opponents of beep-boopism as engaging in their own error, “Plausible Narrativism.”
Tumblr user queenshulamit, known romantic associate of nostalgebraist and fellow advocate for “nicencess” and “reasonableness,” wrote around 4 pm PST that “having a srs opinion on beepboopgate is compulsory now,” in an indication of how quickly and how far the controversy had spread. (The appelation #beepboopgate references previous controversies in the group known as rationalist tumblr or the Tumblr Academic Freedom Club. Most notable among these controversies are #prettygate, #speckgate, and the perhaps most notorious (and certainly most similar to #beepboopgate) #neotenygate. Unfortunately there is no space in this treatment to do them justice.) Her opinion was the the word “beepboop” was conflating several things, which was certainly correct: she named “basing opinions on numbers, being sutistic[sic]/autistic cousin, and disregard for social norms.” (One might call these the severnayazemlya, ozymandias271, and veronicastraszh definitions respectively.)
IV.
An interesting aspect of #beepboopgate as opposed to similar previous scandals is how much of the discourse occurred at the meta level. The original flare-up in July was entirely on the object-level, and the discussion on severnayazemlya’s September 24 post was object-level as well. However, people began discussing it as a controversy before it was a controversy (i.e. nostalgebraist) and as soon as the real controversy began, much of the discourse consisted of pattern-matching it to previous incidents and making jokes.
This is an unfortunately incomplete tumblr historical document here reproduced in its entirety.
It seems polite to tag those mentioned, sorry if I’m missing someone but people change usernames a lot:
@thesublemon @cptsdcarlosdevil @nostalgebraist @veronicastraszh @davidsevera @birdblogwhichisforbirds
Thinking about Ozy’s recent post.
I definitely remember when I first read the Sequences, getting a sort of “world is mad” feeling. But looking back, I’m having a hard time remembering what that feeling was about.
Because … my world was never that mad? I don’t know anyone who buys lottery tickets or homeopathic remedies. Pretty much everyone I know is atheist. My childhood was full of people making jokes about the efficient market hypothesis and the sunk cost fallacy. From an early age my parents taught me things like the difference between correlation and causation, and how to interpret observational vs. randomized studies.
(I mean, I know my world isn’t normal. But since before I can remember I’ve always known I and the people I knew were smarter than most people. The question is where rationality comes in.)
Idk. There are only two things that consistently make me feel alienated from non-rationalists.
One is politics. When normal people are talking about politics I always have the urge to yell, “Can’t you see this discussion is entirely signaling and tribal rock-throwing?” This is the least useful modification because it doesn’t really make me better at anything, it just makes me more annoyed when people talk about politics. 3/10 would not recommend asking the aliens for.
(I’m not even sure how rationality-related this is. Like, I’ve always felt sort of alienated by a lot of political discussions. Maybe because they often rely on emotional appeals to fairness and I’ve always been a degenerate unifoundationist? Maybe because I missed Political Indoctrination Day during Blue Tribe Socialization?)
Another is EA stuff. But I don’t think the main disconnect here is rationality. I think it’s more that most people don’t really care about helping people? Like, I explain GiveWell and people are like, “Cool. I guess I’ll check them out if I ever have any money I really don’t know what to do with.” Or I will explain the case for animal rights to people, and they will nod along and go “Yes, I see that factory farming is basically animal torture and we as a society could end it with little cost to ourselves, but I just don’t really care about animals.”
(This is non-rationalists I’m talking about. Rationalists go “Yes, I see, but I just don’t think animals are sentient.”)
I mean, I’m sure I got some insight from reading the Sequences and discovering the rationality community. I wish I could remember what it was.
Things I’ve heard rationalists say they have no idea how anyone could do:
- support Trump
- be anti-GMO
- think that open borders are bad
- think that jobs are good
- not like cuddle puddles
- not want to fuck their clone
- cheat on someone
- feel jealousy
- consider sexual compatibility in a relationship important
- think that life is worth living
simulationist theodicy is the best thing to confuse people with at parties
person: [in response to a story about something bad happening] well, I guess I shouldn’t be surprised that life isn’t fair.
me: WAIT actually you probably should–
in French, “California” means “the world is going to end and we will go hot tubbing in our underwear and braid LED lights into our hair” and I think that’s beautiful
me: oh hey, Petrov Day is coming up, maybe I should have a party
also me: oh, but if I plan it now North Korea might bomb us before it happens and then what would I do? cancel it? have a really depressing party? turn it into some kind of weird mourning ritual? either way this sounds super awkward–
me: … yes, I agree that in the event of nuclear war the potential awkwardness of any parties I am planning to host should be my first concern
You see, friends, we’re asleep … we’re children of dust … but read the Sequences–and you’re God, you’re pure as on the first day of creation!
Yesterday I downloaded Track Your Happiness, an app that sends you notifications at random times throughout the day, asking you what you’re doing and how happy you are at that moment. While I’ll be interested to see what the data shows after a while, I’ve already noticed something cool.
I think there’s sort of a positive feedback loop to noticing my happiness level. It goes something like (notice current happiness level) -> (update towards average happiness being closer to that level) -> (life satisfaction updates in the same direction) -> (happiness updates in the same direction because either “wow my life is so great, I’m happy all the time” or “oh god despite all my success and privilege I’m still unhappy what even is the point”).
And since I’m more likely to be unhappy when alone/idle/introspecting about my emotions, and more likely to be happy when in a flow state, I notice my unhappiness more than my happiness. Track Your Happiness corrects for that bias.
So yeah. It’s never occurred to me to wonder whether I’m happy while immersed in a conversation at work, because immersion, and because I don’t actually have a choice about whether or not to have most of these conversations. But today I got a notification and realized that you know, I was really enjoying the experience.
I feel like if we do live in a simulation, then praying is not a crazy response. If I were simulating a universe and its denizens started appealing to me to please lay off on the whole suffering thing, I hope I’d at least think twice about it.
I’m kind of tempted to start a simulationist cult, based on worshipping our simulators and thinking about the most effective ways of praying to get their attention. Has anyone done this yet?
I don’t enjoy aspartame,
A mere Diet Coke’s
Little more than a joke
So tell me, why should it be true,
That I’ve found a superstimulus in you.
Some like to play video games,
I’m sure that if I
Were to give them a try
They would bore me terrifically, too,
Yet I’ve found a superstimulus in you.
I thought my reward system hard to hack
Until the day I met you.
I guess the environment of evolutionary adaptedness failed to prepare me for this since
I can’t seem to forget you.
Some find porn a secret shame,
A vacuous scene
On a thirteen-inch screen
Is my idea of nothing to do,
But I’ve found a superstimulus in you.
tip: confuse your nonrationalist friends by being unusually frank about your motivations
“I mean, I do actually think Trump is bad, I just like annoying people by espousing contrarian political views.”
“I … uh … well, yeah, I had noticed that about you …”
Internet! If you have not already done so you must immediately pick up a copy of The Undoing Project by Michael Lewis!
You may have been PREVIOUSLY AWARE of the excellent psychological work of Mr. Kahneman and Mr. Tversky! What you were no doubt unaware of, because you have not read The Undoing Project, is that they are INCREDIBLY SLASHY.
seriously
I have yet to read ANY rpf pairing that was THIS incredibly, ridiculously, purely gay
the ending is just like… it is nicholas sparks shit. the canonical life of daniel kahneman and amos tversky was, in fact, a nicholas sparks novel. I am not gonna spoil it for y’all but I literally said OH COME ON to my computer screen
@worldoptimization you were right about michael lewis writing straight up slashfic
wait has he written OTHER straightup slashfic
or is this his one excusion into the field
no, that was just my characterization of The Undoing Project after reading an excerpt a few months ago
there are depressingly few good pairings in other Michael Lewis books, now that I think about it. if there were actually any high-frequency traders in Flash Boys I would probably ship them with Brad Katsuyama but uh
I had a weird deja vu experience the other day when I somehow ended up in an argument about the existence of God, which hasn’t happened to me since like … 2012?
interestingly my response was to immediately revert to 2012!me and start like quoting the Sequences
“Don’t you think that beliefs should … pay rent … in anticipated experiences? I mean, say you claim there’s an invisible dragon in your garage,
a while ago I was having a conversation with my sister in which I was like “you know that thing nerds do when they talk? how they release the t’s at the end of words?” and she was like “… no”
so I feel vindicated to learn that this is a known fact in the literature (and in addition to it being a nerd thing it is also a Jewish thing apparently)
(also whenever I spend too much time around rationalists/EAs I start doing the /t/ release thing too)
today we were talking about the argument that talking about AI risk publicly was actually a really bad move, because various corporations and governments’ response to being told about AI risk was “hey superintelligent AI? that sounds cool we should try to build one!”
and now I’m wondering if AGCbtIitSH?! is actually part of Eliezer’s brilliant plan to discredit himself enough that everyone will stop taking the idea of superintelligent AI seriously
Consider that since puberty a modern woman may spend almost ¼ of her life under the threat of subtle disorder to their thinking processes due to hormones. It seems clear that for a self-aware woman, the rest of the time they may have an advantage in noting when they are biased.
only in the SSC comment section can you find people arguing that menstruation makes women more rational
still confused by Eliezer’s assertion that Bay Area rationalists will become less annoying when (and only when) more of us have VC funded startups tbh
antisquark said: Where was this assertion?
On Facebook. Someone posted that they found Bay Area rationalists unpleasant to hang out with because of their insularity and tendency to think outsiders not worth talking to. Eliezer responded “It’s a problem I suspect can only be solved in the long term by having more successful startups and VC money flowing through the community.” Everyone was confused and someone asked if he was trolling, but he continued to stand by his statement.
dataandphilosophy said: I think he thinks that people who have or want startup money only hang out with people who have or want startup money, forming a cohesive unit. And this group is supposed to be less obnoxious I guess? IDK.
Yeah that sounds close to right. Rereading what he wrote, the argument seems to be “Bay Area rationalists are insular because they feel marginalized. They are marginalized because in the Bay Area no one wants to talk to you unless talking to you will help them get VC funding.”
And like … I don’t really agree with any of the logical steps in that argument, and more than that I feel like there’s an easier solution and it’s “Bay Area rationalists should be nice to people even if they don’t know what a principal-agent problem is or whatever”
still confused by Eliezer’s assertion that Bay Area rationalists will become less annoying when (and only when) more of us have VC funded startups tbh
To test for political prejudice, Shanto Iyengar and Sean Westwood, political scientists at Stanford University, conducted a large-scale implicit association test with 2,000 adults. They found people’s political bias to be much larger than their racial bias. When Democrats see “joy,” it’s much easier for them to click on a corner that says “Democratic” and “good” than on one that says “Republican” and “good.”
To find out whether such attitudes predict behavior, Iyengar and Westwood undertook a follow-up study. They asked more than 1,000 people to look at the resumes of several high-school seniors and say which ones should be awarded a scholarship. Some of these resumes contained racial cues (“president of the African American Student Association”) while others had political ones (“president of the Young Republicans”).
Race mattered. African-American participants preferred the African-American candidates 73 percent to 27 percent. Whites showed a modest preference for African-American candidates, as well, though by a significantly smaller margin. But partisanship made a much bigger difference. Both Democrats and Republicans selected their in-party candidate about 80 percent of the time.
[snip]
[snip]
ugh yes….you want exactly the same cultural shifts i want…
and i appreciate that you’re being epistemically humble about the value you give to humility—hella meta
but i don’t think that valuing overconfidence is a chesterton’s fence? maybe? i can come up with a believable reason we might overvalue overconfidence: we start out with a world where some people are overconfident in their beliefs, some people are underconfident, it all averages out to basically correct confidence. some people start taking into account the confidence that believers have in a belief, and it helps them determine the truth, it’s a good heuristic. but since being believed gives you social status, as the heuristic spreads, overconfident people do better than underconfident people in situations where social status has more effects than truth, or in situations where social status is easier to see than the truth. bam, the overconfidence fence is built.
also, like, at this point i’ve seen several studies saying that humility has better economic consequences than overconfidence even if it gives you less status (you can see the list of papers in this link, i thought they were good but i don’t know much about statistics), at least when it comes to investing.
idk how useful this is, this is basically just me being like “if there’s one thing i’m SURE of it’s EPISTEMiC HUMILITY!!!”
Haha I did not even intend the meta-ness.
And I like your story of the overconfidence fence. Your link to the papers is broken though :(
oops! here you go. there’s actually a list of papers at the bottom, the article itself is a summary
[snip]
I think I’m pretty good at epistemic humility but according to my mom and Sheryl Sandberg I should probably stop being good at it so I can like succeed in the workforce.
I feel like my favorite way (okay this is like a giant cultural shift so it’s not exactly a solution) to get more women in high-status positions would be to value aggression less and humility more in hiring/promoting decisions. Except I’m not really sure if this valuing of overconfidence is actually rational for reasons I don’t understand?
ugh yes….you want exactly the same cultural shifts i want…
and i appreciate that you’re being epistemically humble about the value you give to humility—hella meta
but i don’t think that valuing overconfidence is a chesterton’s fence? maybe? i can come up with a believable reason we might overvalue overconfidence: we start out with a world where some people are overconfident in their beliefs, some people are underconfident, it all averages out to basically correct confidence. some people start taking into account the confidence that believers have in a belief, and it helps them determine the truth, it’s a good heuristic. but since being believed gives you social status, as the heuristic spreads, overconfident people do better than underconfident people in situations where social status has more effects than truth, or in situations where social status is easier to see than the truth. bam, the overconfidence fence is built.
also, like, at this point i’ve seen several studies saying that humility has better economic consequences than overconfidence even if it gives you less status (you can see the list of papers in this link, i thought they were good but i don’t know much about statistics), at least when it comes to investing.
idk how useful this is, this is basically just me being like “if there’s one thing i’m SURE of it’s EPISTEMiC HUMILITY!!!”
Haha I did not even intend the meta-ness.
And I like your story of the overconfidence fence. Your link to the papers is broken though :(
also, when i do that thing i was just talking about—think about what i said, realize that i’m not sure of it, and offer a qualification—
the motive is rationality/a respect for the truth
but i can tell it sometimes comes across as feminine signalling, like apologizing a lot (something i also do)—qualifying your sentences whether with uncertainty or apology, is super ladylike.
so. two things.
women are frequently told to stop qualifying themselves so much (remember that pantene commercial?). and honestly the apologies can probably go but saying out loud when you’re not sure of something is awesome and good, men should pick it up.*
also, people act like rationality and truth-seeking is something that men are just naturally more interested in…
but (i’ve said it before and i’ll say it again)
men (whether because of genetics or hormones or socialization or whatever) tend to be more into the argumentative, statistical, i’m-sure-i’m-right side of truth seeking, and that kind of fact-bravado is frequently interpreted as rationality
but women are socialized/gened/hormoned into being pretty damn good at epistemic humility and belief updating, which (except in the less wrong community) doesn’t get as much social respect, and isn’t seen as part of rationality
basically….femininity and rationality aren’t opposed, but their connections are lower status than connections between masculinity and rationality……why do you think that is…………………………………………………….
*for instance, see this article, which talks about how feminine-coded uncertainty can actually give better outcomes in male-coded fields, even though that feminine uncertainty is low-status
I think I’m pretty good at epistemic humility but according to my mom and Sheryl Sandberg I should probably stop being good at it so I can like succeed in the workforce.
I feel like my favorite way (okay this is like a giant cultural shift so it’s not exactly a solution) to get more women in high-status positions would be to value aggression less and humility more in hiring/promoting decisions. Except I’m not really sure if this valuing of overconfidence is actually rational for reasons I don’t understand?
Eliezer Yudkwosky: *ends up creating evidence-based self-help community*
ok that onion headline going around really annoys me.
just because you don’t read the news from other countries doesn’t mean that they don’t have senseless, sensationalized mass murders. There was a mass stabbing on the taipei metro two weeks ago. There was a mass stabbing in Kunming the month before. The same day as the UCSB shootings there was a shooting in Brussels.
grouchy about this.
friendly reminder that mass shootings are an extremely rare, extremely dramatic event and making any policy conclusions from them is pretty much just the availability heuristic
In a world where people bet reputation and money on prediction markets, would people who lost a lot on a belief they were particularly passionate about (e.g. political, identity-ish) actually become more convinced of that belief due to the sunk costs fallacy? My co-blogger won’t let me make any science bets until I have some evidence or good arguments either way.
In response to some answers:
People would know and understand the sunk cost fallacy. However, this does not always stop people.
After spending two days studying it in my high school economics class, the teacher had the students enter a sunk cost auction (in which you have to pay whatever you bid, even if you don’t win).
Even after having done a homework assignment on sunk costs and discussed them at length, many students found themselves betting on sunk costs. In the end, one guy payed $20 for a can of coke.
I believe the point of prediction markets is primarily to create a more rational world, and to help people make informed decisions (e.g. politicians) rather than act blindly on irrational pet beliefs. Feel free to correct me if I’m wrong.
What this story tells me, more than anything else, is that people fail to internalize sunk cost fallacies.
This seems to be common of people who learn about biases. Knowing about the planning fallacy doesn’t seem to make a real difference. In Thinking: Fast and Slow Daniel Kahneman tells the story about students learning about the bystander effect, but failing to actually update their thinking.
This tells me that we need a new, better way of teaching people about biases so that the knowledge actually influences their mind and they internalize it.
What does this say about prediction markets? I have no idea.
[…]
I really like these points on steelmanning. I guess generally I feel like there are two goals you might have in reading articles like this:
1) decide if there are good reasons to agree with the author’s point (i.e. self-driving cars are ethically problematic)
2) understand the author’s worldview (i.e. killing people in the trolley problem is bad)
These are both good goals and you have to read the article in different ways for each. Steelmanning is aimed at the first, not the second.
And the first is more practical to do in the short term. For instance, like 90% of what we read in my philosophy class is super deontological. If I had to read all of it as deontological, I guess there’s some small chance I’d come to understand their value system better and convert to deontology at some point. But in the meantime, I wouldn’t be able to believe anything they said, whereas by steelmanning them as consequentialist I can find some parts that make sense in my belief system.
worldoptimization has a good point. I think there are usually good reasons to agree with the author’s point, and good reasons not to; the question is of which outweigh the other.
I am not sure you have to read the article differently; however, since the second goal tends to support the first in many ways. Rather, it helps to understand the author’s worldview when reading ANYTHING because it will give us greater understanding and help keep our minds open, especially if the article discusses ideas that contradict our own.
See, a thing that’s interesting for this is that I’m pretty sure Jim’s objection to steelmanning is a result of his objection to Scott’s post about why people oppose homosexuality [content warning: neoreactionary], which is actually an objection to #1. Scott is saying, “Look, this particular dumb argument makes perfect sense if you just assume some people think such-and-such and other people are using arguments as soldiers” and Jim finds that totally offensive because he doesn’t think such-and-such at all.
So I think we’re steelmanning Jim’s opposition to steelmanning.
Ooh, this post is yielding all sorts of interesting examples of things I wanted to talk about but didn’t have examples for.
I think this is a good example of the limitations of the principle “don’t do things to people without their consent”1. Here Jim has said pretty clearly that he doesn’t want people to steelman his arguments. And here we are, taking one of his arguments and steelmanning it2. We are using something he made in a way he explicitly didn’t consent to.
But I’m pretty sure we’re not doing anything wrong. We are having new and interesting thoughts and discussing them in a way we likely wouldn’t have otherwise; we’re learning3. And I suppose I can see why that would be upsetting; I certainly wouldn’t be too happy if Jim read one of my arguments and then blogged something about how it gave him this great new argument for why homosexuality is bad.
That wouldn’t be a violation of my consent though, even if I told people not to use my arguments in ways incompatible with my beliefs4. I guess mostly I’m just saying that you don’t get to claim ownership of your arguments after making them. Especially as it would be bad to say that Jim believed [your steelmanned argument] — once you’re asserting that he does still own the argument, it’s screwing with his consent to mess around with the argument in a way he doesn’t want.
I feel like this has some bearing on appropriation as well, but I can’t really figure it out. Something about rituals/ways of expressing oneself being part of culture but beliefs not being part of culture, so therefore you shouldn’t appropriate the former but you can the latter? I don’t like calling beliefs not a part of culture though, that seems wrong. Something about not appropriation only being problematic if you do it to a non-dominant culture? That would say we shouldn’t “appropriate” racists’ arguments that black people shouldn’t marry white people to show that arguments against gay marriage are bigoted, since the former belief is not dominant nowadays. That seems wrong too. I guess what I’m saying in this paragraph is that it seems like this example should illustrate some limits on the scope of “don’t appropriate,” but I don’t actually seem to be able to figure out anything new5.
A good principle in general, I’m not trying to say not to use it; I’m just trying to figure out limitations on its scope. And I think the fact that I didn’t have a handy example is that this isn’t a principle that’s applied too broadly very often; “consent” isn’t at the point of something like “freedom” where everyone claims they’re protecting it. ↩
Probably, anyway. I’m not sure we really are strengthening his argument, we mostly seem to be saying it’s not right, but I guess we’re debating against points that are not quite he said but rather something stronger (to us at least), and hence implicitly steelmanning his argument. ↩
I guess I shouldn’t speak for all of us on this point. I, at least, am learning (and having new and interesting thoughts, etc). ↩
Which, by the way, I like my beliefs and think they are right, and if you manage to interpret one my arguments so that it says that homosexuality is bad, it’s very very very likely that I’ve either said something really stupid or that you’re coming from a really messed up worldview. And in the latter case you should change your worldview to something that isn’t wrong. (In the former case you can just tell me that my arguments prove things they shouldn’t, and then I/we/you can try to sort out what the problem is.) ↩
But maybe someone else can? If you have thoughts, I’m interested. ↩
Ooh yay this is interesting. I feel like exchange and cross-pollination of ideas is really important so I’m skeptical of any limits on it except limits necessary to incentivize content creation (you can’t copypaste my book and then sell it). I don’t think Jim is going to stop coming up with ideas just because we steelman them. This is also why I get annoyed at authors who don’t want fanfic of their work written, and why I can’t suggest any new limits on “don’t appropriate,” because I don’t really get what’s bad about appropriation in the first place.
Maybe I just haven’t had the arguments against cultural appropriation properly explained to me? And I also think I lack the “being offended” gene. Like if I were an author and someone wrote terrible fanfic of my characters I don’t think I would mind at all, and on St Patrick’s Day (I was raised Irish Catholic) I recognize that there are lots of appropriation and bad Irish stereotypes going on but I don’t feel bothered by it.
Your statistical power is WEAK, your effect size is SMALL, you will not survive the winter.
The truth is common property. You can’t distinguish your group by doing things that are rational, and believing things that are true. If you want to set yourself apart from other people, you have to do things that are arbitrary, and believe things that are false. And after having spent their whole lives doing things that are arbitrary and believing things that are false, and being regarded as odd by ‘outsiders’ on that account, the cognitive dissonance pushing children to regard themselves as Xes must be enormous.
[…] Those 16,000 lives are unlikely to all be the same ones lost in an alternate world without robot cars. When we say autonomous cars can slash fatality rates by half, we really mean that they can save a net total of 16,000 lives a year: for example, saving 20,000 people but still being implicated in 4,000 new deaths. There’s something troubling about that, as is usually the case when there’s a sacrifice or “trading” of lives.”
—
- “The Ethics of Saving Lives With Autonomous Cars Is Far Murkier Than You Think”, Wired
sometimes I forget how objectively terrible the field of moral philosophy is
[clipped]
(via yxoque)
That article strikes me as a really poorly written version of this article. The argument as I understand it is that, while less people overall would die, it’s not just different people who would be killed, but a specific class of people would become more likely to be killed.
(via nextworldover)
Wow! It seems…. a little too generous to ascribe that stance to the Wired article, but that’s a much more interesting position and more worth discussing. […]
On one of his recent posts, Scott Alexander backed away a bit from steelmanning1, in response to the comment
James A. Donald (external link)
Please, no more steelmanning.
Person A says something that Person B finds strange and unreasonable, something that makes no sense given the rest of Person B’s worldview. So, Person B, “charitably” invents a somewhat similar position that makes sense given person B’s worldview Which worldview person A massively disagrees with, and is indeed violently insulted by.
Whenever I see the word, I assume, usually correctly, that you are about to demonize your opponents with some version of their argument that only makes sense, only is steel, given that everything that they believe is wrong, everything you believe is correct, and they are evil and stupid.
And even if you have the best of intentions, and I doubt your intentions are as pure as you think, you are going to wind up inadvertently doing that.
This comment seemed a bit off to me, and with this example I see why. If you work under a moral system where the correct answer to the trolley problem is “don’t kill some small number of people to save a larger number of people”2, then this does look like perverting an argument to some version that only makes sense if everything you believe3 is wrong.
On the other hand, for people who think the answer to the trolley problem is “do kill some small number of people to save a larger number of people” this steelmanned argument is important, interesting, novel, and potentially true where the previous argument seemed pretty clearly false4.
And I don’t think the point of steelmanning is to present someone else’s points the way that person would most like them presented. I think the point is to make the argument as strong as you can. And that’s very likely going to be by making it match your own belief framework5.
James A Donald is probably right that this doesn’t quite maximize charity. The person whose argument you’re rewriting will likely think it worse6. If you are valuing charity above all else7, this will likely fail. In most cases just republishing the other person’s argument will be more charitable.
But if you value being correct above all else8, I think steelmanning is a pretty good idea
He only mentioned it as a tangent to his main point, so I’m not sure how much this is a general shift in his approach and how much it was just about that one post. ↩
Which is presumably the Wired author’s moral system, judging by the article. ↩
or at least one thing you believe. ↩
and not in a particularly interesting way, either. ↩
Both because that’s probably where you’ll end up if you try to make the argument as strong as you can (since you believe what you believe), and also because each person has the most comparative advantage for strong argument writing under the “using own belief system” condition. ↩
This is not guaranteed; they might be persuaded by you or they might think you were better able than them to make the point they wanted to make. If one of these happens, everyone wins, and the rewrite will almost definitely be considered charitable by the involved parties. ↩
Valuing one thing above all else is usually a bad idea. ↩
See previous footnote. ↩
I really like these points on steelmanning. I guess generally I feel like there are two goals you might have in reading articles like this:
1) decide if there are good reasons to agree with the author’s point (i.e. self-driving cars are ethically problematic)
2) understand the author’s worldview (i.e. killing people in the trolley problem is bad)
These are both good goals and you have to read the article in different ways for each. Steelmanning is aimed at the first, not the second.
And the first is more practical to do in the short term. For instance, like 90% of what we read in my philosophy class is super deontological. If I had to read all of it as deontological, I guess there’s some small chance I’d come to understand their value system better and convert to deontology at some point. But in the meantime, I wouldn’t be able to believe anything they said, whereas by steelmanning them as consequentialist I can find some parts that make sense in my belief system.
worldoptimization has a good point. I think there are usually good reasons to agree with the author’s point, and good reasons not to; the question is of which outweigh the other.
I am not sure you have to read the article differently; however, since the second goal tends to support the first in many ways. Rather, it helps to understand the author’s worldview when reading ANYTHING because it will give us greater understanding and help keep our minds open, especially if the article discusses ideas that contradict our own.
See, a thing that’s interesting for this is that I’m pretty sure Jim’s objection to steelmanning is a result of his objection to Scott’s post about why people oppose homosexuality [content warning: neoreactionary], which is actually an objection to #1. Scott is saying, “Look, this particular dumb argument makes perfect sense if you just assume some people think such-and-such and other people are using arguments as soldiers” and Jim finds that totally offensive because he doesn’t think such-and-such at all.
So I think we’re steelmanning Jim’s opposition to steelmanning.
[…] Those 16,000 lives are unlikely to all be the same ones lost in an alternate world without robot cars. When we say autonomous cars can slash fatality rates by half, we really mean that they can save a net total of 16,000 lives a year: for example, saving 20,000 people but still being implicated in 4,000 new deaths. There’s something troubling about that, as is usually the case when there’s a sacrifice or “trading” of lives.”
—
- “The Ethics of Saving Lives With Autonomous Cars Is Far Murkier Than You Think”, Wired
sometimes I forget how objectively terrible the field of moral philosophy is
[clipped]
(via yxoque)
That article strikes me as a really poorly written version of this article. The argument as I understand it is that, while less people overall would die, it’s not just different people who would be killed, but a specific class of people would become more likely to be killed.
(via nextworldover)
Wow! It seems…. a little too generous to ascribe that stance to the Wired article, but that’s a much more interesting position and more worth discussing. […]
On one of his recent posts, Scott Alexander backed away a bit from steelmanning1, in response to the comment
James A. Donald (external link)
Please, no more steelmanning.
Person A says something that Person B finds strange and unreasonable, something that makes no sense given the rest of Person B’s worldview. So, Person B, “charitably” invents a somewhat similar position that makes sense given person B’s worldview Which worldview person A massively disagrees with, and is indeed violently insulted by.
Whenever I see the word, I assume, usually correctly, that you are about to demonize your opponents with some version of their argument that only makes sense, only is steel, given that everything that they believe is wrong, everything you believe is correct, and they are evil and stupid.
And even if you have the best of intentions, and I doubt your intentions are as pure as you think, you are going to wind up inadvertently doing that.
This comment seemed a bit off to me, and with this example I see why. If you work under a moral system where the correct answer to the trolley problem is “don’t kill some small number of people to save a larger number of people”2, then this does look like perverting an argument to some version that only makes sense if everything you believe3 is wrong.
On the other hand, for people who think the answer to the trolley problem is “do kill some small number of people to save a larger number of people” this steelmanned argument is important, interesting, novel, and potentially true where the previous argument seemed pretty clearly false4.
And I don’t think the point of steelmanning is to present someone else’s points the way that person would most like them presented. I think the point is to make the argument as strong as you can. And that’s very likely going to be by making it match your own belief framework5.
James A Donald is probably right that this doesn’t quite maximize charity. The person whose argument you’re rewriting will likely think it worse6. If you are valuing charity above all else7, this will likely fail. In most cases just republishing the other person’s argument will be more charitable.
But if you value being correct above all else8, I think steelmanning is a pretty good idea
He only mentioned it as a tangent to his main point, so I’m not sure how much this is a general shift in his approach and how much it was just about that one post. ↩
Which is presumably the Wired author’s moral system, judging by the article. ↩
or at least one thing you believe. ↩
and not in a particularly interesting way, either. ↩
Both because that’s probably where you’ll end up if you try to make the argument as strong as you can (since you believe what you believe), and also because each person has the most comparative advantage for strong argument writing under the “using own belief system” condition. ↩
This is not guaranteed; they might be persuaded by you or they might think you were better able than them to make the point they wanted to make. If one of these happens, everyone wins, and the rewrite will almost definitely be considered charitable by the involved parties. ↩
Valuing one thing above all else is usually a bad idea. ↩
See previous footnote. ↩
I really like these points on steelmanning. I guess generally I feel like there are two goals you might have in reading articles like this:
1) decide if there are good reasons to agree with the author’s point (i.e. self-driving cars are ethically problematic)
2) understand the author’s worldview (i.e. killing people in the trolley problem is bad)
These are both good goals and you have to read the article in different ways for each. Steelmanning is aimed at the first, not the second.
And the first is more practical to do in the short term. For instance, like 90% of what we read in my philosophy class is super deontological. If I had to read all of it as deontological, I guess there’s some small chance I’d come to understand their value system better and convert to deontology at some point. But in the meantime, I wouldn’t be able to believe anything they said, whereas by steelmanning them as consequentialist I can find some parts that make sense in my belief system.
1) A post about liking cheese became a giant debate on utilitarianism and what it means to make the universe happy. 2) A cute post about how cool/cute internet people tend to live far away from you became a giant proof of the people on tumblr actually existing and/or trying to explain the effort someone had to put in to fake all that.
And also, of course, it has made me an objectively better, more effective person.
I’ve never seen the Icarus story as a lesson about the limitations of humans. I see it as a lesson about the limitations of wax as an adhesive.
At every period of history, people have believed things that were just ridiculous, and believed them so strongly that you risked ostracism or even violence by saying otherwise. If our own time were any different, that would be remarkable. As far as I can tell it isn’t.
Pressing the ‘I win forever’ button is not a good idea, because it’s also the ‘all progress stops with me’ button.