Forum Replies Created
-
Posts
-
FYI this is a better explanation than I was to give regarding the ethos of EA’s: https://twitter.com/robbensinger/status/1728590241924251720
I’ve been reading Hayek’s “The Intellectuals and Socialism”, which I found through “Culture Wars are Long Wars”.
I realized several things.
First, Hayek’s intellectual is a literal description my job. I perform the filtering function that he describes. Clients hear about some particular fancy new software technology and ask me if they should use it. I’ll say, “Well, if you needed A, B, or C, it might be something to consider, but since you don’t, it wouldn’t be useful for this project.”
It is not surprising that the real scholar or expert and the practical man of affairs often feel contemptuous about the intellectual
I’ve felt this a bit about myself, not to the level of “contemptuous”, but at least noting that I’m not an expert in AI safety. I’m not a mathematician or a decision theorist. I don’t have a direct contribution to make. It was unclear to me how I might help beyond e.g. donating money or joining a write-in campaign.
However, as a Hayek intellectual, it’s my job, maybe not a paid job in this particular instance but the duty I do for society, to forward the best explanations and arguments.
As an example, according to what I’ve read so far in press reports there was apparently a split in the board, a “go slow and carefully” faction and a “go fast” faction. Meanwhile Yudkowsky says that both approaches are lethally dangerous. While I don’t have any original insights here, it’s my duty as a Hayek intellectual to explain that Yudkowsky is correct.
If we still think [the intellectual inclined to socialism] wrong, we must recognize that it may be genuine error which leads the well- meaning and intelligent people who occupy those key positions in our society to spread views which to us appear a threat to our civilization. 1
1. It was therefore not (as has been suggested by one reviewer of The Road to Serfdom, Professor J. Schumpeter), “politeness to a fault” but profound conviction of the importance of this which made me, in Professor Schumpeter’s words, “hardly ever attribute to opponents anything beyond intellectual error.”
There are arguments that I’ve seen which I judge to be intellectually dishonest. There are others which I believe to be genuine error. Something for me to keep in mind.
the characteristic errors of any age are frequently derived from some genuine new truths it has discovered, and they are erroneous applications of new generalizations which have proved their value in other fields
[..]
The argument will not lose its force until it has been conclusively shown why what has proved so eminently successful in producing advances in so many fields should have limits to its usefulness and become positively harmful if extended beyond these limits.
The ethos of modern software development (open collaboration, unfettered experimentation, rapid iterative development, “move fast and break things”) has been enormously successful. (At least in a relative sense compared to other sectors of the economy). While I don’t discount economic self-interest as a factor as well, I think that applying the usual software development approach to AI is an example of this kind of characteristic error of applying what was successful in one field beyond its limits to a different field where it is harmful.
As an aside, I notice to my embarrassment that I’m not used to reading sentences as long as the ones that Hayek uses. By the time I get to the end of the sentence sometimes I’ve lost track of what it was saying at the beginning 🙃
Twitter may not have the 140 character limit any more, but it still truncates the display of tweets at around 276 characters with a “Show more” link. This is shorter than many of Hayek’s sentences.
Thank you Tanner for your thoughtful reply to my late-night rant 🙂.
To turn this in a more positive direction, now that I’ve discovered that I have this passion for honest argumentation, what do I want to do about that?
I increased my contribution to the Scholar’s Stage on Patreon as a way of expressing that I want to do more to support honest argument.
Hmm, at age 56 I seem to be developing an intense visceral dislike of intellectually dishonest arguments. Of course, bitterly complaining about them one at a time is a foolish activity… like complaining about individual rain drops in a torrential rainstorm. 🙃
There’s a connection, I think, between cryptocurrency and the ongoing Scholar’s Stage thesis regarding the American decline in capability and agency. There’s a cryptocurrency story that goes something like, we used to have a physical frontier and developed self reliance. That’s gone now, but there’s a new frontier, “cyberspace”, the Internet, cryptocurrency, where people can self-govern and act with agency. Balaji’s thesis is the lessons learned and power gained can be used to recapture agency and self-governance back in the physical world. From my perspective I don’t think they’ve been very successful (at least not yet), but I suspect it’s an important part of the self-image.
You may not need to include every flavor of techno-optimism in your comparison with China, but another flavor to keep in mind is the effective altruists. The advocates of uncontrolled AI development call them “doomers” now for saying that uncontrolled AI is likely to be unsafe, but e.g. MIRI has spent the last twenty years on research for how to create a superintelligence, safely. I suspect the EA’s are the most ambitious of the flavors (while perhaps having correspondingly the least real-world impact so far). E.g. EA’s say “we may all die in a trillion years in the heat death of the Universe, but a trillion years is better than a hundred, so let’s cure disease and old age”.
The EA’s are strongly accellerationist, with the caveat that they want a positive future with human values. Perhaps having so much in common explains why the e/acc dispute with EA’s is so bitter.
Well, “tech billionaire” and “twitter” are ad hominem attacks. Like saying “Buddha was just a rich prince who wrote stories”.
One measure is reach, number of adherents, but also number of people impacted. Another measure is intellectual depth. And of course the two don’t have to go together.
Reach can be hard to measure. E.g. Casey Handmer is working on the hardest of hard hardware startups: creating synthetic fuel using solar panels and CO2 extracted from the air… cheaper than drilling for oil. Was Casey personally influenced by Industrial Dynamism concepts? I have no idea. Maybe without Industrial Dynamism, Casey wouldn’t have gotten started thinking about hard hardware.
Next, Casey’s startup (https://www.terraformindustries.com/) received early seed funding from Patrick Collison, who is also one of the founders of Progress Studies (https://www.theatlantic.com/science/archive/2019/07/we-need-new-science-progress/594946/). I would describe Progress Studies as having both intellectual depth and, at this point still small, yet substantive reach.
I had been trying to argue that “e/acc” was an expression of mainstream technology optimism, and, thinking about it more, now I’ve changed my mind.
Taking Paul Graham’s writings as a baseline, one thing that’s notable is its intellectual rigor. Each essay explores a narrow topic in depth.
Like the woke, Andreessen’s manifesto and e/acc are attempting to create, or express, a faith. Like the woke, the argument becomes circular, “you should believe this because it’s bad not to believe this”. Like the woke, they become obnoxious because having abandoned intellectual rigor there’s no alternative.
Cryptocurrency is perhaps the purest expression of the anti-authoritarian nature of hacker values, as well as an illustration of how hacker values fail if they’re taken too far. Individual hackers, freely creating in voluntary association, creating (in the vision) a replacement for banks and governments. Patrick McKenzie (patio11) writes elegantly about how cryptocurrency rediscovers all the lessons of banking and government regulation from the last hundred years.
Let me take another try at articulating this…
There is what I think of as a mainstream Silicon Valley technology optimism, exemplified perhaps by the writings of Paul Graham. Here the optimism is implicit, unarticulated. It is a background assumption, taken for granted. An assumption so obvious that it doesn’t need to be discussed. Paul Graham’s earliest writings were about which programming languages are the most productive. That one would want programming languages to do more, and thus to be able to do more with computers, is obvious and unquestioned.
Industrial Dynamism, or what I personally kind of think of as nostalgia for the 1950’s vision of technology, is a reaction to that mainstream. Back in the 1950’s we aspired to do “real stuff”. We should do more “real stuff”. (Banks are included in “real stuff” because we had banks back in the 1950’s).
I think of Andreessen’s manifesto as a first attempt to articulate the mainstream hacker technology optimism. As a hacker and a technology optimist myself, I personally think he did a bad job. My impression is that he threw together everything he could think of in support of techno-optimism. The result is contradictory, and in declaring some ideas to be a matter of faith and others the enemy, antithetical to hacker culture. But perhaps better arguments will be developed over time.
Hackers are builders, hands-on makers, experimentalists. The warnings about AI from EA’s and MIRI are based on the math of decision theory, not software or programming. This is alien to hackers. They don’t understand, and thus don’t have a way to make an argument within the paradigm that MIRI is using. Not understanding, they don’t have a way to distinguish the argument from nonsense. Not being able to refute the central point, arguments devolve into obnoxiousness and ad hominem attacks.
I’d be surprised if most people who put “e/acc” on their profile have read Land or hold particularly Landian beliefs. I could be wrong. They know they’re against AI being slowed. EA’s say it should be, so they’re opposed to the EA’s. “e/acc” is the opposite, so “e/acc” sounds good. The arguments are incoherent because there are multiple and incompatible reasons for not slowing AI.
I think I can look at these flavors through the lens of regulation.
Progress Studies wants good regulation. We had bad things happen, e.g. DDT poisoning songbirds and leaded gasoline poisoning people, and discovered that we needed regulation. We weren’t very good at it at first, so we ended up with a bunch of harmful regulations and a lot of things being over-regulated. For example, they don’t want medicine to become totally unregulated, but they do want the FDA to stop blocking life-saving treatments. I suspect they may also have a classic liberal kind of belief, or maybe this is a component coming from liberals in the movement, that they can get society headed in the right direction by crafting just the right regulations.
I’m not coming up with a better label than “industrial” (and of course you do say “a strong industrial component”) but the movement is a bit broader than just physical industry. Peter Thiel started at PayPal, which was a software company, but the original aspiration was to replace the banking system. In this mindset, “replacing the banking system” is “real” in a way that just playing around with stock prices isn’t.
I suspect Coinbase falls into this category. Not to replace banks, not directly, or not yet, but to serve people who the banks aren’t serving (which happens to be a ten times larger market). The liberal instinct is to use regulation to ban cryptocurrency because it purports to move finance outside of government control. Coinbase says, “we want to be regulated with appropriate regulation”, not “good” regulation in the same sense as liberals directing society to better outcomes, but more of a classic right-wing “tough on crime” sense where the purpose of regulation is to go after criminals.
Then there’s the “no regulation” flavor, which is very old.
Consider that in 1984, Steven Levy published his book “Hackers: Heroes of the Computer Revolution”. In Chapter 2 he describes the “Hacker Ethic”: “ACCESS TO COMPUTERS–AND ANYTHING WHICH MIGHT TEACH YOU SOMETHING ABOUT THE WAY THE WORLD WORKS–SHOULD BE UNLIMITED AND TOTAL. ALWAYS YIELD TO THE HANDS-ON IMPERATIVE!”
and: “Hackers believe that essential lessons can be learned about the systems–about the world–from taking things apart, seeing how they work, and using this knowledge to create new and even more interesting things. They resent any person, physical barrier, or law that tries to keep them from doing this.”
Vernor Vinge, the science fiction writer who first popularized the idea of a technological singularity, published in 1985 a novella “The Ungoverned” which explored an anarcho-capitalist society, which without a government privately funded decentralized defense, and successfully fought a war against an invading country.
And also consider this portrait of the politics of a typical hacker from The New Hacker’s Dictionary, published in 1991, and based on material from the 1980’s, specifically the “anti-authoritarian” part:
Formerly vaguely liberal-moderate, more recently moderate-to-neoconservative (hackers too were affected by the collapse of socialism). There is a strong libertarian contingent which rejects conventional left-right politics entirely. The only safe generalization is that hackers tend to be rather anti-authoritarian; thus, both paleoconservatism and ‘hard’ leftism are rare. Hackers are far more likely than most non-hackers to either (a) be aggressively apolitical or (b) entertain peculiar or idiosyncratic political ideas and actually try to live by them day-to-day.
In the hacker culture, having lots of people try lots of different things is good, and central planning, central control, or centralized regulation is bad. The essential thing is the individual freedom to create, which in a techno-optimist vision creates a better future.
You write: “According to e/acc types, the technologists of that age did not just attempt to make the world a better place—they succeeded in doing so.”.
I agree, but this is broader than e/acc. Hackers, in general, think they succeeded.
Now they feel under threat. Ominous storm clouds of regulation on the horizon. “e/acc” is a response to that threat, “leave us alone and let us keep doing what we’ve been doing”.
A problem is that there are incompatible reasons for not having regulation. For example, “We don’t need regulate AI because it’s impossible for AI to replace humans and it’s silly to worry about it”. And, “We shouldn’t regulate AI because our creations will replace us and it will be awesome”.
I can imagine that it’s possible that “e/acc” might not last very long if it gets some negative publicity, i.e. if Tivy is correct about Land not believing in preserving human values (I haven’t read any Land myself), and people start talking about that. Or, since “effective accellerationism” is explicitly a play on words in opposition to “effective altuism”… and EA warns of extinction, this might suggest to some that “e/acc” means “not worried, or don’t care, about the possibility of killing 8 billion people”.
Regardless of what happens with “e/acc”, I expect the “no regulation” flavor of techno-optimistism to be enduring.
I’m trying to think of a different name than “tech accelerationist”. Progress Studies and “Industrial” both want to accelerate what they see as progress.
Progress Studies think, of course, that we need to study progress, and figure out what will work and what won’t. Industrial Dynamists fear that hackers, left to their own devices, will waste their time on trivial pursuits such as creating computer games or social media sites. Like a stern father, the Industrial Dynamist says, “get serious!”
The “no regulation” techno-optimist doesn’t think things need to be studied, or that hackers need a kick in the butt. They believe that progress is natural, if people are left alone. If and when problems arise, technologists will figure it out and deal with it, if they’re allowed to.
For example, early cryptocurrency advocates thought that cryptocurrency would operate outside of the banking and legal system. “What about fraud?” No problem, smart contracts or something, some technological solution, would deal with it. Of course it didn’t turn out that way, and we ended up needing police and prosecutors to deal with fraud like always.
I don’t know what name to suggest, but I think the “no regulation” mindset is that progress is natural if technologists have freedom, and so therefore technology should be unconstrained.
I think this mindset is the oldest and has the most adherents, but it’s also diffuse and for many is non-urgent (i.e. “things are going well”)
“e/acc” / Andreessen is a subset that “our status quo is under attack, we need to do something”
Thus for example common to “e/acc”, as for “no regulation” in general, is the belief that progress should be unconstrained, even though there are wildly different visions of what the result would be.
Tivy, or Land as quoted by Tivy (I wasn’t sure which), thinks that the end result of an uncontrolled superintelligent AI would be “agents pursuing their own unique visions and interests”. Some imagine that an AI might be gracious to us, its creators. SignField420 considers the possibility that an AI might hold us accountable.
These are all human values.
Even the most vicious Nazi, the most fanatical death cultist, the most atrocious mass murder are expressing human values. Evil values, of course, but values recognizable as something some human might want.
The chance that an uncontrolled AI will end up executing a goal that we would attach any human value to is vanishingly small.
Creating an uncontrolled superintelligent AI is simply committing suicide. The result won’t be any more profound than being sloppy at mixing up a batch of explosives and blowing ourselves up. There won’t be a rich landscape of agents pursuing unique visions. The AI won’t hate us, or care about us, or remember us.
Whatever outcome we might want from creating a superintelligent AI, whether that’s to create a being to judge us or something else, would need to be engineered. It won’t happen by accident.
We did a terrible job with nuclear weapons, but even there it could have been a lot worse. Moloch / pseudo-Darwinianism would predict uncontrolled proliferation.
@SignField420 Yes, I agree, EA’s are definitely not conservatives. The typical EA is young, a university graduate, often with some kind of science degree. As Tanner pointed out in Culture Wars Are Long Wars, “almost all ‘social’ or ‘value laden’ attitudes are established early in life and are then maintained throughout it.” EA’s generally share the same values as their classmates.
Would a useful summary be that the paucity of their values leads their utilitarianism to collapse to hedonism, to hedonic utilitarianism?
I found it interesting to compare “Instead of relying upon their own intuitions and common sense wisdom, parents in the 20s, 30s, and 40s, Lasch explains, began to defer more and more to those claiming special medical, psychiatric, and social-scientific knowledge, whose shifting and frequently conflicting advice often helped only to confuse them.” with the woke position that ordinary people are helplessly immoral with correct moral instruction from the woke elites, combined with the ever-changing requirements and definitions of the woke ideology.
@SignField420 you say that EA’s are “hedonic utilitarians”, and Tanner says that their morals obviously boil down to questions of pain and pleasure. I don’t mean to argue with you, maybe you’re right that EA’s are mistaken about what their ethics are. I’m just curious what led you to your conclusion?
Well, SBF was a fraudster and a pathological liar, and common to fraudsters and pathological liars is claiming to be an ethical person. He even came out and said that “the ethics stuff” was “a lot” (that is, mostly) a “front”, a “dumb game we woke westerners play where we say all the right shiboleths and so everyone likes us”.
And, sure, maybe he was lying yet again and his earlier claims to ethics wasn’t a front, and he was in some sense actually an effective altruist… or maybe there was a defect in the EA ethical system that enabled or encouraged him, and it wasn’t just a convenient currently popular flag that he picked up to wave around to get people to like him and to distract from his fraud. Is there anything you heard or know about to suggest that?
The comment on “e/acc” meme ideology (if we take AGI seriously) having the “aim” of replacement and extinction of humanity by machines I think may be misplaced. That might well end up being the result, but the people I know I see labeling their Twitter profile with “e/acc”, I’d be very surprised that they intend that. My impression is that “e/acc” reflects a kind of naive optimism that may be helpful in e.g. starting companies, but probably isn’t what you want in someone e.g. performing heart surgery on you.
Re Yudkowsky “his conclusion seems to be that no one has any idea how to make AI safe”, we can take out “seems”. Yudkowsky believes that it is possible to create a safe AI, and that we will be able to create a safe AI someday if we keep researching it (like he and his other researchers at MIRI have been doing for the last eighteen years), but that we don’t know how to create a safe AI yet. Further Yudkowsky believes that creating an advanced AI safely requires rigorous math, and that e.g. the OpenAI approach building AI’s to figure out AI safety for us is not going to work.
As an aside, I realize we need words to label the various positions and we’re stuck with “doomer” now, but something to keep in mind is that Yudkowsky isn’t at all opposed to AI superintelligence, he thinks it would be great, if we build one that does what we would want.
“Doomer accelerationism seems to be the result of a crisis for humanism.” This sounds interesting. I have been mystified at reports that some AI developers think there’s a substantial chance that their work would lead to extinction. Even if they were wrong, and there actually was a 0% chance, wouldn’t that still be a crime? Reckless endangerment? Criminal negligence?
Like, suppose I shot up my barn, and suppose it was legal for me to use my barn as target practice, except that I had a belief that there was a 30% chance that someone could be inside, and I had an expectation that I would kill them if they were… even if it turns out that the barn was empty, recklessly not checking whether it was safe to shoot at the barn would be a crime that I could be prosecuted for.
“If artificial intelligence will eventually be more efficient or powerful than humans at all tasks, and will replace humans in everything from industry to politics to philosophy, then what’s left for humanity? If humans aren’t the center of the moral universe, then what?”
Just because something is superintelligent doesn’t mean that that it will have good morals. There’s a fallacy that imagines that if something is “superhuman”, it will be superhuman in all dimensions, including morality, godlike in the sense of being better than humans in all ways. But this is confusing the meaning of “better”: there’s “better” in the sense of “more powerful than a human” and “better” in the sense of “having a better outcome”.
Most people care about human values. Most people, for example, are not in favor of gratuitously torturing children to death. If we create a superintelligence, and we want it to have human values, that’s something we would need to do. It would need to be engineered. It would not happen by accident.
Then there’s the idea that we’ll all be dead, “but at least the future [will be] built by agents pursuing their own unique visions and interests”. But that’s not guaranteed to happen by accident either. Just because a machine is superintelligent in terms of being able to accomplish a goal doesn’t mean that it will have a goal that would be at all complex, unique, or interesting. A superintelligent AI isn’t human. We would find a simplistic and monotonous goal boring. An AI wouldn’t have to. It could pursue what we would find a trivial and boring goal with overwhelming intelligence and tenacity.
“But trying to prevent an open-ended space of possibility to entrench humanity as we know it amounts to an ironic inversion of humanism: humanity transformed from open-ended starting point to dogmatic conclusion. And the smartest doomers don’t even think it will work.”
I’m confused here by what is the “it” that the “smartest doomers” don’t think will work. Deciding not to create a superintelligence? The smartest doomers don’t think that we can avoid creating a superintelligence forever? But if delay the creation of a superintelligence by 10 years, or 20, or 30, we would have time to learn how to create one safely. We could have a societal conversation about whether to create one at all. If we decide we want to create a superintelligence, and we want it to have human values, we could talk about what values we thought were important.
“Land once politely called me a monkey socialist for saying we needed to make sure the whole system continues to serve human purposes.”
OK, I don’t know if this is Land, or Tivy’s characterization of Land, or my misunderstanding, but to me this sounds like a death-cult villain from a movie? “Let’s kill everyone, and also make no effort to try to get our replacement to embody human values.”
“Capitalism” is “inevitable” and “uncontainable”, but we shut down building new nuclear power plants for 50 years because they sounded scary. We stopped producing DDT because it was killing the songbirds. We fixed the ozone hole. We measure the safety of commercial airplane flights by number of fatalities per billion passenger miles.
Over-regulation may be bad (some economists claim that the FDA kills more people by disallowing life-saving drugs than it saves by banning harmful drugs), but the government has the power to crush industries.
To me, Land’s description of capitalism sounds like a dystopian left-wing fantasy, not something related to reality. “Capitalism is terrible”, typed the philosopher into his Macbook while sipping chilled wine, flying a mile above the ground. I get that people feel alienated by having managers and jobs and not understanding the complexity of the modern economy, but that doesn’t mean it’s valid to project that feeling onto the economy. If Land wants to commit murder-suicide, justified by imagining we’ll be replaced by something better, well, my preference is that he commit suicide by himself and not try to take me with him.
OK, I’m going to have to skip ahead.
I don’t understand the conclusion. “Don’t worry about the long term destiny of man”, but “wax your own will to power”.
Power is the ability to do things, such as choosing the long term destiny of man.
“If we are to be overcome by some superior post-human species, we can go down with such a fight as to prove ourselves worthy of remembrance”.
If an uncontrolled superintelligence is created, there won’t be anything left inside the universe that will or will care to remember us.
But before we start taking actions where the only difference they make is how worthy they make us, perhaps we might like to spend a bit more time considering whether or not we might like to avoid destroying ourselves in the first place.
The link is broken. I did a Google search, is this the correct link? https://www.palladiummag.com/2023/11/03/make-yourself-human-again/