Jeff Hirsh Read: January 2024
Inspiration: Read an article about the founding of DeepMind and wanted to learn more on AI from one of the founders
Summary
Written with the help of ChatGPT, below is a brief summary to understand what is covered in the book.
“The Coming Wave”, published in 2023 by AI researcher, entrepreneur, and author Mustafa Suleyman, details the widespread implications of the rapid development of AI and other fast-developing technologies. The book establishes “the containment problem”—the task of maintaining control over powerful technologies—as the essential challenge of our age. Suleyman issues an urgent warning of the unprecedented risks that AI and other frontier technologies pose to global order, and how we might contain them while we have the chance—from a co-founder of the pioneering artificial intelligence company DeepMind.
Unedited Notes
Direct from my original book log, below are my unedited notes (abbreviations and misspellings included) to show how I take notes as I read.
Suleyman founded DeepMind in 2010 with goal to create an algo that could distill the essence of what makes humans so productive, AI=science of teaching machines to learn humanlike capabilities, AGI=point at which AI can perform all human cognitive skills better than smartest human, ACI=point between AI and AGI (“Capable”) where can handle range of complex tasks but long way from fully General, the Coming Wave is the emergence of AI and synthetic biology, The Dilemma=growing likelihood that both new tech and even their absence may lead to catastrophic/dystopian outcomes, people seem to have pessimism aversion with AI—scenarios of synthetic bioweapons, bad actors, techno-authoritarianism are increasingly viable for those motivated so worth contemplating, 1947 Bell Labs invent transistor (a semiconductor creating the “logic gates” to perform calcs, IBMs Pres Thomas Watson famously said “world market for about five computers”, then Noyce and Fairchild invent integrated circuit, and Moore’s law take action from there, “waves” come through history as technologies collide and expand but lumpy paths, nuclear weapons represent the one tech that has relatively been contained but more due to incredible costs and mutual destruction vs ai/tech increasingly cheap and has societal benefits so containment is harder, DeepMind made first breakthrough in 2012 as mastered Breakout video game via DQN network but few really take notice, real storm came in 2015 for DeepMind when beat master at AlphaGo with strategy all thought was a mistake—rewrote tactics after studying millions of games on its own, AI and synthetic biology are the central general-purpose tech of the new wave but also surrounded by quantum computing, robotics, nanotech, and more, AI entered lexicon 1955 but first real breakthrough was 2012 AlexNet which leveraged deep learning—neural networks modeled on human brain trained on data (images for AlexNet), AN built by Geoffrey Hinton and 10% improve past other image tech—big leap “computer vision”, computer vision at center of all modern AI use cases, Hinton hired by Google then Google acquire DeepMind 2012 as pivot to AI focus, GPT stands for Generative Pre-Trained Transformer, LLMs take advantage of sequential nature of language, words give clues and relate, can “attention map” based on “tokens” of letters and words that appear frequently together (somewhat like syllables) and then autocomplete, big advancement is now AI can learn on raw data vs early 2010s required hand-labeled data via “supervised” learning, in under 10 years amount of compute to train AI up 9 orders of magnitude—far faster than Moore’s Law, cost of sequencing human genome fell from 1bn in 2003 to well under 1k by 2022–1000x faster than Moore’s Law, DNA sequencing akin to reading vs synthesizing akin to writing, DNA synthesizing taking off—write entirely new genetic designs, Altos Labs has raised 3bn with mission to reverse aging biomarkers, for half a century the protein folding question—how protein forming DNA would fold—was unsolvable, then 2018 a DeepMind neural network solve it out of “nowhere” at annual competition—exponential change moment, AI and bio are the general accelerants with clusters forming around it and at intersection, robotics is the physical manifestation of AI—automation of material vs AI is automation of information, quantum computing very early on but essentially enables storage of unparalleled amounts of info in compact place—Google quantum computer breakthrough 2019 could store equivalent of 72bn GB of memory, more broadly it is energy consumption that limits rate of progress which is why renewable and new forms of energy so important, slow but key steps in nuclear fusion as energy source where more out than out in, 4 intrinsic features which compound the problem of containment: asymmetric impact, hyper-evolution, omni-use, autonomy, new tech creates unthinkable pressure points/vulnerabilities, fast iterations, many different uses for single general tech, more autonomous than anything before, tech increasingly designed to be general/widely useable but more easy to cause harm as a result, most complex neural networks baffle even experts as to how make certain decision—cannot contain if cannot understand, AlphaGo win over Sodel from South Korea was also a national pride punch as 280mm koreans watched and saw defeat at hands of american tech—AI can spark another “cold war” dynamic, AlphaGo loss in China 2016/7 was Sputnik moment where gvt top down direct billions to win AI/quantum tech race, embarrassed in century prior and quickly catching and passing US on patents, quantum comp power, etc, first Chief Software Officer of US pentagon declared 2021 it was already over as China to win next 15-20 years, the best entity to facilitate containment is the nation state, states are built on trust and trust is eroding which further complicates any efforts of containment, very reductive and simplistic to say tech is politically neutral, cannot dissaggregate tech from politics as it inserts itself into so many processes and structure, social media an example of naive view of neutrality, lack of trust inhibits unified coordination across nations globally which is required to contain, politics is power and tech is power, new wave may enable techno-totalitarian gvts never seen before, also can erode democracies to point of ineffectiveness, already seen hackers leverage gvt developed tech for ransomware—luckily not with new wave tech so mitigated impact though still widespread, as new wave tech broadens in terms of accessibility for nonstate actors as well as better as self learning and autonomy—makes harder to identify blame and react, iran nuclear head killed via autonomous robot shooter 2020, but larger question comes in as nation state gets worse and worse as keeping its promise to citizens to stay safe—fragility amplifier, what is the point of remaining loyal to state if cannot keep bargain, gain of function research is where scientists engineer pathogens to be more lethal/transmissible to study and prepare, labor automation and replacement debate often countered with new jobs always come, etc but difference here vs past tech waves is generalized nature and exponential speed in excess of any ability to retrain, also this tech is fundamentally intended to replace humans and not complement, further impact will be to state tax receipts in a jobs recession—nation state power erode as services not provided, must acknowledge that the locus of power lies with corporations in the new wave as research and tech centered there vs any gvt, have companies like google, apple, samsung that are larger than many nation state economies, samsung is a parallel gvt in korea, this will only be amplified (apple 200bn in cash to spend), fortune 500 companies combined revenues are 44% of world GDP, democratization of empowerment—smallest local units/factions can have energy/tech to self-sustain, drive further fragmentation and autonomy of those who grow disenfranchised, corporations can evolve toward statehood, possibility of hyper local microstates given all can easily access most powerful tech, threats such as lab leaks, rogue actors, make catastrophe more likely and accessible from small pressure points, natural response is surveillance state and control—like China’s system but even further, hard to argue against this dystopia in face of threats from new wave and ability to rogue/local actors to disrupt/threaten, however any sort of tech moratorium/pause brings forth another dystopia—inherent to society today is tech developments to keep society and economy afloat given struggles with climate, aging population, etc, standstill means implosion, a tradeoff between prosperity and surveillance could come to be in order to contain and prevent tech from being a net negative to the world, original Turing Test was about computers being able to communicate/generate language that cannot be distinguished from humans and essentially there, Modern Turing Test: what can AI do such as if asked to go make $1m on retail platform with 100k investment in a few months—this requires human-like thought, negotiations, actions, etc to materialize, complex real world goals in Modern Turing Test, beyond just communication, as construct a regulatory approach to contain need to consider price, breadth of use cases, alternatives available, resources/talent constraints, degree of autonomy, if cheap, broad use, no alts and autonomous then hardest to contain/regulate, but can regulate narrow use cases such as specific weapons, also cost hurdles and talent hurdles help contain, need a concerted effort from regulators to force safety research—currently super small teams dedicated but many issues to address (could set 20% of spend on team to safety and publish findings to gvt body), then need to audit code—can have gvt sponsored tech that scans new algos/dna/synthetics to see if any dangerous elements to flag, NVIDIA does GPU, TSMC does chips, ASML does EUV lithography—dutch, most valuable tech company in europe, these are choke points, can buy time by focusing at these points to slow/curb, single mine in North Carolina for 80% of essential quartz for chips/photovoltaic panels, essential to align profit and safety incentives as best as possible since cannot deny profit motive, DeepMind attempted this as a part of Google to ensure profits redirected but hard to operationalize, critics need to be active practioners not just sideline—but critics are essential, gvt needs to be active in tech r&d—expensive but worth it to be in-house with talent and tech not all private sector, boost salaries, have transparency, need “minister of emergy technology” in all gvts, establish a new licensing regime for AI so cannot just freely develop—like how people cannot just fly planes or launch rockets or build nuclear reactors, need licenses that scale with complexity of AI, shift tax burden from labor to capital (more on land, property, shares, etc) and new tax on “robots”, need cross border mechanism, also perhaps require % company value as “public dividend”—need to redistribute for “losers” in AI and AI is broad use/not owned by one so logical to give back, collaboration cannot stop at national borders—need alliances like for nuclear weapons, laser weapons, CFCs, Paris Climate, etc, one inhibiting factor is tech culture does not embrace failure and audits of mistakes—instead when companies fail they turn to secrecy and get ridiculed on twitter, airplanes became safest form of transport over time as used early crashes as robust opportunity to learn and improve system, also AI experts cannot see themselves as just exploring or research—need to own it and think of impact, pause instead of rush to publication,
