💡 Is Singularity Near?

Ray Kurzweil proclaimed that "the singularity is near" in 2005, and reconfirmed his prediction in 2024. "The Singularity" in this case is a somewhat nebulous concept of a point where we the humankind will set on a new trajectory of development nearly overnight. The very idea of a "new trajectory" makes the singularity hard to discuss. Basically, you can imagine it the way you like: we’ll be immortal, we’ll kill ourselves, we’ll create superintelligence, the superintelligence will turn us into superhumans, the superintelligence will kill us all, etc.

The important point here is that the trajectory being imagined is not something like invention of computers, universal voting rights or whatever some of us might call "a new era" in the history of humankind. All these developments were conceivable, and I’d say none of them shook the core of human existence. Kurzweil believes that the pace of technological advancements is getting faster. This not a new idea: indeed, we see that the adoption of, say, Internet, took much less time than the adoption of a telephone. The spread of modern generative AI is so fast that it even breaks the usual pattern of starting expensive and then gradually getting cheaper: it’s racing down the price from the very start. If the trend continues, we’ll indeed stumble into certain "super-advancements" very soon.

As a side note, if we buy this argument, the last decades were more packed with visible technological progress than any preceding age. I certainly witnessed a lot of changes in my lifetime, but I wouldn’t say the pace of changes was too uncomfortably fast to my taste. It was mostly fine. Sometimes it is hard to figure out which technology is worth investing into, and which one is just a passing fad. Am I old-fashioned if I don’t like the newest Windows interface? I hope not. I personally skipped devices like pagers and PDAs, and I don’t feel awkward talking about it, as they proved to be short-lived.

To me, the most unsettling word in the title of Kurzweil’s books is "near". Doing groundbreaking work is exciting, but there is also much dignity in contributing to a certain future-facing faraway cause. People worked on building medieval cathedrals knowing very well they won’t see them complete in their lifetime. Likewise, the abolition of slavery or communist revolution are lifetime projects. Sometimes we gloriously drive our carriage to the bright future. Other times are merely the nameless builders who pave the road for our successors.

"The Singularity" is not the kind of project where merely paving the road fills fulfilling. At least, to me there is little enjoyment in being a part of the "transitional" generation that somehow "contributed" and died away. The singularity is hard to imagine, so it has to be experienced first-hand. It is fine to be a cathedral builder who can get satisfaction by envisioning the complete structure in his mind. It is perhaps even better to be an early communist activist who dies fighting for the utopia instead of actually experiencing the fruits of his labor. In the case of singularity, there is a different, "didn’t make it" feeling: die from pneumonia a couple of years before penicillin arrived; get recognition as a great artist when it does not matter anymore to you; spend your whole life in prison by enraging a tyrant. There is nothing noble about it. These are the stories of pure sadness.

When authors like Kurzweil describe exponential progress of technology, they often make the case that the new, post-singularity reality is just around the corner. However, the issue is that the exponential nature of progress is only visible in the long run.

Our species has been around for 150,000 years or even longer. We typically think of our present world as of "the age of science", having its roots in 16th-17th century or so. However, earlier times knew great inventions such as writing, governments, laws, art. Keeping in mind that the earliest known art is around 50,000 old, and the earliest texts are around 5,000 old, it’s easy to draw a chart of exponential progress going back in time to prehistoric human societies. The trouble is that even relatively large fluctuations do not really matter in the grand scheme of things. Deservedly or not, European Middle Ages have a reputation of being dark and backwards, a sort of age that produced little to match earlier Greek and Latin works. And yet this millennium-long period is just a wrinkle compared to earlier millennia that left almost no visible changes behind. Likewise, later "dark" periods that lasted 50 or even 100 years might have meant much for an individual but not for the "exponential progress" theory.

Even the most recent history shows how it works. Imagine someone living closer to the end of the 19th century. In 20 years they are about to witness cars and airplanes, telephone, motion pictures, and radio, just to take the most obvious examples. They might live long enough to see early computers and space rockets. When I watch a movie made in 1930s, I essentially see the modern world: cities of high-rise buildings, office workers, traffic of cars and buses, and even clothes are often roughly the same (such as male business suits). If I am transported back in time, I won’t be puzzled too much, I guess.

I personally witnessed the widespread adoption of computers in regular homes (offices had them before), the birth of the modern Internet and dotcom companies such as Facebook and Google. Almost all software products we use now appeared within a short span of 10-20 years, and by early 2000s everything was basically there. In 2007 Apple released its first iPhone, which provided a template for a modern smartphone, and in early 2020s we saw the rapid development of generative AI. These are cool and transformative technologies, but I don’t have the feeling of "increased pace" in comparison with the situation 30 years ago. However, it does not change anything: the progress is very real and visible, and if a certain decade happened to be especially fruitful, it does mean the next decade must match it to keep up with the exponential law. Some world events like wars or economic recessions also can delay progress, but as we see from history, even the worst of them did not really change the overall trajectory of human development.

Technological developments go in bursts. Something new arrives. It is cool but rough around the edges, and it takes decades to polish it and reach perfection. Then the cycle repeats. There are many examples: film cameras, CRT displays, mechanical calculators, cassette players. Some late cassette players, for instance, are true technological marvels: they are often surprisingly miniature yet packed with features like double bays, auto-reverse, and equalizers. Same can be said about mechanical calculators. It was great while it lasted, but then many of such marvels became obsolete nearly overnight. However, not all technologies follow this pattern. Civil aviation is roughly the same as 50 years ago — there is little incentive for a breakthrough. Modern cars aren’t the same as 50 years ago but they are a product of a lengthy series of incremental improvements, so their lineage is clearly visible. In contrast, a modern smartphone has very little in common with a telephone of the early 20th century.

Transformative technologies take much of media attention and hide the fact that many ongoing developments are incremental at best. These don’t make headlines, so just by reading news, it is easy to form a picture of one revolutionary change after another. After some decades of rapid progress we don’t see many improvements in laptops anymore, and a 5-year old car is as good as the newest model. News about all kinds of AI technologies hide, for example, that there is relatively little progress in robotics. Very little progress in curing cancer or moving the upper limit of human life. Even many fashionable technologies that did make headlines just a few years ago, such as virtual reality glasses or self-driving cars, are somewhat stuck in their tracks. Consumer virtual reality devices fail spectacularly every time. Self-driving cars are "almost there", but we wouldn’t read about it every month, would we? As a side note, "incremental phase" is not bad, it is usually a sign of maturity.Building a cathedral is an incremental effort, just like improving public transportation, education, or healthcare.

I am not sure how to fit this observation into the idea of "exponential technological progression". One possible argument is just the one I used above: a decade or two of lackluster improvements do not change anything in the grand scheme of things. Another argument is that there is a promise of progression somewhere, not everywhere. Say, we saw many great inventions in the 18th century England, but its medicine remained almost medieval. In hindsight, we know that certain inventions have hard prerequisites. To make a modern computer, you need electronics. To advance medicine, you need to understand genetics. These were out of question in the 18th century, so anyone who would try to build a programmable computational device (like Babbage a century later) was undertaking a doomed effort. Therefore, we should not be too harsh on cancer researchers who can’t crack it. Let’s focus on successes in other fields. There are spillover effects, so maybe these successes will help us to crack cancer as well.

The practical meaning of this picture of reality is very simple: if progress in a certain field has been incremental so far, it will likely stay incremental. We should not expect breakthroughs merely due to the "exponential law". Our only (semi-)realistic hope is for spillover effects from other fields that do experience exponential progress. To take the obvious example, advancing computer technologies affected nearly all branches of science and areas of modern life.

As of today, do we have any looming technology capable of delivering such spillover effects? I think there is only one possible candidate: artificial intelligence. In other words, any "moonshot" dream like life extension, cancer medicine, human-like robots or Mars colony has only one way to get fulfilled: we create superintelligence, and this superintelligence helps us with our problems. I have two issues with this conclusion. First, there is an uncanny deus ex machina air around it. There are no clear paths we have to take to achieve our goals. We do incremental science, and no increments would bridge the gap between our current state of the art and the imaginary shiny future. Thus, it is not even "superintelligence" (something smarter than us) that we need, it is a magic wand. You wave it, and it somehow grants you your wishes. It is possible to imagine such a development, but it’s hard to discuss it seriously. Second, while so much depends on this "AI the magic wand" idea, and so much is at stake, in reality all we have today is the established bag of technologies: deep learning, LLMs, convolutional networks, variational autoencoders, and the like. Are they up to the task?

Frankly speaking, I have no idea. There are too many unreliable signals, and too much is pure guesswork. By unreliable signals I mean that we have to base much of our judgement on hype, advertisement, wishful thinking, fanboy talks and ungrounded criticisms. Much of we hear is about share prices, investments, data centers, beaten benchmarks, US vs China, OpenAI vs Anthropic, etc., etc. How on earth to make sense of all that? There is no way. NVIDIA would sell their chips to whoever pays, so of course they are happy to increase production. OpenAI thinks they must invest more to beat Anthropic and vice versa, and they both want to beat China. Large businesses receive daily pie-in-the-sky promises, and who am I to teach AI firms' sales departments on how to do their work. You know the stories they feed — embrace the new tech or become obsolete, and companies do embrace it with very varying degrees of success. There are wildly different estimates of the actual impact of AI technologies. I believe impact on software development will be profound. However, software development is just a small part of worldwide economy. How much would it affect construction or mining or medicine? No idea. For now we can see that AI can greatly aid people in certain scenarios, but this alone does not sound like a disruptive technology. Say, car number plate detection has been around for a while. It got adopted for ticketing overly fast drivers, and that’s it. Is this AI? It used to be treated this way. Are, for example, LLMs any different, and if yes, how different?

Pure guesswork is much of the current discussion around potential limits of the current AI technologies and their potential impact. The current intellectual landscape surrounding AI is a complete mess. Maybe I am asking for too much, but there are irreconcilable disagreements over very basic points. Acemoglu believes AI technologies will "increase [worldwide] GDP by 1.1% to 1.6% over the next 10 years". Not trivial, but nothing really to write home about. Yudkowsky and Soares predict that we are on the track to build a superhuman AI that will kill us all. Other thinkers do not expect our demise, but definitely expect profound changes brought by AI, definitely outside the estimates of "1.5% GDP". I think Acemoglu misses much in his analysis, but at least he tries to employ mathematical models, so when I disagree with him, I can point to specific factors not being accounted for. Most essays on the subject don’t have any formulas, falling into the land of rough speculations of the most general sort: will burst like a bubble should, will have much or little impact, will kill us all, will propel us to the singularity. Naturally, many of these predictions are based on certain pre-existing beliefs of what the current technology can and cannot do. Sadly, these beliefs, in turn, are often riddled with pseudo-religious arguments. For example: humans are self-conscious because we asked them and they said so, and I, the author, also think I am self-conscious. AI is not self-conscious, because we looked under the hood and it is a statistical parrot. Of course we won’t look under human’s cranium to check what’s there, and we won’t ask AI if it is self-conscious, because why on earth would we use the same criteria to judge them. Anyway, because AI is not self-conscious, it cannot do X, Y, Z, because it is self-evident that only a conscious being can do X, Y, and Z. I find such writings frustrating. (Writings proclaiming the birth of AGI on the basis of yet another beaten benchmark are no less frustrating.) I won’t go now into my own understanding of what kinds of arguments are valid and correct, but clearly intuition won’t work here.

One cool thing about LLMs is not their sheer power. It is unexpectedness of their power. We do not really know what is the limit here, but I think it would be accurate to say that if you asked a group of experts 10 years ago what a massively trained "statistical parrot" should be theoretically able to achieve, very few of them would have portrayed anything close to today’s Claude Code. I think the strong Church-Turing thesis is accurate: in principle, computers can do anything that we can do. Software is the real challenge. There is a concept of "Behavioral modernity" which tries to explain the following riddle. We know that biologically modern humans appeared 150,000 years ago or even before that. However, the earliest evidence of "modern" behavior traits (art, burials, cooking) are at most 50,000 years old. Why the gap? Why people spent roughly 100,000 years living like animals before they developed anything that resembles uniquely human activity? Some theories directly connect "modern behavior" with certain genetic "brain upgrades". I am not qualified to talk about this topic, but apparently we are dealing with some subtle fine tuning of the brain, somewhat akin to software rather than hardware upgrade of our "wetware".

I am saying this because it’s surprisingly hard to predict the limits of a Turing-complete technology. An LLM can write a Python script, so in a sense it can do anything. Yes, it technically has seemingly severe constraints, e.g., it always deals with linearly organized data (a single-dimensional sequence of tokens), but do we rely on intuition when we proclaim such constraints deal-breaking? I readily agree that LLMs have a greatly limited core, but we still do not know where is their ceiling, and in any case we see much progress in AI technologies beyond LLMs: speech and handwriting recognition, image processing, video generation, you name it. We don’t need one single universal technology to solve everything. Our current LLMs use conventional tools like calculators and Python scripts, and this is exactly as it should be. I also use a calculator when I need to multiply numbers.

Personally I think there is not much of a choice but to push forward. We are entering the era of vast publicly available computational resources and vast data sets. In this reality, it does not really matter whether LLMs, deep learning or any other existing technology would propel us into some new stage of existence. We do have resources to research intelligent algorithms. These algorithms can be of great help to us. Let me be blunt. Humankind faces countless challenges. For many of them, solutions are known but not applied due to the lack of political power, motivation, ideological brainwashing, etc. I greatly sympathize with people who got into the wringer, but they don’t need AI to get out of the rut: it is enough for the society to stop acting stupidly. It is easier said than done, but I am not in the business of fixing human flaws. However, there are challenges we do not how to solve. We do not know how to fight aging or cancer, we do not know how to create cheap and eco-friendly energy to power cargo ships and airplanes, we do not know how to make people’s lives happy and enjoyable, and so on. I think the root cause of our inability to do it is not the lack of resources, it is the lack of our intelligence in the general sense. We lack understanding of the world, we don’t have the talent to develop breakthrough technologies, and our cognitive capacity is limited. We do advance, but the biggest hindrance is our inability to find solutions quickly. Everything takes way too long. You can’t make carrot grow faster by pulling it upward, and you cannot fight aging simply by throwing more people and money on the task. We have to wait a bit longer — maybe 100 years, or maybe 500 years? Another option is to get smarter. I think this simple idea, in combination with all pure money making incentives, is one of the reasons why people are ready to throw this much resources at the AI project. The temptation is just too high. Do we have better goals to pursue? Okay, we can fund more of cancer research and get more of the same (pretty lackluster) incremental progress as we had till the present time. Alternatively, we can build tools that would make us much smarter and help us to attack the problem at hand more efficiently. This is clearly a far mor exciting proposal, even with unclear chances of success and numerous murky people doing their murky business along the way.