LLMs: Subroutine or Kernel

While the ideas were mine and the prompt was long, everything below is unedited GPT4 output. Here is a fun exercise: Guess in whose style I instructed GPT4 to write it?

A Shift in Computational Paradigms

In a recent tweet, Andrej Karpathy eloquently laid out the idea of large language models (LLMs) evolving from mere chatbots to becoming something quite similar to the kernel process of a new Operating System (OS). As someone who has spent a lifetime pondering the fundamental principles of computation, I find this to be a tantalizing concept that is profoundly disruptive to our established understanding of software engineering.

Traditional Computing: A Model of Control and Determinism

In traditional computing, the software we design runs atop a robust operating system that governs the machine's I/O, computational abilities, and memory management. We, the software engineers, are in full control; we instruct the machine in a deterministic manner through a set of precise and efficient programming languages.

Emergent AI: Subroutine or Sovereign?

But AI, particularly LLMs, are introducing a seismic shift in this established order. The natural progression would be to treat AI merely as a subroutine—a tool that functions under the directive of a software written in a conventional programming language.

However, what Karpathy and others envision is not merely a slight modification but an inversion of this relationship. The AI becomes the kernel, the orchestrator, which in turn utilizes traditional software modules as its subroutines.

The Implications: Two Divergent Paths

AI as a Tool: A Loyal Servant

By keeping AI as a subroutine, we retain control. These AIs are then optimized to execute specific tasks, adding value by making our machines smarter, more human-friendly, and more efficient. They are loyal servants, doing precisely what we've instructed them to do. If something malfunctions, it is transparently trackable back to human-authored code.

AI as a Kernel: An Adaptable Sovereign

On the flip side, the “AI as a Kernel” approach results in a system that is more resilient and capable of adapting to unforeseen circumstances. Here, the AI is not just a tool, but a managing entity that makes its own decisions based on its pre-trained models and real-time data. This has the potential to be more flexible but also more opaque, relinquishing some amount of human oversight.

The Inevitability and Pragmatism

One might argue that relinquishing control to an AI kernel could be perilous. Yet, the fact remains that if there is intrinsic value in the "AI as a Kernel" paradigm, it's not a question of 'if' but 'when' someone will implement it.

Use Cases: Navigating Choices

Given the varied landscape of computational needs, the approach we choose may indeed be dependent on the specific use case. For example, the operating kernel for habitat construction robots on Mars might be better managed by an LLM. It can adapt to unforeseen environmental conditions and make real-time decisions. In contrast, when it comes to our international banking system, where transparency, traceability, and control are of paramount importance, traditional software architectures may be preferable.

Final Thoughts

As we navigate these dual pathways, we face profound questions about the nature of control, adaptation, and human-machine interaction in the computational universe. Each approach has its merits and risks, but one thing is certain: the computational landscape is on the cusp of a radical transformation. The challenge—and the opportunity—lies in wisely choosing our path.

---

Final Final Thoughts, after the human reflected on GPT4’s writing:

It probably does make some sense for the good actors to focus more (or mostly) on AI as a tool rather than as the kernel, just because of the path dependencies here: Perhaps the kernel approach has some pitfalls that we don’t see yet, and perhaps we can do a better job building it safely once we are better equipped with AI tools.

Also, to be clear - the LLM subroutines I’m talking about can easily themselves use software tools, which in turn can use LLM subroutines, etc. The discussion here is just about the top level — who or what is ultimately in charge, at a level that is more fine grained than just the OFF switch.