AI’s Real Divide: Engineering or Software?
The Red Queen problem
Jensen Huang has a simple philosophy: study the inputs, understand the constraints and engineer the output you want.
It helps explain Nvidia’s success. It also points to an important divide in the debate about artificial intelligence.
Is advanced AI fundamentally an engineering problem or a software problem?
The distinction matters because the two traditions produce quite different attitudes to risk.
The engineering view
The engineering view starts from the proposition that complicated technologies can be made progressively safer. Aircraft, power stations and industrial systems fail, but engineers respond through testing, redundancy, standards, monitoring and analysis of failure.
On this view, AI is an exceptionally difficult but ultimately tractable engineering problem. When a model behaves badly, determine why. Improve the training, change the architecture, strengthen safeguards and test again.
The software view
The alternative view comes from software, where complexity has a less reassuring history.
Large software systems routinely behave in ways their designers did not anticipate. Individual components can work as intended while interactions between them produce unexpected outcomes. AI compounds the problem because developers do not explicitly program every capability of a modern model. They train systems and then discover some of what those systems can do.
That changes the question from: Did we build what we designed? to: What exactly did we build?
| Engineering view | Software view | |
|---|---|---|
| Model of AI | A complex system that can progressively be understood and controlled | Complex software whose behaviour cannot always be specified in advance |
| Response to failure | Diagnose, redesign and test again | Failure may expose behaviour absent from the original model of the system |
| Greater capability | Can improve both usefulness and safety | Can introduce new behaviours and failure modes |
| Scaling | Manageable if engineering controls scale with capability | Can turn small weaknesses into systemic problems |
| Development instinct | Build, test, learn, improve | Increase capability only as understanding and control improve |
| Main risk | Excessive caution sacrifices innovation and its benefits | Capability advances faster than the ability to control it |
| Key question | How do we engineer AI safely? | How do we know it will continue to behave as intended? |
The case for engineering confidence
The engineering argument has considerable force.
Modern economies already depend on systems too complicated for any individual to understand completely. Their reliability comes not from eliminating failure but from designing systems that can tolerate it.
AI could follow the same path. More capable models could also become important safety tools, helping identify vulnerabilities, improve infrastructure, accelerate medical research and analyse problems beyond the practical capacity of human teams.
There is therefore a cost to excessive caution. The technology capable of creating new risks may also help address existing ones.
Where software changes the equation
The software objection becomes more important as capability increases.
A bridge does not discover a new way to be a bridge. Its components deteriorate in ways governed by physical processes that can usually be modelled and tested.
Software is less accommodating. Bugs can propagate across millions of machines almost instantly. Complex interactions can produce outcomes that were neither designed nor predicted.
AI adds another dimension. Increasingly capable systems can generate software, interact with other systems and undertake sequences of tasks. The concern is not that such systems must become uncontrollable. It is that the traditional engineering cycle — deploy, observe failure, redesign — becomes less satisfactory as the potential consequences of the first significant failure increase.
Nvidia straddles the divide
Nvidia sits squarely across this divide.
Its expertise is engineering: making the computational machinery underlying AI faster, more reliable and more capable. Its commercial opportunity lies in extending that infrastructure into industries from robotics to health care.
But better hardware also makes more ambitious software possible.
The processors may behave almost exactly as their engineers intended while the AI systems running on them display capabilities their developers did not explicitly specify.
That is the tension: engineering success can accelerate software uncertainty.
Who sets the speed?
The practical argument is therefore less about whether AI development should accelerate or stop than about which discipline should determine its speed.
If AI follows the history of aviation, continued development combined with better engineering and progressively stronger safeguards makes sense.
If advanced AI has more in common with highly complex software, the threshold should be different: increases in capability should depend on corresponding increases in our ability to understand, constrain and, where necessary, stop the system.
That suggests a useful principle for companies and regulators alike: capability should not compound faster than control.
The Red Queen problem
Huang’s formula still applies: study the inputs, understand the constraints and engineer the desired output.
The unresolved question is whether, with advanced AI, our ability to impose those constraints can keep pace with the capability of the systems themselves.
Lewis Carroll supplied an unexpectedly apt metaphor in Through the Looking-Glass. The Red Queen tells Alice that “it takes all the running you can do, to keep in the same place”.
AI safety may face its own Red Queen problem. Engineers can make models safer, improve testing and build stronger controls. But if capability is advancing just as quickly — or more quickly — those improvements may amount to running faster merely to maintain the same margin of safety.
That reframes the argument between the two schools. The engineering view is confident that better technology will produce better tools for controlling technology. The software view worries that each increase in capability also expands the problem those tools must control.
Hardware has taught us that extraordinarily complicated machines can become extraordinarily reliable. Software has taught us that extraordinarily complicated systems can continue to surprise their creators.
The question for advanced AI is therefore not whether the engineers can keep running. It is whether they can run faster than the Red Queen.