Impaired Judgement is Downside of Tech Innovation

Ahuge departure from the collections of slogans masquerading today as serious non-fiction, Professor Gene Rochlin's book, Trapped in the Net, is a fundamental critique of the premises of computerization, written in an original and reflective way that reveals the sort of understanding-both broad and deep-that is disappearing as we slip further into the skein of computerization.

Anyone whose life is contained within the interminable web of computers can read this brief but enlightening book with enormous profit-if they're willing to slog through serious ideas presented in a serious way.

Rochlin's main argument: The relentless computerization of everything that can be computerized, in the name of efficiency and economic performance, is creating a world not only in which the basic skills necessary to operate without computers are lost, but also in which human intervention or control during crises is difficult at best. The increasing transfer of complex processes to computer control creates even more complicated systems, beyond complete human understanding, and those computers can only deal with the conditions for which they're programmed. So the danger that some external shock could create uncontrollable chaos is increased.

The secondary effects of such transfer of control to machines from the human mind, however, are at least as scary as the idea that instead of working on the machines, we are strapping ourselves into the machines- subordinating our abilities to their computing engines.

Among the secondary effects: The loss of slack-the breathing space humans need to adjust to and deal with shocks. While slack seems unnecessary in the computer world, he says, its loss in the real world can actually have deadly consequences.

In finance, to chose just one apt example, technology is forcing people, encouraged by the computer's supposed infallibility, to shuck any true intuitive feel for markets, and to make decisions based solely on what computers tell them to do. Knowing what to do when computers don't have the "intelligence" to solve business problems is no longer part of people's training. After all, computers can only manipulate pre-set algorithms under predetermined conditions.

Seasoned operators in financial markets, on the other hand, not only know what they know, but also what is unknown. This allows them to exercise the art of management and deal with unexpected, external shocks; but people unschooled in anything but operating computers and acting on the outcomes can never develop this skill. What Rochlin falls short of saying is that, as time goes on, judgement will be in short supply as more and more executives will come from this technically adept, but essentially "deskilled," talent pool.

This phenomenon is yet another unanticipated result of computerization, says Rochlin. We see this everyday in fast food restaurants; but it's just as common on the trading floor. Traders rely on algorithms pre-programmed into hand-held computers to tell them how to price futures and options, yet these traders may never have heard of the Black-Scholes model-the foundation of options trading.

Rochlin finally concludes that " if markets can be made any more virtual they might perhaps trade purely conceptual instruments that relate only to what people (and other computers) think might happen, or should happen, making and losing money as a pure computer game instead of being tied to the slow and cumbersome processes of real economic activity." Sound like derivatives?

This is a methodical and informed book that not only tells its readers what is happening, but also how we got there. The first four chapters form a sort of history of the development of the environment of networked computers, explaining how we got to a world in which we more or less blindly accept computer products handed down by scientists who are more concerned with perfecting the machine than making something useful.

As relevant examples go, consider Rochlin's discussion of chess programs. Correctly predicting that computers will eventually be able to beat the best players, he asks: So what? "Chess as a human activity will remain an exploration of the ability to recognize patterns in the face of near limitless possibilities and learn to act into an uncertain future, not recklessly, but with forethought and insight."

This book is important not just because it's a thoughtful critique of a seemingly unstoppable phenomenon, but also because those deeply involved in applying the benefits of computers to the real world need to stop once in a while and ask what they're really doing. This capacity to reflect, to weigh, to judge, and decide not just what's bigger, or higher, but also better, is a purely human activity. No computer can exercise it. But good books, which can only be created by people, can help us reach such perspectives.

-reinbach tfn.com

For reprint and licensing requests for this article, click here.
MORE FROM AMERICAN BANKER