Re: rambling thought about complex systems

[Roberto Verzola <rverzola@phil.gn.apc.org>]

Can we survive a genetic version of Y2K?
by Roberto Verzola


I am not a molecular biologist, but an engineer who had worked in
hardware and software design. So when biologists claim that they are
doing engineering of the genetic variety, comparing genes to computer
programs, I am naturally curious. I'd like to share some insights
about genetic engineering, therefore, from an engineering point of
view.

Machines: testing still misses problems

We engineers often know inside-out the systems we designed from
scratch. But by the time our design involves thousands of components
(or millions - in the case of the most complex hardware or software
designs), the entire system has become a delicate piece of equipment.
If we change the specifications of even a small portion of the whole
system, that change must be thoroughly tested because it can lead to
undesirable side-effects, which may not show up immediately, but
perhaps only under certain rare conditions. Often, an error introduced
by a design change would lie undetected despite rigorous testing, only
to show up under field conditions.

I should repeat this important point: problems like this happen even
with systems which we designed ourselves from scratch and where,
because full specifications and documentation exist, we can
theoretically claim 100% knowledge, given enough effort and time.

Plants: more complex, less understood

A soybean, potato, corn, or rice plant appears to me to be a lot more
complex than the most complex systems ever designed by engineers like
us. Those who fancy themselves to be genetic engineers admit that they
do not quite understand many of the mechanisms that make these plants
work. They have neither the complete specifications nor the designs
nor the drawings. So, a lot remains a mystery to them. And, of course,
since they didn't design it themselves, they really do not know all
the various considerations that went into the design.

Approaching these plants as an engineer, I would say that I'm faced
with a nicely working system that is better left alone. It is true, I
can perhaps make a slight change somewhere, and see its immediate
effect - possibly useful - elsewhere. But I would have NO idea at all
how the change I introduced would affect the system as a whole. In a
system with a million or more parts interacting with one another, in
perfect working condition, I know from experience that the slightest
change can introduce a "bug" -- an unintended side-effect that may not
show up immediately but will spring a surprise one of these days, when
perhaps a rare set of conditions occurs.

More changes, greater risks

The change, however, slight, introduces an element of risk which
reduces the system's overall reliability and its
mean-time-between-failures (MTBF). The more changes you introduce,
particularly when you don't really understand how the whole system
works, the greater the risk you are running, and the shorter is your
MTBF.

Sloppy engineers try shortcuts, hacks and all kinds of tricks for the
sake of efficiency. Careful engineers will consider efficiency, but
they know that the more important measure of a system's fitness is its
reliability. We engineers have settled this a long time ago, and
whenever we forget this lesson, the subsequent failure of an
"efficient" design will again hammer the lesson home hard.

If we sacrifice reliability for efficiency, we can easily miss risk
factors that stare at us in the face, like the programmers who used
the last two instead of all four digits to represent the year. Because
of this simple change, they embedded a bug which will create, in the
year 2000, a spectacular problem and technology's greatest crisis --
the Y2K software bomb.

Modular is reliable

By the way, to design systems of high reliability, we engineers almost
always use the modular approach, ie, we design a complex system by
breaking it up into smaller, relatively autonomous subsystems (or
modules), which interact only through well-defined interfaces. Then we
create barriers between modules, to prevent unnecessary interactions
among components in different modules, and to ensure that the
interactions go through the modules' interfaces. This approach results
in very reliable engineered systems. The opposite approach of allowing
any component in one module to interact with any other component of
another, forming one large humongous system, has led to buggy,
unreliable and crash-prone systems.

Modularization actually provides a theoretical basis (from systems
theory, not economic theory) for arguments against economic
globalization and in favor of local-sufficiency.

Biology has its equivalent of the modular approach: living systems do
not exist as one humongous community, but in the form of separate
species, which "interact only through well-defined interfaces". If
species barriers are broken down, unrestricted DNA exchange can result
in a dramatic increase in the number of potential biochemical
interactions which is presumably undesirable. Indeed, from my vantage
point, the geometric increase in the number of unforeseen interactions
is bound to result in the engineering equivalent of a buggy,
problem-ridden and crash-prone system.

Naive genetic "engineers"

To an engineer like me, today's cocky genetic "engineers" are
incredibly -- and dangerously -- naive if they think that the changes
they are introducing into perfectly-working systems will not cause
some undesirable side-effects, which may someday show up as genetic
"bugs". How can they be so blind to risk factors that stare at them in
the face (like using virus and bacterial genes or antibiotic
resistance markers)? Ask any genetic "engineer" -- they also call
themselves molecular biologists -- and they will deny the problem.
They are no different from the programmers who denied the Y2K problem
and kept postponing the solution until it was too late.

I pray that the cockiness of molecular biologists will not lead to the
genetic equivalent of the Millennium Bomb.