Make complexity work for you
- By John K. Waters
[PROGRAMMERS REPORT, OCTOBER 22, 2002] -- Medical scientists are using complexity science to understand the patterns
that underlie health and disease. Sociologists are using it to figure out human
society. Business analysts are using it to gain insights into market structures
and to develop competitive strategies. And some software engineers are beginning
to use it to quantify the differences between good and bad software.
''The missing piece of the software engineering puzzle,'' Alex Iskold told
Programmers Report recently, ''is understanding complexity. That is also what's
missing from the existing tools and methodologies.''
Iskold is CEO and co-founder of Information Laboratory, a New York-based
analytical software solutions provider. During his career, Iskold has designed a
number of large-scale software systems, and he has taught software engineering
at New York University. He and co-founder/CTO Dan Kogan started Information
Laboratory in 2000 to apply the new ideas of complexity science to many
different areas, in particular, to software engineering.
''When we started the company, we had this clear vision that complexity
science had a lot to say about how to do software development,'' said Iskold,
''and in particular about large-scale software development.''
Complexity science is described as a multi-disciplinary approach to the study
of complex systems. It evolved out of General Systems theory and quantum physics
in the mid-20th century. Basically, it is the study of systems composed of many
and varied parts that interact in complex and non-linear ways. Complexity
science recognizes that such systems can be understood only by looking at the
interactions among their parts and the consequences of these interactions.
Iskold explains the concept this way: ''Complex systems are made out of
smaller systems, and small systems interact and form a network; the complex
system is essentially the emergent property of the interacting subparts.
''The basic insight [of complexity science] for us,'' Iskold said, ''is that,
because any kind of large-scale software system is a model of something very
real -- a trading system, a car, a supply chain -- it is itself a complex system
and must obey the laws of complexity.
''And then we asked the question, 'Can we, by looking at the structure of this
complex system, actually distinguish good software from bad software?'
''And, sure enough,'' he asserted, ''we found that you can.''
All software systems have very definite ''structural signatures,'' said Iskold.
Good systems are highly modular, with component dependencies kept to a minimum;
bad systems have components with excessive responsibilities and involve
large-scale cyclical dependencies.
''Once we looked into this closely,'' Iskold said, ''we began to identify simple
structural patterns that occur in bad software. For example, when people write
large-scale systems, a lot of companies suffer from the problem that code is
intermingled. Intermingled code means that every component has many
dependencies, most of which are unnecessary -- extraneous dependencies,
components that are hooked into tangled webs. This is structurally very bad,
because when you make a change to any component it starts traveling and spills
all over the system. And that's when people say, 'I fixed this and something
Looking at large-scale software systems within the context of complexity
science has led Information Laboratory to invent a new discipline, which the
company calls Structural Analysis of Large Scale Systems. ''We are basically
claiming that it's not a matter of your technology,'' Iskold said. ''It doesn't
matter if it's .NET or J2EE. It boils down to dependency management and the
complexity of the system.''
Along with a new way of thinking about software, Information Laboratory's new
discipline has produced a tool for analyzing and visualizing the structure and
dynamics of large-scale systems. First introduced in September of last year,
Small Worlds is one of the first tools to take on the problem of structural
dependencies and system complexity.
''What we've done [with Small Worlds] is to create a framework for taking an
arbitrary software system and transforming it into a gigantic dependency web,
parsing it and then modeling it,'' explained Iskold. ''We also created several
structural metrics that help us to very quickly assess the overall health of the
Small Worlds provides users with a variety of visual models and statistics,
including views for managing component and package dependencies, tracking the
effects of change, system coherency and summary reports. The views help users to
identify problem areas in the code that may need system-level refactoring.
The current 2.0 version of Small Worlds allows for analysis of C++ and
Java-based systems, Unix shell scripts and a Forte plug-in module. Future
releases are planned for database analysis and for systems written in C, C#,
Visual Basic, COBOL and J2EE. The 3.0 version, which has no projected release
date, will be a ''leap,'' Iskold said, providing overall views of the entire
For more information about complexity science, Iskold
recommends a book he says changed his life: Complexity: the Emerging Science
At the Edge of Order and Chaos
, by M. Mitchell Waldrop (Touchstone Books, 1992).
To find out more about the Waldrop book at Amazon.com, go to http://www.amazon.com/exec/obidos/tg/detail/-/0671872346/qid=1034902168/sr=1-1/ref=sr_1_1/103-6737774-1962217?v=glance
To find out more about Small Worlds software, go to http://www.thesmallworlds.com/
For other Programmer Report articles, please go to http://www.adtmag.com/article.asp?id=6265
John K. Waters is a freelance writer based in Silicon Valley. He can be reached