Upfront versus Iterative/ Evolutionary design

This is one of the issues I grapple with every day as a software architect. Upfront design offers the comfort of being able to partition systems into sub-systems with some sense of control on the end design. However, complex solutions that we try to build need iterative design.

It makes far more business sense to develop these solutions within an agile framework where the design and architecture themselves evolve and the code is refactored constantly. This also provides external stakeholders visibility into what is being developed and provides them a greater sense of control.

I feel that iterative development, also provides the software architects and engineers validation for the design choices made. This is important to ensure that you do not end up looking like the self-absorbed architect who was afraid to ask for directions.

Another important aspect is that as new information becomes obvious to you and the end client, different stakeholders start zooming in on aspects of the system they had not thought before. As requirements start getting re prioritized, architectural requirements also evolve.

It should be clear by now, which side of the debate I increasingly see myself on. However, I was looking for ways that would provide me the comfort of upfront design while allowing a solution to develop in an incremental fashion. An approach to analyze the sub-systems in the design while not getting locked in to sub-system choices made initially. In short, I was looking for ways to reconcile Upfront design with Incremental design.

This lead me to a rather interesting article by Steve Jurvetson in the July 2006 issue of MIT technology review. Titled Technology Design or Evolution. To quote Steve...

"Designed systems offer predictability, efficiency, and control. Their subsystems are easily understood, which allows their reuse in different contexts. But designed systems also tend to break easily, and they have conquered only simple problems so far. ...

By contrast, evolved systems demonstrate that simple, iterative algorithms, distributed over time and space, can accumulate design and create complexity that is robust, resilient, and well adapted to its environment. In fact, biological evolution provides the only "existence proof" that an algorithm can produce complexity transcending that of its antecedents. Biological evolution is so inspiring that engineers have mimicked its operations in areas such as genetic programming, artificial life, and the iterative training of neural networks.

But evolved systems have their disadvantages. For one, they suffer from "subsystem inscrutability." That is, when we direct the evolution of a system, we may know how the evolutionary process works, but we will not necessarily understand how the resulting system works internally."

Steve goes on to quote Wolfram to imply that it is not easy to discover rules for evolution without going through all the steps of evolution. This is interesting. It means that there are no easy solutions.

One idea that I am toying with, is to focus constantly on maintainability and extensibility of the solution each iteration. The goal, to ensure that resulting designs can be actually partitioned into sub-systems rather than ending up as spaghetti code that cannot be drawn on the white board.

Open source projects: The right client

At the inception stage of open-source projects, it is important to identify the right client. However, the traditional view that open-source projects automatically benefit from unfiltered requirements and guidance is probably not true. There is a lot of benefit that traditional projects gain from working with people in a vertical structure.

Traditional commercial projects are conceived and implemented by the "caretakers" in any organizational setup. Organizational caretakers are people who represent the organization at different levels. They solve a vital problem of representing a large group of people when they organize to deliver a good or service.

Human society organizes itself at different levels. Earlier societies were defined by traveling distances. Communities and villages were defined by walking distances, cities or towns by driving or riding distances. Regions were defined by primary markets and the areas that served them. This could involve distances that could be covered when transporting perishable goods on a routine basis.

Later on, as the early empires were formed, administrative regions were defined. These were defined by areas where tax could be uniformly collected as well as welfare tasks such as education, health, law and order could be provided so that people did not rebel.

At the outset of nations in 18th and 19th century during the industrial revolution, the distance that could be covered by railroad determined the areas that could be joined to create the first nations. These new entities started sharing defence, budgets, currencies, foreign policy and international trade policies.

Why this does not work for open-source projects

Open source projects represent a larger playing field than most nations or empires can hope to achieve. However, open source projects work at the level of the individual end user. This means that requirements do not come from organizational representatives but individuals. This provides a new challenge. Key individuals have greater insight into an organization due to their functional roles and large amount of time these individuals spend in analysing the organizational needs from their functional perspectives.

There needs to be a way to bring these perspectives into open-source projects. Functional requirements from the people who understand them best in addition to a shared peer review is probably the best way to develop open-source code.

Knowledge is power or is it?

It is my premise that knowledge is power that destroys those who wield it against common good of people.

The samurai in real history, the jedi in fiction, were destroyed because they attempted to control means and production of knowledge. Hindu mythology talks about the great brahmin king - Ravana of Lanka who was destroyed primarily because everyone hated the knowledge and control he exercised over others. Drawing parallels to present knowledge based economy is something I wanted to understand.
According to the laws of commerce, when hard products are created - value is actually created in transaction at the point of consumption.

With this premise, knowledge would also only create value at the point of consumption.

So, how it all relates to software architecture? As architects, we have to ensure that systems we build are not exclusionary and are available to everyone. This will allow value to be created at multiple points of consumption, thus increasing the overall value of a system.

What allows a system to be non-exclusionary? Allow it to be accessible, and available on memory constrained and a vast variety of hardwares. However, each segment of users should be studied in isolation of each other. That is the only way a complete offering can be prepared for the end-user

The "Science" of modern management

For a very long time, I have been debating the role of managers in any organization that has a lot of people that deal with tacit knowledge such as software, medicine, architecture, etc. My own thoughts with respect to my employer have been ranging between extremes. On one hand, I prefer a slightly open environment where I can learn and grow. On the other hand, I would have preferred that there was some structure to day to day workplace battles. If on a day when I am dragged into different roles- the kinds I think would be handled by at-least 4 different persons in a more structured organization, I feel that I am subsidizing the employer's business model or lack thereof.

As I was looking for answers, I came across a book titled False Prophets: The Gurus Who Created Modern Management And Why Their Ideas Are Bad For Business Today by James Hoopes. The editorial review claimed that most modern day management gurus are simply misleading people with management fads. The review claimed that the book exposes the deeper conflict between the authority of managers and the freedom granted by Jeffersonian ideals of a free-man, a concept on which most modern democracies are based.

Intrigued, I finally picked up the book and read it cover to cover. Overall the book offered many interesting facts and insights on the people whose theories most management students study as a part of the curriculum. Based on the reviews, I expected a lengthy discourse on why management conflicts with a persons' freedom as an individual.

What I found was a lot of historical and well-researched facts about the gurus themselves. Another bonus was a sort of comparative discourse of different management theories. Most importantly, it traces the individuals along their own professional and personal paths, as they came up with the theories, something I always am curious about.

I would recommend this book because, I feel a lot of management theories have to be studied in the context they were proposed, which may or may not be valid today.

While many of the critiques I felt were justified, I felt the author over-demonized FW Taylor and got off easy on Peter Drucker. The author built the most convincing case against Edwin Mayo, who conducted the famous Hawthorne experiments. To management students, the Hawthorne experiments are presented as conclusive evidence for morale based management, just like Michelson-Morley experiments are to students of physical sciences as evidence of the absence of ether. Mr. Hoopes evidence proves that the experiments were a farce and the results were mainly fudged.

As for Frederick Taylor, father of modern management, I thought was over-demonized. That his motives were not altruistic is easy to understand, however, by raising production, he was able to offer workers a more decent standard of living.

Some names that the author presented in a good light were Peter Drucker, Deming and Mary Parker Follett. I was familiar with both Drucker and Deming (thanks to management courses and software quality workshops, respectively), the name that surprised me was Ms. Follett.

I must say, I was intrigued by a woman who articulated so many things that I have been trying to resolve in my own head for quite some time. Not only did she answer so many questions almost 90 years before they were bothering me, she took the discussions to a new level and provided answers that are as elegant as E=mc^2.

As a follow up, I read her 1918 book - The New State about which I will write in a later post.

Preventing climate change - software and other strategies

Governments have now accepted that the earth's climate is changing and that we need to do something about it. Nicholas Stern, a british diplomat submitted a report (The Economics of Climate Change: The Stern Review) on the economic and financial impact of global warming (or more appropriately climate change) to the British chancellor and prime minister.

The results of the report caused quite a stir when they reported facts that were already known to the scientific research community - that the earth will heat up 1-2% or even 5-6% in the coming years.

Mr. Stern has given tight deadlines to governments around the world to divert at least 1% of global GDP towards mitigation measures and related costs.

Need for new institutions

An institution is something to which people belong, however something that endures across generations. Common institutions known to us include family, school, religious community, profession al institutions among numerous others.

To address the impact and take both preventive and remedial measures we need new institutions. For instance, these institutions may look at carbon flows in an area and determine if carbon levies need to be applied.

To explain this scenario a bit more, lets say a producer in China (a new plastic toy factory), is able to sell its toys to kids in California. The total carbon costs of that toy would be a sum of the carbon emissions towards production in China, emissions in extraction and transportation of raw materials to the factory and costs in transportation of toy to California. Such a scheme not only puts the cost on the factory in China but serves as a demand control mechanism by taxing the consumer.

Will such a scenario happen? For sure, countries that have export driven economies will feel the impact, and will possibly use WTO to rally their cause. But eventually, once Chinese have mastered emission technologies, they will support these scenarios. Another corollary is the need for improvement in global logistics technology that is available to all economies.

Other impacts

We may see changes in siting of carbon based energy production facilities to places like Africa or even Sahara. On software side, new software forms will be needed. We will see a proliferation of environmental monitoring and logistics solutions. Carbon emission models may become open-source libraries that can be easily downloaded. For software architecture, we may need new institutions like LEEDS is for building architecture in choosing technologies (materials) and software processes (automated build and testing scripts) that reduce the energy cost of software solutions. Software deployed on virtualized and shared environments may be promoted to reduce energy footprints.

Open source vs. commercial software: The case for freedom

I have been thinking about the whole open source versus commercial software and realized that at their heart both offer the same thing : freedom!

The whole argument actually goes back to economics or rather formulation of economies: the concept of "gift economy" versus "money economy". Money economy promises freedom from patronage and privilege - "Any body can buy it as long as they have the money". On the other hand, Gift economy promises freedom to those, who for some reason cannot or do not wish to participate in the money economy.

In practice, a healthy mix is needed. It is no different from public and private mix required in business or providing mobility in a city. Consider a city where there is no public transportation, only form of mobility offered is through personal vehicles. You can already start guessing the mix of people who will live in that city. A typical modern suburbia.

On the other hand, a city where public transportation is the only means of mobility, chances are people will get stranded when the system breaks down, such as when operators or employees go on strikes. While the comparison is not entirely valid, many open-source projects are like "Soup nazi" kitchens, where your influence rests on your personal relationship with the committer. However, for a lot of customers with less than adequate money, it offers the ability to deploy technology within their budgets.

As for me, I would like to have the ability to switch a commercial stack with an open-source one, and vice-versa, based on the needs of the client. That is a luxury, however, that I can only dream about....

Responsive buildings and software context

Building architects are now thinking of making responsive buidings, that change shape based on the context. Software architects have battled such designs since the advent of authorization policies, where the software looks, feels and responds differently in different contexts.

Finally the students of architecture are teaching their building masters a trick or two.

Runtime software structures

Software architects need to start worrying about runtime software structures on a larger scale than they do currently.

The software landscape is changing from simple client-server models, to a mesh of connected systems, linked by real time calls across the network. As the world increasingly becomes a network of live systems feeding data to each other in real time, the software architects' emphasis will need to change from static structures to runtime structures. This is because devices and computational structures will be needed to constantly monitor several raw streams, for data that needs to be aggregated, compiled and analyzed.

Traditionally software architects focus on static structures while conceiving solutions. Classes, frameworks, libraries, etc, are all static structures that focus on how the written code is structured or layered. Runtime structures, in contrast, are dynamic and may change based on certain system events.

User community world over will start to realize, how their everyday choices impact real time supply chains. This may lead to a change in people's interaction behavior as well. As systems are getting linked, more and more users are interested in getting a dashboard view of the system.

Interaction design then changes from active (initiated by the user) to reactive (response to an event). In his book In the Bubble, John Thackara refers to this as "In the flow" and asserts that a process control paradigm will increasingly become more important.

This also involves a change in user mental models. An important aspect being talked increasingly in solution design is the concept of personal dashboards that would provide people with information they should be looking at or acting on.

All this is great. However, in my humble opinion, the system should force the users at some point to review past data and performance and make some critical decisions. This could be in form of change in policies, or change in the process itself.

An important question to ask is what will be this sub-system and how will vast amounts of information be analyzed and sorted. Lastly, but not less important,is the fact that if everything is reactive, then how do we engage the human operators to make decisions and develop foresight. In the environment where every action is a reaction to an event, what kind of institutions will this create and what will be the impact on social structures.

Design options for reducing environmental costs of mobility

In my previous blog, I mentioned what I thought were key parameters in determining reducing overall mobility related costs in North America and rest of the world.

At a policy level, there is already a change in investment patterns made by departments of transportation. Departments of Transportation are responsible for investments in state highway systems. There is a growing realization that highway investments have to change from the single dimension of preservation of asset value to multiple dimensions of providing mobility, ensuring safety and reducing environmental costs.

However, that is no longer enough. In urban America, a very large number of people have commute distances that can be best described as intra-regional and not intra-urban. The same is true for container traffic. The option is then, to provide regional multi-modal systems. To be successful, regional multi-modal systems require integration of transportation systems at both ends of the system. Fo most commuters, it means a viable road based personal and public transportation systems for their sub-urban environment.

A similar strategy is required for goods transport. For intra-regional and inter-region transportation of goods, rail systems could become the system of choice rather than the huge fleet of road and air vehicles maintained by logistics organizations. It could also mean developing a market for smaller and more nimble logistics companies that are more regional in nature. These could offer collection, sortation and delivery systems at either end of the system.

Software architecture

How does it all tie back to software architecture? The question obviously is to design a software architecture to meet this requirement. The following stakeholders needs need to be met to meet the above goals.

1. System owners and operators - These are typically the owners of the system such as state departments of transportation.
2. Policy makers - These are national transportation bodies.
3. Local system owners and operators - These are urban bodies responsible for planning and running the system.
4. Individual system users - These are individuals who make travel decisions for planning commutes and making them.
5. Commercial system users - These are organizations who use the system for commercial reasons such as freight and logistics companies.

In the Bubble

Just finished reading "In the Bubble: Designing in a Complex World" by John Thackara. The author's premise is that the design profession has to change to respond to the impending crisis in the future. Mr. Thackara proposes that design profession has to change to offer a more sustainable future. A future which is "people-centred" rather than "gadget-centred".

While I found myself agreeing to most diagnosis and detailed analysis, it is the over-simplification and sometimes complete absence of a solution to the problem that I found nagging. Another question I have is, who exactly is Mr. Thackara's audience. Most designers (in their own realms) have a very good grasp of the issues that the author raises. Agreed, there is a lot of interesting trivia, but trivia is exactly that, trivial.

Taking an example. In chapter 3, the author discusses how the mobility question needs to be re-defined for a more sustainable solution. Most planners and urban designers very well know the difference between sprawl and a high density development. The forces that cause sprawl are not in the designers' domain but in the domain of politicians and interest groups. As long as the automobile is a subsidised means of transport, and gasoline is cheap (like in North America), people will prefer to own their own automobiles. It is this low density development suited to the automobile, with cul-de-sacs and private 3 car garages that make urban transit services such as buses unviable.

In developing countries where vehicle ownership is low, and density is higher , already public transport operates at near capacity. So what solutions remain to various designers while considering options for design. The following comes to mind.

1. Change pattern of investment in state highway systems
2. Make public transport services more demand responsive and attractive
3. Create services that allow automobile owners to integrate with public transportation systems.

We will take a look at each of these in the upcoming blogs.

Identity - of the architect and the project

As any architect can certify - it is far easier to approach the design of a building whose identity can be understood as an abstraction or even a caricature. Ask any kid in the developing world to draw a cowboy. You will get a drawing of a Marlboro man or a Clint Eastwood clone, straddling down the cattle ranch. Ask the same question to a 12 year old growing up in Alberta countryside and it is a not so easy question. The question of identity comes up - does he ride a horse, or a 4X4. Does he then need to wear spurs on his shoes. Does he carry a guitar or an iPod?

It is my experience that as you start approaching a new project - whether a building or a software project, it is these questions of identity that first need to be answered as an architect. The questions being -

1. Who is the client ?
2. Why do they need the project?
3. What do they aspire that the project be?
4. Is this aspiration sustainable?
5. Do the project goals align with my own morals and belief systems?
6. Do my own belief systems require re-interpretation?
7. Am I the qualified to do this project for this client? Will I hold true to the client's objectives?
8. Has the client chosen me as the architect for some reason? What are these reasons?

Initially, I struggled with these questions myself, even to the point that should I be asking them at all, or should I move on and the answers will come to me? I found that the unanswered questions kept coming back. So it is in everyone's best interest that the architect answers these questions affront.

Privacy Policy for syntharch.blogspot.com

Privacy Policy for syntharch.blogspot.com 

If you require any more information or have any questions about our privacy policy, please feel free to contact us by email at arthgallo.wachs@gmail.com. 

At syntharch.blogspot.com, the privacy of our visitors is of extreme importance to us. This privacy policy document outlines the types of personal information is received and collected by syntharch.blogspot.com and how it is used. 

Log Files
Like many other Web sites, syntharch.blogspot.com makes use of log files. The information inside the log files includes internet protocol ( IP ) addresses, type of browser, Internet Service Provider ( ISP ), date/time stamp, referring/exit pages, and number of clicks to analyze trends, administer the site, track user’s movement around the site, and gather demographic information. IP addresses, and other such information are not linked to any information that is personally identifiable. 

Cookies and Web Beacons 
syntharch.blogspot.com does use cookies to store information about visitors preferences, record user-specific information on which pages the user access or visit, customize Web page content based on visitors browser type or other information that the visitor sends via their browser. 

DoubleClick DART Cookie 
.:: Google, as a third party vendor, uses cookies to serve ads on syntharch.blogspot.com.
.:: Google's use of the DART cookie enables it to serve ads to users based on their visit to syntharch.blogspot.com and other sites on the Internet. 
.:: Users may opt out of the use of the DART cookie by visiting the Google ad and content network privacy policy at the following URL - http://www.google.com/privacy_ads.html 

Some of our advertising partners may use cookies and web beacons on our site. Our advertising partners include ....
Google Adsense


These third-party ad servers or ad networks use technology to the advertisements and links that appear on syntharch.blogspot.com send directly to your browsers. They automatically receive your IP address when this occurs. Other technologies ( such as cookies, JavaScript, or Web Beacons ) may also be used by the third-party ad networks to measure the effectiveness of their advertisements and / or to personalize the advertising content that you see. 

syntharch.blogspot.com has no access to or control over these cookies that are used by third-party advertisers. 

You should consult the respective privacy policies of these third-party ad servers for more detailed information on their practices as well as for instructions about how to opt-out of certain practices. syntharch.blogspot.com's privacy policy does not apply to, and we cannot control the activities of, such other advertisers or web sites. 

If you wish to disable cookies, you may do so through your individual browser options. More detailed information about cookie management with specific web browsers can be found at the browsers' respective websites.