Good engineers make terrible leaders

Yuval Noah Harari in his highly celebrated book Sapiens says that human values are shared myths and that there is no objective basis in humanism. I agree with him.

In several separate exchanges on Twitter, I’ve debated the claim that technology has caused progress in human society or that there’s never been a better time to live than now.

I personally agree with the evidence that’s presented in favor of progress: infant mortality rate has been reducing, education levels have gone up and poverty has been reduced world over. (I follow Human Progress handle on Twitter, believe in Effective Altruism and donate to GiveWell’s recommended charities).

But I also hold two other views:

• a) the claim that human society has made progress (because of tech or anything else) cannot be made objectively;
• b) it’s meaningless to say human society had made progress without stating areas that you’re considering and not considering when it comes to assessing such progress.

This perspective is not unique and has been discussed by social scientists and philosophers for a long time. However, the paper Dilemmas in General Theory of Planning (PDF) by Rittel and Webber takes this idea and expands it by distinguishing problems that involve humans and human society (which they term as ‘wicked problems’) and problems of sciences, mathematics, and engineering (which they term as ‘tame problems’). The main distinguishing feature between them is as follows:

• Tame problems are closed in formulation (e.g. building a bridge using known rules) while wicked problems are comprised of openly interacting systems (e.g. reducing poverty in a city)
• Tame problems have objectively right or wrong solutions (e.g. we can calculate if a proposed design for bridge will take the expected load) while wicked problems have subjectively good or bad solutions (e.g. if a solution proposes raising tax rate for universal basic income to reduce poverty, is that a good or a bad solution?)
• Tame problems can be stated independently of solution (design a bridge using X material that takes Y load) while for wicked problems, defining the problem is coincident with searching for solutions (to “solve” for poverty, you have to define poverty and state the metrics using which you’ll measure poverty. Once this description of poverty is clear, the solution is usually trivial: reduce one or more identified metrics)

Authors of the paper argue (and I agree) that it’s a big mistake to apply engineering or scientific method for solving societal problems (or problems in any other human system like a company) and believing that there’s a right or wrong solution to that problem (when in reality there are only opinionated good or bad solutions). For determining goodness or badness of a solution, the scientific method of putting our hypothesis and testing it against evidence fails because full consequences of a solution in an open interacting system is never captured in a limited time. All such solutions (say increasing tax rate to fund education) leave a trace in history and impact future in unmeasurable ways. This is because society is an open system where the solution to one formulation of a problem (poverty happens because of lack of education) can lead to more problems (more taxation for funding education, rising costs due to expensive labor, and so on). Thus anyone claiming an objective basis for a problem in society is taking a simplistic view. And that’s my issue with saying technology is causing progress.

It isn’t just theoretical that you cannot solve wicked problems in a scientific manner. Psychologists have found out that intelligence doesn’t correlate with perceived leadership abilities. (Perceived leadership abilities peaks at IQ of 120 and then starts going down with an increase in IQ. Surprised?) My hypothesis is that perceived leadership abilities decreases because there’s no right or wrong solutions to wicked problems, only good or bad solutions. And that’s where persuasion abilities of a leader come in. While a leader with an engineering mindset works hard at finding at a better solution, other leaders use their personality, power or charm to persuade people that their solution will be in everyone’s benefits, even if it actually isn’t. This is why despite his IQ, Donald Trump got elected as the president of US.

This insight will seem to be unpalatable to readers of a scientific bent, but it’s what the reality is. Internet is connecting more people in the world with each other, and that’s increasing the plurality of opinions and values of subgroups in the world. Some argue drugs should be legalized, some argue against it. Some refuse to believe Earth is round while others make fun of them. The issue isn’t why certain people believe what they believe, it is that do and we have to live with the fact that values and opinions of others are as valid to them as ours to us.

How to resolve this diversity in opinions and values to get measures and methods of pursuing society’s progress? Now that’s the wickedest problem of all.

Notes, highlights and observations from the paper

I recommend reading the original paper as it’s full of insights, but if you want you can read my notes. Hat tip to Alan Klement for sending me this paper.

Notes from General Planning Dilemma:

• Society is composed of variety of groups where one’s output becomes another’s inputs
• Increasingly, getting a consensus on what-ought-to-be is becoming difficult because different sub groups have difference of opinion
• Wherever there’s a system,
  • Defining desires outcome becomes difficult (what system ought to be)
  • Defining and locating problems becomes difficult (where you think a problem is may not really be a problem)
  • Knowing which actions to take from what-is to what-ought-to-be
• Wicked problems v/s tame problems
  • “The kinds of problems planners deal with – societal problems – are inherently different than problems that scientists and engineers deal with”
    • Problems in natural sciences / engineering are definable, separable and have solutions that are findable
    • Problems of society are ill-defined, and they rely on political judgment for resolutions (not “solution”. Social problems are never solved. Only re-solved again and again)
• 10 characteristics of wicked problems
  • There is no definitive formulation of wicked problems
    • The information needed to understand the problem depends on one’s idea of solving it. (Is this because societal problems are human preferences / moral framework? And since no two people agree on preferences / moral framework, there’s no objective understanding of the problem (only subjective interpretation) This is in contrast to the problem of designing a bridge, where Physics serves to be common objective ground
    • For example, what is the problem of poverty? (People usually feel free to define it like themselves)
    • The process of formulating a problem and conceiving a solution is identical, since every direction in which the problem is explored is also a direction where solution potentially lies.
  • Wicked problems do not have stopping rule / they’re never fully solved (that is why utopias are delusional dreams?)
    • Wicked problems cannot be stated in an exact manner because of the interacting open systems. Literally, an exact specification of a wicked problem would contain entire universe as an environment/ system. This is a solver can always do better if she puts more time and investment, and that is why a solver stops wrt to an external criteria: “I have run out of time” or “this is the best that can be done with these resources”, or “unemployment levels are below 1% and that’s good enough”
  • Solutions to wicked problems are not true-or-false, but good-or-bad
    • As there’s no independent, objective criteria for such problems, different people / expert will have different opinions on a “solution” to be good or bad, depending on their moral framework and personal values.
  • There is no immediate and no ultimate test of a solution of a wicked problem
    • Solutions to wicked problems are interventions to open interacting systems, and these interventions impact many lives / people in many different ways, that have further consequences, and so on. So there’s no way we’ll know full repercussions / consequences ahead of the time
  • You don’t get multiple chances to solve a wicked problem, every trial counts
    • For contained problems of science and mathematics, if we’re wrong, we can usually try again to solve the same problem. But for wicked problems, every attempt at a solution matters significantly as every solution has consequences that cannot be easily undone, and reversing such consequences created its own wicked problems
    • Example, the decision for a new highway has a long half-life (people have been displaced, money has been spent, political careers spent, cities connected, traffic reduced / increased).
    • When actions are effectively irreversible and whenever the half lives of the consequences are long, every trial matters.
  • Solutions to wicked problems cannot be comprehensively listed
    • In pursuit of wicked problems, a number of solutions come up; and a number of solutions do not come to mind. Then it’s a matter of judgement whether to expand that list or not, and of course which solution to pursue
  • Every wicked problem is unique
    • >Despite seemingly related characteristics of wicked problems (say poverty problem in Delhi and Mumbai), there could be always one or more significant difference in characterization of the problem that ends up having major consequences for any solution that’s proposed or accepted
      • E.g. in Delhi or Mumbai difference could be of rate of immigrants, municipality effectiveness (that’s entangled with local political situation), cultural norms (poverty may be accepted as normal or not normal in these cities)
    • Because all wicked problems are unique, the engineering mindset of “I recognize the problem before, here’s a solution” might do more harm than good
    • The art of dealing with wicked problems is not knowing too early which type of solution to apply
  • Every wicked problem is a symptom of another wicked problem
    • Wicked problems are related other wicked problems, at a higher level
      • Say the problem of crime. We can say it’s a symptom of moral decay, wealth inequality, which further is a problem of media, democracy and so on
      • There’s no “right” level on which a wicked problem should be solved, it’s a matter of judgement
      • People usually think the solution of wicked lies at one level below them
        • Teachers think students aren’t studying, parents think teachers aren’t teaching, administration thinks parents aren’t creating right conditions for studying, people think administration isn’t adopting latest scientific research
  • Worldview of the one who is analyzing the wicked problem is strongest determinator of its explanation, and hence resolution of the wicked problem
    • Since exact controlled experiments cannot be done for wicked problems, and since all wicked problems are unique (and rich in complexity as they’re embedded in open interacting systems), any argument for or against particular viewpoints can be argued and defended. This is unlike science where hypotheses can be definitely accepted or rejected.
    • In social problems, solutions are agreed on through discourse or power, and not through rightness or wrongness of solutions.
  • The one who’s responsible for “solving” a wicked problem must live with its consequences
    • Since, unlike scientific, mathematical or engineering problems, there’s no right or wrong solution, only good or bad, the solver is held responsible if the “solution” has bad consequences
• As technology makes more people connected to each other, the plurality of opinions and values in society will grow and there could not be an aggregate measure of societal progress (of a highly diversified society).
  • Solutions to problems of one group maybe problem generators of one group
  • The de facto approach so far has been individualism, but we all live in an interconnected interacting systems. If an individual’s actions driven by his values has an external impact (say pollution), someone else bears the cost. Some groups (say extremists) might have a value system that do not recognize individualism. So let alone deciding that individualism is a preferred mode of policy, getting everyone to agree on individualism is a wicked problem in itself.
  • Moreover, there’s no escape from the knowledge that even an expert solving a problem is promoting his/her preferred worldview.
  • The heart of the issue of wicked problems lies in subjectivity of morals and values.
  • There’s no theory that can tell the right measure of progress or societal welfare, only tell good or bad and the audience is free to agree and disagree.

What are YOUR metrics for measuring progress in human society?

Dear reader, if you have read so far, I have a question for you that’s related to this article.

Tweet your response to me as a reply to this thread and I’ll retweet the most intriguing responses. In the same thread, you can also check out and comment on what others proposed.

Join 150k+ followers
Follow @paraschopra