Tuesday, April 24, 2012

Random Jottings 7

For several years now, I've published an annual personal magazine, Random Jottings. In various issues, I've told the story of my dealings with the Samaritans, published my father's World War II memories, and most recently collected the cognitive biases material that originally appeared in this blog.

The seventh issue of Random Jottings is now available. Articles include "Have Spacesuit, Will Travel," "Why Did the Samaritan Cross the Road?", "Wikipedia Mon Amour," and "A Modest Proposal." All of them originally appeared here, but they've all been substantially rewritten and most of them expanded.

Random Jottings 7 also includes a portfolio of artwork by my good friend and ten-time Hugo nominee Steve Stiles, who also did the cover and back cover.

Because I caught the amateur publishing bug from science fiction fandom, each issue of Random Jottings is published to coincide with Corflu, the annual convention of fanzine publishers. As I write this, I'm just back from Corflu Glitter in Las Vegas. Next year's Corflu will be in Portland, Oregon.

The electronic edition of Random Jottings, both current and back issues, appears courtesy of Bill Burns' website eFanzines.com. There's amazing stuff here, and it's worth a look around. Random Jottings itself can be found here. I have a limited supply of printed copies; contact me if you'd like one.

Please feel free to leave comments here or send me a letter of comment for publication. I hope you enjoy the issue.

Tuesday, April 17, 2012

In re Zimmerman

Stages of a Project

In project management, we frequently say that the stages of a project go like this:

1. Enthusiasm
2. Disillusionment
3. Despair
4. Panic
5. Search for the Guilty
6. Punishment of the Innocent
7. Praise for Non-Participants

The key problem comes in Step 5, “Search for the Guilty,” as opposed to the alternative, “Fix the Problem.” The overwhelming need to punish the offender outweighs the need to fix the problem.

In re Zimmerman

These days, I’ve mostly sworn off political topics. I hate that I’ve lost friends that way. But the shooting of Trayvon Martin by George Zimmerman also raises issues relevant to project management, cognitive bias, and fallacies of logic, all of which are legitimate subjects of this blog.

As is too often the case in any sensational media matter, the professional outrage machinery is in full swing. The most extreme and definite opinions on both sides receive the majority of coverage, even though they are a minority of the population, and in response, both defenders and accusers try to delegitimize the comments and opinions of those who disagree.

My goal in writing this is not to start an argument about the relative culpability of either Zimmerman or Martin — and in fact, I’d vastly prefer not to have one. There are many other places you can go to have that argument. Right now, let's look at the nature of those opinions.

In the case of Zimmerman — or, for that matter, any criminal suspect — there are four separate questions:

  1. Should Zimmerman be suspected?
  2. Should Zimmerman have been arrested?
  3. Should Zimmerman be put on trial?
  4. Should Zimmerman be convicted?
The Decision Hierarchy

Decision-making processes are normally hierarchical. There are earlier, tentative decisions we make before we reach our final conclusion. Should we, for example, do a particular project? The initial decision may be to do a feasibility study, or a pilot test. Only when those results are in will we make our final choice and commit resources to the solution.

The legal concept of level of proof is particularly meaningful. Depending on the stage of the process, different levels of proof are required to support a given decision.

Should Zimmerman be suspected? The legal standard for suspecting someone of a crime is a low bar: reasonable suspicion. This standard is based on "specific and articulable facts", "taken together with rational inferences from those facts," to distinguish it from an "unparticularized suspicion" or a hunch. In Zimmerman's case, there's no real doubt that Zimmerman shot Martin. Even Zimmerman admits as much.

In project management, a related question is whether the environment contains opportunities or problems worthy of addressing. Coming up with a target list doesn't mean you've made a final decision, so the burden of proof is low.

Should Zimmerman have been arrested? The standard for an arrest is probable cause. That standard is higher than "reasonable suspicion," but much lower than the standard required for a criminal conviction. The best-known definition of probable cause is "a reasonable belief that a person has committed a crime." Another common definition is "a reasonable amount of suspicion, supported by circumstances sufficiently strong to justify a prudent and cautious person's belief that certain facts are probably true." 

This question, not the question of whether Zimmerman is ultimately found guilty, is the core of the current controversy. Frankly, had Zimmerman been charged and tried at the time — even if he were subsequently acquitted — it’s hard to believe that this case would have ever achieved national prominence. It’s not exactly as if shootings are an uncommon occurrence in the United States.

"Probable cause" also has a role in project management. Feasibility studies aren't free, so you have to pick your shots. Establishing a proof level equivalent to "probable cause" helps you focus on the issues that matter most.

Should Zimmerman be put on trial? Prosecutors can bring charges based on no more than probable cause, but a grand jury, which reviews the case prior to trial, is supposed to make its judgment based on the preponderance of the evidence, that is, whether is charge is more likely to be true than not true. Prosecutors also have the benefit of a more comprehensive investigation than is normally performed at the time an arrest is made. The additional information may well alter an initial determination.

A full-fledged project is the equivalent of a trial, because you have to do all the work, and in the process, you may discover surprises and risks not part of the initial process. The initial project plan rests not on final determinations, but rather on the preponderance of the evidence available in the initiating and planning stages.

Should Zimmerman be convicted? Depending on the charge, two standards may apply: clear and convincing evidence, or proof beyond reasonable doubt. The first standard applies in some civil cases; the second in criminal cases. Zimmerman clearly falls into the second category. Our adversarial legal system is designed to put the strongest burden on the shoulders of those who would convict someone of a crime.

There is a final standard of proof, proof beyond a shadow of a doubt, but that is often an impossible burden to meet, so that standard doesn't apply in a criminal case. Indeed, given that there are only two people with full knowledge, and one of them is dead, the "shadow of doubt" may never be completely eliminated.

At the end of the project, the results must necessarily speak for themselves. Did you solve the issue that led to the project? Perhaps clear and convincing evidence is all you need, but sometimes a higher burden of proof rests on the project manager.

Justified and Unjustified Opinions

If there's one fundamental American right, it's the right to an opinion, but not all opinions are equal. To have a legitimate, justified opinion, you need to have some actual facts and apply an appropriate standard of proof. As long as you're clear about what stage of the decision is currently on the table, saying "yes" to one level doesn't necessarily equate to a final determination.

In an imperfect world, armed with imperfect knowledge, we cannot escape the reality that we must necessarily make some decisions and hold some opinions in advance of all the facts. And that raises the question about what makes an opinion justified and legitimate.

It's often argued that [insert side of the argument] has already convicted [Zimmerman] [Martin], rather than wait for the judicial machinery to work its course. And, of course, some extremists have already rushed to final judgment on the matter. But the vast majority of us have not.

That doesn't mean judgments, even in preliminary stages, are inappropriate. As noted, there's no reasonable doubt of the basic fact that Zimmerman shot Martin. It's hard for anyone to argue that the police should not have looked into the matter.

Can a "prudent and cautious person" reasonably conclude, at this stage of the investigation, that Zimmerman should be arrested? Clearly, the answer is yes. Even without the discredited claim of a racial slur, the 911 call, in which Zimmerman disregarded the recommendation of the dispatcher not to get out of his car, casts a "reasonable amount of suspicion" on Zimmerman's actions.

Does the preponderance of the evidence lean in favor of putting Zimmerman on trial? The Florida prosecutor assigned to the case has determined the answer to be yes, so it's not unreasonable or unjustified for an outsider to agree with it.

It's possible, and even legitimate, to have a definite opinion on the questions of arrest and trial. It's much less justified to have a definite and final opinion about Zimmerman's ultimate guilt. The standard of "reasonable doubt" has not yet been overcome, and will not be until all the evidence — including evidence presented by the defense — is out in the open.

The claim is often made that Trayvon Martin's supporters have already convicted Zimmerman, but that's an exaggeration. It's true that Martin's supporters have generally concluded that Zimmerman should be arrested and tried, but those opinions are valid at a much lower standard of proof. It's not "convicting" Zimmerman to argue that the evidence supports arrest and trial.

Bias and Judgment

When it comes to the final decision, a prudent person should be cautious. You not only need evidence, but the evidence itself often needs to be challenged and validated. But that doesn't mean you can't hold a preliminary opinion, as long as you're willing to modify it in the face of persuasive information to the contrary.

If you rate opinions about Zimmerman on a scale of 10 (certain he's guilty) to 0 (certain he's innocent), people with scores of 10 or 0 are clearly making decisions ahead of the facts. For such people, the outcome of a trial will mean nothing: if the verdict goes their way, they knew it all along, but if the verdict goes against their opinions, it will mean that the trial was rigged and thus invalid. People who are unwilling to revise their opinions in the face of new facts aren't reasonable. Fortunately, their number isn't large.

More common is people whose opinions are 8-2, strongly convinced of Zimmerman's guilt or innocence, but not so locked into their positions that persuasive contrary evidence is incapable of changing their minds. People in this category need to be extremely careful of the actions of cognitive bias in their information processing and decision process. Even if you are trying to be open-minded, once a mind's made up, inertia takes hold. Change is difficult.

People with opinions in the 4-6 range are in less danger from cognitive bias, because their opinions are inherently more tentative. In my case, I am not paying detailed attention to the story, because of the general unreliability of most of what's published at this point. That's not the same as saying I have no opinion, or that I don't lean toward one side, but I'm fully aware that the eventual factual record may not support my tentative ideas. Even so, cognitive bias can have its effects even on people of more moderate opinion, so it's important to stay on one's guard.

But that's opinions about Zimmerman's guilt. Opinions about arrest or trial fall into different standards of proof.

The Process of Proof

After watching innumerable crime shows, we all should have a pretty good idea how the police and trial process is supposed to work. When it goes according to plan, it does a reasonably good job of establishing a factual record in support of a decision. Of course, your mileage may vary.

Even if the story seems straightforward at first glance, investigators still go through the process of reconstructing the story, gathering physical evidence, and taking initial testimony. Raw evidence, of course, is of little use until it’s processed — the body examined, witnesses interviewed in detail, a timeline reconstructed, DNA tests performed, etc. In processing evidence, investigators create a story, a timeline of events and people that ideally reveals the truth of a situation.

Stories, of course, always begin as outlines, and as they take shape and form, you can fill in greater detail. Sometimes, stories surprise you, and you find yourself in an unexpected place. Minor characters (“persons of interest”) become suspects — people with motive, means, and opportunity. Some suspects are ruled out as the process moves forward; other suspects are eventually charged with a crime.

But all that assumes police and prosecutors are doing their jobs properly. To me, the most important question is not whether George Zimmerman committed second degree murder in the shooting of Trayvon Martin, but whether he should have been arrested in the first place.

What's the Real Problem?

Questions about the process will probably not be part of the trial, because it’s Zimmerman, not the police department, who is its subject. One hopes that the Justice Department investigation will address these matters. Again, I don’t claim to know the right answers. But as a management consultant, my job is to ask the right questions. Here are some things I'd like to know.

First, was the behavior of the Sanford Police Department appropriate? To measure "appropriate," consider the following: Did the department follow established protocols? Were those protocols adequate to the situation? Did any special circumstances make it more difficult to follow the protocols? What can we do about that in the future?

If protocols were not followed, why not? Are there management or organizational issues, problems with internal culture, changes in the environment, or other factors? And if so, what can we do to address those issues? Even if the more serious allegations of political interference by Zimmerman’s father (a retired Virginia magistrate) or charges of institutional racism turn out to be true, the reflexive “search for the guilty” is a much less effective response than fix the problem.

Wasting a Crisis

I have a stronger opinion about process in this case than I have about guilt. One of the facts of management consulting is that if you have a really bad outcome, you need to adjust your process as necessary to keep it from happening again.

The Sanford police were, no doubt, shocked at the public response and degree of national interest. Regardless of the assignment of fault or blame, they have to adapt to the new reality that their work will receive increasing scrutiny in the future. The 1982 Chicago Tylenol murders were clearly not the fault of the manufacturer, but the ubiquitous tamper-evident seals on all that we eat or drink date from that crime. And if they truly are at fault, then there’s all the more reason to work harder and better in the future.

Too much emphasis on determining guilt can, unfortunately, detract from the more important matter of change. In the cognitive biases series, I covered my version of the “Semmelweis Effect," the reality that peoples’ views harden when you accuse them of terrible crimes. Moral indignation can backfire, and that does no one any good.

Because it’s so easy to identify the most inflammatory and outrageous pieces on either side, it’s equally easy to miss the large amount of reasonable, proportionate commentary—also on both sides. A crisis, as has been noted, is a terrible thing to waste. That doesn’t mean we want a crisis or welcome it when it comes, but if we waste the crisis, we’re often doomed to experience it again. That’s the worst possible outcome.

It’s clear that something went terribly wrong in the Trayvon Martin case. Less clear is what went wrong, and the purpose of trial and investigation is to establish the narrative in an authoritative manner. Whether the trial and investigation accomplish that goal remains to be seen.

But it’s truly the most important part of the matter. Because once we answer that question, we can move forward to the two questions that matter in the long run:

Why did it happen?

And how can we keep it from happening again?

Tuesday, April 10, 2012

Why Projects Fail

The following piece is from my book with Ted Leemann, Creative Project Management, published in 2010.
 * * *

If project management is so good, why do 70% of all projects, including those led by experienced and capable project managers, fail to get done within the Triple Constraints of time, cost, and performance — or in layman’s language, on time, on budget, and to spec?

Here are a few instructive examples.

  • In 2006, a $400 million purchasing system for Ford Motor Company was simply abandoned.
  • Software errors in a UK Inland Revenue system resulted in a $3.45 billion tax-credit overpayment.
  • The infamous automated baggage system at Denver International Airport burned through $250 million before being abandoned as unworkable.
  • The Department of Defense’s $6 billion Kinetic Energy Interceptor program was terminated in 2009 after it was determined that it would not achieve its goals.

That’s not all. Let’s look at some numbers on project performance.

The Standish Group has tracked project performance since 1994. Every two years, it issues the CHAOS Report. In the 2009 CHAOS Report, they reported these abysmal numbers:

  • 32% of projects were delivered on time, on budget, and with the required features and functions.
  • 44% were finished either late, overbudget, or only partially completed
  • 24% failed altogether, and were cancelled or abandoned.

There’s good news and bad news here. The good news is that in 1994, when the Standish Group began tracking, only 16% of projects succeeded in meeting the trifecta of the Triple Constraints (on time, on budget, to spec). On the other hand, the 2009 report shows that there’s been a downtick in success (34% to 32%) and a significant uptick in failure (from a low of 15% to today’s 24%).

For challenged projects, those that succeed in some elements and fail in others, the good news is that average budget overrun has dropped from 180% to only 43%. On the less positive side, time overruns have gone up 30%, and the percentage of features that made it into the final product has dropped from 67% to 52%.

During this time, nearly 260,000 project managers earned the prestigious Project Management Professional (PMP®) designation from the Project Management Institute (PMI). But the track record of improved project performance is lackluster at best.

What’s going on?

A significant amount of study and reporting has attempted to discover the reasons for these failures.

  • The 1998 Bull Survey, conducted by the French computer company of the same name, identified the major causes of IT project failure as a breakdown of communications, a lack of planning, and poor quality control. 
  • KPMG Canada, in 1997, identified the core issues as poor planning, weak business case, and a lack of top management involvement and support. 
  • The Standish 1995’s Chaos Report named incomplete requirements and lack of user involvement. 
  • The OASIG Study, also published in 1995, cited lack of attention to the human and organizational aspects, poor project management, and poor articulation of user requirements.

But poor planning, weak business case, and inattention to human and organizational aspects aren’t causes, but rather symptoms of a much large systemic shortcoming.

Treating the symptoms isn’t the same as treating the underlying medical conditions. We know some of the root causes. People with poor interpersonal or team leadership skills and stakeholder conflict create friction in the project environment, and friction increases inefficiency and waste. The mass and complexity of the organization increases its moment of inertia, and getting anything to move takes enormous effort. People come and go, missions mutate, information goes missing, and that increases entropy, the tendency to move from order toward chaos.

Things fall apart.

It’s been said that there are only two reasons for project failure:

  1. Things that happened that nobody thought of and that they accordingly didn’t prepare for.
  2. Things that happened that everybody knew was going to happen, but nobody did anything about.

How often have you experienced project problems because something came along halfway through your project and they had to pull two people off the team ? How about someone wanting a major change in one or more of the triple constraints when the project is  three-quarters completed? Somewhere in your project environment, there’s probably some recurrent problem that happens every single time. Things take longer than you expected. Not everybody is really on board. There’s always a layer of technical complexity no one expected. Important people don’t really know what they want, or expect you to figure it out magically.

But do you account for these situations in your project planning? For a few outstanding project managers, the answer is at least a partial “yes.” For most of us, the answer rests somewhere between seldom and never.

Maybe it’s something you can’t afford to recognize officially because the organization’s in denial. But maybe it’s something in your project management blind spot.

If you take the list of reasons from the studies above, you can boil them down into the following four (seven, if you count clauses) often unasked and unanswered questions:

  • Why are we doing this? (Business case)
  • Who has an interest in what we’re doing, and what do they each want and need? (Human and organizational aspects)
  • What do we have to do, and how are we going to do it? (Project management, including planning and quality control)
  • Who needs to be involved, and in what way? (Top management and user involvement and support)

The official standards of professional project management are designed to make sure these issues get appropriate consideration. But let’s face it — these are pretty obvious. It shouldn’t take a PMP to grasp these concepts.

And all too often, having a PMP doesn’t mean someone does.

Tuesday, April 3, 2012

Some Notes About Decision Making

Some of this material appears in my chapter in Applied Space Systems Engineering: Space Technology Series, which I wrote a few years ago. While originally aimed at making mission-critical decision in a complex engineering environment, some of the ideas are more generally applicable.

“A decision,” wrote author Fletcher Knebel, “is what a man makes when he can’t find anybody to serve on a committee.”

Committees and teamwork are often an essential part of decision-making, but even in that framework, each of us must sooner or later take our stand, knowing full well the range of potential consequences. In an organization, a decision-making process must often be open and auditable. We must know not only the decision we make, but also the process that led us to that decision.

Decisions often require tradeoffs. A perfect solution may not exist. Each potential choice may have a downside, or may be fraught with risk. In some ways, making the least bad choice out of a set of poor alternatives takes greater skill and courage than making a conventionally “good” decision. Napoleon Bonaparte observed, “Nothing is more difficult, and therefore more precious, than being able to decide.”

The outcome of a decision, whether positive or negative, is not in itself proof of the decision’s quality, especially where probability is concerned. The odds may be dramatically in favor of a positive outcome, yet the dice may come up boxcars. Equally, if someone makes a stupid decision but gets lucky, the decision is no less stupid in spite of a good outcome. A good decision process improves our odds and results in the desired outcome the majority of the time.

Decision-making is tightly woven into the risk management process, but one does not follow automatically from another. An engineering evaluation might tell us that there is a 42% probability of an event happening, and that the consequence involves the loss of $20 million and three lives. What the evaluation does not tell us is whether the risk is worth running. Values—organizational values, mission-related values, ethical values—address the consideration of worth.

There are two types of complexity in decision-making. The first, and most obvious, is the technical complexity of the issue and the tradeoffs that may be required. The second, though not always openly addressed, is the organizational complexity: the number of people involved, the number of departments or workgroups that must be consulted, the existing relationships that shape communication among people and groups, the organizational culture, and the pressure of the political process.

Decisions also vary in their importance. Importance can be measured in terms of the consequences of the decision and the constraints imposed on the decision process.

There are two slogans about decision-making: “Don’t just stand there, do something!” and its reverse, “Don’t just do something, stand there!” Both can be worthwhile pieces of advice; the trick is deciding which philosophy applies to the decision at hand. The first question is whether an actual problem exists, and, if so, the nature of that problem. The second is whether an action should be taken, and if so, the nature of that action.

At different times in the decision-making process, consider the opportunities as well as the negative consequences that can result both from the decision to act and from the decision to wait. If the consequences of a missed opportunity are greater, then the appropriate bias is in the direction of action. If an inappropriate decision could cause greater harm, the bias should fall in the direction of delay: gather more information and reassess.

Threats and opportunities both require proactive management, but opportunities even more so. Good luck and bad luck operate differently. If a person, say, loses $100, it’s gone, and all the consequences of that loss flow automatically. If, on the other hand, there’s a $100 bill somewhere in the area, it’s possible to miss it, there is no requirement to pick it up, and no obligation to spend it wisely. Exploiting opportunity requires observation, initiative, and wisdom.

A good process does not necessarily result in a good consequence. Predicting the future is never an exact science, especially when probabilities are involved. Evaluate the decision process separately from the decision outcome. Hindsight is a useful tool, though teams must remember that what seems so clear in light of actual events looked different to the decision-makers.