Can optimism ever be bad?

"Self-Portrait. Between the Clock and the Bed" by Edvard Munch. Needless to say, the symbolism here is quite pessimistic, given his apparent age and positioning.

Our psychology

Taking a first step into management from an individual contributor background can be tough. Not only is the role a step up in terms of accountability and responsibility, it also widens the interfaces in which one operates: increased debate and discussion, more exposure to opposing views both inside and outside of the team, and deeper emotional connections, tensions and breakthroughs.

An engineering manager can find themselves as a conduit and chair for all sorts of situations: debates about scope and priority with Product, dealing with clashes of ideals about which framework should be used for a new application, through to fallout in the team after missing a deadline, ongoing performance issues, and believe me, I could go on.

In addition to continuing to utilize your technical strength, you also need to also develop a strong mindset that can identify and work with the psychology of those on your team. Having worked with a great number of people over the years, I have seen that conflict can manifest in so many ways beyond the prescribed roles that people work in.

Yes, of course, there is a natural tension between Commercial, Product, and Engineering. But even those with the same skillset can find themselves clashing with each other due to the psychologies that frame the way that they think about the world.

Let's look at the play between pessimism, optimism, realism and idealism. We face this more than we might think; both within ourselves and in the disposition of others that we work with. How does it manifest and what can we do about it to help our teams succeed?

The outlook matrix

Now, before I go any further, I must admit that the matrix I am about to present to you is not my original work. I stumbled across it on a quiz website aimed at what I can only assume to be schoolchildren, given that some of the questions were asking my reaction to someone telling me that my crush likes me, or whether the teacher has yelled at me for being late to class. Sometimes inspiration comes from the strangest places. Don't worry, though: the actual origin of the image differs.

But anyway, to the matrix. It represents the tension between optimism and pessimism contrasted with the tension between idealism and realism.

Thanks to @eleventhleft for recreating this image for me. I've marked my default position with the coffee cup.

Both you and your team will naturally gravitate to a default position on this matrix: our natural mindset. Yet, that default is not fixed. Often we will find ourselves temporarily at other positions on the matrix in reaction to particular situations. When faced with the planning stage of a project we may inhabit a mindset of optimism and idealism, inspired by the CEO's vision of how the feature is going to completely revolutionize the application. During the tail end of a death march project we may find ourselves channelling a combination of pessimism and realism.

Let's explore the nuances of the two axes with respect to managing software teams. We'll discover that the extremes of the psychologies can actually drive both positive and negative behavior. There is no perfect place to be for all situations.

Optimism and pessimism

Some basic terminology that you'll already know: optimism is the general tendency to expect positive outcomes, and pessimism is the general tendency to expect negative outcomes. Initial categorization of these traits could lead to thoughts that optimism is entirely beneficial and that pessimism should be entirely avoided. There is some truth in this, however there are subtleties between optimism and pessimism and two related traits - credulity and skepticism - that are important to understand and separate.

Pessimism, of course, can drive a lot of negative behaviors. A pessimist can shut down conversation and throw water on to the fire of a great idea before it is fully formed. A pessimistic team can sap the creativity from a product owner and lead to an aversion to try anything innovative and challenging. Pessimism can make estimating work a nightmare: you just wish you could change the team's mind, but their psychology is defeating you.

Yet, it is important to recognize that pessimism is related to, but quite different from, skepticism. Whilst a pessimist will naysay ideas due to their mindset, a skeptic will apply questioning to the idea to ensure that it is as fully formed and correct as it can be. Pessimism comes from the wrong place, but skepticism comes from the right place. Don't get them mixed up, as a healthy skeptic can inject more rigor into your ideas and improve them as a result. Skepticism should be thoroughly encouraged and praised. It is essential for critical thinking.

Conversely, optimism can in many situations be extremely useful: a person with an optimistic outlook on life can drive many positive behaviors in themselves and others. An optimistic manager, especially in times of high stress and difficulty, can be a calming and balancing influence.

Optimism should be commended, but it is not to be confused with credulity, which is the tendency to be too ready to believe that something is real or true, without sufficient evidence. A credulous leader can make irreversible decisions based on false premises. A credulous team can make a poor technology framework choice without properly evaluating it. That same credulous team can underestimate the length of time it takes to complete a project and overpromise and repeatedly miss deadlines.

Drive towards optimism but watch out for credulity. Equally, reward and encourage skepticism, but don't confuse it with pessimism. The subtleties are critical.

Idealism and realism

Idealists believe that reality is constructed from ideas: that reality is itself a mentally constructed concept. Realists believe they see the world objectively as it is. Without straying into the realm of early Greek philosophy, we can place idealism and realism at two ends of the vertical axis in our diagram.

Idealism and realism can both drive positive and negative behaviors. Both have their place and time. Idealism within a founder or leader can drive everybody towards a vision of the future, no matter how challenging it is to get there. Think of Steve Jobs and his reality distortion field.

Idealism can drive innovation; it can rally a team behind a cause much greater than themselves. However, idealism can also create issues - it can become radicalism. It can create standards that may be entirely unachievable. Radicalism can cause heated arguments over pull requests. It can cause deadlocks in thinking and endless arguments. Idealism should serve as a guide, but the path towards idealism can never be entirely pure: the fact that we are just humans and are trying to get stuff done to a deadline means that there will always be hacks, shortcuts and compromises. We must be pragmatic. A healthy pragmatism between idealism and realism means that things actually get done.

Too much realism, on the other hand, can have negative consequences. Extreme realists can judge idealists as uninformed, irrational or biased, which can kill innovation. Someone has to come up with that crazy moonshot after all. Extreme realism may prevent a department with a crumbling legacy technology stack from ever trying to modernize because it is seen as too difficult, which could ultimately be the death of that company.

Your team will likely have a combination of idealists and realists. Harness the vision of the idealists to see where the team should move towards, but use the realists to help create the steps needed to get to that destination. A pragmatic balance of idealism and realism creates an actionable plan that moves in the right direction over time.

A balancing act

We need optimism without credulity, skepticism without pessimism, and a balance of idealism and realism. Where do you naturally sit in the matrix? What about the individuals in your team? What about the team as a whole? Is the group dynamic different to the sum of the individuals? Why do you think that is?

Are you able to change your position to counter-balance extreme opinions within the team, even if that mindset is not the natural one in which you reside? Also, are you able to notice when your own default position can cause biases in your own judgement?

Personally, I tend towards optimism but I can recognize my pessimism which can be higher on days that I am stressed. I sometimes suffer from credulity when talking to someone very persuasive or senior. In these situations I tend to have to remind myself to apply more skepticism. I would like to think that I have a good balance between idealism and realism, but try asking me that when I'm in a pinch near a deadline!

How about you?