Wednesday, July 21, 2010

The Difficulty of Self-Assesing Competency

There was a thought-provoking but poorly organized essay by Errol Morris over at the New York Times a few weeks ago. There's so much in it and so much that comes out of thinking about it that I'm having difficulty coordinating everything I want to say about it. This first post on it is going to be a little dry, so bear with me. The resulting discussions and insight will be worth the initial dryness.

The essay itself is broken into five parts. I found it worthwhile to read the first, fourth and fifth parts as well as the first four paragraphs of the second. I strongly encourage you to read those portions, but I will do my best to summarize the relevant parts as we go.

Before I begin the summary and clarification, I want to offer a brief criticism of the article for those who do read it. Many people will probably find the second and third sections of the essay interesting as well but I found that the majority of the second portion was a somewhat obscure history lesson that confused rather than illuminated what Morris was driving at. Along the same lines, I found that while part 3 was a very vivid and historically important example of anosognosia, it also didn't illuminate Morris's points in any significant way and was more of a sidetrack. The presence of these sections forced Morris to, somewhat awkwardly, resummarize the beginning of the essay again in parts 4 and 5. And the digression away from the core of the problem of anosognosia leads me to my second criticism of the essay. Until the epilogue, Morris doesn't answer any of the questions he so provocatively raises. In fact, by the time we get to the epilogue, Morris is answering different questions from those he had been asking throughout the course of the essay and, to my mind, giving very unsatisfactory answers to those questions he does directly address.

Connected to this, the most interesting parts of the essay by far are the responses he elicits from the people he interviews. While they sometimes answer the questions he raises, their answers are at best incomplete. I know some people like the journalist to stay out of things and just report (whatever they think that means in an essay piece) but I think when you're writing an essay on philosophy, psychology and neuroscience like this, you need to give an opinion. The majority of the time you will know the subject far better than your reader and by not telling him what you learned and where you stand, you are robbing him of your expertise. Most people will just let an article like this pass in and out of their head and you cheat them of the biggest insights when you deny them the answers to the questions you raise, even if your answers are poor and incomplete. The advanced reader will still be able to see past your conclusions and come to his own.

I have other, more quibbling, problems with the essay which I'll get to as they come up. 

To briefly summarize the article, Morris is chiefly interested in what's called the Dunning-Kruger effect, which is summarized as follows:

"When people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it." 1

To illustrate this phenomenon, Morris offers the case of a bank robber who was told that if he rubbed lemon juice on his face, the cameras in the banks wouldn't be able to record his face. Not quite believing this, he rubbed lemon juice on his face, took a picture, and found that his face was not in the picture. Turns out that, with lemon juice in his eye, he mis-aimed the camera and snapped a photo of some part of the room where he wasn't. Regardless, now a convert to the powers of lemon juice, he proceeded to rob two banks squinting all the while and looking squarely in the cameras. He was caught rather quickly. This is certainly not the only case you'll run into in your life of people being almost unbelievably stupid, but it does raise some pressing questions.

How can people be so stupid? And, by extension, are even the most successful of us doing really stupid things and simply not realizing it? Or, perhaps more directly, what really stupid things are even the most successful of us doing and how do we find out?

The need to explain the Dunning-Kruger effect brings Morris to the concept of anosognosia. There are many definitions he uses throughout the article, but I'm going to go with this one:

"The state of not knowing something you should know."

To illustrate anosognosia he cites an example of a woman paralyzed on her left side who is not aware that she is paralyzed. He asks her to touch his nose with her right hand, which she does. Then he asks her to touch his nose with her left hand. Though she claims that she is raising her left hand towards his nose, it actually remains lying in her lap. She really has to know at this point that her arm isn't moving, but she's in a state of anosognosia. He leaves her for a time and then returns, again asking her to touch his nose with her left hand. This time, she uses her right hand to pick her left arm up so that her left hand touches his nose. You can see the confusion in her mind. Some parts of her clearly know what is going on, and some parts clearly don't.

In the fifth part of the essay, Morris quotes Dunning tying anosognosia and the Dunning-Kruger

The road to self-insight really runs through other people. So it really depends on what sort of feedback you are getting. Is the world telling you good things? Is the world rewarding you in a way that you would expect a competent person to be rewarded? If you watch other people, you often find there are different ways to do things; there are better ways to do things. I’m not as good as I thought I was, but I have something to work on. Now, the sad part about that is — there’s been a replication of this with medical students — people at the bottom, if you show them what other people do, they don’t get it. They don’t realize that what those other people are doing is superior to what they’re doing. And that’s the troubling thing. So for people at the bottom, that social comparison information is a wonderful piece of information, but they may not be in a position to take advantage of it like other people."

I think this is one of those areas Morris screws up in not refining Dunning's answer. Dunning basically says that incompetent people don't realize the difference between a good way to do something and a bad way. But I don't think that is quite right. I think first that the incompetent have trouble figuring out whether or not they are being rewarded as a competent person ought to be rewarded. They ought to know, for instance, that they are poor or are getting bad grades. But they don't want to admit it, they're convinced that the problem is not them but the world's perception of them.

That's one way to see it. Another way is that competent people realize when the screw things up, and do things so as not to repeat the same mistake in the future. I don't think competent people need outside confirmation to know when they've done things poorly or well. On the other hand, incompetent people don't realize when they screw things up. Perhaps another way of putting it, the incompetent don't see the difference between the better way to do things and the way they are doing things. And if you don't see the difference, of course you can't realize one way is better than another. But that is only part of the answer. I think some part of them does know the difference, but for a variety of reasons, they don't want to acknowledge the difference.

In failing to make this explicit, in leaving Dunning's incomplete answer as the last word, Morris does his essay and his topic a major disservice.

Before I move on to what all of this has led me to think about, I want to make explicit some things Morris confuses a bit. First, there are three relevant types of knowledge which Morris doesn't break out but which he probably ought to have:
  1. Factual knowledge
  2. Knowing how to do something
  3. Knowing about oneself, having an accurate self-representation
These three types have a good bit of overlap, but they also have particular differences. I'm not going to go into even more dry detail about why I've separated these types of knowledge as I have, hopefully I'm competent enough that my meaning is clear.

Second, Morris specifically breaks potential questions into different types. Expanding on his essay a little we have:
  1. Answered questions: questions whose answer we either have memorized or know where to find
  2. Unanswered questions: questions we have thought to ask but don't yet know how to answer
  3. Unasked questions: questions we haven't yet thought to ask
  4. Unanswerable questions: questions that we will never be able to answer
There are of course, several permutations of these (for example, are there unaskable questions?, partially unanswered questions?) but the above is sufficient for our purposes.

1. Justin Kruger and David Dunning, “Unskilled and Unaware of It: How Difficulties of Recognizing One’s Own Incompetence Lead to Inflated Self-assessments,” Journal of Personality and Social Psychology, 1999, vol. 77, no. 6, pp. 1121-1134.

No comments:

Post a Comment