The power of doubt in research
(because even second guesses deserve a second chance)

(20:35:21 CST)

The power of doubt in research
(because even second guesses deserve a second chance)

When I do an Amazon.com search for self-doubt, let's see, I get about 60,000 books (no clue if they are separate titles or different editions in there, I'll let you do that if you care to know) and at least 16-20 per page, for the first couple of pages, are about how self-doubt is so bad and holding you so back and you are goshdarn good enough. The idea that all suffering is caused by not understanding how awesome you are and how self-doubt is the biggest sin. I suppose it makes something of a sense. If we are gods, now, and self-help books largely suggest that either we are gods or God's chosen; then doubting ourselves is heretical. That's fascinating. More study is required.

Of course, this is all poppycock. Of a sort. Self-doubt can be limiting, I admit. I have seen out of shape schlubs of the schlubbiest order manage to pull something like a back-flip after they realized they could, where self-doubt led them to tense up or pull short and not put their all into it. It is like diving for a baseball. It is going to hurt, but if you believe it is worth it you can have your hand out while if you think it is all for naught, you'll tighten up and miss the catch. What about that one kid, though, that died because he was completely convinced that he could do a backflip and didn't? What about all those petty dictators that were absolutely sure that God had spoken and it was there time to rule for a millennia? All those petty dictators who shot themselves, or were exiled, or did nothing outside of a couple of small countries. A little bit of self-doubt would have done them good, and the millions they killed.

Because, ladies and gentlemen, self-doubt is first and foremost a reasoning mechanism. You feel fear at the edge of a precipice because the fall will kill you. Not because you do not understand your godhood. No, because there is good biological and mental reason to doubt yourself some time. Learning your doubt, and when and how to use it, is just as powerful a tool, if not more so, than any self-help affirmation. I'll give you two specific ways that I see it being useful.

Self-Doubt helps us to overcome confirmation bias. If we are unsure of where the experiment is going, we are less likely to get a set of results and go, "But, these are wrong!" We are more likely to let the results speak for themselves. Eventually, we have to translate the language of the lab into results, but any attempt to name nature before then is a guess about logos. As informed and educated, hopefully, as it can be, but still a guess.

Self-Doubt helps us to overcome the Principle of Least Effort. We walk into a lab or a library or up to our computer desk and study after study has predicted that we will research until we feel we have found adequate, more so than correct, information. In practical terms, this means we look at the first page or two of results, we read the first couple of articles that seem to be a match, and then we stop. Presumably, if you are a research assistant being paid by the hour, then you might just click on page three of the Google results, but most of us don't. Librarians and students alike, we want our answer pre-packaged. The Internet and Web 2.0 has just made this worse, because they are not designed to give the best results, not really; they are designed to make it look like we know what we are doing. These are all part of the Principle of Least Effort.

Self-doubt means we get those two pages of results, and then go "Maybe that's not enough." Sure, there comes a point where page 86 is enough, but self-doubt will get us further than presuming that we are awesome.

As for limiting doubt, well, are you ready for some right up Buddha? You have to doubt it. Literally. You have to question why you doubt yourself? Is it because of embarrassment if you fail? Is it because of imminent death? Because the self-doubt of one is a problem doesn't mean the self-doubt of the other one is.

I am pretty sure I will have more to say on this later.

Si Vales, Valeo

PS: I would remiss if I did include the fact that doubt helps us to do more than one test, more than one pass through, more than one calculation. It also inspires us that what we have might not be the best, that something better than be had. In these two cases, self-doubt starts to become necessary for scientific progress.

file under (...on Life, Law, & Society)
and (...on Science)

• See other entries for February 10
• See Journal Archives (sorted by year then month)