In his March 1996 column, Al Davis, the editor of IEEE Software suggests that gut feeling is enough when adopting new software technology; experimentation and data are superfluous[3]. He even suggests ignoring evidence that contradicts one's intuition.
However, instinct and personal experience sometimes lead down the wrong path and computer science is no exception. Here are some examples. For about twenty years, it was thought that meetings were essential for software reviews. However, recently Porter and Johnson found that reviews without meetings are neither substantially more nor less effective than those with meetings[11]. Meeting-less reviews also cost less and cause fewer delays, which can lead to a more effective inspection process overall. Another example where observation contradicts conventional wisdom is that small software components are proportionally less reliable than larger ones. This observation was first reported by Basili [1] and has been confirmed by a number of disparate sources; see Hatton [6] for summaries and an explanatory theory. As mentioned, the failure probabilities of multi-version programs were incorrectly believed to be the product of the failure probabilities of the component versions. Another example is type checking in programming languages. Type checking is thought to reveal programming errors, but there are contexts when it does not help [12]. Pfleeger et al. [10] provides further discussion of the pitfalls of intuition.
What we can learn from these examples is that intuition may provide a starting point, but must be backed up by empirical evidence. Without grounding, intuition is highly questionable. What one thinks obvious may turn out to be dead wrong sometimes.