This is actually partly true. Some established computer science journals have difficulty finding editors and reviewers capable of evaluating empirical work. Promotion committees may be dominated by theoreticians. The experimenter is often confronted with reviewers who expect perfection and absolute certainty. However, experiments are conducted in the real world and are therefore always flawed somehow. Reviewers may also build up impossibly high barriers. I've seen demands for experiments to be conducted with hundreds of subjects over a span of many years involving several industrial projects before publication. That smaller steps are still worth publishing because they improve our understanding and raise new questions is a thinking that some are not familiar with.
However, this situation is changing. In my experience, publication of experimental results is not a problem of one chooses the right outlet. I'm on the editorial board of three journals; I review for quite a number of additional journals and have served on numerous conference committees. All non-theory journals and conferences that I've seen would greatly welcome papers reporting on solid experiments. The occasional rejection of high-quality papers not withstanding, I'm convinced that the low number of good experimental papers is a supply problem.
The funding situation for experimentation is more difficult,
especially in industry/academia collaborations. However, it helps to
note that experimentation may give industry a three to five year lead
over the competition. For example, suppose an experiment
discovered an effective way to reduce maintenance costs by using
software design patterns. The industrial partner of such an
experiment could exploit this result immediately, especially since the
experiment prepared the groundwork for adopting the
technology. Given a two-year publication time lag and various other
delays (such as the results being noticed by others, let alone adopted), the
industrial partner in such an experiment can exploit at least a
three-year lead. Lucent Technologies
estimates that it is presently benefiting from a five-year lead
in software inspection methods based on a series of in-house
experiments,
apparently despite (or because of)
vigorous publication of the results.
On the negative side I fear that the ``systems researcher'' of old will face difficulties. Just building systems is not enough unless the system demonstrates some kind of a ``first,'' a breakthrough. Computer science continues to be favored with such breakthroughs and we should continue to strive for them. The majority of systems researchers, however, works on incremental improvements of existing ideas. These researchers should try to become respectable experimentalists. They must articulate how their systems contributes to our knowledge. Systems come and go; insights about the concepts and phenomena underlying systems are what is needed. I have great expectations for systems researchers who use their skills in setting up interesting experiments.