Where is the joy in research?

Dennis Overbye has a rather odd essay in the NYT: "The Joy of Physics Isn’t in the Results, but in the Search Itself ." It starts off reading as a conventional justification for pure research based on the unanticipated but fundamentally useful technological products it produces, products that applied scientific or technological research would not have produced. You know the things: hypertext and the WWW, digital camera sensors, MRI and PET scanners, Velcro, pens that write when upside down, memory foam beds.

But then it veers off to tackle, as far as I can tell, the slow and bumpy road along the path of scientific progress from Overbye's perspective as a science writer in 2009. The pre-servicing mission Hubble contrasted with post servicing mission glory, the Large Hadron Collider's commissioning woes, and the pre-announcement hype of CDMS-2's decidedly ambiguous one (or two, if you're generous) sigma pseudo result. This is more in line with the essay's title.

I presume that Overbye's point was to highlight that the scientific method is not all about predictable results that appear in a regular and preplanned way. That both the results and process of science are unpredictable, and that's where the "joy" of it lies.

Its all very well, admirable even, for a science writer to tackle societies naive and preconceived views of how science works, but I must say I think Overbye's essay veers to much in the opposite direction to ignore the many years of routine operation by telescopes (the result of careful planning and hard work) and particle colliders, where experiments are planned and go roughly as expected. Unanticipated surprises happen, but the surprise us because they're rare and contrast so vividly with the larger edifice of scientific progress built slow accumulation and refinement.

Returning to the title of Overbye essay, I'd agree that doing science is fun. Seeing or discovering things that no-one else has ever seen or thought is a thrill. The mental challenge itself is pleasurable, even when you're treading scientific ground where others have gone before. But getting a result - answering a question - even if it isn't the answer you expected, is the payoff and culmination of all the hard work. So some of the joy of physics is certainly in the search, but a lot of it is in getting results.

Brian Switek reviews the Templeton Foundation's "Test of Faith" DVD series

Brian Switek has a characteristically thoughtful review of the Templeton Foundation's DVD miniseries "Test of Faith" on his blog. As Brian is not one of those evil "divisive" (or worse: "uncivil") atheists like Dawkins, Coyne or PZ Myers I'd hope that his critique is not dismissed out of hand by pro-religion accommodationists.

Unfortunately for an organization that claims to represent a serious, sober and moderate vision of (some form of) religion and science not in being in conflict, the DVDs engage in much the same sort of God-of-the-gaps teleology and straw-man argumentation that creationists normally engage in.

Nor does the Templeton Foundation's vision of religion sound particularly deist or ecumenical. Brian writes:

I would not have been so aggravated with the program if it presented scientists who said something akin to "I am a Christian/Muslim/Buddhist/Pastafarian/&c. I believe [insert belief system here] on the basis of faith, and I feel what I have come to understand about the nature of the universe is consistent with the faith I practice. Rather than make nature conform to my beliefs, however, I would rather understand the world as it is. If it turns out to be inconsistent with my faith then I will have to question what I believe." I could at least respect that. Instead the Test of Faith series trots out scientist after scientist who believe that they have some special glimmer or proof of God in nature; it is going at the whole thing backwards. The impression the series gives is that the natural world justifies and supports a particular religion, Christianity, rather than stating that some liberal forms of that religion could accept the science of evolution. (Whether evolution is reconcilable with religion depends on what brand religion we're talking about.)

And people wonder why many scientists have little or no respect for the Templeton Foundation?

PS: The 'physicist Katherine Blundell [who] says that there are "truths" in the universe that science does not detect' (*) mentioned is most probably the Oxford astronomer Katherine Blundell, as she is associated with some organization called The Faraday Institute For Science and Religion.

* Oh really. How do you know?

Myths about Peer Review

Michael Nielsen has an interesting article discussing three myths about scientific peer review.

I certainly wasn't aware how little peer review was used until recently. For example, perhaps only one of Einstein's 300 papers was subject to what we modern scientists consider as peer review, and Einstein was far from pleased at the (correct) negative report he obtained.

I'm less than convinced by Nielsen's claim (Myth 2) that peer review is not reliable. Its not perfect, but just because its not perfect does not mean that it doesn't serve some useful function at all. All practicing scientists certainly know the system has a large random component. With respect to papers destined for Journals I'd say on peer review has improved all my papers (although often in very minor ways) and that my reviews of other's papers have improved them. Some of the stuff in a Referee's report turns out to be a waste of time, other parts may be good suggestions or advice, and other parts total misreadings or misunderstandings of the paper (in which case you know other readers will probably also misunderstand it too, so you'd better explain yourself better).

In the two cases where I have rejected a paper in the 14 years of my scientific career (including my time as a PhD student) both cases had demonstrable fatal flaws. Both of those were theory papers with mistakes that a non-expert almost certainly would not have identified themselves, and thus letting them enter the literature would have been detrimental. Bad observational papers are harder to reject as the data may be ultimately useful even if the Author's analysis is sub-par or their interpretation is merely crazy speculation.

Peer-review, as applied to grant or observing time, certainly has a larger random component that reviewing papers, but that is understandable given the broader range of science you'd encounter on a peer review committee (i.e. subjects further from your own specialized area of expertise), and the large number of proposals the committee has to read, rate and debate in the short amount of time allocated. A committee may like a proposal one year that another committee didn't like a previous year. There is a large random component in the absolute ranking of proposals, but generally people can come to agreements on what are the best and worst proposals. At that imperfect level peer review works.

Nielsen's Myth 3 is correct. Just because a paper passes peer review does not mean it is correct!

But I do think peer review is a necessary component of modern science, even if (as Nielsen argues) science functioned acceptably without widespread peer review in the 19th and early 20th centuries. Modern science is vastly larger and more complex in scope and depth that it was even 50 years ago. The degree of specialization is such that even the smartest and most widely-read scientists can not fully judge papers outside their own narrow range of expertise themselves. Peer review is an imperfect filter, but without it there would be even more wrong science out there. There is no way that increasing the amount of flawed, bad or fundamental wrong papers can be beneficial to science, can there?

Ultimately peer review requires that you have informed, qualified, peers (I suppose they're not technically peers if they aren't qualified). If science becomes so specialized that no-one else understands anyone else's work then peer review will fail. Similarly "peer" review by unqualified reviewers doesn't work (e.g. the slashdot comment rating system - as the number of readers increased and their average technical knowledge decreased the system devolved in to pure uninformed opinion. Eventually the signal-to-noise ratio decreased so far that I personally consider slashdot unreadable). Reviewers have to know something to make useful reviews!

Its also worth remembering that astronomy/astrophysics journals have relatively low rejection rates compared to other sciences. If I were in a field where my papers suffered a 50% chance of being rejected I might have a less positive view of peer review!

Scientists need History and Philosophy of Science classes

I doubt I can remember the names of all the classes I took in my undergraduate Physics with Astrophysics BSc at Birmingham in the early 90's, but I know for sure which classes I always wanted to take but were never offered.

We didn't have any class that dealt with the Philosophy of Science, and we didn't have any class that covered the History of Science. If you had any interest in those topics you had to go find a books in the library yourself. The closest we had was Introduction to Astronomy, which briefly covered some historical ideas in Astronomy (Olber's Paradox, etc). (Later, as a PhD student in the UK we didn't have classes at all unlike the US PhD system.)

There are some people who say "Why bother telling students about all the wrong things people believed in the past? And even things that are right, e.g. electromagnetism, are much more easily taught from a modern standpoint using modern mathematical techniques, and not in the tortuous and overly complicated ways they were originally derived."

There is some truth to this. Certainly we wouldn't teach the laws of motion and dynamics in the way Newton presented them in Philosophiæ Naturalis Principia Mathematica. It uses geometrical proofs that most modern students lack the training to appreciate, and that can be done much more cleanly and simply with more modern math (to understand Newton's Principia as it was written you really need another genius, Chandrasekhar, to explain it to you).


But knowing what people once believed, why they believed it, and why they then moved on to believing what we now believe, is still valuable and is good scientific practice (its also intellectually interesting, which one would hope real scientists would appreciate for its own merit). Furthermore, understanding the method and philosophy of science is important to being a good scientist - you can go a long way just doing things they way you were trained to in grad school, but without being taught the Philosophy of Science you can easily get off in the wrong direction.

All of this becomes more important when scientists need to communicate clearly and accurately with the public, in particular in cases where science is struggling against anti-scientific social groups (e.g. creationist/ID, anti-vaccinationists, global-warming-denialists).


As evidence that even people trained as scientists often not understand the metaphysical foundations of science, James Williams of the University of Sussex has studied the perceived meaning of words such as fact, theory, hypothesis, etc, in graduate students with existing science degrees who plan on going on to become science educators.

• 76% equated a fact with 'truth' and 'proven'

• 23% defined a theory as 'unproven ideas' with less than half (47%) recognizing a theory as a well evidenced exposition of a natural phenomenon

• 34% defined a law as a rule not to be broken, and forty-one percent defined it as an idea that science fully supports.

• Definitions of 'hypothesis' were the most consistent, with 61% recognizing the predictive, testable nature of hypotheses.

The results show a lack of understanding of what scientific theories and laws are. And the nature of a 'fact' in science was not commonly understood, with only 11% defining a fact as evidence or data. Here are just a few of their definitions of a scientific theory: "An idea based on a little evidence, not fact"; "an idea about something, not necessarily true"; "unproven ideas."

From "Is Something Missing from Science Education" [io9.com]. The io9 article continues

Although this survey is hardly proof of widespread epidemic of ignorance among recent science grads, it does suggest that many science programs educate students only in the technical aspects of their field, failing to provide them with the underlying context and purpose of scientific study.

Williams fears that failing to educate science graduates in the history and philosophy of science, these grads are ill-equipped to educate a public whose lives increasingly depend on a basic understanding of how scientists operate and what scientific findings really mean. Both he and Gallagher believe that introducing such courses to undergraduate science curricula could go a long way toward making science education more complete.

If people with science degrees have trouble using even the most basic and fundament words correctly then we have a problem - and there is little hope that the general public is going to learn if we can't explain it to them.

I don't teach, but I certainly support the argument that those majoring in the sciences need Philosophy of Science classes at the very least, and preferably some reasonably detailed class on the history of their particular science.

Fred Hoyle and the Venusian Pox

In which we muse on panspermia and the decline of once-great astronomers into crackpots.


[Approximate true-color image of the Venusian surface taken by the Soviet Venera 13 probe in 1982. Images take from the NASA NSDCC photo gallery.]

While browsing the science section of the BBC news online last week I came across the provocatively titled article "Life from Venus blown to Earth".

Many people are aware that temperatures at Venus's surface are a blistering and thoroughly inhospitable 480 °C (896 °F) or so, too high for any life as we know it (the lack of water is also a major problem) thanks to a run-away greenhouse effect. However, high in Venus's atmosphere there is a layer that with temperatures and pressures that are closer to that of Earth, and it has been seriously suggested that primitive life akin to bacteria might survive in the atmosphere (I haven't managed to track down a reference to the originator of this hypothesis).

Furthermore, panspermia (the idea that primitive life might be spread through the solar system or even the galaxy on material ejected by asteroid/comet impacts, among other mechanisms) is a not a totally kooky idea - a moderate amount of sober and serious scientific research is published in reputable peer-reviewed scientific journals each year (it is also a staple of science fiction, most often used to excuse the author's extreme laziness and lack of imagination in populating their fictional universe with large numbers of species of intelligent bipedal tetrapods that somehow often look more similar to humans than humans do to chimpanzees. They also want to mate with the captain of the human's space ship and occasionally rip his thin tee-shirt - this is in fact a robust prediction of many published models of panspermia). Wikipedia's article on panspermia is of reasonable quality, and so worth a read if you're interested.

This ADS abstract query returns all the peer-reviewed articles containing "panspermia" in their abstracts, and it appears to generate of order 5 papers per year, mainly in Astrobiology journals (IJAsB, AsBio) but occasionally in Astrophysics and Space Science (Ap&SS) which is often really a form of conference proceedings rather than a traditional peer-reviewed journal.

I would hazard a guess that most professional astronomers consider panspermia to be unlikely to be true (in particular on scales larger than the Solar system), but interesting and worth some research none the less on the off chance it might be important - that is my view of the subject in any case. Anyway, I followed the link to the BBC article in the hopes of reading something interesting...

Life on Venus could be blown to Earth by powerful winds, scientists claim. Previous research has considered the possibility of micro organisms existing in Venus's atmosphere despite extreme temperatures on its surface. But two scientists at the Cardiff Centre for Astrobiology say microbes from Venus could actually be blown into the Earth's atmosphere by solar winds. Their findings follow analysis of data from the European Space Agency's Venus Express probe, launched in 2005.

All seemed well, until I read the next sentence, at which point alarms bells started ringing in my head and my skepticism levels surged from mild to extreme.

Prof Chandra Wickramasinghe and Dr Janaki Wickramasinghe claim Venus's clouds contain chemicals that are consistent with the presence of micro organisms.

So let me explain why I'm not going to bother reading the actual Ap&SS article that sparked the news story.

Professor Chandra Wickramasinghe is quite a famous astrophysicist, and was a long-time colleague and collaborator with the even-more-famous (or infamous) Sir Fred Hoyle. In fact as a teenager I read popular science books co-authored by Hoyle & Wickramasinghe that were in the public library. Let us discuss Hoyle for a while, but as you'll see Chandra Wickramasinghe will naturally enough re-enter the story later.

Hoyle is most famous in general conception as the originator and main proponent of (now very thoroughly disproven) Steady State Cosmology, and ironically as the originator of the term "Big Bang" for the competing cosmological theory that now bears that name (it may or may not be the case that he used Big Band in a derisory sense). \

Hoyle is an interesting, even tragic, character - his early work, in particular on nuclear astrophysics and the generation of the elements (e.g. Burbidge, Burbidge, Fowler & Hoyle, 1957, Rev. Mod. Phys. 29, 547 - 650), is still very much the foundation of modern astrophysics, and Hoyle is remembered favorably and fondly in the astrophysical community for that work.

But by the later half of his career he ended up being considered also as something of a kook or a quack. Some of that was because of his (and a very few others) refusal to see or admit that Steady State Cosmology was increasingly inconsistent with the growing body of observational data in favor of the Big Bang. Hoyle continued to flog the dead horse of SSC well beyond the point the paradigm shift in favor of Big Bang cosmology had occurred, and in doing so made himself appear unscientific and contrarian (rather than being some sort of principled non-conformist).

That is not to say we astronomers do or must all think the exact same things. At the "working edge" of astronomical research disagreement between different astronomers is very much the order of the day. Astronomers can quite reasonably and rationally disagree about General Relativity verses Modified Ordinary Newtonian Dynamics, whether the Big Bang had a actually beginning, or whether it was preceded by a Big Crunch. But all professional astronomers agree that the Earth orbits the Sun, and those people that don't believe that are almost certainly not scientists.

Similarly scientists prefer clean simple theories that don't need lots of fine-tuning, because in general anyone can made an arbitrarily complex theory fit any data and at that point you've lost all hope of actually having either useful predictive power and/or hope of falsifiability. By the 1990's the additions Hoyle and colleagues had made to the Steady State Cosmology (now renamed the Quasi Steady State Cosmological Model: Hoyle, Burbidge, & Narlikar, 1993, ApJ, 410, 437), e.g. the precisely shaped metallic needles required to create a 2.7K microwave background without a Big Bang, appeared to the rest of us as little better than the epicycles upon epicycles in the flawed Ptolemaic system of planetary motion.


The other reason that many scientists (and not just astronomers) started to view Hoyle as a crack pot was that by the 1980's Hoyle was saying all sorts of really weird stuff. One telling example is that Hoyle, Wickramasinghe and Spetner
claimed that the feathered and winged dinosaur fossil Archaeopteryx was a fake (they seem to have ignored the fact that there where least six Archaeopteryx specimens known at the time). Why would they claim that?

Others might be deterred but Hoyle and Wickramasinghe have replied with Archaeopteryx, The Primordial Bird a book published in London by Christopher Davis in 1987. In it they repeat their claims in addition to advancing the notion that evolution proceeds in sudden fits and starts as a result of genetic storms of viruses carried to the earth from outer space. "Egads," you might think "where is the line between science and science fiction?" Molecular biologist have reacted with embarrassment to these mystical outpourings and have replied that there exists not an iota of evidence to support these wild theories. In a review of the book in New Scientist (10 September, 1987) Beverly Halstead writes;

"This contribution [is] one of the most despicable pieces of writing it has ever been my misfortune to read. It displays utter contempt for minimal standards of scholarship ... [and] will remain for a long time a stain on the reputation of both authors."

Not an ambivalent response.

Further info regarding the claims and evidence against can be found in the TalkOrigins FAQ "On Archaeopteryx, Astronomers, and Forgery".

Basically Hoyle and Wickramasinghe thought that Archaeopteryx must be a fake because its a fully functional intermediate (or "missing link") between reptiles and birds, i.e. consistent with the scientific theory of evolution, and hence (if not fake) would be evidence against Hoyle's pet idea that viruses from outer space (see, we are back to panspermia) change one type of animal into something very different ala hopeful monsters (only monsters from outer space!). One minute you're happily a standard four legged lizard, then you get a cold (from space!) and sprout wings, no fully functional intermediates allowed. I kid you not.

So perhaps you can now see why I suddenly lost confidence in the plausibility of the "Bacteria from Venus" story (or as I call it, the Venusian Pox, as it has a nicer ring to it).

In a later post I'll discuss more interesting-but-horribly-wrong Hoyle ideas, and the possible reasons behind them.

Norman Levitt Deconstructs Steve Fuller’s Postmodernist Critique of Evolution

Steve Fuller, a self-proclaimed leftist sociologist at the University of Warwick (just south of Birmingham, which is where I did my undergraduate and graduate degrees) was a witness for the defense in the Dover School board trial (i.e for the pro-ID school board). He has a new book out ("Science v. Religion? Intelligent Design and the Problem of Evolution") defending himself and critiquing evolutionary science.

Skeptic magazine has a review of the book by Norman Levitt, a mathematician and author with Paul Gross of several books on the academic culture wars (e.g. "The Flight from Science and Reason", "Higher Superstition" and "Prometheus Bedeviled"), which is worth a read (more for the insight into the unlikely alliance between the leftist and rightist anti-science groups than for a detailed critique of Fuller's book).

http://www.skeptic.com/eskeptic/07-12-19.html#feature

How not to get tenure

Speaking of the Templeton Foundation, it is claimed that the TF(*) was one of the few sources of funding that Guillermo Gonzalez managed to obtain. Indeed, it seems he only managed to bring in $170,000 spread over 5 years, compared to the average of $1,300,000 other ISU physics and astronomy faculty had brought in while they were on tenure track!

GG's failure to bring in normal levels of funding, in addition to a dramatic drop in publication rate and lack of grad student/postdocs, were major factors in him being denied tenure. These are, along with undergraduate teaching, major aspects of being a professional scientist in tenure-track although tenure is not based on a fixed set of rules.

Many good scientists do not get tenure (e.g. Rob Knob of the Galactic Interactions blog), and many good scientists never even get onto tenure-track in the first place. Given that GG appears to have failed to satisfy the requirements in many ways it is totally unsurprising that he was denied tenure at Iowa State University.

That he espoused an unscientific astronomical version of Intelligent Design and had close links to the Discovery Institute was also, quite rightly and quite fairly, another aspect of concern for the the faculty in the Physics department he was attempting to get tenure from. The Discovery Institutes's anti-secular and anti-scientific agenda, coupled with its manifest dishonesty is no secret. The recently revealed emails clearly show that GG's DI/ID links were known and were (entirely fairly) viewed negatively but were not used as a litmus test to "discriminate" against him. Of course, this will not stop Gonzalez and the DI is hijacking a routine and just decision in order to play politics.

After all, the faculty must have been aware of his views when they offered him tenure-track in the first place - they were no secret in the astronomical community - yet he did get a tenure track position. I have no doubt that while the faculty may have viewed GG's views with distaste they would have given him tenure had he satisfied the standard requirements of all tenure track faculty: bring in funding, mentor students and postdocs, be scientifically productive. Guillermo Gonzalez has only himself to blame for his current position.

Gonzalez could have used those tenure track years to engage in peer-reviewed research to develop the concept of galactic habitability and turn it from a poorly-constrained hypothesis into a robust theory. He could easily have applied for grants to pay for several grad students and postdocs to to work with him to expand our knowledge of the role galaxies play in habitable planet formation and evolution.

But Guillermo Gonzalez didn't attempt to further science. Instead he decided to present his speculative and religiously-distorted views of Galactic Habitability to the unsuspecting public as scientific fact through his book "The Privileged Planet", bypassing peer review altogether.

In the mean time real science, done by real scientists, went ahead and left Guillermo Gonzalez behind. The ADS abstract service records 440 astronomy-related abstracts with the word habitable in 2006-2007 alone. If we repeat the search requiring the surname Gonzalez be one of the authors we get 1 abstract, and its a M. Gonzalez, not Guillermo. I have to expand the search to 2000-2007 before the Guillermo Gonzalez(**) appears, and then only in three abstracts, two of which are reviews rather than new work. Again, by way of comparison 107 abstracts contained "Galactic" and "Habitable" in the abstract between 2000 and 2007 (1353 with the word Habitable alone).

If Guillermo Gonzalez really believed that astrophysics did indeed provide convincing evidence of a Designer (specifically a Christian God) why would he have abandoned actual research?

(*) I am informed that the TF fund many good studies the interaction between science and religion, and that the TF is opposed to the "culture war" spin presented by fundamentalists such as the DI. Some of the people they occasionally fund are less rational though.

(**) There is also another Guillermo Gonzalez in professional astronomy.

Anyway, all this politics is tiring. In my next post we'll be back to discussing really interesting stuff. Yup, more on galactic winds!

Bob Park on Paul Davies and the Templeton Prize

Bob Park discusses Paul Davies's infamous Op-Ed (previously mentioned here) and the Templeton Prize in his November 30th "What's New" column.

Component analysis, causal inference, and general intelligence

The aim of astronomy is astrophysics - we observe to Universe with the hope of using the resulting data to understand the fundamental physical processes that give rise to its observed properties.

As with many sciences the data obtained from observation (experimentation, in other sciences) itself does not uniquely tell you the physics or what caused what. Instead one normally investigates to look for correlations between different aspects of the data.

For example it is known that the surface brightness, effective radius and velocity dispersion of the stars in elliptical galaxies are strongly correlated, a result now called the fundamental plane. Another example is that in starburst galaxies the soft X-ray luminosity is linearly proportional to the galaxies far-IR luminosity because, causually, the FIR traces the formation rate of massive stars, the same stars that very rapidly die and whose supernovae heat the ISM to X-ray-emitting temperatures.

Various methods of investigating correlations between multiple variables exist (e.g. principal component analysis), now often referred to as "data mining." The problem is that these methods, while useful at recasting the data in ways that aid visualization of any correlations in the data variables, do not necessarily tell you what caused what.

An interesting discussion of these often-forgotten issues and complexities, one is applicable even to astrophysics, can be found in Cosma Shalizi's article on the myth of g, the so-called general factor of intelligence. Indeed, he argues that while factor analysis is perfectly valid for data exploration or model testing, as a method for finding causal structure it is not reliable (it can be right, but often its completely wrong and can fool you).

All very interesting, and rather important to understand in the wake of a certain elderly Nobel-prize winner's recent counter-factual comments.

Wilkins on Feyerabend

John Wilkins at Evolving Thoughts has an interesting essay-length article on the philosopher most guaranteed to make a scientist role their eyes in scorn: Feyerabend. Well worth reading, especially regarding the origin of Feyerabend's ideas and their consequences in todays era of special interest denialist "think tanks".

[Image of Paul Feyerabend from https://webspace.utexas.edu/cokerwr/www/slides/philosophers.html]

Overcoming Bias rips Francis Collins a new one.

Overcoming Bias rips Francis Collins a new one. That is not the title of their blog post (its actually "Outside the Laboratory") but that is the subtext. And more importantly its a good examination of the basic question: Can you be religious and still be a good scientist? Go read O.B.'s argument for why you can't.


[Hat tip to PZ for this one.]

Failure in the self-correcting system? Maybe not.

The NYT magazine has a fascinating article by Janeen Interlandi on the case of Eric Poehlman, a medical researcher at UVM who was discovered faking his data by a student at his lab. Eventually he was sentenced to 1 year and 1 days jail time for scientific fraud, but he'd been getting away with it for years - indeed he was hired to UVM based on work he'd faked and seemed never to have stopped faking data until he was investigated.

The scientific process is meant to be self-correcting. Peer review of scientific journals and the ability of scientists to replicate one another’s results are supposed to weed out erroneous conclusions and preserve the integrity of the scientific record over time. But the Poehlman case shows how a committed cheater can elude detection for years by playing on the trust — and the self-interest — of his or her junior colleagues.

... Not only does any research touched by tainted data have to be re-examined, but high-profile cases of misconduct can also shake public confidence. “We already have a large subculture in society of people who don’t trust science to begin with,” says John Dahlberg, one of the Office of Research Integrity investigators who oversaw Poehlman’s case. “This doesn’t help at all.”

Peer review and repeatability in science are self correcting, but deliberate fraud is harder to guard against. If you get a result that disagrees with a colleagues you don't immediately think "fraud" - you check your analysis, try to see how they might have analyzed or interpreted the data differently. It might even be a typo in their manuscript, or put in the journal. And thats if you're using the same data, often you're not. Deliberate fraud of the "making up data" variety is probably rare, but this case shows that it is caught... eventually. The self-correcting part of science is necessarily speedy.

Another interesting part of the article is when it asks how did he get away with it for so long?

The length of time that Poehlman perpetrated his fraud — 10 years — and its scope make his case unique, even among the most egregious examples of scientific misconduct. Some scientists believe that his ability to beat the system for so long had as much to do with the research topics he chose as with his aggressive tactics. His work was prominent, but none of his studies broke new scientific ground. (This may also be why no other scientists working in the field have retracted papers as a result of Poehlman’s fraud.) By testing undisputed assumptions on popular topics, Poehlman attracted enough attention to maintain his status but not enough to invite suspicion. Moreover, replicating his longitudinal data would be expensive and difficult to do [emphasis mine].

“Eric excelled at telling us what we wanted to hear,” Matthews, Poehlman’s former colleague, told me.“ He published results that confirmed our predisposed hypotheses.” Steven Heymsfield, an obesity researcher at Merck Pharmaceuticals in New Jersey, echoed Matthews’s sentiments and added that Poehlman’s success owed more to his business sense and charisma than to his aptitude as a scientist.

Debate or Denial? What constitutes a valid argument?

I was not going to discuss the Hopkins/Iraqi mortality study recently published in the Lancet (as its methodology is outside my experience in statistics, and there are already a lot of good analyses of the fallacies in the naysayers arguments), but I saw an interesting comment by Zeyad of Healing Iraq (that This ModernWorld had picked up on) that is actually quite a nice explanation of what makes for a real scientific debate verse simple (unscientific) denial:

One problem is that the people dismissing – or in some cases, rabidly attacking – the results of this study, including governmental officials who, arguably, have an interest in doing so, have offered no other alternative or not even a counter estimate. This is called denial. When you have no hard facts to discredit a scientific study, or worse, if you are forced to resort to absurd arguments, such as “the Iraqis are lying,” or “they interviewed insurgents,” or “the timing to publish this study was to affect American elections,” or "I don't like the results and they don't fit into my world view, therefore they have to be false," it is better for you to just shut up.

This is just as true in the physical sciences, e.g. astrophysics, as it is in the medical/social sciences. In astronomy, much of it an observational science, there are as many claims made based on human interpretation of images ("by eye" as we'd say) as there are based on quantitative measurements based on the data.

I've just been mulling over one such issue, which I'll present part of just for fun as to get some real superwind stuff into this blog (and as an antidote to all the complaining about the woes of the space program).

For me the issue is whether the soft X-ray emission in superwinds (which are galaxy sized winds of gas flowing out of star-forming galaxies, I'll do a better into some time later) comes from the stuff drivin the wind itself, or is just a tracer of its interaction with the ambient medium. By way of analogy, think of a dust devil or tornado. What you see in a tornado is only a tracer of the actual thing powering it, you see debris (dust, water vapor, leaves, bits of houses etc) carried along by the motion of the wind - you don't actully see the air molecules themselves.



This image is of a dust devil. Notice how there seems to be more dust a the left and right-hand edges of the devil than in the center? In astronomy we would call this limb-brightened (limb as in edge, not as in arm or leg). If you imagine looking down on the devil from above you might seem an approximately circular structure, with a central circle of nearly clear air (little dust) surrounded by an annulus of dusty air.

Now that was an interpretation based purely on a qualitative argument - I looked at the image and interpreted it based on my scientific experience and came up with an hypothesis based on the dust devil looking (by my eye) to be limb brightened.

But could I test this hypothesis rather more quantitatively? Well, I could construct a mathematical model of the 3-dimensional geometry and distribution of dust, calculate how that would look projected into 2-dimensions (i.e. an image) and compare that to the data (the image).

What if someone said that the dust devil didn't look limb brightened to them? Say they though it was pretty uniformly dusty from left to right and that they though this meant it was uniformly filled with dust. Well, rather than a meaningless he said/she said argument what we'd have to do is measure the amount of dust from left to right [Note: A uniformly filled cylinder would actually appear to have more dust in the center when seen in projection].

Just for fun I've rotated the dust devil image by 5 degrees (to make the dust devil more nearly vertical) and written it out as a FITS file using the gimp, so I can use some standard astrophysical software (the wonderful fv) to probe the image quantitatively. Then I've just taken a horizontal cut accross the evil and plotted the brightness in a X-Y graph. As the dust is brighter than the sky then the image brightness is roughly a measure of the amount of dust.



As you can see the edges are brighter (have more dust) than the center, so this is evidence for limb-brightening. One side is slightly brighter than the other, which tells us there are assymetries in the dust distribution, but to first order its OK just to say its limb-brightened.

In astronomy we often have to compare data taken with difference telescopes at different times, which makes things a bit more complicated. The different telescopes often will differ in sensitivity, in the wavelengths of light they're most sensitive too, and in their spatial resolution (how sharp the images are), not to mention that the astronomical source itself might have actually changed its state in the time between the different observations.

Imagine if we'd taken a picture of the dust devil with a rather blurry camera. Then it would be harder to see evidence for limb-brightening.



Just for fun, here is that image again, now blurred so as to represent a lower spatial resolution observation (I haven't added the noise you'd actually get in a new observation). You can see the same general shape as before, but note that the evidence for limb-brightening is weaker - the peak to trough amplitude (dust devils edges to center) in the profile on the right hand side is much less than the earlier full-resolution image. If I'd blurred the image even more you wouldn't see any evidence for limb brightening. But it doesn't mean it isn't there - in this case by blurring the image we're using an effective spatial resolution too low to be able to test the hypothesis of limb-brightening.

Could we use data from the second camera to argue against the limb brightening seen in the first camera? The answer, which might surprise you, is a qualified yes. Sure the instrument itself might be not as well suited to the question we're trying to anser, but nonetheless if we got really good data we could try to quantitatively compare the two images, if we make sure to accounht for the difference in instrument capability. But you would need to work harder to prove it, you couldn't just say "well, doesn't look limb brightened in our lower resolution data, therefor the interpretation using the better camera is wrong".

Of course, no practising professional scientist would be so lazy as try a flat-out unsubstantiated denial, would they?