Recall that this past March results from BICEP2 were announced, claiming a greater than 5 sigma detection of a primordial B-mode signal in the CMB polarization. This result received a huge amount of world-wide attention (and see some commentary here). Yesterday saw a very curious situation, with Adam Falkowski at the Resonaances blog claiming that the BICEP2 foreground analysis was flawed, and that “Various rumors place the significance of the corrected signal between 0 and 2 sigma.” Adrian Cho at Science magazine has a story about this, quoting Clement Pryke, co-PI of BICEP as saying “We stand by our paper”, while acknowledging that, with respect to a plot of Planck data they used to estimate the foreground “It is unclear what that plot shows”.
The controversy surrounds slide 6 of this presentation, with the BICEP foreground analysis evidently relying on scraping data from this slide. The claim at Resonaances is that they didn’t take into account the “Not CIB subtracted” notation on the slide:
However, it seems they misinterpreted the Planck results: that map shows the polarization fraction for all foregrounds, not for the galactic dust only (see the “not CIB subtracted” caveat in the slide). Once you correct for that and rescale the Planck results appropriately, some experts claim that the polarized galactic dust emission can account for most of the BICEP signal.
This is backed up by David Hogg’s report of a talk by Raphael Flauger at NYU yesterday:
At lunch, Raphael Flauger (NYU) gave a beautiful talk on foreground uncertainties related to the BICEP2 results. He built his foreground models as did the BICEP2 team by scraping data out of Keynote ™ presentations posted on the web! I have to say that again: The Planck team showed some maps of foregrounds in some Keynote presentations and posted them on the web. Flauger (and also the BICEP2 team before him) grabbed those presentations, scraped them for the all-sky maps, calibrated them using the scale bars, and worked from there. The coolest thing is that Flauger also simulated this whole process to account in his analysis for the digitization (scraping?) noise. Awesome! He concludes that the significance of the BICEP2 results is much lower than stated in the paper, which makes him (and many others) sad: He has been working on inflation models that produce large signals.
It sounds like this issue is not going to get resolved until there is something more substantial from Planck about this than a slide suitable for data scraping. In the meantime, blogs are your best source of information. Or maybe Twitter, where Erik Verlinde tweets from the Princeton PCTS workshop Searching for Simplicity that:
News from Princeton: BICEP2 polarization data are due to dust foreground and not caused by primordial gravity waves.
Update: There’s also a New Scientist story here. It should be emphasized that the BICEP team are denying that there is any need to revise what is in their paper, with New Scientist quoting John Kovac of BICEP as follows:
Kovac says no one has admitted anything. “We tried to do a careful job in the paper of addressing what public information there was, and also being upfront about the uncertainties. We are quite comfortable with the approach we have taken.”
See comments in the comment section here from Sesh Nadathur and Shaun Hotchkiss explaining why there may not be very much significance to this issue.
Update: Sesh Nadathur has a detailed post up now about this, New BICEP rumours, nothing to see here. Bottom line is:
The BICEP result is exciting, but because it is only at one frequency, it cannot rule out foreground contamination. Other observations at other frequencies are required to confirm whether the signal is indeed cosmological. One scenario is that Planck, operating on the whole sky at many frequencies but with a lower sensitivity than BICEP, confirms a gravitational wave signal, in which case pop the champagne corks and prepare for Stockholm. The other scenario is that Planck can’t confirm a detection, but also can’t definitively say that BICEP’s detection was due to foregrounds (this is still reasonably likely!), in which case we wait for other very sensitive ground-based telescopes pointed at that same region of sky but operating at different frequencies to confirm whether or not dust foregrounds are actually important in that region, and if so, how much they change the inferred value of r.
Until then I would say ignore the rumours.
Peter Coles also has a blog post here, with bottom line
I repeat what I’ve said before in response to the BICEP2 analysis, namely that the discussion of foregrounds in their paper is disappointing. I’d also say that I think the foreground emission at these frequencies is so complicated that none of the simple approaches that were available to the BICEP2 team are reliable enough to be convincing. My opinion on the analysis hasn’t therefore changed at all as a result of this rumour. I think BICEP2 has definitely detected something at 150 GHz but we simply have no firm evidence at the moment that it is primordial. That will change shortly, with the possibility of other experiments (specifically Planck, but also possibly SPTPol) supplying the missing evidence.
I’m not particularly keen on the rumour-mongering that has gone on, but then I’m not very keen either on the way the BICEP2 result has been presented in some quarters as being beyond reasonable doubt when it clearly doesn’t have that status. Yet.
Update: There will be a talk about this issue in Princeton tomorrow morning, see here.
Update: Slides from the Flauger talk at Princeton are here. I’ll leave discussion of the results presented to the better-informed, but will comment that this work appears to definitely involve new heights in the technology of data-scraping from Keynote presentations.
Update: Video of the Flauger talk is here. Quite interesting are the introductory remarks of Paul Steinhardt, and the concluding remarks of Lyman Page. See also new blog posts from Jester and Sesh Nadathur. Sesh (via Eiichiro Komatsu at Facebook) includes a transcription of part of Page’s comments on the situation:
This is, this is a really, peculiar situation. In that, the best evidence for this not being a foreground, and the best evidence for foregrounds being a possible contaminant, both come from digitizing maps from power point presentations that were not intended to be used this way by teams just sharing the data. So this is not – we all know, this is not sound methodology. You can’t bank on this, you shouldn’t. And I may be whining, but if I were an editor I wouldn’t allow anything based on this in a journal. Just this particular thing, you know. You just can’t, you can’t do science by digitizing other people’s images.
From looking at all this, and seeing what the people in Princeton are saying, my non-expert opinion is that the BICEP2 result should be interpreted as an observation of B-mode polarization, but there’s no convincing data yet about the crucial question of whether this is foreground or cosmological. The BICEP2 data could not address this, and the relevant Planck data is not yet available (other experiments will soon also provide the data needed to resolve this question). The BICEP2 press release claiming “the first direct evidence for cosmic inflation” now looks like it may have been a highly premature claim.
Update: Talks ongoing at Caltech today about this at a workshop, videos later here. On Twitter, you can follow the BICEP/Planck fight via Sean Carroll:
At Caltech CMB workshop. #BICEP2 folks seem completely unconcerned about recent worries about galactic foregrounds. Wait for Planck paper…
Zaldarriaga on CMB grav waves vs. dust: sane answer is “let’s just wait.” On the other hand… we just can’t. No scientist is that patient…
MZ: Planck hasn’t measured dust in #BICEP2 region. But extrapolating from where they did measure, apparently can fit B-mode signal.
MZ: “I’m not happy this is on Facebook and Twitter.”
Seems to me we’re now stuck with Planck saying they think this is dust, BICEP saying they think it’s not. Planck is the side that has data about dust, BICEP is the side that has something they scraped off a slide of a Keynote presentation…
Update: Excellent article about this in the Washington Post from Joel Achenbach: Big Bang backlash.
Update: Zaldarriaga: I believe the case in favor of a detection of primordial B modes is not convincing (hopefully just temporarily). See more here and here.
I don’t see that this rumour has materially affected our knowledge about BICEP results in any way. In particular, the quoted value of r=0.2 was obtained assuming zero foreground dust contamination (which everyone knew was unrealistic), and that certainly hasn’t changed. They also tried a handful of different foreground dust models, all expected to be only approximations to the real dust contamination – only one of which was extracted from this Planck slide – and found that all these approximate dust models gave a foreground contamination that was an order of magnitude too small to account for the B mode signal seen.
Now, nobody really believed these dust models were entirely accurate (at least I hope not). One of the handful turns out to probably be slightly less accurate even than that. So what’s new? The most probable answer to the question of how much of the signal is due to dust contamination is still “not a lot”. And the answer to the question of how precisely we trust any one of the dust models they used is also “not a lot”.
What’s interesting is that it now seems unlikely that Planck is going to be able to tell us what the real dust contribution in the BICEP region is, but this doesn’t fit with the rumoured narrative of “BICEP shown to be wrong”.
I am not a physicist (computer engineer), but am very much interested in this area. Normally I read through a lot of this stuff and sometimes research more, and sometimes just say wow. On this particular issue though, I really have to ask to ensure I’m reading what I think I’m reading. They took a slide from a presentation and scraped points off of it, and used that as their data points?! Why would someone even bother to do that? Isn’t the original data that was used to generate the slide in the first place available in a more appropriate form, so that screen scraping wasn’t required? Couldn’t the whole exercise be repeated again, but use the original data (instead of the flawed screen scrape data) to validate the results or get more accurate results?
Nick,
As you can see from Sesh’s comment, the argument is that this was being used to show that this source of foreground was an order of magnitude too small to account for what they were seeing (to be clear, I have no idea who is right about this, other people are making other claims). So, from this point of view, it didn’t matter much whether they had an accurate version of the Planck data, the slide was good enough.
This does show that the relationship between the Planck and BICEP teams isn’t exactly one of close cooperation…
Erik Verlinde only tweets the science magazine blog post. So no new information really.
Peter (and Shesh),
Thanks for the addition clarity. I think the big item here (beyond the accuracy of the original claim) is actually that screen scraping was required at all. Saying close cooperation doesn’t exist seems to be an understatement if people are willing to go to those types of extremes to get good data.
This is pretty silly behavior for scientists. Apparently the Planck team would not release the raw data to the BICEP2 team which caused them to go for the screen scrape:
So here we have two very high profile groups with talented researchers bickering and competing resulting in ‘digitizing’ a screen scrape off a PDF file.
The things they work on are cosmic in scope and the results of potentially huge import for the future of science, but the behavior of the individuals is decidedly middle school in scope and (should be) embarrassing for scientists.
My personal bet is that at the end, we will meet mid-way, at rho ≈ 0.1 – anyone interested?
The two BICEP talks that I’ve heard were a bit opaque on their relationship with the Planck collaboration. Certainly, the slides would be good enough for an order of magnitude estimate, or better than an order of magnitude. To get fine detail and the level of confidence they claim, though, requires better information than that. But as Sesh points out, the BICEP team has a number of other ways of checking their result, and they all agree, within uncertainties. Of course, they agree even better if the uncertainties are larger than BICEP initially claimed. The BICEP talks all stressed the multiple independent checks of their results.
The refusal of the Planck team to share their raw data is disturbing. This is potentially one of the most important scientific discoveries of the last century. I assume, as scientists, we outgrow this behavior in high school or college.
Slightly further to Sesh’s comments it should be stressed that within the patch of sky BICEP has measured, the dominant noise in Planck’s measurement is Planck’s instrumental noise (or so I have been led to believe). That this is probably true can be seen by the fact that in Planck’s recent paper on polarisation (http://arxiv.org/abs/1405.0871) the BICEP patch is part of the region that has been masked because it lacks sufficient signal (dust) to noise (instrument).
Therefore, any attempt to reconstruct the expected dust foreground for BICEP by analysing Planck’s preliminary “polarisation fraction” maps slide is probably going to end up mistaking a fair bit of Planck’s instrument noise for dust. Obviously, Planck’s instrumental noise has nothing to do with the BICEP detector. So, as Sesh stressed, Planck might not be able to, even in the long run, determine the dust foreground along BICEP’s line of sight accurately enough to have an impact on BICEP’s measurement. This seems kind of obvious to me in hindsight, or Planck would have detected BICEP’s signal long before BICEP did!
Planck will only ever be able to make a statistical detection of tensors by analysing B-modes over the whole sky and saying the average signal isn’t consistent with zero, but it won’t be able to say that along one specific line of sight the B-mode signal is x \pm y, like BICEP has.
At least, this is the picture as I’ve understood it over the last week of discussion.
There can be good reasons not to share your “raw” data: you are possibly not done with the analysis and what you have shown up until now are preliminary results, or you think that the raw data would likely not be interpreted correctly by the other experiment (for example, raw data from a large particle physics experiment could not (should not) be used by another experiment). If the Planck results were published in a paper and hence not preliminary, then to give the the numbers used to make a plot used for the publication should be a simple courtesy however.
Some of these comments are really quite silly. At the time of the BICEP announcement no Planck polarisation papers had appeared. Preliminary results that appear in talks are just that, preliminary. No experiment should be under any obligation to release its results or data before they are ready to their own satisfaction.
Once Planck releases its fully analysed polarisation data, uncertainties on foregrounds will come down. Until then, the uncertainties are higher. The basic obligation experimentalists have to the community is to be correct.
BICEP chose to announce their results via press conference, using the full amplification of the news media. IMO publishing in this manner means you have to be right – there can be no excuses if foregrounds turn out larger than your best estimate, it is either Stockholm or go home.
I hope it’s the former, as the science is enormously exciting if they are correct.
Shaun Hotchkiss
even if Planck can’t get an accurate dust map for the specific patch of sky measured by BICEP2 the fact that the polarization signal (from dust) is so high in the rest of the map would make it hard to argue how the small BICEP2 patch might avoid such contamination.
Adam T,
situation in astronomy is much better than in HEP.
At least they release their data after some years of proprietory period.
In HEP experiments no one releases ANY data to outside world even decades
after experiment has ended.
piscator,
The news reports suggest that Planck scientists are passing around rumors tsk tsking the BICEP2 folks for screen scraping and saying that the raw data interpreted correctly lowers confidence in BICEP2’s results. If that is true and the they refuse to let the BICEP2 folks have access to the raw data, then this is very bad form from the Planck investigators.
This is not middle school. The science should be held as a higher goal than trying to pass around rumors and withholding data from a rival experiment that might illuminate a discovery of this magnitude.
JG: That argument could be quite simple actually, since the level of dust contamination varies across the sky – there’s obviously more dust in the line of the galaxy than in other regions. BICEP’s window was specifically chosen to be in a region that was believed to have low dust emission. The fact that Planck is perhaps unable to distinguish dust emission from instrument noise in that part of the sky could even be an encouraging sign that dust emission is actually small there (I’m obviously making some assumptions about noise properties which could be wrong, but that’s kind of the point – people making assumptions without knowing the details can wind up at any kind of conclusion).
As several others have said, it’s silly to blame the Planck team for not releasing preliminary data: they’re still working on it and checking it, that’s why it’s preliminary.
Could Planck’s failure to release data be related to the fact that
Planck 2013 results. XXXI. Consistency of the data
has NOT yet been released – although the http://www.sciops.esa.int web site says
that it has been in 2014 “Submitted to A&A”
but NOT yet Accepted by A&A.
Tony
Just a point of psychology. Anybody who thinks that the same instincts, impulses and motivations that impel us in middle school aren’t still at work in middle age hasn’t been paying attention. Read some history, or better, Watson’s version of the Double Helix.
If Watson and Crick hadn’t surreptitiously swiped Franklin’s X-Ray diffraction, they wouldn’t have won the Nobel. If Pauling had seen it, he would have collected his third Nobel Prize.
Thanks Sesh
and also I noticed from reading other blogs and comments that Shaun Hotchkiss is fully aware of my trivial point (and has stated it more accurately)
I also agree that Planck have no moral obligation to release raw data before they have accurately modelled influences from instrument noise and other contamination sources
There is one factor nobody mentioned: some BICEP2 scientists are on the Planck collaboration! and they have therefore access to all the raw data.
It seems to me that Planck is not our best hope to settle this issue mainly due to the fact that it is a satellite, and information from certain ground-based telescopes might be more credible.
Since the CMB polarization level is obtained from differencing two intensity measurements toward the same direction on the sky, any optical imperfection of the detectors can potentially leak the dominant intensity to the faint B-mode polarization. A few of these spurious signals can be potentially mitigated by the telescope’s scanning strategy; basically, each pixel in the field is observed multiple times with (ideally) different orientations of the polarimeter. Ideally, this would suppress a large fraction of the systematic but not entirely.
Most ground-based telescopes benefit from the earth rotation, others use half waveplate, and in general the scanning strategy could be optimized for minimizing the intensity-to-B-mode leakage. Satellites in orbit, however, are limited and their typical scanning strategy is sub-optimal if not poor. We know, as a fact, that Planck never published their B-maps, and I guess that this is partially due to the issue of B-mode systematics which is always a challenge, for satellites in particular. I’m not sure how much would the Planck team have to say in October.
My best bet is that a joint effort of ground-based instruments (which are located off the pole) might ultimately provide a conclusive answer to this thorny issue. The problem with this alternative, though, is that ground-based experiments are limited to a relatively narrow frequency window. Hopefully, two or three frequency bands will suffice for the construction of a reliable polarized dust model, but this is not apriori guaranteed.
CIP,
Just because some people haven’t moved on from middle school mentality does not negate that some of is have and that more should.
All defending Planck for not releasing prelim data,
Fine if it is prelim and unsure they should stop spreading nasty rumors about he competing project. Put up or shut up.
AdamT
Planck aren’t spreading rumours, it’s casual blogosphere discussions causing the stir, the fact that some pretty well respected bloggers are involved has made it a bigger story than it should be, but your reaction is unjustified.
Planck have “put up” their results in an orderly professional manner, and they will deliver the final results later this year
The title of the talk at Princeton tomorrow sounds like something from the social sciences or philosophy: “Towards an Understanding of Foregrounds in the BICEP2 Region” – maybe there is a little humour/sarcasm implied by the title?
The PDF has just come out for the Princeton talk on BICEP2 foregrounds.
http://www.pctp.princeton.edu/pcts/SpecialEventSimplicity2014/Simplicity.pdf
Read it and frown if you’re a BICEP2 advocate.
Go to page 39 of the presentation. One corrected model shows 7 out of 9 BICEP2 observation points within the area predicted by a combination of dust foregrounds and gravitational lensing. The other corrected model also shows 7 out of 9 BICEP2 observation points within the area predicted by a combination of dust foregrounds and gravitational lensing.
If you use the BICEP2/Keck combination for the observation points, then it’s basically 9 out of 9, with nothing that needs to be explained by cosmic inflation.
The video of the Princeton talk is now available too.
One thing to remember from this story will be for sure Flauger’s work on reproducing the data from the pdf file and “correcting” for the pipeline (starting on healpix maps) effect. Bravo! (note to myself, go back to transparency slides in your next talk)
Yesterday, Matt Strassler finally weighed in on the current BICEP2 controversy with his blog post “Will BICEP2 Lose Some of Its Muscle?” A very interesting read, and, as always, a very balanced presentation. (And please forgive me, Peter, if this link has already been provided elsewhere in your article, or in subsequent visitor postings.)