In the aftermath of our press conference last Friday, there have been some unfortunate and misleading statements about Flintwaterstudy, our experimental design, conclusions and funding sources. We herein set the record straight about some of these issues.
Also, for anyone interested, the lead data for all five rounds has been made publicly available. Download it here.
Claim: Our study was funded by the State of Michigan who has a stake in getting “good” news.
As our press conference and presentation very clearly indicated, the 1st and 5th round of water lead sampling was funded by Virginia Tech and rounds 2-4 were funded by EPA. The State of Michigan has never funded any of our water lead tests in collaboration with Flint residents. In each round of sampling we have been focused on collecting high quality data and releasing results publicly as soon as it is available. We have done so.
Claim: The declining number of samples in each Virginia Tech sampling round, indicates that fewer residents in Flint are willing to open their homes to us.
Our study design, aims to resample homes tested in prior rounds, using the exact same protocol. In this manner we are generating a consistent dataset to determine trends in Flint water lead levels over the last two years. If a resident misses a single round of sampling, for any reason, or did not properly follow the sampling protocol, they have to be dropped from the study to accurately evaluate trends. The fact that 50% of the original participants from our first round in August 2015 have sampled in all five rounds, is actually a very impressive achievement that we are very proud of. It is a testament to the extraordinary effort of our Flint Citizen Science team, who executed all elements of sample kit distribution and collection, and personally drove sample bottles to many homes of shut-ins (or those who could not otherwise collect/return their own samples).
Claim: The declining number of samples in each Virginia Tech sampling round introduces a bias, that makes lead look lower (or higher).
Because anyone who drops out of the study is also excluded from the analysis of the prior rounds, there is no inherent bias introduced to our trend analysis when participants drop out. For instance, consider a home that tended to test high for lead, that dropped out in round 5 despite our best efforts. Losing that home would decrease absolute lead results in round 5, but we also exclude it from our calculations for the results we present in rounds 1-4. The final dataset still reflects sampling data for 138 homes in five sequential rounds of sampling.
While it does not affect our conclusions, based on the 90%’ile first draw lead value in August 2015, homes that dropped out did have higher lead than homes that stayed in the pool (27 ppb versus 22 ppb). But the median lead was higher for those who stayed in the pool versus those who dropped out (3.9 vs 3.2 ppb).
Claim: The press conference made a blanket statement that Flint water is now “safe.”
This is false. We have repeatedly stated, our belief that no tap water in the U.S. can be made completely safe for all people at all times. All claims about safety must be made relative to existing federal regulations or levels of contaminants found in water of other cities. We concluded that Flint water now has lead levels in the range of other cities with old lead pipes. We also clearly stated that this is nothing to be proud of, and remind everyone to use filters or bottled water provided by the State, to further reduce consumer exposure to water lead in Flint (and other cities with lead pipe).
Claim: Flintwaterstudy has not been sharing data or has not been transparent.
Ever since we launched our effort, we have conducted open science, and have shared all our data with every party who asks to see it in contradiction to many academic norms. In so doing we have routinely given up “credit” that would benefit our careers. We have also answered every question formally posed to us. And to show you our strong commitment to these principles, we will even honor a data request by Jordan at Young Turks, who has had an adversarial relationship with us in the recent past.
Claim: Flintwaterstudy was “disrespectful” to the residents of Flint, because we did not consult certain individuals who felt they should be consulted, before releasing the most recent citizen collected sampling data.
We have used the same open science approach to disseminate and publish our results for two years. Anyone can access our processed data. We have also provided the raw data to anyone asking for it more than 100 times at last count. We do not feel a need to consult with anyone before releasing our data that is collected in partnership with Flint residents, back in August 2015 or now, and we would not change our approach if we conduct another sampling round. While we understand that some parties with an agenda do not like what certain data show, as was the case for the State of Michigan in August 2015 when our results showed problems, or other groups who simply do not like what the data show today, we do not accept that our actions on Friday were disrespectful to anyone. Thankfully, neutral parties can form their own opinion about our methods and approach, by viewing the entire recorded press conference here.
Claim: The press conference declared that the Flint water crisis was over
There is no accepted definition as to when a water crisis begins or when a water crisis ends. The only statement we made on that particular issue, was in response to a reporter’s direct question at the end of the press conference, to which Dr. Edwards responded:
“If you define the end of the water crisis as having water quality parameters back in the range considered normal for other cities with old lead pipes, the answer is yes,” Edwards said in response to a reporter’s question. “Obviously, there’s still a crisis of confidence amongst Flint residents that’s not going to be restored anytime soon. It’s beyond the reach of science to solve, but it can only be addressed by years of trustworthy behavior by government agencies who, unfortunately, lost that trust, deservedly, in the first place.”
We stand by our nuanced response to this reporter’s question.
Claim: The problem is not anything you said, it is how the press conference is being reported.
We do not control what other people say about our results. If another reporter said something that was factually in error about our work, we would address that by writing to the reporter. We have done so on several occasions and we provide written clarifications on our website.
Claim: Some residents were wrongly excluded from the most recent sampling round.
We have worked very hard, to sample every home of those who participated in all rounds of sampling using the Virginia Tech protocol. One person who willfully deviated from the Virginia Tech protocol in the 4th round of sampling, in a manner known to give false high water lead results, was informed in April 2017 that they would not be part of future sampling rounds. While this person has purportedly claimed she was unfairly excluded in order to bias the results, our team’s written decision to exclude this resident was documented with U.S. EPA Region V back in April 2017.
Claim: Your protocol does not have at least half lead pipes in your pool, so unless I am missing something, you are not following the guidelines to calculate a correct 90%’ile lead value.
Most cities, including Flint, have poor records of service line materials. Obviously, when our team first sampled in August 2015 and were operating in crisis mode in potential conflict with authorities, we did not have this information and instead focused on responding to residents and sampling as many homes distributed across the city as we could. Because we now have information on the suspected service line material, courtesy of open science conducted by some great researchers at the University of Michigan-Flint, we can “back calculate” what a 90%’ile lead level for a LCR sampling pool with minimum 50% lead pipe, would look like in all five of our sampling rounds. The rationale was explained during our press conference (see slide 24). Rightly or wrongly, most states allow old and imperfect records on service line material to construct an official LCR sampling pool. Verification of actual materials by plumbers is not required. When we calculate hypothetical results for a representative high risk EPA pool with 50% lead pipe using this approach, our results are not inconsistent with the “official” results recently published by the State of Michigan. Those official results recently determined that Flint now meets the EPA action level. We have always made clear that our work was not an official LCR sampling event, but this analysis shows that we no longer have reason to scientifically doubt the State of Michigan data. Unlike August 2015, when our results could not be reconciled with official claims of meeting the action level.
Question: What is the distribution of samples by Ward?
This table provides the distribution by Ward. Our conclusions are not dependent on an even distribution of samples, but there is reasonable representation across all 9 Wards.
Claim: If the State sampling pool was constituted to examine the worst of the worst case homes, rather than just a normal legitimate sampling pool, the 90%’ile level in Flint would exceed the action level.
This is a truthful statement that we have frequently made ourselves. We remind everyone, living in Flint or in cities all over the U.S., that one cannot rely on the EPA LCR to protect you from elevated lead in water. In a given city, compliance with the LCR can still mean that 50-70% of homes, will have some water samples well over the 15 ppb action level in first, second or third draw, on some occasions. Little pieces of lead plumbing, with very high water lead values, do detach from the plumbing on a semi-random basis. We roughly estimate that, dependent on sampling pool and site selection, “legitimate” 90%’ile lead values in Flint could still range from 2-50 ppb dependent on site selection. That is right. That is the maddening reality of the LCR, as we have been arguing for the last 14 years. We need to update the EPA LCR, in order to make sampling results more consistent, rigorous and meaningful. In the meantime, we strongly advise people to continue use of bottled water or lead filters to reduce lead exposure, which can be very significant even in cities meeting the LCR.
FAQ: Dr. Marc Edwards
Acknowledgements: Mr. Siddhartha Roy, Dr. Amy Pruden, Dr. William Rhoads, Dr. Min Tang, Dr. Kelsey Pieper, Dr. Jeff Parks, Mr. Anurag Mantha
