Citizen Science in Flint: Triumph, Tragedy and Now Misconduct?

The story of how a coalition led by concerned residents, activists, whistleblowers and others (including worked to expose the Flint water crisis, has become an epic case study illustrating the power of citizen science and ethical science action. But we have also uncovered examples in which citizen science has been abused, to harm people and promote false narratives.

Citizen Science Tragedy.

After the Federal Emergency was declared and the responsible agencies admitted there was a problem and began working together to fix it, opportunists came to Flint in droves. In early 2016 we called out one abuse—businessmen who were claiming to be scientists, asserting unprecedented dangers with bacteria and chloroform and other contaminants that were an artifact of their flawed proprietary sampling methods. We later demonstrated they had a clear financial conflict of interest, as evidenced by attempts to recommend overpriced filters to Flint residents, many of whom understandably live in fear of tap water.

The pseudo-scientists and unscrupulous activists, found a willing accomplice with “journalists” at The Young Turks Network, who also came to Flint after the federal emergency was declared, seemingly intent on undermining any remaining trust residents have– long after the government wrongdoing had been exposed and acknowledged. In a remarkable video posting last week, TYT’s Jordan Chariton revealed his approach and journalistic standards, when he stated “ be honest with you, I do not understand why journalists are so worried about being wrong.  […] if you make assertions and it is found out later you were incorrect, there is nothing wrong with saying…..I was wrong.” Is making assertions “based on [your] intuitions” consistent with the first ethical canon of journalism “to seek truth and report it”? 

We also witnessed, an unfortunate case where a Flint citizen deliberately used improper water lead sampling methods, that made lead in their water appear much higher than it otherwise would have been. Even after we had a conversation acknowledging this lead result was not valid, it was shared with the press, to support a narrative that Flint’s water quality was getting worse and worse– despite massive relief efforts and expenditure of millions and millions of dollars to improve the situation.

Citizen Science Misconduct?

We herein document the case of another Flint resident, who has been broadcasting data showing that their water lead test data was also getting much worse with time.

Note: If you cannot see the interactive graph above, click here for an image. Only lead levels shared on NBC25 and the resident’s Facebook posts (privacy settings: public) have been included. As a further check, several of these were confirmed with available Residential testing Excel sheets from MDEQ.

This story has now been featured in numerous media (ABC12, Detroit Free Press, NBC25, TYT, other outlets), because this resident’s water lead level climbed to thousands of ppb this summer. As in the earlier case, it was a scientific mystery as to how this could happen without having any lead pipe to the home — the publicly stated implication was that water coming from the City of Flint was already contaminated with lead from the water main. This resident also posted data showing very high blood lead, and claimed their lead exposure probably occurred in the shower. This defies the conventional wisdom that bathing and showering, even at relatively high water lead levels, does not tend to elevate blood lead.

We became aware of this case in late July, when ABC12 Flint presented us with the case. At that time we offered to personally visit the home to investigate, but our offer was refused and the story ran July 27th, 2017. A few weeks later we heard rumors that Scott Smith (formerly of Water Defense) and Harold Harrington investigated, took samples, and tried to diagnose the problem. In the course of that work, it was claimed that lead fishing sinkers were found inside this person’s plumbing. In a response to an EPA FOIA related to the case of high water lead in this persons home, we have discovered disturbing documents (see below) and a video showing there is something to this rumor.

At a minimum, it seems that the mystery of astronomically high lead in this particular consumer’s home plumbing, is not as simple as the narrative that has been presented in the media. The documentation we have obtained to date, raises more questions than it answers.

Let us first compliment Mr. Smith and Mr. Harrington, because it appears they appropriately reported a discovery of lead sinkers in the plumbing of this resident to the EPA.  But did they also have an obligation to speak out, and correct the very public and misleading reports of dangerous lead in this resident’s water? What was their motive to not openly acknowledge, that there are some complexities to the story presented in the media, that are not being discussed?

We certainly have not heard the last of this story. Citizen science efforts in Flint, which in our experience and practice brought out the very best in people, have also occasionally been abused to bring out the worst in people. After the water crisis was acknowledged, a few reporters, academics, actors, activists and pseudo-scientists have attempted to exploit the tragedy for personal gain—it seems that their ends, sometimes justifies use of citizen science misconduct as a means. As these cases indicate, these tactics can create a spiral of distrust, that genuinely harms the community while undermining those who want to help the recovery.  Our intensive collaborations with hundreds of Flint residents over the last 2.5 years, in a quest to discover the truth about water quality whatever it may be, has shown us that the “good” of citizen science overwhelmingly outweighs a few “bad” examples. But the story of Flint has now become a cautionary tale, that includes citizen scientific misconduct.

Emails obtained via FOIA to EPA on finding lead sinkers from inside the resident’s plumbing

Download (PDF, 436KB)

Write-up: Dr. Marc Edwards and Mr. Siddhartha Roy

FOIA: Dr. Marc Edwards

Background research and data: Mr. Siddhartha Roy

A heartening letter from Flint

[published with permission under condition of anonymity]

Sep 19, 2017

Hello everyone,

This is ____________, I am __________ in Flint. I’m writing just to say thank you for all the amazing and thankless work you do and have done for residents and families in Flint. You are clearly in a no win situation with some activists, reporters, residents, and miscellaneous malcontents. We have been in contact with you since early last year and so appreciate your dedication to the science and to doing so in an honest and transparent fashion, unlike some of the pseudoscience and grifting being done by the circus known as ____________ and others.

This is a job you couldn’t pay someone to do in a climate so toxic and a political environment so surreal and yet you have stayed the course and continued your work. You and your incredibly dedicated colleagues and students have done a great service to the residents, families, and children of Flint. Of that, there is no doubt. You have my heartfelt and sincere thanks for all you do. At some point we hope to undertake some new initiatives in Flint to bridge the biggest remaining gaps in this crisis which you aptly called the Trust issue. That remains the single biggest impediment to weening people off bottled water and getting them to begin to use municipal tap water again. It is an endeavor we hope will bear fruit but we have no illusions and also know it may likely be decades before we see any substantial shift in how people use their water and can again trust what comes our of their faucets. We hope to approach your team in the near future to get your input, feedback and insights on how best we might undertake such an effort and achieve some lasting positive outcomes from such an effort. But for now we at _____________ are grateful for having you in Flint. Your actions are nothing short of heroic and in a place and time where few others actions can be labeled such.

Thank you all. We support you wholeheartedly and without reservation and if we can be of service to you in any way in Flint or beyond feel free to reach out to us.

Take good care


Correcting some misconceptions about our 9/15/2017 Press Conference: Lead Data

In the aftermath of our press conference last Friday, there have been some unfortunate and misleading statements about Flintwaterstudy, our experimental design, conclusions and funding sources. We herein set the record straight about some of these issues.

Also, for anyone interested, the lead data for all five rounds has been made publicly available. Download it here.

Claim:  Our study was funded by the State of Michigan who has a stake in getting “good” news.

As our press conference and presentation very clearly indicated, the 1st and 5th round of water lead sampling was funded by Virginia Tech and rounds 2-4 were funded by EPA.  The State of Michigan has never funded any of our water lead tests in collaboration with Flint residents. In each round of sampling we have been focused on collecting high quality data and releasing results publicly as soon as it is available. We have done so.

Claim:  The declining number of samples in each Virginia Tech sampling round, indicates that fewer residents in Flint are willing to open their homes to us.

Our study design, aims to resample homes tested in prior rounds, using the exact same protocol. In this manner we are generating a consistent dataset to determine trends in Flint water lead levels over the last two years. If a resident misses a single round of sampling, for any reason, or did not properly follow the sampling protocol, they have to be dropped from the study to accurately evaluate trends. The fact that 50% of the original participants from our first round in August 2015 have sampled in all five rounds, is actually a very impressive achievement that we are very proud of. It is a testament to the extraordinary effort of our Flint Citizen Science team, who executed all elements of sample kit distribution and collection, and personally drove sample bottles to many homes of shut-ins (or those who could not otherwise collect/return their own samples).

Claim:  The declining number of samples in each Virginia Tech sampling round introduces a bias, that makes lead look lower (or higher).

Because anyone who drops out of the study is also excluded from the analysis of the prior rounds, there is no inherent bias introduced to our trend analysis when participants drop out. For instance, consider a home that tended to test high for lead, that dropped out in round 5 despite our best efforts.  Losing that home would decrease absolute lead results in round 5, but we also exclude it from our calculations for the results we present in rounds 1-4. The final dataset still reflects sampling data for 138 homes in five sequential rounds of sampling.

While it does not affect our conclusions, based on the 90%’ile first draw lead value in August 2015, homes that dropped out did have higher lead than homes that stayed in the pool (27 ppb versus 22 ppb). But the median lead was higher for those who stayed in the pool versus those who dropped out (3.9 vs 3.2 ppb).

Claim:  The press conference made a blanket statement that Flint water is now “safe.”

This is false. We have repeatedly stated, our belief that no tap water in the U.S. can be made completely safe for all people at all times. All claims about safety must be made relative to existing federal regulations or levels of contaminants found in water of other cities.  We concluded that Flint water now has lead levels in the range of other cities with old lead pipes. We also clearly stated that this is nothing to be proud of, and remind everyone to use filters or bottled water provided by the State, to further reduce consumer exposure to water lead in Flint (and other cities with lead pipe).

Claim: Flintwaterstudy has not been sharing data or has not been transparent.

Ever since we launched our effort, we have conducted open science, and have shared all our data with every party who asks to see it in contradiction to many academic norms. In so doing we have routinely given up “credit” that would benefit our careers. We have also answered every question formally posed to us. And to show you our strong commitment to these principles, we will even honor a data request by Jordan at Young Turks, who has had an adversarial relationship with us in the recent past.

Claim: Flintwaterstudy was “disrespectful” to the residents of Flint, because we did not consult certain individuals who felt they should be consulted, before releasing the most recent citizen collected sampling data.

We have used the same open science approach to disseminate and publish our results for two years. Anyone can access our processed data.  We have also provided the raw data to anyone asking for it more than 100 times at last count. We do not feel a need to consult with anyone before releasing our data that is collected in partnership with Flint residents, back in August 2015 or now, and we would not change our approach if we conduct another sampling round. While we understand that some parties with an agenda do not like what certain data show, as was the case for the State of Michigan in August 2015 when our results showed problems, or other groups who simply do not like what the data show today, we do not accept that our actions on Friday were disrespectful to anyone. Thankfully, neutral parties can form their own opinion about our methods and approach, by viewing the entire recorded press conference here.

Claim: The press conference declared that the Flint water crisis was over

There is no accepted definition as to when a water crisis begins or when a water crisis ends. The only statement we made on that particular issue, was in response to a reporter’s direct question at the end of the press conference, to which Dr. Edwards responded:

“If you define the end of the water crisis as having water quality parameters back in the range considered normal for other cities with old lead pipes, the answer is yes,” Edwards said in response to a reporter’s question. “Obviously, there’s still a crisis of confidence amongst Flint residents that’s not going to be restored anytime soon. It’s beyond the reach of science to solve, but it can only be addressed by years of trustworthy behavior by government agencies who, unfortunately, lost that trust, deservedly, in the first place.”

We stand by our nuanced response to this reporter’s question.

Claim:  The problem is not anything you said, it is how the press conference is being reported.

We do not control what other people say about our results.  If another reporter said something that was factually in error about our work, we would address that by writing to the reporter.  We have done so on several occasions and we provide written clarifications on our website.

Claim: Some residents were wrongly excluded from the most recent sampling round.

We have worked very hard, to sample every home of those who participated in all rounds of sampling using the Virginia Tech protocol. One person who willfully deviated from the Virginia Tech protocol in the 4th round of sampling, in a manner known to give false high water lead results, was informed in April 2017 that they would not be part of future sampling rounds. While this person has purportedly claimed she was unfairly excluded in order to bias the results, our team’s written decision to exclude this resident was documented with U.S. EPA Region V back in April 2017.

Claim: Your protocol does not have at least half lead pipes in your pool, so unless I am missing something, you are not following the guidelines to calculate a correct 90%’ile lead value.

Most cities, including Flint, have poor records of service line materials. Obviously, when our team first sampled in August 2015 and were operating in crisis mode in potential conflict with authorities, we did not have this information and instead focused on responding to residents and sampling as many homes distributed across the city as we could. Because we now have information on the suspected service line material, courtesy of open science conducted by some great researchers at the University of Michigan-Flint, we can “back calculate” what a 90%’ile lead level for a LCR sampling pool with minimum 50% lead pipe, would look like in all five of our sampling rounds.  The rationale was explained during our press conference (see slide 24). Rightly or wrongly, most states allow old and imperfect records on service line material to construct an official LCR sampling pool. Verification of actual materials by plumbers is not required.  When we calculate hypothetical results for a representative high risk EPA pool with 50% lead pipe using this approach, our results are not inconsistent with the “official” results recently published by the State of Michigan. Those official results recently determined that Flint now meets the EPA action level.   We have always made clear that our work was not an official LCR sampling event, but this analysis shows that we no longer have reason to scientifically doubt the State of Michigan data. Unlike August 2015, when our results could not be reconciled with official claims of meeting the action level.

Question:  What is the distribution of samples by Ward?

This table provides the distribution by Ward.  Our conclusions are not dependent on an even distribution of samples, but there is reasonable representation across all 9 Wards.

Claim:  If the State sampling pool was constituted to examine the worst of the worst case homes, rather than just a normal legitimate sampling pool, the 90%’ile level in Flint would exceed the action level.

This is a truthful statement that we have frequently made ourselves. We remind everyone, living in Flint or in cities all over the U.S., that one cannot rely on the EPA LCR to protect you from elevated lead in water. In a given city, compliance with the LCR can still mean that 50-70% of homes, will have some water samples well over the 15 ppb action level in first, second or third draw, on some occasions.  Little pieces of lead plumbing, with very high water lead values, do detach from the plumbing on a semi-random basis. We roughly estimate that, dependent on sampling pool and site selection, “legitimate” 90%’ile lead values in Flint could still range from 2-50 ppb dependent on site selection.  That is right.  That is the maddening reality of the LCR, as we have been arguing for the last 14 years.  We need to update the EPA LCR, in order to make sampling results more consistent, rigorous and meaningful. In the meantime, we strongly advise people to continue use of bottled water or lead filters to reduce lead exposure, which can be very significant even in cities meeting the LCR.

FAQ: Dr. Marc Edwards

Acknowledgements: Mr. Siddhartha Roy, Dr. Amy Pruden, Dr. William Rhoads, Dr. Min Tang, Dr. Kelsey Pieper, Dr. Jeff Parks, Mr. Anurag Mantha

[Complete Dataset] Lead results from Tap Water Sampling in Flint, MI Rounds 1-5

In response to multiple requests, we are releasing lead-in-water testing results from all five citizen-led sampling events (Aug 2015, Mar 2016, Jul 2016, Nov 2016, and Aug 2017) for public use and dissemination.

Suggested Citation: (2017) “Lead Results from Tap Water Sampling in Flint, MI during the Flint Water Crisis – Citizen Science Rounds 1-5”

Download (PDF, 103KB)

Data checks (for one or more sampling rounds): Min Tang, Kelsey Pieper, Rebekah Martin, William Rhoads, Jeffrey Parks, Anurag Mantha, Siddhartha Roy

Flint Water Study Press Conference 09.15.2017

Today is the two-year anniversary of the first press conference organized by Flint residents and activists outside Flint Town Hall where results from the 2015 citizen-led water sampling effort were first presented. We now provide updates on the 5th round of citywide lead-in-water testing led by Flint citizen scientists as well as Legionella monitoring by our team.

Speakers: Dr. Marc Edwards, Ms. Min Tang, Dr. William Rhoads and Mr. Siddhartha Roy

Download (PDF, 1.25MB)

Watch the press conference here:

[Update 9/25/17 4:08PM]: Slide 24 of the PDF should say “Randomly select 17 homes from the remaining 88 homes without lead pipes but built before 1986”