Google+ Followers

Tuesday, 26 November 2013

20 Things politicians should understand ... (Part 3)

Continuing the previous two postings here is the third set of 5 more "Things politicians need to know about shale gas science", inspired by the recent Guardian article entitled "Top 20 things politicians need to know about science" from an original article in Nature.  

It is not just politicians that need to know this stuff - without it the whole debate is not possible.

11. Seek replication, not pseudoreplication

Results consistent across many studies, replicated on independent populations, are more likely to be solid. There is nothing better than good quality data and lots of it from different locations. Unfortunately data is often man-power intensive and hence expensive. However, government and companies must be prepared to spend money on collecting that data if the general public are to trust their operations (and here).

Moreover, data from different scenarios or locations can often be combined in a systematic review or a meta-analysis to provide an overarching view of the topic with potentially much greater statistical power than any of the individual studies. This requires that data is made freely available between companies and to the general public as well as academics.

Since data is expensive and represents a commercial advantage, companies are not likely to share it or make it available on their own, however enlightened they are. Interestingly Cuadrilla have released a large amount of water quality testing data here and here because they recognise that it represents part of the community patrimony. It is hoped that this will continue. The government should take a central role in coordinating the archiving and publication of all shale gas data through, for example, the British GeologicalSurvey, but is currently avoiding it.

12. Scientists are human

It is not a case of companies bad, politicians bad, activists bad, scientists good – scientists are human too. Although most scientists take extreme care in balancing evidence and following a scientific rationale, a few are less than candid. One must always remember that scientists have a vested interest in promoting their work, often for status and further research funding, and occasionally for direct financial gain. This can lead to selective reporting of results and occasionally, exaggeration. Peer review is not infallible: journal editors might favour positive findings and newsworthiness.

All this adds up to the statement that scientists should not be believed blindly nor their statements regarded dogmatically. If shale gas extraction is to be carried out successfully, it needs the informed consent of the local communities – informed consent means listening to the statements of a range of scientists and others to form a balanced evidence-driven view upon which solid decisions can be made.

13. Significance is significant

Opinion is not important. The only way of testing data is by using valid statistical tests.
One of the most common ways of stating whether an effect, such as whether hydraulic fracturing has contaminated an aquifer, is real is the statistical significance or P-value. The P-value is a measure of how likely a result is to occur by chance. Thus P = 0.01 means there is a 1-in-100 probability that what looks like a link (say fracking and aquifer contamination) actually occurred randomly. We would call P=0.01 very significant as it also indicates that there is a 99-in-100 probability that the link is real. Usually P<0.05 is taken as the limit where a link is considered to be proven.

So far we have no data in the UK that can be used to carry out a test like this because there has been no fracturing where back-ground data is available (no fracking was carried out at Balcombe). Similarly, no background data is available in the USA and so proper statistical tests cannot be carried out there either. However, such tests will be common in future, in the UK at least, because companies are committed to carrying out before and after water quality tests on aquifers.

14. Separate no effect from non-significance

The lack of a statistically significant result (say a P-value > 0.05) does not mean that there was no underlying effect: it means that no effect was detected. A small study may not have the power to detect a real difference. For example, tests of local wild-life around the Balcombe drilling site may suggest that it suffered no adverse effects from disturbance by the drilling operation. Yet if the tests sampled too few animals it would not have the power to detect impacts had there been any. Even then, it would be extremely difficult to distinguish between disturbance by the drilling operations and disturbance by the large number of protestors.

15. Effect size matters
Small responses are less likely to be detected and may fall below the measurement sensitivity of whatever instrument is being used. However, a study with many replicates might result in a statistically significant result but have a small effect size (and so, perhaps, be unimportant).

Let’s take drilling or fracturing induced earthquakes. When hydraulic fracturing is carried out it results in thousands of tiny earth tremors by definition – the whole process is designed to make fractures in the rock and each fracture formation is an earthquake, however small. These earthquakes are mapped in the sub-surface by microseismic methods, and it is possible to see where each one occurs and to delineate the fracture network that forms. If one correlated these earth tremors with the hydraulic fracturing process, there would be, not surprisingly, an extremely significant result -  an apparent smoking gun! However, all of these earthquakes have such a small magnitude that they are never felt at the surface, and are hence unimportant – in fact, a smoking pop-gun!

However, occasionally one earthquake might be big enough to be felt at the surface, but it would not materially alter the significance of the correlation. We must try to correlate problem earthquakes with hydraulic fracturing, but so far there are just too few for this to be possible (only two in the UK, and few in the USA where most of the bigger earthquakes associated with shale gas are not due to hydraulic fracturing, but the irresponsible and thankfully obsolescent habit of disposing of old fracking fluid by deep underground injection).

No comments:

Post a Comment