1. Differences and chance cause variationThe real world varies unpredictably. For some branches of science such as physics, the questions may be reduced to very simple experiments whose results are more straightforward to interpret. In Earth Sciences, as in Life Sciences, we cannot simplify the complexity of Nature, and hence scientific results may seem more open to interpretation. The important thing is to recognize that there is a natural complexity and variability and take that into account in the interpretation of scientific observations. There is, for example, a variation of the amount of natural methane in aquifer waters. We need to understand that before we can attribute methane in drinking water to shale gas extraction.
2. No measurement is exactPractically all measurements have some error; and let's be candid here, errors are not bad things but a recognition that there is a limit to what we can do.
Some things cannot be measured very well. Imagine you want to weigh 10 grammes of salt but you only have a 5 kg kitchen scale, the chances are that you might weigh out anything between 1 gram and 50 grammes even if you are careful as can be.
Some things can be measured with incredible accuracy: Arsenic in drinking water can be measured accurately to about 10 parts per trillion! (The US EPA sets its safety threshhold for arsenic in drinking water at 10 parts per billion because it believes that arsenic is cummulatively dangerous at higher levels and it knows it can accurately measure these amounts.)
The good news is that measurement errors can be quantified and quoted easily. You should NOT trust any measurement unless it has an associated measurement error especially if the argument rests on the value of the measurement.
3. Bias is rifeExperimental design or measuring devices may produce atypical results in certain circumstances. The corollary is that it is not sufficient to just take the results of a study, but to understand how it was carried out.
For example, a study of gas in drinking water may show that there is more methane within 100 m of a shale gas well. An interpreter (politician, activist, scientist) might then say "shale gas is leaking into the aquifer and contaminating it." This is wrong. There has been no distinction made between thermogenic methane (shale gas, formed at depth by heat) and biogenic methane (naturally occurring methane in aquifers formed at shallow layers by bacteria).
Perhaps if the analysis showed that most of the gas was biogenic (which is actually the case), the interpreter may then say "As the gas is biogenic it was not caused by shale gas extraction." This may also be wrong because the drilling, though not contaminating the aquifer with shale gas from a deep provenance, has disturbed shallow biogenic gas in a way that it has entered the aquifer temporarily.
The results of studies should be carefully studied for interpretation bias.
4. Bigger is usually better for sample sizeThe average taken from a large number of observations will usually be more informative than the average taken from a smaller number of observations. That is, as we accumulate evidence, our knowledge improves. The problem with shale gas is two-fold:
- Most of the concerns come from the practice of shale gas extraction in the USA.
- Almost no scientific studies have been carried out there, although the situation is slowly improving, most opinion is not based on evidence.
5. Correlation does not imply causationIt is tempting to assume that one pattern causes another. However, the correlation might be coincidental, or it might be a result of both patterns being caused by a third factor – a “confounding” or 'lurking' variable. For example, it is tempting to believe that methane exists in aquifers because of shale gas drilling, and it is important to find out if that is true. However, we have already seen that an inability to discriminate between two types of gas (thermogenic and biogenic) can lead to misinterpretation, and acts as a 'lurking' variable.
Early studies in the US were not very good because they had not measured the methane in aquifers before shale gas extraction started and hence could not be sure that what they were measuring was natural or as a result of the drilling. These studies relied on the association of a rise in groundwater methane close to wells (other more recent studies have also found the opposite).
However, even if it were true that there is a correlation between well position and high levels of groundwater methane it does not imply that the drilling caused the groundwater gas concentrations. It may simply be that the wells were placed to extract shale gas at a position where gas has been reaching the surface naturally for millions of years. In other words, a well placed well.
In this example correlation does not imply causation, though:
- causation may exist too - more study needs to be carried out if this is suspected, and
- if gas has been reaching the surface naturally (not caused by the drilling), how is this the case? Are there natural pathways, fractures and faults that ease the transport of the gas? A responsible producer would be using science to have the best solution to these questions to ensure that the drilling operations did not exacerbate the effect.