Guest post: How inferential science works and why it matters
Guest post by James Garnett.
How inferential science works, episode one: the null hypothesis.
Ever wonder what things like medical studies are actually showing, and why they are sometimes (often?) disproved?
Inferential studies attempt to demonstrate a correlation between two things, generally speaking. That correlation is stated in a way that can be tested, through what is called a null hypothesis. Think of it as the default assumption. For example, in simple (aka not rigorous) terms: “the amount of cholesterol in the food that a person consumes is correlated to the amount of cholesterol present in their blood”. A statement of that nature can be tested, and disproved.
But null hypotheses cannot be proved. There are simply too many factors at play. It’s sort of like jury trials in the USA: we don’t prove someone innocent, we prove them “not guilty”. You can never prove innocence, you can only show that you don’t have evidence to prove guilt.
Moreover, a proper null hypothesis can be very hard to formulate. The example that I used above is a bad one, for example, because there are different types of cholesterol (among other reasons).
So a lot of studies start out with improper null hypotheses, and review plus later studies show their results to be unreliable. (Plus there is this problem of pressure to show positive results, rather than negative ones. Nobody cares if a scientist shows that bird feathers don’t cause prostate cancer, to indulge in a bit of hyperbole. But negative results matter.)
This is why the idea that “GMO’s are bad” is a faulty starting point. Which GMO’s? What methods of engineering? You have to be specific. If you’re not being specific, you’re reacting upon emotion and intuition—that is not science. It may be a starting point for science, but it’s not conclusive or reasonable in and of itself. You may believe that corn seed variety X that is GMO is bad, but what does that say about the oil from GMO olives? Nothing. Specificity matters.
This is one of the major reasons why GMO labeling is a bad idea, and why it keeps losing when brought up during popular election initiatives. The labels don’t tell you anything of value—they only play upon your fears.
And yes, I’ve been called a “shill for Monsanto” for stating this kind of opinion in the past! Still waiting for my first Shill Royalty Check, though.
All farmed foods are genetically modified, the only difference is that GMO’s are lab created whereas most of our food is selectively bred one sowing at a time. Any changes engineered in a lab can be replicated by selective breeding the old-fashioned way, it just takes a lot longer.
It’s general ignorance and fear of science that scares people about GMO’s; they don’t even realise that horticulture is itself a science, but rather think that traditional farming is natural and safe whereas lab-made changes are unnatural and dangerous.
Acolyte of Sagan, “it just takes a lot longer” could be a feature rather than a bug. It gives more time for side-effects to become apparent before the new product is in widespread use. My worry about GMO’s is that, based on experience with the pharmaceutical industry, we can’t trust big business to look rigorously for side-effects or to report them honestly when found.
Well the other problem with selective breeding is that it selects for other things as well (thus all the associated problems with pure bred animals). With genetic engineering we get the benefits of targeting discreet traits while also potentially installing traits that could not have otherwise be imparted with selective breeding.
As a result we’ve got Polar apples, golden rice, and BT corn, which is good because the world’s population continues to explode.
David Evans, How so?
Lets take wheat breeding, but any similar crop would be a good example. over the breeding cycle to create a new variety the plant breeder initially selects on fairly arbitrary agronomy criteria. Stem to short or too tall? Stem breaks easily? Seed head to small? Seeds drop out too easily or not easily enough? Looks nice or nasty? susceptible to know regional diseases. You get the idea. It’s a long list and a lot of the items used as selection criteria are very subjective.
Around about year 4 you get enough seed to sacrifice half of it to do some simple functional tests. Somewhere between year 5 and 7 there will be enough seed to perform more complex tests such as demonstrating that you can bake a good quality (miniature) loaf of bread. Any crosses that make it this far will start to be bulked up for trials and eventual commercial release, during which time baking, protein and amylase tests are performed each year for each trial plot to determine whether the cross is actually going to be a commercial success.
What does not happen during this period is any testing to see if eating this cross will render you infertile, give you cancer, or make your willy drop off. For all anyone knows there may be a natural mutation that renders the plant dangerous to eat. That won’t be picked up because no one screens for such things. We assume the food to be safe to eat.
Getting to commercial release can take anywhere from 10-14 years from first cross. This is such a commercial burden that many countries cannot run their own breeding programs and instead rely heavily on basic stock provided by others, with mixed success. The costs are compounded by the fact that commercial cultivars generally do not last more than around 10 years, by which time the performance ‘in the wild’ has begun to drop off and disease begins to become a problem again.
I have to admit to some sneaking (concerns about GMO food (with no foundation whatsoever) and far greater concern about the business practices of companies such as Monsanto and big agriculture generally (with some foundation). However, I have no actual rational basis to question the food safety of any GMO crop I know of in current release. On the contrary, as noted by Blood Knight, there are examples of GMO crops that serve as distinct examples of foods that are safe, economic and beneficial.
Yes, lets be a little cautious. Yes, lets clean up some of the poor agricultural and business practices in the industry. But lets also admit that a rational understanding of the mechanism and specificity of the change induced using GM methods is at least no more dangerous than completely randomly induced widespread modifications induced by traditional breeding.
No. No no no no no. The null hypothesis is not “the default hypothesis” in the sense you mean, and it is not difficult to formulate because you don’t have to because it is always the same: that there is no correlation. No, none, nothing, nada, null.
If your hypothesis is that X correlates with Y, you do an experiment not to prove that hypothesis, but hoping to _fail_ to get results that support the null hypothesis. Yes, I know it sounds backwards, but this is _important_. It’s why “significant at a .05 level” doesn’t mean that there’s a 95 % chance you’re right, it means that if X and Y are uncorrelated in the population/world as a whole, you’d only expect that experimental result 5 % of the time. Stating it like that is important because it reminds you that you should expect a result like that in 1 in 20 tests (not 20 experiments, but 20 instances of checking if an X and Y correlate) even if it’s all random, so you’re bound to get a lot of false positives.
amrie@5: no. H_0 speaks to the formulation of a relationship between the two variables under consideration. There is no requirement that it state that there be a negative correlation. H_0 can easily (and often does) state a positive relationship between the two variables. Do you think that the mathematics differs? You may be accustomed to always formulating H_0 as a negative in your own work, but that simply means that you’re accustomed to following a specific formula that that has worked in the past. It doesn’t mean that it’s the only formula that is right, and it also doesn’t mean that informal presentations of what it all means need to follow how you do things.
Too, the wordings of null hypotheses are notoriously difficult to get right. And your comment @6? Actually reinforces the point of the post.
MrFancyPants @7:
You’re confusingly using negative in “negative correlation” as lack of correlation, which is indeed the default H_0 in standard uses of frequentist test statistics (or the collateral and very similar “lack of difference between sample means”). I may add that you seem to have confused yourself too and this is not was amrie @5 & @6 was actually claiming. Their comments can be argued upon on minor grounds, but they are true with regard to the nature of statistics.
Also, amrie comment @6 does not reinforce the point of the post, since the point of the post actually reduces to argumentum ad voodoo statistics and is a weak argument against labelling.
Moreover, in my opinion, labelling is a fair option. Even if there’s no evidence that any GMO were harmfull (or even if it was demonstrated that no harm could ever result from their use), labelling option would allow consumers to make choices free of constraints and independently of truth statements about technoscience. And that’s just fair: people should have a right to decide for themselves, just like they have currently a choice about brand, size of packaging or expected content of the good (e.g. sugar free or nature of oil composition, etc. )
It’s nice that you feel entitled to be perceived as a “Monsanto shill”, this means you deserve it. If you were truly following the line of your argument that takes on GMO will be very case specific, then the correct position is that GMO will be good or will be less good in a quite case specific fashion. Not that one can free oneself and claims to be pro-GMO once and for all. I understand that this position is the consequence of strong partisanship within this important issue, with most anti-GMO arguments being fallacious or anti-science. But this leads to pro-GMO arguments being sometimes fallacious and actually anti-science too. There are important aspects of the issue that end up dismissed to the benefit of pro-GMO industry. If I were into plot theories, I’d say this is a really good marketting and ideological move from those who’ll get the monney (while none of the full-lie advertising arguments such as reducing starvation in the world will actually do it).
Rob @4 expressed doubts on economic consequences, but there are others, such as the kind of agriculture we want to have (and its environmental consequences), the future of agrodiversity (and future of breeding), the technoscience self exhaustion in seeking expensive solutions only when alternative choices might do it as well. The list of collaterals is actually longer than what proponents might admit to themselves. And this is merely if ever discussed in the current debate framed simplistically into a either/or binary.
#7: a negative correlation is still a correlation. If you think blondes have more fun, and want to test that, the null hypothesis is not that they have _less_ fun, but that hair colour has nothing to do with how much fun you have.
L: The problem with labeling is that the existence of a label has become a red-flag for people, disconnected from any significance of the label itself. In particular, food labels have an established function of alerting people to potential allergens, dangerous elements within the food, dietary information and other safety issues. Ignoring that context is a disingenuous approach to the situation.
If we must have GMO labeling, I propose that the label read, “This food enhanced with SCIENCE!”
Freemage:
This is a much stronger argument to discuss labelling itself. I hope you realise it.
I find it amazing that between the original argument and mine, you chose to claim it’s mine that is disingenuous. I wonder why. (Nah, I know why, I’m just kidding).
I can find at least one scientifically sound argument against your take, just for the sake of being contrarian: when you take a gene from an organism which will alter the nutritious quality of food because you alter the protein composition (yes, these are restrictions to the argument, but we are willing to discuss rationally and fairly, aren’t we?) which basically translates with a certain protein finding itself in higher concentration than in the initial product. Even if the original protein wasn’t triggering allergies you cannot claim that it won’t in the GMO because it wasn’t first allergen. There’s a simple reason for that: the three domains of life (to simplify) -microbes, plants, animals, differ quite a lot in how structural post transcription modifications are processed within each group: the nature and rate of methylation, the addition and nature of sugars in glycoproteins etc. And guess what? these are known processes that translates as food allergies. So we should clearly test whether post transcription processes will alter allergenicity of the end product whenever they are transfered to another reign. Of course, it is a case specific example. An example showing that when we have already answered anything that we shouldn’t ask, it’s not science anymore.
The thing is, if all what you do is “pfft, you’re just anti-science, there’s no reason to investigate because we all know blablibla”, you’re just posing. Pro-science folks can be mere poseurs, yes. Why enquire and debate issues, since it’s just anti-science propaganda?
L: Your argument supports the idea of testing these things to determine if they are allergenic; it says nothing about whether or not labels are communicating useful information in the absence of those tests. The part where you’re being disingenuous is where you step up and claim that, in the absence of any evidence, we should be putting a scare-label on the food because it might, in some theoretical fashion, be potentially harmful, and that such labels will not be misinterpreted as being an absolute warning of demonstrable risk.
If this is the threshold for which we demand labels, then expect food to get very, very expensive, as each product is going to need a book attached to it.
If you want to increase product and health testing before a new GMO product is brought to market, I’m fine with that (I generally support more testing requirements, both before and after a product goes to market, in a general sense, be it food or toys or furniture). If an individual product is demonstrated to have an allergen similar to the ones we require allergy warning labels for (eggs, nuts, etc), then by all means, label it accordingly. But labeling the process, rather than the result, is short-sighted and foolish.
Freemage @12:
Yes, we disagree that the label is a bad move, quite strongly. Currently consumers can chose between organic or non-organic, and the organic label is anything but a scarecrow. Some people might be scared about the potential impact of pesticide residuals, and guess what, I won’t treat them as irrational. People may well be scared about GMOs, but it’s not as if there should be no reason for this. My personal take is that it’s completely the fault of the way the issue is framed: scientists asking for total trust while we accumulate cases where being cautious enough should have been the rule (contaminated blood scandal, mad cow disease, use of amiante in buildings etc.)
If you can’t communicate efficiently enough, it’s your problem, not because of uneducated masses.
We have many labels, organic, fair trade, environmentally friendly, health friendly. None of this cannot be said to be based upon non scary issues.
A label “This food enhanced with SCIENCE!” is foolish too and once more just marketting. There is a strong science of food conserve and food process. The result is proposing processed food that’s safer than it was in the past (when one think of can and preserve science), and safer to go with long term conservation. But processed food is also a danger in itself, just looking at health effects on people that cannot afford anything else than living on cans.
There’s isn’t any issue with science, it’s just a tool. But there are many issues about what use is made of it. It has a simple name, capitalism (today incarnated as financial capitalism). The whole anti-science frame is just serving seed and pesticide industries in the name of science. The GMO debate is just doing that, and I wouldn’t expect anything else than another disaster enrooted in unregulated use of technology gone bad.
The strength of science is not some secret autority power to be blindly trusted. It is critical thinking. The narratives about GMOs are not particularly based on critical thinking, else scientists would clearly state the constraints and limits of the technology. It’s both good and bad, and the system in which it’ll be used is not known to manage consequences in a fair way.