By Michelle Marvier

As the world's most abundant and widespread pollinator, honey bees (Apis
mellifera) play a critical role in our food security and make an important
contribution to the human economy. Because of their global importance to
agriculture, there has been a great deal of consternation surrounding the
widespread recent decline of honey bee populations.

Amid the sometimes wild speculation about what might be causing these
declines, it has been suggested in several popular media outlets that pollen
from genetically engineered Bt crops might be poisoning honey bees2, 3. It
should be emphasized, however, that there are not any scientific
publications indicating that Bt crops have anything to do with honey bee
declines. All studies performed to date support the contention that Bt
toxins, called Cry proteins, are in fact toxic to only a narrow range of
insect groups. For example, the Bt transgenes that have so far been
incorporated into crops such as corn and canola are toxic to Lepidoptera
(butterflies and moths) or Coleoptera (beetles). Although there is the
possibility that these Cry proteins are toxic to certain taxa, such as
caddisflies (Trichoptera), that are closely related to the target group4,
hymenopterans (bees, wasps, and ants) are not closely related to beetles or
butterflies and moths, and in the laboratory no toxicity test has ever shown
Cry proteins to cause any harm to hymenopterans.


Nonetheless scare stories about Bt crops and bees keep circulating. The
question is whether, given the absence of evidence implicating Bt crops in
honey bee declines, is there anything more that can be done to eliminate
this worry once and for all? In order to fully lay to rest the worry about
Bt pollen and honey bee declines, there is indeed an additional analysis
that needs to be done?one that in a quantitative way "adds up" all evidence
from independent experiments that have assessed impacts of Bt toxins on
honey bees. This additional step, called meta-analysis, has gained
prominence in clinical trials and the medical arena, where one has to be
very careful before deciding that a treatment is relatively risk-free. The
crux of meta-analysis is the realization that an absence of significant
effects in a collection of individual studies is not necessarily as
convincing as it might first seem. The problem is that the individual risk
assessment or toxicity studies may be poorly replicated and thus have low
statistical power. For example, a study might expose three groups of honey
bees to a Cry protein incorporated into a standard diet and three groups of
honey bees to a control diet, lacking the Cry protein. No matter how many
honey bees are in each "group," the replication of the study is only n = 3.
With such low replication, only a large and very consistent difference
between the two treatments in the survival, development, or growth of the
honey bees could be detected as statistically significant. The weak
statistical power of these studies means that a finding of no significant
effect is not very convincing.


A sample size of just three replicates per treatment might sound
unrealistically low, but the reality is that three replicates meets the EPA
standards for assessing risks to honey bees and other nontarget invertebrate
species. Many studies do use more than three replicates, but in general the
level of replication used in industry studies for nontarget invertebrates
such as honey bees is only n = 2 ? 6 replicates per treatment5. However,
wouldn't it be reassuring if there were a dozen or so of these poorly
replicated studies, all indicating no significant effect of Bt pollen? The
answer is no. In fact, a simple tally of the results (the number that found,
versus didn't find, significant effects) from a collection of weak studies
is not much more convincing than the findings of each individual study on
its own. Among statisticians, such tallies are called "vote counts" and if
one thinks about it a bit, it is pretty obvious that even a dozen studies,
all with poor replication, finding no effect would not constitute convincing
evidence that no effect actually exists.


Meta-analysis to the rescue

Fortunately, meta-analysis provides an alternative to vote counting. By
statistically combining the observed differences between treatments and
controls across a group of independent studies, and weighting the results of
each experiment by the variance in the data, one comes up with an estimate
of the general effect size across experiments. This resultant effect size is
much richer than simply stating that 9 of 11 experiments or even 11 of 11
experiments found no significant impact on honey bee survival. It is
possible that, by applying meta-analysis to a set of poorly replicated
studies, a more reassuring picture may emerge. Of course, it is also
possible that a meta-analysis will draw out small but potentially
biologically important effects that went undetected by any individual study.


Meta-analysis of clinical trials has revolutionized health care, and in
ecology, meta-analysis has produced some of the clearest general evaluations
of predation, competition, and herbivory. In conservation of endangered
species, meta-analysis is just now being adopted as a way to assess the
effectiveness of alternative management actions. Risk assessment of
genetically modified crops should similarly embrace this paradigm.


To facilitate meta-analysis of risk assessments that have examined the
nontarget effects of Bt crops, my colleagues and I have created a searchable
database, publically available at [delphi.nceas.ucsb.edu].
Between April 2007 and February 2008, this database was queried by visitors
from 215 unique IP addresses, and we hope scientists around the world will
use the data to ask questions that summarize all of the evidence available
about the nontarget effects of Bt crops. Initial meta-analyses of the field
(as opposed to laboratory) studies included in this database have recently
been reported6, but to date there have been very few field studies that have
recorded the abundance of honey bees.


How about those honey bees?

The question of whether Bt crops might be contributing to honey bee declines
was clearly in need of a more definitive answer. To provide that answer, and
also to demonstrate the general utility of a meta-analysis approach for risk
assessment, my coauthors and I assembled and analyzed a collection of 39
independent assessments (appearing in a total of 25 separate publications or
unpublished industry reports) that examined the direct effects of Bt Cry
proteins on the survival on honey bee larvae and adults in a laboratory
setting7. Our meta-analysis of the data from these studies revealed no
adverse direct effects of Bt Cry proteins on the survival of either larval
or adult honey bees. I should note, however, that the studies synthesized in
our meta-analysis were all laboratory experiments, so there is still some
possibility that different results might be seen in the field, where the
stresses of weather, disease, and so forth might alter the susceptibility of
honey bees to Bt toxins. On the other hand, the studies from which we drew
these data were all so-called Tier I studies that exposed honey bees to
extremely high concentrations of Bt toxins?at least an order of magnitude
greater than the concentrations that bees would encounter in nature. Given
these caveats, we believe that our meta-analysis strongly supports the
conclusion that the Cry proteins expressed in the current generation of Bt
crops are unlikely to have adverse direct effects on honey bees.


Evidence-based risk assessment

We often hear about all the experience and trials and tests that have been
done to assess the safety of genetically modified crops, but there is no
single place for concerned citizens or even scientists to turn to in order
to see if they are themselves convinced by the accumulated evidence. We
believe that creating large open-access databases that include data from all
relevant risk assessment studies is the future of risk assessment. With such
databases at hand, the power of meta-analysis and of a global community of
scientists can be turned loose.


A formal synthesis of what scientists had learned about the effects of Cry
proteins for honey bees had been lacking until recently. Once we put
together the database, the meta-analysis was straightforward. However, there
are many other questions about impacts on other groups of organisms, or
concerning how effects might correlate with differences in everything from
body size and growth rates to the experimental methods used to assess the
risks. The construction of databases detailing the methods and results of
all sorts of risk assessment studies, followed by the creative application
of meta-analyses to these data, offers the clearest path to the sort of
transparent cost-benefit analyses that society deserves.


Meta-analyses have the potential to move the debate about the safety of
genetically modified crops beyond a situation in which competing sides argue
that "study X shows this" only to be countered with "yes, but studies y and
z show the opposite." Indeed, no single study should, by itself, be taken
too seriously until other studies have confirmed the findings. Yet there are
so many scientists doing so many different experiments and risk assessments
that the information has the potential to overwhelm decision makers or cause
the debate to zig-zag around. If meta-analyses and large databases of
completed studies were to become a routine part of risk assessment, then
there would not be the distraction of single experiments capturing media
attention and inappropriately alarming or comforting the public and policy
makers. An investment in the creation and maintenance of risk assessment
databases will have high payoff in terms of improved transparency, increased
public confidence in the process, and more rapid advancement of scientific
understanding.

The creation of the "Nontarget effects of Bt Crops" database was supported
by EPA grant CR-83214701 awarded to Michelle Marvier and Peter Kareiva of
Santa Clara University.

References
1. Stokstad E. (2007) The case of the empty hives. Science 316, 970-972

2. Latsch G. (2007) Collapsing colonies: are GM crops killing bees? Spiegel
Online International March 22, 2007

3. McDonald J. (2007) Could genetically modified crops be killing bees? San
Francisco Chronicle March 10, 2007, F4

4. Rosi-Marshall EJ, Tank JL, Royer TV, Whiles MR, Evans-White M, Chambers
C, Griffiths NA, Pokelsek J Stephen ML. (2007)
Toxins in transgenic crop byproducts may affect headwater stream ecosystems.
Proceedings of the National Academy of Sciences USA 104, 16204-16208

5. Marvier MA. (2002) Improving risk assessment for nontarget safety of
transgenic crops. Ecological Applications 12, 1119

6. Marvier M, McCreedy C, Regetz J Kareiva P. (2007) A meta-analysis of
effects of Bt cotton and maize on non-target invertebrates. Science 316,
1475-1477

7. Duan JJ, Marvier M, Huesing J, Dively G Huang ZY. (2008) A meta-analysis
of effects of Bt crops on honey bees (Hymenoptera: Apidae). PLoS One 3(1),
e1415. doi:10.1371/journal.pone.0001415

Michelle Marvier

Associate Professor of Biology and Environmental Studies
www.scu.edu