Author Archives: ascelin

New publication: Metaresearch for Evaluating Reproducibility in Ecology and Evolution

Over the last few years we have learned a lot about the reliability of scientific evidence in a range of fields through large scale ‘meta-research’ projects. Such projects take a scientific approach to studying science itself, and can help shed light on whether science is progressing in the cumulative fashion we might hope for.

One well known meta-research example is The Reproducibility Project in Psychology. A group of 270 psychological scientists embarked on a worldwide collaboration to undertake a full direct replication of  100 published studies, in order to test the average reliability of findings. Results showed over half of those 100 replications failed to produce the same results as the original. Similar studies have been conducted in other fields too—biomedicine, economics —with equally disappointing results.

It’s tempting to think that this kind of replication happens all the time. But it doesn’t. Studies of other disciplines tell us that only 1 in every 1,000 papers published is a true direct replication of previous research. The vast majority of published findings never face the challenge of replication.

As yet, there have not been any meta-research projects in ecology and evolution, so we don’t know whether the same low reproducibility rates plague our own discipline. In fact, it’s not just that the meta-research hasn’t been done yet, it is quite unlikely to ever happen, at least in the form of direct replication discussed above. This is because the spatial and temporal dependencies of ecological processes, the long time frames and other intrinsic features make direct replication attempts difficult at best, and often impossible.

But there are real reasons to be concerned about what that meta-research would show, if it was possible. The aspects of the scientific culture and practice that have been identified as direct causes of the reproducibility crisis in other disciplines exist in ecology and evolution too. For example, there’s a strong bias towards only publishing novel, original research which automatically pushes replication studies out of the publication cycle. The pragmatic difficulties of experimental and field research mean that the statistical power of those studies is often low, and yet there are a disproportionate number of ‘positive’ or ‘significant’ studies in the literature—another kind of publication bias towards ‘significant’ results. The rate of full data and material sharing in many journals is still low, despite this being one of the easiest and most obvious solutions to reproducibility problems.

In our paper, we argue that the pragmatic difficulties with direct replication projects shouldn’t scare ecologists and evolutionary biologists off the idea of meta-research projects altogether. We discuss other approaches that could be used for replicating ecological research. We also propose several specific projects that could serve as ‘proxies’ or indicator measures of the likely reproducibility of the ecological evidence base. Finally, we argue that it’s particularly important for the discipline to take measure to safe guard against the known causes of reproducibility problems, in order to maintain public confidence in the discipline, and the important evidence base it provides for important environmental and conservation decisions.

Paper citation:

Fidler F., Chee Y.E., Wintle B.C., Burgman M.A., McCarthy M.A., Gordon A. (2017) Meta-research for Evaluating Reproducibility in Ecology and Evolution. Bioscience. doi: 10.1093/biosci/biw159l available at https://doi.org/10.1093/biosci/biw159

Nature comment on biodiversity offsetting

Some colleagues and I have a Comment piece that has come out today in Nature. The article outlines the risks associated with using offsets to achieve pre-existing commitments, such as those to which nations have committed under the Convention on Biological Diversity and the World Heritage Convention.

We recommend that while it is often appropriate for offsets to create and manage new protected areas, these outcomes should be accounted for separately from progress towards existing commitments such as the Aichi targets, in order to avoid offsets simply replacing government funding for protected areas. We argue that future international agreements should require separate accounting of conservation gains that were possible only because of equivalent losses, and benefits from the new protected areas funded by offsets should always be reported alongside the losses that triggered their protection.

Here is the link to the article: http://www.nature.com/news/conservation-stop-misuse-of-biodiversity-offsets-1.18010.

Citation: Maron, M., Gordon, A., Mackey, B. G., Possingham, H. P. and Watson, J. E. M. 2015. Stop misuse of biodiversity offsets. Nature 523, 401–403; doi:10.1038/523401a

In addition the ABC has written an article on this topic which you can see here: http://www.abc.net.au/science/articles/2015/07/23/4278534.htm

Could perverse incentives undermine biodiversity offset policies?

We’ve just had a published in Journal of Applied Ecology that examines potential perverse incentives resulting from biodiversity offsetting. We outline some of the ways in which even best-practice offsetting could end up being bad for biodiversity, and discuss how to reduce the risks of perverse outcomes.

The paper is available here: http://onlinelibrary.wiley.com/doi/10.1111/1365-2664.12398/abstract

Elsevier has bought Mendeley!

Elsevier has bought Mendeley! Just when Menedely had ironed out most bugs and was getting really useful. This is possibly sad news. See their blog entry here and the public reaction in the comments!

There is more info in this blog post

I’m going to be investigating moving to Zotero as my main reference manager. It’s open source and unlikely to ever be purchased by a large multi-national. I’ve also heard good things about it.

Is driving an electric car really better for the environment?

Is driving an electric car really better for the environment? An interesting article in the Sunday Times argues that when a full life cycle assessment is done, electric cars are not so green after all. I’ve sometimes wondered about this. I’ve also heard others saying that driving a small mass produced petrol car produces less emissions then an electric car (over say 10 years) when you do a full life cycle assessment. Though I’m also aware that the technology used for electric cars will continue to improve and this might not always be the case…