New publication: Metaresearch for Evaluating Reproducibility in Ecology and Evolution

Over the last few years we have learned a lot about the reliability of scientific evidence in a range of fields through large scale ‘meta-research’ projects. Such projects take a scientific approach to studying science itself, and can help shed light on whether science is progressing in the cumulative fashion we might hope for.

One well known meta-research example is The Reproducibility Project in Psychology. A group of 270 psychological scientists embarked on a worldwide collaboration to undertake a full direct replication of  100 published studies, in order to test the average reliability of findings. Results showed over half of those 100 replications failed to produce the same results as the original. Similar studies have been conducted in other fields too—biomedicine, economics —with equally disappointing results.

It’s tempting to think that this kind of replication happens all the time. But it doesn’t. Studies of other disciplines tell us that only 1 in every 1,000 papers published is a true direct replication of previous research. The vast majority of published findings never face the challenge of replication.

As yet, there have not been any meta-research projects in ecology and evolution, so we don’t know whether the same low reproducibility rates plague our own discipline. In fact, it’s not just that the meta-research hasn’t been done yet, it is quite unlikely to ever happen, at least in the form of direct replication discussed above. This is because the spatial and temporal dependencies of ecological processes, the long time frames and other intrinsic features make direct replication attempts difficult at best, and often impossible.

But there are real reasons to be concerned about what that meta-research would show, if it was possible. The aspects of the scientific culture and practice that have been identified as direct causes of the reproducibility crisis in other disciplines exist in ecology and evolution too. For example, there’s a strong bias towards only publishing novel, original research which automatically pushes replication studies out of the publication cycle. The pragmatic difficulties of experimental and field research mean that the statistical power of those studies is often low, and yet there are a disproportionate number of ‘positive’ or ‘significant’ studies in the literature—another kind of publication bias towards ‘significant’ results. The rate of full data and material sharing in many journals is still low, despite this being one of the easiest and most obvious solutions to reproducibility problems.

In our paper, we argue that the pragmatic difficulties with direct replication projects shouldn’t scare ecologists and evolutionary biologists off the idea of meta-research projects altogether. We discuss other approaches that could be used for replicating ecological research. We also propose several specific projects that could serve as ‘proxies’ or indicator measures of the likely reproducibility of the ecological evidence base. Finally, we argue that it’s particularly important for the discipline to take measure to safe guard against the known causes of reproducibility problems, in order to maintain public confidence in the discipline, and the important evidence base it provides for important environmental and conservation decisions.

Paper citation:

Fidler F., Chee Y.E., Wintle B.C., Burgman M.A., McCarthy M.A., Gordon A. (2017) Meta-research for Evaluating Reproducibility in Ecology and Evolution. Bioscience. doi: 10.1093/biosci/biw159l available at https://doi.org/10.1093/biosci/biw159

Advertisements

ICSRG at the Banksia Awards

Last week, I attended the Banksia Awards dinner in Sydney, hopeful of bringing home the Sustainable Cities Award for our entry Biodiversity Sensitive Urban Design.

gg-at-banksia-awards

ICSRG Researcher Georgia Garrard at the Banksia Awards

Unfortunately, we didn’t win – the gong was taken by 202020 Vision, who have been working towards a target of 20% more green space in Australian cities by 2020. But I thought I’d take a moment to reflect on some of the benefits of the experience, which lies outside of the day-to-day experiences of most researchers.

First, it was fun! We got to dress up in cocktail/lounge wear, which is otherwise pretty much non-existent for conservation researchers.

But jokes aside, although Awards like this are not necessarily recognised by the reward structures we are used to, a number of potential benefits became pretty obvious very early on.  The Banksia Awards are held in very high regard by industry and local government.  Winning one (and perhaps even being a finalist, as we were) could be very helpful when trying to secure industry partners for research grants. In addition to (or perhaps BECAUSE of) this, recognition by the Banksia Foundation is an indication of the relevance of research beyond academia and therefore helpful in demonstrating research impact. And finally, the Awards attract a large number of applicants and I found it to be a great way of learning about what is going on in my own field OUTSIDE of academia. And there’s a LOT. The Awards dinner was a great opportunity to engage with a different group of people who are potential collaborators, stakeholders and end-users of our research.

Biodiversity Sensitive Urban Design by ICSRG researchers Georgia Garrard and Sarah Bekessy was a Finalist in the Banksia Sustainable Cities Award.  We’d like to acknowledge the great work done by other finalists and winners, as highlighted here.  We’d also like to thank The Myer Foundation and RMIT’s Centre for Urban Research for supporting our research and application.

How satisfied and motivated are landholders with conservation covenants?

plc_mh

Landholders who have a conservation covenant on the title of their property (sometimes known as “covenantors”) have taken on the responsibility of managing their land for nature.

As with many things, the enthusiasm of landholders to continually manage their land in ways that benefit biodiversity is driven in large part by why they got involved in the first place, and stay involved (their motivations) and how satisfied they are with participating (satisfaction).

Following a similar study in South Africa, our group has helped develop and send out a survey on motivations and satisfaction to covenantors across Victoria, New South Wales and Tasmania, in conjunction with members of the Australian Land Conservation Alliance. It is part of a broader initiative to better understand how landholders feel about participating in private land conservation initatives, and will help guide the development of these programmes.

A summary of the results will be made available early next year, so stay tuned…

Oh, and if you are a covenantor in NSW, Victoria or Tasmania and you have been sent a link to the survey, we politely urge you to fill it out and have your say!

National Feral Cat Management Survey

Photo by Fredy MercayDid you know?

Australia has the highest extinction rate of native species on earth. The impact of feral cats has been recognised as one of the major threats to Australia’s native and endangered wildlife. In many cases, feral cats could be the final threat that causes a species to become extinct.

This is the first study of its kind.

Researchers from RMIT University are looking for volunteers to participate in a survey that asks questions about feral cat management in Australia. The information collected will help to generate a better understanding of feral cat management across the nation, including how to make improvements.

You can help by participating in this survey.

The survey should take around 5 mins. There are no identifying questions asked – you will remain anonymous. You will be asked a couple of non-identifying demographic questions (e.g., age range, occupation and State/Territory of residence). You are under no obligation to answer any questions.

If you would like to participate in this survey and help with this research please visit the following link: National Feral Cat Management Survey

Beyond Advocacy – a new take on the advocacy debate

Georgia will be presenting this work at the North American Congress for Conservation Biology next week in Madison, Wisconsin (Tuesday, 19th July, 8AM, Hall of Ideas Room E), please come along if you’re going to the conference.

James Kenyon Cartoon_signedLate last year, we published (in collaboration with colleagues from The University of Melbourne) an article in Conservation Letters, which we hope will open up a little more space for conservation scientists and ecologists to engage in public debates without the fear of being labelled an advocate and, by association, having their scientific credibility questioned.

We were motivated to write the paper by what we considered to be a general reluctance by conservation scientists to join public debates about conservation issues and policy.  Without the voices of scientists, public conversations about conservation are dominated by vested interest groups – business and industry on the one hand, and NGOs and lobby groups on the other.  As a result, public debate about these important issues is impoverished.

However, we believe that the reasons conservation scientists choose not to engage are in large part based on misconceptions about the relationship between scientific integrity and objectivity.  In our paper, we set out to unpack this relationship a little bit.  Our key point is that values have a role and a place in science. It is not possible nor advisable for an individual scientist to be value-free.

But thankfully, objectivity isn’t maintained by individuals. It is an emergent property of a collective.  And greater diversity in the scientific community helps to ensure scrutiny and self-correction.  So, in other words, objectivity is maintained by the whole community of scientists, not individual scientists or established statistical thresholds.

Once you accept this, many of the common arguments against advocacy by scientists (ie. that advocacy will damage your credibility, or that advocacy is outside the scope of science) simply don’t make sense.

Of course, it is not the case that ‘anything goes’ when it comes to advocacy by scientists.  There are some value judgements (eg. what is a tolerable level of extinction risk?) that can and should be disentangled from judgements that are more factual in nature (eg. what is the probability of extinction?).  And scientists should aim to avoid inadvertent advocacy (which occurs when a scientist presents personal preference as a scientific judgement) or advocacy by stealth (in which values are deliberately dressed up as facts).

Drawing on precedents in medicine and the social sciences, we provide some guidance for scientists and science in general for responsible advocacy in order to reclaim some space for scientists to engage in informed public debate about conservation issues, in a way that does not deny their value-system.

What is the fate of Victoria’s flower-strewn plains?

ISCRG’s Georgia Garrard and Sarah Bekessy discuss the fate of Victoria’s native grasslands as part of The Conversation’s Ecocheck series.

L1050112

The native grasslands of the Victorian Volcanic Plain are one of Australia’s most endangered ecosystems. Productive and fertile, these grasslands were quickly converted to grazing pastures by early European settlers, and a notable degradation in their quality was documented by the beginning of the 20th century.  Since then, the addition of fertilisers, and clearing for cropping and development have led to further losses. Now, less than 1% of the original extent of these native grasslands remains.

Native grasslands are intriguing ecosystems. Historically, they provided habitat for a wide array of native animals, including rufous bettongs and eastern barred bandicoots, and were an important food source for Aboriginal people. Today, native grasslands are still home to fascinating native species, such as the grassland earless dragon and striped legless lizard, and native wildflowers continue produce a dazzling array of colour during spring (although you might have to get up close to see them!).

Conservation of these systems must occur alongside human-dominated landuses, such as urban development and agriculture. Community engagement is critical. Grasslands in other parts of the world, such as North America’s prairies or the African savannah, are viewed with romanticism and awe. In the Australian consciousness, grasslands take a back seat to the mythical outback. But the future of the grasslands of southeastern Victoria may well depend on our capacity to generate the same public profile for this truly remarkable but critically endangered ecosystem.