Peer review? Don't mind if I do.

The best peer reviewer I know.

I am an avid movie watcher. I love them. I quote them. I get lost in them. Finding Nemo is one of my favourites. As a kid, I fell in love with the story, the characters, and the world Pixar created.

What does this have to do with science? Pixar has an internal process called Braintrust that reflects a foundational building block of science - peer review (cue scary movie music - Jaws might be appropriate).

Getting others to critique the work that I poured my heart and soul into? No thank you, I shall pass. But part of me knows that it is a process well worth the discomfort. In Ed Catmull's Creativity, Inc., he describes Pixar's Braintrust meetings, where the team gathers to provide honest feedback on films. "Put smart, passionate people in a room together, charge them with identifying and solving problems, and encourage them to be candid with one another" (1).

The goal isn't to tear down, it's to foster excellence. We cannot do our best work entirely by ourselves. Guidance from those who have been there, done that, or from those outside of our area of expertise offers new insights, creative problem solving, and better results. Finding Nemo is a gem because many brains worked together to iterate until they created something magical.

Now, let's say a scientist discovers something super cool, like a hair product that controls bangs (I'm talking a big impact finding), how do we know it's "real"? The person who made the discovery can be biased (even if they used objective evidence to draw their conclusion).

So we use peer review to assess their work. Other experts in the field come together as reviewers to assess the study. Normally at least 3 other experts, but I've had more.

If the work is submitted to a good quality journal, then the invited reviewers will be well respected academics who can evaluate the work. They might ask questions like, were all the right controls included? Is the sample size large enough? Did the researchers use the best statistical methods for what they are assessing?

The point is to properly vet the work before it is released to the world. If the study is not ready, it may be rejected or the team could be given suggestions from the reviewers to

make the study more robust, increasing the evidence to support their conclusions.

Peer review makes creative work better and science is a creative endeavour.

Papers that have not been peer reviewed, have not been properly evaluated. This means be cautious, be very cautious about their findings (cue the scary movie music, I see a shark fin in the water).

Are all peer reviewed papers accurate? No, unfortunately not. Peer review doesn't allow us to blindly trust scientific data. It's not a perfect solution, but it gets us started on the right track.

This is a big topic that we are just skimming the surface of. How should we tackle this? Well, I think we should "just keep swimming".

Until next time, dream big, love fiercely, laugh often, and sparkle.

References:

1. Catmull, Ed, and Amy Wallace. Creativity, Inc.: Overcoming the Unseen Forces That Stand in the Way of True Inspiration. Random House, 2014.