APS Presentations

APS was in Chicago this year, so the replicators I have been supervising were out in full force.

Clinton Sanchez presented his replications of a study claiming that analytic thinking promotes religious disbelief. (10.1126/science.1215647). His manuscript is having a rough time, but we’re hoping it will be out soon. Clinton is now in a MA program in Clinical Counseling at DePaul. Data from his project is here: https://osf.io/qc6rh/

Elle Lehmann presented a poster of her replications of a studies showing that red enhances perceived attractiveness of men rating women (10.1037/0022-3514.95.5.1150) and women rating men (10.1037/a0019689) . Elle’s paper is in submission–she found little to no effect for either gender. She’s now working on a meta-anlaysis which has become quite a project, but really interesting. She has graduated and will be applying for a Fullbright in the fall. Data from here project is here: https://osf.io/j3fyq/

Last but not least Eileen Moery presented a poster of her replications of a study which claimed that organic food makes you morally judgemental (10.1177/1948550612447114). Eileen’s studies were recently published (10.1177/1948550616639649). She found little to no effect of organic food exposure on moral judgements. She’s starting an MA program in clinical psych at IIT in the fall!. Data from here project is here: https://osf.io/atkn7/

Photos came out a bit blurry (new phone, but crappy camera!).

Elle and me at her poster.
Elle and me at her poster.
Clinton and Elle at his poster.
Clinton and Elle at his poster.

Bibliography

Rest easy — organic food probably does not make you into a jerk…

My student Eileen Moery and I have a new paper out today in Social Psychology and Personality Science. It’s a replication paper that I’m quite proud of (10.1177/1948550616639649). It represents some evolution in how I’m supervising replication projects.

The new paper replicates a study purporting to show that being exposed to images of organic food produces a strong decrease in prosocial behavior and a strong up-tick in being morally judgmental (10.1177/1948550612447114). This is a potentially fascinating phenomenon–something like ‘moral licensing’, the ironic effect of good behavior fostering subsequent bad behavior.

The original paper caught fire and the media covered these findings extensively. Rush Limbaugh even crowed about them as evidence of liberal hypocrisy. I noticed the media coverage, and this is how the original study made it onto my ‘possible replication’ list. Eileen found it there, read the paper, and developed a fantastic honors project to put the initial study to the test.

For her project, Eileen contacted the original author to obtain the original materials. She planned and executed a large pre-registered replication attempt. She included a positive control (Retrospective Gambler’s task) so that if the main study ‘failed’ we would have a way to check if it was somehow her fault. She also devised a nice memory manipulation check to be sure that participants were attending to the study materials. She conducted the study and found little to no impact of organic food exposure on moral reasoning and little to no impact on prosocial behavior. She did find the expected outcome on the positive control, though–so sorry, doubters, this was not an example of researcher incompetence.

One of the things I don’t like about the current replication craze is the obsessive emphasis on sample size (this paper is not helping: (10.1177/0956797614567341)). Sure, it’s important to have good power to detect the effect of interest. But power is not the only reason a study can fail. And meta-analysis allows multiple low-power studies to be combined. So why be so darned focused on the informativeness of a single study? The key, it seems to me, is not to put all your eggs in one basket but rather to conduct a series of replications–trying different conditions, participant pools, etc. The pattern of effects across multiple smaller studies is, to my mind, far more informative than the effect found in a single but much larger study. I’m talking about you, verbal overshadowing (10.1177/1745691614545653)

Anyways, based on this philsophy, Eileen didn’t stop with 1 study. She conducted another larger study using Mechanical Turk. There are lots of legitimate concerns about MTurk, so we used the quality controls developed in Meg Cusack’s project (10.1371/journal.pone.0140806 )–screening out participants who don’t speak English natively, who take way too long or too short of a time to complete the study, etc. Despite all this care (and another successful positive control), Eileen still found that organic food produced about 0 change in moral judgments and prosocial behavior.

Still not finished, Eileen obtained permission to conduct her study at an organic food market in Oak Park. Her and I spent two very hot Saturday mornings measuring moral judgments in those arriving at or leaving from the market. We reasoned those leaving from had just bought organic food and should feel much more smug than those merely arriving or passing by. Yes, there are some problems of making this assumption–but again, it was the overall pattern across multiple studies we cared about. And the pattern was once again consistent but disappointing–only a very small difference in the expected direction.

Although Eileen and I were ready to call it quits at this point, our reviewers did not agree. They asked for one additional study with a regular participant pool. Eileen had graduated already, but I rolled up my sleeves and got it done. Fourth time, though, was not the charm–again there was little to no effect of organic food exposure.

With all that said and done, Eileen and I conducted a final meta-anlysis integrating our results. The journal would not actually allow us to report on the field study (too different!?), but across the other three studies we found that organic food exposure has little to no effect on moral judgments (d = 0.06, 95% CI [0.14, 0.26],N=377) and prosocial behavior (d=0.03, 95% CI [?0.17, 0.23],N=377).

So–what’s our major contribution to science? Well, I suppose we have now dispelled what in retrospect is a somewhat silly notion that organic food exposure could have a substantial impact on moral behavior. We are also contributing to the ongoing meta-science examining the reliability of our published research literature–it gives me no joy to say that this ongoing work is largely painting a relatively bleak picture. Finally, I hope that we have now gained enough experience with replication work to be (modestly) showing the way a bit. I hope the practices that are now becoming routine for my honors students (pre-registration, multiple studies, positive controls, careful quality controls, and synthesis through meta-analysis) will become routine in the rest of replication land. No, strike that–these are practices that should really be routine in psychology. Holding my breath.

Oh – an one other important thing about this paper–it was published in the same journal that published the original study. I think that’s exactly as it should be (journals should have to eat their own dog food). Obviously, though, this is exceptionally rare. I think it was quite daring for the journal to have published this replication, and I hope the good behavior of its editors are a model for others and a sign that things really are changing for the better.

Bibliography

Grant Awarded to study the mechanisms of sensitization maintenance and decay

Woot! The Slug Lab has just been awarded a 3-year R15 grant from NIH to study the transcriptional mechanisms of sensitization memory and decay. What does that mean? It means that Irina and I will continue to be working our a**’ off trying to understand what genes are activated as an animal stores a long term memory, and even more importantly, as a long-term memory is forgotten.

Here’s the screen grab from ERA commons:
grant*

Big thanks to our dedicated and amazing students, and to the incredibly supportive colleagues and administrators we have here at Dominican University. We’re looking forward to crushing it with this project.

*Technically, that’s not the actual notice of the award, but of our priority score from a peer review of our grant proposal by a panel of esteemed scientists in the field. We got the award letter via email last week.

Sluglab Strikes Again – New paper tracing dynamics of learning-induced changes in transcription

A nice way to wrap up 2014–we have a new paper out (25486125) where we trace learning-induced changes in transcription over time and over different location in the CNS. We think it’s a nice follow-up to the microarray paper, because:

  • We show that some transcriptional changes are likely occuring in interneurons and motor neurons, not just in the VC nociceptive sensory neurons.
  • We found some transcripts which, like Egr, are rapidly *and* persistently up-regulated by sensitization training (GlyT2, VPS36, and an uncharacterized protein known for now as LOC101862095). We’re interested in such transcripts because they could be related to memory maintenance
  • We were able to better test the notion that CREB supports memory maintenance. So far, our evidence continues to go against this hypothesis, with no long-lasting changes detected in the VC sensory neurons nor in the pedal ganglia.
  • As a methodological point, we found that microdissecting out the VC cluster really really improves signal:noise for identifying transcriptional changes induced by learning. This is exciting–most work on the molecular mechanisms of memory uses tissue samples representing homogenous cell types. Zooming in on a single cell type of known relevance for storing the memory really enhances the power of the analysis.
  • We re-rested the four novel transcripts identified in our microarray paper from earlier this year (25117657). All four validated again! Moreover, all 4 were specifically up-regulated in the VC nociceptors (and some elsewhere as well). Another good indication that we’re on the right track with our microarray approach.
  • Another 3 student co-authors on this paper! We’re especially proud of Sami, Catherine, and Saman.
  • The paper is free on PLOSE ONE: http://dx.plos.org/10.1371/journal.pone.0114481. Also, you can download our raw data to examine for yourself at the Open Science Framework: https://osf.io/ts9ea/.

    Bibliography

    New Publication – Microarray analysis of sensitization

    We’ve got a new paper out (25117657) with the first of what we hope will be a series of studies using microarray to track the transcriptional changes following long-term sensitization training. This paper looks at the changes that occur immediately (1 hour) after training. It provides lots of details and data to validate the microarray design we developed, but also identifies a set of 81 transcripts that are strongly regulated after learning. Best of all, for a microarray paper, we use a large sample size (n = 8) and show using a subset of transcripts that most generalize to a completely independent sample. Among the changes we fully validated are up-regulation of a c/ebp-gamma (what the what!?), a glycine transporter, and a subunit of ESCRTII. The rest of the gene list that we’re working on has some exciting possibilities, too.

    Another thing to be proud of, is our three student co-authors on the paper.

    The paper is free for the next 50 days via this link, then it goes behind a paywall for 305 days, then it will be in PubMedCentral for free again (strange, right?). All the raw data is available on the Open Science Framework: https://osf.io/8pgfh/.

    Bibliography

    Travel award for Gerry Holmes!

    Congratulations, Gerry!  Junior DU student, bio major, and all-around Slug Lab superstar won a Faculty for Undergraduate Neuroscience Travel Award to attend this year’s Society for Neuroscience Meeting in San Diego, CA.  It was a competitive and international field, but Gerry nabbed one of these prestigious awards to present her ongoing work on the transcriptional mechanisms of long-term habituation.  Shown in the photo on the left is Gerry (left), Dr C-J (middle) and lab alumni Kristine Bonnick (right) who visisted the poster and the meeting from Loma Linda medical school where she is now enrolled.  In the photo on the right, Dr. Bob makes an appearance.  Go sluglab!

    IMAG0774IMAG0776