I think it is fair to say that commenting on scientific papers is broken. And with commenting I mean online comments that are publicly available, not informal discussions in journal clubs or at meetings. This definition would include discussions of papers on social media such as Twitter or Facebook. Why do I think that commenting is broken?
- the number of papers with online comments is low. For PLOS Biology we have comments on the journal platform for 11% of articles, tweets for 14% of articles and Facebook activity for 22% of articles (Fenner, 2013). The numbers for Twitter and Facebook are much higher for more recently published articles, but are nowhere close to every article having at least one comment.
- even though there is a fair amount of social media activity around articles, the quality of the discussion is varied. Twitter for example seems to work mostly as an alerting service for interesting articles with little more than the title of the article in the tweet text and not much discussion.
- when comments are made, they are really hard to find coming from the article. Unless they are made on the journal platform, or the publisher tracks article-level metrics and links out to these comments.
What can be done to address these issues, i.e. increase the number of comments, increase the depth of the discussion, and make it easier to link comments to articles? Some of the thoughts that I and others have had include the following:
On Tuesday the US National Library of Medicine launched PubMed Commons (http://ncbiinsights.ncbi.nlm.nih.gov/2013/10/22/pubmed-commons-a-new-forum-for-scientific-discourse/?) as a New Forum for Scientific Discourse:
We hope that PubMed Commons will leverage the social power of the internet to encourage constructive criticism and high quality discussions of scientific issues that will both enhance understanding and provide new avenues of collaboration within the community.
PubMed Commons is still a pilot project and in order to read or write comments you have to be a PubMed Commons participant and be signed in with your My NCBI account. PubMed Commons has some important features:
- PubMed is probably the place where most life sciences researchers search for literature. Having comments and discussion there makes perfect sense, and is probably a better place than a publisher platform that only targets particular journals. PubMed also has a reputation that is very different from social media tools that are popular, but not really familiar to most scientists.
- Access to PubMed Commons is restricted to researchers, and this is one strategy to have the comments focus on scientific discourse. It has to be seen whether the process of registering for PubMed Commons (which currently is a bit more involved than most commenting systems) is a barrier for scientists to take part in the discussion, or whether it generates an audience that makes it more likely that scientists contribute.
- For people signing in with their My NCBI account (I don’t know the percentage of PubMed users that do that on a regular basis), commenting is really easy and the interface is straightforward. The comment editor uses markdown, which makes it easy to format comments and to include links.
10/26/13: added link to the recently launched PeerJ Questions (http://blog.peerj.com/post/62886292466/peerj-questions-a-new-way-to-never-publish-forget?) which uses a question and answer format (thanks to Jason Hoyt for reminding me).
Fenner, M. (2013). What can article-level metrics do for you? PLoS Biol, 11(10), e1001687. doi:10.1371/journal.pbio.1001687 (http://doi.org/10.1371/journal.pbio.1001687?)
Carmeliet, P. (2005). Angiogenesis in life, disease and medicine. Nature
(7070), 932–936. https://doi.org/10.1038/nature04478
Contopoulos-Ioannidis, D. G., Alexiou, G. A., Gouvias, T. C., & Ioannidis, J. P. A. (2008). Life Cycle of Translational Research for Medical Interventions. Science
(5894), 1298–1299. https://doi.org/10.1126/science.1160622