I have been interested in peer review of literature searches for quite some time.  At the International Clinical Librarian Conference in 2017 I was intrigued by a conference poster which described risk of bias in published systematic reviews due to avoidable failures in searching. At the same event I attended an excellent workshop on assessing search strategies, where more fascinating, amusing and frankly shocking details of published search strategies were aired. Kate Misso and Janine Ross from Kleijnen took us through using the ROBIS tool[1] to judge risk of bias from common inadequacies found in reported search strategies in systematic reviews. Then we used the PRESS Evidence Based Checklist[2] to evaluate a search strategy.  Using what I’d learnt I facilitated a peer review programme at Doncaster & Bassetlaw where once a quarter we each did the same search and then looked our approaches to spot differences, similarities and discuss what worked and what didn’t etc. It was useful, if more than a little artificial, but often it was too hard to justify spending the time on a “fake” search when there were plenty of the real thing coming in thick and fast!

Last year I heard about an initiative taking place in the London region from a colleague down south and it sounded ideal – a structured buddying scheme complete with prompt sheet to support a discussion about real literature searches and search strategies.  There was no pressure to report to anyone else what was discussed, no-one listening in and judging, just an opportunity to talk tactics and sticky bits with someone who understands. The lovely London librarians sent me their documentation lock, stock and barrel, all I had to do was sort out a sign up link and an evaluation survey.

The Yorkshire & Humber network (YoHHLNet) committee were very supportive of the idea, so after lots of discussion with Helen Swales, Library Manager at Leeds Community NHS Trust (YoHHLNet’s CPD lead and my co-conspirator in this venture), I sent my first email to local library managers in November 2021.  I explained we would like to launch a literature search peer review buddy scheme across YoHHLNet as a way of helping librarians conducting searches to form connections to one another, encourage networking, and also to enable expertise sharing between individuals to improve the quality of their searches.  I approached managers first because I was acutely aware that not everyone who would benefit from an arrangement like this would necessarily volunteer without encouragement, either due to a lack of, or indeed an abundance of confidence, and I didn’t want people to miss out on the opportunity if a gentle push / inclusion in their CPD objectives might do the trick! 

I followed this up with a general email to all staff in Yorkshire & Humber region inviting them to sign up, and 16 people took the plunge.  I paired them up roughly in the order they signed up with only minimal tweaking necessary to avoid people working in the same organisation being paired together, and sent the pairs an email outlining the steps with a prompt sheet to guide the discussions and a link to a post meet up evaluation questionnaire.  In the event, two sets of pairs didn’t manage to meet up, and not everyone completed the evaluation, but 9 people responded to the survey; their feedback confirmed that the scheme had value and also that they would welcome it as a regular event.  Things that participants found particularly beneficial:

  • An opportunity to meet new colleagues and establish a connection
  • Having a framework to discuss searches in a supportive manner
  • Sharing ideas and swapping techniques, discussing problems with someone who understands
  • Identifying similarities and differences in search approaches

The things that people felt they would take away from the meeting were fascinating and varied, including:

  • More use of geographic search filters
  • Remembering to use Cochrane
  • Restructuring results formats
  • Future plans for collaborative work
  • Take on board helpful feedback.

From an administrative point of view it was really easy to sort out as all of the emails and documents were available as templates already.  It’s a scheme with multiple benefits – not least of which is a connection with a friendly fellow professional who, thanks to COVID, you haven’t had the chance to bump into over coffee at a face to face event in years. The feedback was overwhelmingly positive and round two for Spring 2022 is currently underway. If anyone regrets not taking part in rounds one and two, never fear, I expect further rounds to take place and details will be available via the general YLKS mailing list and on the YoHHLNet CPD webpage.  If anyone from YoHHLNet or further afield has queries, please don’t hesitate to contact me and I’ll happily evangelise in your direction. You have been warned!

Sarah Gardner
sarah.gardner11@nhs.net

Clinical / Outreach Librarian
Doncaster & Bassetlaw Teaching Hospitals NHS Foundation Trust

 

[1] Whiting, P., Savović, J., Higgins, J. P., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., Churchill, R., & ROBIS group (2016). ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of clinical epidemiology, 69, 225–234. https://doi.org/10.1016/j.jclinepi.2015.06.005

[2] McGowan, J., Sampson, M., & Lefebvre, C. (2010). An Evidence Based Checklist for the Peer Review of Electronic Search Strategies (PRESS EBC). Evidence Based Library and Information Practice, 5(1), 149–154. https://doi.org/10.18438/B8SG8R