The new version of RobotReviewer has been live on our website for a couple of months now, but we've just released all the code as open source, which is a good opportunity to write a post!
The goal of the RobotReviewer project is to speed up evidence-based medicine, by automating, or semi-automating the extraction of data from clinical trial reports.
To date we can to automatically extract information about bias (using the Cochrane Risk of Bias tool), and additionally can extract information about the participants, interventions, and outcomes studied in a trial. We are actively researching and adding new machine learning models to the system, and aim to extract all the pieces of information from a clinical trial needed to produce a systematic review.
The first version of RobotReviewer processed one paper at a time. The most important change is in how RobotReviewer can process multiple trial PDFs at once. This will allow us in future to move beyond summarising individual trials, and towards synthesis.
The first screen of the new RobotReviewer interface. Simply drag a bunch of PDFs of clinical trial reports to the dashed box to start the process
Thinking time... takes a minute or two
Then comes our new Report View, which is the main user interface. We've taken small steps towards synthesising the results from individual studies, with the Risk of Bias table being one example.
Automatic Risk of Bias tables! Note in the top right we now have download buttons, you can download the report in HTML, MS Word, or JSON formats.
In the background, RobotReviewer now automatically recognises the identity of an uploaded trial, and is able to retrieve relevant related information from PubMed, and ICTRP. For the moment, we simply use this to make nicely formatted citations, but clearly lots more is possible.
Further down the report, characteristics of the study population, interventions, and outcomes is extracted, plus the justifications for the judgements in the Risk of Bias table.
One key feature is that we retain links to the source PDF. One important problem in research synthesis is that the provenance of data is often not clear. By contrast, by clicking any of the PDF icons next to the extracted data, RobotReviewer will take you to the exact place in the PDF where the data came from.
the source PDF with the text describing the trial population highlighted in green
We hope you enjoyed this tour of our new RobotReviewer! We're excited by the possibilities our new framework gives us, and will post updates as they happen.
Our code is available here on GitHub.