Frequently Asked Questions.
PI Questions
What is required from the PI to start a new project?
  1. A brief description of the systematic review (abstract)
  2. Prospero registration link (if available)
  3. CSV (MS-DOS format) document containing your citation list. The CSV file should be formatted in seven columns titled: Paper_ID; Paper_Author; Paper_Year; Paper_Title; Paper_Abstract; Paper-Citation: PMID)
  4. Identify the number of assessments for abstracts and full text (i.e. how many reviewers do you want to review each citation)
  5. Inclusion and exclusion criteria for abstract and full-text screening.
  6. On your project page, you may also wish to include: (i) a description of the PI and research team; (ii) tips and advice for reviewers; (iii) whether you are offering any incentive to reviewers; and (iv) upload the systematic review protocol and search strategy
How do I upload the citation list to my project?

To upload your citation list to the project, you will need a CSV file containing the list of citations.

The CSV file should be saved as a CSV (MS-DOS) file.

The CSV file should be formatted to include seven columns:

1) Paper_ID

2) Paper_Author (the authors of the citation)

3) Paper_Year (the year the citation was published)

4) Paper_Title (the title of the citation)

5) Paper_Abstract (the abstract if available)

6) Paper-Citation (the full citation)

7) PMID) (PubMed ID number)

The citation list can be uploaded to your project in two ways:

On the Project Creation Page: Under the heading “File Uploads”. Click on the “Choose” button under “Select Abstracts”, select your CVS file and upload.

On the Project Management Page: In the Quick Actions tool bar , click on the “Upload Abstract” button (left-hand side), select your CSV file, then click on the green arrow to upload the file.

Can I test a reviewer's capacity before starting their review?
You can set up a Test Set of citations to assess a potential reviewer’s capacity before granting them access to the project. Usually 5-10 citations are enough for a Test Set. To prepare the Test Set:

  1. Upload your citation list (see FAQ above)
  2. On the Project Management page or from the Project Dashboard, click on “Edit Test Set”
  3. Assess the citations that you wish to include in your Test Set. You can use the Search Bar at the top of the screen if you have specific abstracts you wish to include in your test set.
  4. You can also enter comments in the “Screening Criteria Comments” box as you assess each citation to explain why you chose to exclude or retain. This information is fed back to the potential reviewer in real-time as they are completing the test set to help them understand how the screening criteria should be applied.
How can I provide the PDFs documents for the citations?

Once a citation is retained at the Title and Abstract level, it will appear under Upload Documents. Find the required PDF, then press “Upload” beside the citation. Once the PDF has been uploaded to the website, the citation will move forward to the Full Text screening level. 

Papers that were retained at Level 1 (Title and Abstract) will not be shown in Level 2 (Full Text) until a PDF has been attached.

On the Project Creation page, the PI needs to indicate the “Number of Assessments” and whether “Single Retain Moves Forward” for the abstract and full text screening level. What do these mean?
Numbers of assessments: This is how you indicate how many  you assessments you want for each citation. For example, if you want your citations to be reviewed in duplicate, you would indicate “2”.

Single Retain Moves Forward: Number of reviewer assessments indicating that the citation is to be retained before it is allowed to move to the full text screening level. PIs might indicate that “Yes”, a single retain moves the citation forward in order to allow the screening process to be more sensitive to eligible trials, however, this might increase the amount of work required in the next level.

Can I have someone join the project without completing the test set?

Yes. On the Project Creation page, you will have the option of creating a Secret Key. The Secret Key can be any combination of letters and/or numbers. Reviewers can request to join a project and “Apply with Key” instead of completing the test set, then enter the Secret Key to submit their request.

Once a reviewer has requested to join the project with the Secret Key, they will automatically be given access to Title and Abstract screening.

Alternately, in the Quick Actions toolbar on the Project Management page, you can invite reviewers to your project by entering their email address. Once they create an account and login, they will have access to Title and Abstract screening.

Reviewer Questions
Creating a Profile
When you first visit the website, you will see a button that says “Sign Up” near the top of the page. There is also a “Sign Up” option in the toolbar at the top of the web page. Click on the “Sign Up” button and you will be prompted to enter your name, email address, and to set up a username and password.

Once you have submitted your information, you will receive an email notification. You must confirm your request within the email notification before you are able to log in to the website.

Are reviewers evaluted before starting their review?
The PI of the project may set up a test set to evaluate reviewers before giving them access to the project. The test set will consist of a smaller number of citations and will give you a chance to evaluate the citations using the systematic review screening criteria.

After you assess each citation in the test set, you will see how your assessment compares to the assessment by the PI.

Accessing the Website

When you first login to the website, the screen that you see is your Dashboard. On your first access, you will not find any projects listed here. Once you join a review, it will show up on your dashboard. 

I have just signed-up to the website, how can I find projects to be involved in?

Once you login to the website, you will see a “Projects” button in the toolbar at the top of the web page. Clicking on “Projects” will bring up a list of existing projects. Clicking on a project title will bring you to the project homepage where you can read details about the systematic review and decide if it is a project you are interested in joining.

At the bottom of the project description, you will see a section titled “Test Set Link” Typically, you will be required to complete a test set before you can join a project. Click on the “Apply to this Project” button to start the test set. 

Alternately, if the PI has provided you with a Secret Key to join their project, you can select to “Apply with Key”

What should I do before starting on a project?
Before starting to assess citations, it is recommended that you read over all of the systematic review details on the project home page.

Depending on what is provided by the PI, you might be able to see an overall study goal, information about the PI and study team, comments and instructions for reviewers, the SR abstract, the full systematic review protocol and/or a Prospero link providing further details on the study.

Assessing Citations
Once you have been granted access to a project by a PI, the project will appear on your dashboard when you first log into the insightScope website.

From here, select “Title and Abstract” for Level 1 screening. Only the citations that you have not previously assessed would show a visible “Assess” button them. For Level 2 screening, select Full Text Review from the dashboard.

Which citations should I screen first?
Is recommended to start screening citations from the top of the table one by one. Citations are sorted to show the ones with the least input first, to make sure all get equal assessments.
What are the different stages of a project on insightScope?
Depending on the SR, there are the 5 stages:

Test Set: Reviewer assessment.

Title and Abstract: Includes screening only the abstract for specific criteria.

PDF Upload: all retained Title and abstracts are uploaded to the system

Full-Text Review: Includes reading the complete article (PDF).

Extract Data: retrieve all the documents retained and the analysis data of the SR

Will my name be shown to other reviewers?
Only your username will be shown, your full name will only appear to PIs.
I made a mistake while inputting one assessment. What should I do?

If you made a mistake while entering an assessment, click on the “Completed by You” tab. Use the search function to find the citation. Press the green “Completed” button beside the citation. You can change your answer and re-submit your assessment. 

I did not fully understand one of the screening criteria, and I have been using it incorrectly for 10 citations, what should I do?

If you made a mistake while entering an assessment, click on the “Completed by You” tab. Use the search function to find each citation. Press the green “Completed” button beside the citation. You can change your answer and re-submit your assessment.

You may also want to review the criteria and/or protocol, or discuss with the PI to make sure you now understand the criteria correctly.

What’s in it for me?

This is completely between you and the PI. Some PIs might offer monetary compensation for your work, while other systematic reviews are done on a voluntary basis. Sometimes the PI will indicate what they are offering to reviewers on the project’s homepage.

If you are involved in other parts of the research project (ie: Analysis, writing …), you might be able to become other on future manuscripts. For a definition on who qualifies for authorship, you can review the following link: ICMJE Authorship Criteria

Still need help? Send us a note!

For any other questions, please write us at kohearn@cheo.on.ca