In the administration the review process can be activated.
The review process can be activated under Administration. If activated, all questions follow a process to allow for quality assurance. As a prerequisite, you must first define so-called taxonomy level (subjects). All questions will be assigned to a subject. In addition, personnel using the Question Pool must be given access rights (teaching competence) for their teaching field. Only then the teaching fields will be visible. In the example below it is the subject "mathematics".
In the following graphic the review process is shown exemplarily.
The following steps are depicted in the question bank review process:
- Questions are created under "My Questions" or directly under the assigned subjects (or teaching field, competence...). The question has the status "Draft".
- The question is added to the review process, the status changes to "Review".
- During the review the question is assessed. Review is only possible for questions of other teachers of the same subject (or teaching field, competency...). The author of a question can not review the own questions.
- a) The star rating is below the defined lower limit. The question status changes to "Revision" and the question is put back to the author to be edited again. After editing the review process for this question can be started again.
b) The star rating is equal or above the defined lower limit. That also means the defined number of ratings per question is reached. The question status changes to "Final". - Questions with the status "Final" can be integrated in tests.
On the left the different parts of the review process are shown.
1) The subject Mathematics is visible under "My question bank". Here you can find all the questions of this subject you created yourself. The questions are listed in order of their status.
2) Under "Review" there is also the subject Mathematics. All questions of other authors of the subject Mathematics with the status "Review" are listed there. You can not see your own questions here, because authors can not review their own questions. The questions are ready to review.
3) If a question meets all the defined review process criteria its status changes to "Final". The block "Final" is only visible to you if you have the corresponding rights. Nevertheless questions can be reviewed as specified under 2).
Status
If you choose a subject under "My question bank" the review process with all its different status is shown (marked in red). By clicking on each status you can see which of your questions have this status.
In the review process questions may have one of the following states:
- Draft: The question is ready to be reviewed and linked to a subject, but not in the review process yet.
- Review: The question is being assessed by another teacher (not author of question).
- Revision: The question must be reworked. If a question rating is lower then the lower limit for positive decision, the question status is immediately changed to "Revision", even if the minimum number of ratings is not reached yet. The lower limit can be defined under Administration.
- Final: The question meets the review process criteria. Questions with this status survived the quality assurance and can now be integrated in tests.
- End of life: Question with the status “final” which are set to the status "end of life" by administrator or manager. These questions can be deleted or just left there. To edit the question again it has to be copied. The copy will have the status "draft" again, so the review process can start from the beginning.
Review question
Here you can review the question of another teacher from the same subject (teaching field). The question is "read only" mode, so you can view the question including score, feedback and metadata, but you can not edit the question.
Start the review with "Review question".
The following dialog appears:
Now an assessment can be made by giving a comment and rating via stars. If the rating is below the defined lower limit a comment is essential. So the author of the question is able to improve the question. With the button "Rate" the assessment is saved.
