Evaluation algorithm
Each of your questions has a maximum amount of points a candidate can receive for it.
For example, if a numeric input question has maximum 3 points and the candidate answers it correctly, they receive 3 points.
When calculating the test score, answers to questions are evaluated and for each answer two values are set:
Maximum score: the highest amount of points the candidate can receive for the answer
Obtained score: the amount of points the candidate received for the answer
The test score is then calculated as obtained points divided by maximum points.
How are different question types evaluated?
Free Text and Code Input Questions
Free text and code questions require manual evaluation. Until they are manually scored by a reviewer, they do not affect the test score. Close-ended questions are always scored automatically.
Here’s how scoring works step by step:
Before Manual Evaluation
All close-ended questions are automatically scored as soon as the test is submitted.
The score reflects only these questions at this stage, if open ended questions have been answered.
If a free text/code question is answered, it is not scored yet. It is pending manual evaluation.
If a free text/code question is left blank, it is scored as zero points and included in the score automatically.
🧠 Important: After submitting the test, candidates see an automatic initial score based only on questions that can be auto-graded. Once open-ended questions are manually reviewed, the score may go up or down depending on those evaluations.
After Manual Evaluation
Manually reviewed answers (free text/code) are added to the total score.
Example:
The test has 10 questions: 7 close-ended and 3 open-ended.
Candidate scores 7/7 on close-ended → Initial score = 100% (7/7).
After manual evaluation, 1 open-ended question is marked correct, 1 is marked as incorrect, and 1 is left ungraded → Final score = 8/9= 89%.
Single choice and picture choice
If the correct option is selected:
Candidate receives full points.
If any other option, or no option is selected:
Candidate receives zero points.
Multiple choice
For each correctly chosen option 1 mark is added
For each incorrectly chosen option half a mark is taken
If the total number of marks is less than 0, the answer receives no points. But if the total number of marks is more than 0, the number of marks is divided by the number of correct options and multiplied by the maximum points for the question.
Numeric input
If the answer is correct
Candidate receives full points.
If the answer is incorrect
Candidate receives zero points.
More details on how Multiple choice questions are evaluated:
The equation used is: points x [(number of correct options chosen) - 0.5 x (number of incorrect options chosen)] / (number of correct options)
If only incorrect options are chosen, then the candidate will not get a negative score. It will be zero.
The reason we take points away for incorrect answers more than non-selected answers is to make sure candidates don't select every option available without consequences. If this wasn't the case, selecting all options would result in answering a question with 100% accuracy because all correct answers would be selected.
Have any further questions? Feel free to reach out to our support via chat or email.