It is of great importance to have your judges evaluate consistently and effectively, and still show candidates appreciation for their involvement.

By setting all your judging criteria, your judges will have everything they need readily at hand during grading, and you’ll have all their scores consolidated in an easy overview. Not only that but with our feedback feature, you can quickly elaborate on the performance of each of the solutions with prewritten feedback as well as personal feedback. It is easy to set up.

Setting up Assessment Criteria

To get started head to the Case -> Criteria option in the dashboard. It is best if you create the criteria in the order, you would like your judges to assess them in.

• Click ‘Add’ to create a new criterion.

• Now select whether you want a Yes/No or Scale type criterion.

  • The Yes/no type is best suited for binary assessments.

  • The Scale type allows for a more nuanced grading, on a numbered scale from 1-10. With this type, you also have the option to write feedback.

• Fill in the name and description of what you wish the judges to assess. The description will be shown right above the voting scale.

• If you are creating a scale criterion, you have the option to add feedback to a range of scores for a criterion. Read more about that below.

• Judging criteria must be set up before the submission deadline. After the submission deadline, you will not be able to add new criteria.

Automatic Feedback

Candidates always appreciate feedback on their work, but it is often time-consuming to give relevant feedback to everybody. You are, however, able to resolve this, by using the feedback option for scaled criteria. This is how you do this.

When you have created the scaled criterion itself, click ‘Add’ in the automatic feedback section.

• Now you see a box where you can select the range of score you want to write feedback to. You decide how wide the ranges go, but you always must start with the lowest range, by clicking and dragging the point on the line.

• Now write the feedback.

• Click ‘Submit’ to add the feedback.

• Now repeat the process by hitting ‘Add’ and selecting the next range.

• Once you have written feedback for all values on the scale, you’ll be able to save the criterion with feedback, by hitting ‘Submit’.

Solutions Overview

As the votes come in, you can keep track in the Solutions overview. You’ll see each of your criteria and the average scores, team names and the numbers that the teams have been assigned to avoid any unconscious bias from the judges.

Sending Feedback

Under the feedback option, you will be able to send feedback to candidates. You do, however, need to decide whether you want to be able to always send feedback, or if it should require that all judges have graded all solutions. As you can only send feedback once and because this can’t be undone, we recommend that ‘Feedback always is disabled.

Be sure to review feedback before sending it out to candidates.

Before you send the feedback, we encourage you to review the feedback text by using the ‘Review Text’ button. When you are ready, hit ‘Send Feedback’.