Evaluate
Evaluate is a wireless 'in class' tool to capture Student Evaluation of Teaching (SET) and Student Evaluation of Modules (SEM) data. Evaluate has been available since September 2014 and is used on all 3 campuses.
In view of the significant changes to teaching practice due to Covid-19, in 2020/21 Student Evaluation of Teaching (SET) and Student Evaluation of Module (SEM) were combined into a single SEM survey as an interim measure for UNUK. UNM and UNNC have continued to use separate SET and SEM surveys.
The SEM survey that ran in 2020/21 and 2021/22 will continue to be used in 2022/23 for one further interim year, pending the replacement of the Evaluate system. In the meantime, a new approach is being developed to replace SET and SEM. This new approach will include the creation of Teacher Development e-Portfolios and Student Evaluation of Learning surveys.
Although we hope that you will find this a positive development, we recognise that historically the evidence relating to teaching that was used in promotion cases relied on SET scores and that recent SET scores will continue to be absent. New guidance for promotions is available through the HR website.
The UNUK SEM Policy for the 'Collection and Use of Student Evaluation Surveys in 2020/21' will remain in use for this academic year. This can be found in the Documents section of the Evaluate SharePoint site.
The policy for colleagues in China and Malaysia is available in the Quality Manual.
For further information about Evaluate, please see the Evaluate SharePoint site. This includes FAQs and user guides for staff and students.
A video has been produced about the importance of Evaluate for staff and students. This could be used in lectures to encourage students to complete the questionnaires.
Evaluate (Blue Castle) Frequently Asked Questions
What is the difference between student evaluation of teaching (SET) and student evaluation of modules (SEM)?
The University-wide Student Evaluation of Teaching (SET) is designed to gather feedback on all staff in their performance as teachers for the purposes of appraisal and promotion. (SET is not being used on the UK campus from March 2020 onwards.) Student Evaluation of Modules (SEM) enables Schools to collect information on their modules from students and discuss the outcomes freely amongst staff and students and to share the analysis with external reviewers. From September 2014, SET & SEM information has been created and stored in Evaluate via
https://bluecastle-uk-surveys.nottingham.ac.uk (UK),
https://bluecastle-cn-surveys.nottingham.ac.uk (China) and
https://bluecastle-my-surveys.nottingham.ac.uk (Malaysia).
What is the role of the SETSEM Coordinator?
Each school is asked to put forward a point of contact for SET/SEM in their School. There can be several SETSEM Coordinators for each school. Although the role of the coordinator varies from school to school, there are three primary functions:
- Point of contact: The coordinator is available to advise teachers in their school on SET / SEM policy issues.
- To create SET and/or SEM surveys: In accordance with University policy and School guidelines.
- Additional question usage: In addition to the fixed University questions, the SETSEM Coordinator will have access to add up to three additional School based questions to surveys. Approval from the Head of School must be sought before amendments to School-specific questions are made.
- Disseminate information: Regarding policy changes and amendments to current practice.
I am (or I am going to be) a SETSEM Coordinator and I can’t access Blue Castle / Evaluate – what do I do?
What should I do if I receive offensive comments?
When asking students to complete Evaluate, they should be informed that while they may make comments, favourable or unfavourable, any personally offensive comments are unacceptable and the data will be deleted. Staff in receipt of results with offensive comments are entitled to have the data deleted and should report the abuse to a person agreed by their school (e.g. Head of School, School Manager or Director of Learning and Teaching). If it is agreed that the comment is abusive, this should be reported to the Evaluate Team who can delete the comment and the associated ratings.
1. History of the scheme
Why do we need a University-wide scheme of student evaluation of teaching?
It is accepted that students must have the opportunity to give their views on teaching in the University. The adoption of a University-wide scheme is intended to promote fairness amongst all staff applying for promotion or for performance review. There is a need for some consistency in the evidence presented by all applicants.
How can the University enable its students to understand the scheme?
The video shown above can be used in teaching to highlight the importance of students providing their feedback. The
Student user guide also details how students can complete surveys. In addition, schools can discuss the scheme directly with their students through the Staff Student Consultative Committees and can include information about evaluation at Nottingham in their Student Handbook.
Does the University consider forms of evidence in addition to SET in the evaluation of teachers?
Student evaluations of teaching are only part of an individual's portfolio which is built up over several years. The portfolio should fairly reflect the full range of teaching. Additional evidence may include peer observation, award of grants to enhance teaching and learning, articles on teaching and learning in refereed journals, comments from external examiners, Subject Reviewers and other assessors.
How can SET results be linked to training and staff development?
SET results can be discussed during appraisal / ADC sessions where an individual's training and staff development needs can be reviewed. HR have provided some
guidelines identifying other forms of evidence of teaching.
2. Teaching, learning and assessment
Why are there compulsory questions?
These are questions that should provide an element of comparability across the University.
Is it possible to ask less than 3 questions in addition to the compulsory questions?
Yes. Schools will be able to add up to 3 optional questions. You can ask any number from 0-3 and each survey automatically contains the University compulsory questions.
Will this scheme encourage staff to set easier exam questions and coursework and mark them more leniently?
School systems for setting and maintaining standards, including moderation by external examiners, should discourage such a trend.
3. Administration of SET / SEM Surveys
Who should be a SETSEM coordinator?
SETSEM coordinators are now mainly based in Student Service Centres and create SET / SEM surveys for particular school(s).
What should the SETSEM coordinator do if the module code is not recognised when generating questionnaires?
As part of the survey creation process, the SETSEM coordinator should initially search for a module using the in-built search feature within the Evaluate system, either by using the module code or title. If the module exists in Campus Solutions but isn’t being displayed in Evaluate, please contact the
Evaluate Team. If it's known that the module doesn’t exist in Campus Solutions then a Non Standard survey should be created.
Can individual teachers generate their own surveys?
No. Only the school's SETSEM Coordinators can create surveys and add additional questions.
Can a school generate more than one set of optional questions?
Yes. For each survey there is the opportunity to add up to 3 optional questions. Any number of optional questions can be added to the school's question bank in Evaluate.
Can I have questions on the questionnaire that relate specifically to my teaching?
Yes, this is possible; however, they must be submitted for approval by your Head of School. Only the SETSEM Coordinator can add optional questions.
What should be done if the teacher cannot access the Evaluate system?
I’m team-teaching – can I still use Evaluate?
SET surveys can only be used to evaluate one teacher, so where modules are team-taught, one SET survey needs to be produced for each of the teachers. Students need to be reminded that they are evaluating a particular teacher’s input to a module. If a module has been taught by multiple teachers it is sometimes more appropriate to use module evaluation (SEM) alone.
4. Confidentiality
Are the results from the surveys confidential?
SET / SEM data are confidential however they will be made available to relevant groups and individuals to support a range of processes including (but not limited to) publication, promotion, staff development and ADC.
5. Feedback from SET and SEM
When will staff get their feedback?
Should the outcomes of both SET and SEM be discussed at an appraisal / ADC interview?
An appraisee would be expected to discuss the outcomes of both processes with his / her appraiser.
Should feedback from SET be given to students?
It is up to the teacher as to whether they discuss any specific or general points with students.
Should the feedback from SEM be published?
Yes. The feedback obtained from SEM surveys should be put onto the Moodle module page.