Evaluate Guidelines
In view of the significant changes to teaching practice in 2020/21, Student Evaluation of Teaching (SET) and Student Evaluation of Module (SEM) were combined into a single SEM survey for one year as an interim measure.
At a meeting of the University’s Teaching and Learning Committee it was agreed that the SEM survey that ran in 2020/21 should be used in 2021/22 for one further interim year. The reason for this is that the Evaluate system is due to be replaced in 2022 and we wish to take this opportunity to have a complete re-think about the way in which we obtain and make use of student survey data, focusing more on “learning” than on “teaching”, and introducing plans relating to teacher development which will be relevant to promotion cases. The new approach should be in place for September 2022 to coincide with the implementation of the new software.
Although we hope that you will find this a positive development, we recognise that historically the evidence relating to teaching that was used in promotion cases relied on SET scores and that recent SET scores will continue to be absent. New guidance for the 2021/22 promotion round is now available through the HR website.
The Policy for the 'Collection and Use of Student Evaluation Surveys in 2020/21' will remain in use for 2021-22.
Senate decision regarding SET and SEM publication
At the meeting of Senate on 11 June 2014, Evaluate and the publication of quantitative data from Student Evaluation of Teaching (SET) and Student Evaluation of Modules (SEM) was discussed. It was agreed that SEM results should be published, but SET should not be publised. More recently Senate have agreed that SET results can be publised if schools/lecturers wish. Please see the Senate Minutes from 2014 for more information:
Senate Minutes
Guidelines
1.1 The Evaluate processes are designed so that the University can assure the quality of the education provided for students at Nottingham.
1.2 Academic staff are required to gather student evaluations on their teaching (SET). SET results are used for promotion, appraisal and management assessment procedures. From March 2020, due to the COVID-19 pandemic SET results are not being collected at UoN UK.
1.3 Schools are also required to gather feedback on their modules for curriculum development and make the results available to QAA Subject Review teams. Student Evaluation of modules (SEM) is one way of satisfying this requirement.
1.4 Student evaluations of their course (SEC) enables schools to develop the courses they offer, it is optional and not required for any other purpose. These data cannot currently be collected via the Evaluate system.
2. Staff selection for Student Evaluation of Teaching (SET) (N/A for from March 2020 for UoN UK)
2.1 Evidence from student evaluation of teaching is required for all University of Nottingham Research and Teaching staff who have either a full-time or part-time contract with the University.
2.2 All Research and teaching staff are to be evaluated at least once per academic year by students for the purposes of appraisal.
2.3 For promotion: Applicants for promotion are advised to build up a portfolio over several years so that they can provide evidence of their teaching performance, fairly reflecting the full range of that individual's teaching. Evidence may be derived from any relevant source, including: peer reviews, SET and SEM and comments from external examiners, Subject Reviewers and other assessors. Staff are encouraged to include comments setting this evidence in context.
2.4 It is accepted that the above arrangements for student evaluation of teaching can become difficult to apply where the bulk of teaching is delivered by large teams and where each individual makes a small contribution to modules. Please see the Evaluate Policy for further details.
2.5 Modules in which the teacher's contribution is so small that the collection of evidence would be disproportionately burdensome may be left aside from all the evaluation requirements set out in this document. In the first instance, please seek advice from the Evaluate Team.
2.6 A Head of School may require their staff to have their teaching formally evaluated by the students under the SET scheme.
3. Administration of SET & SEM
3.1 In cases where a teacher teaches a module in another school, the following terms apply. A 'guest teacher' is defined as a teacher who is teaching in a school other than their own. A 'lead school' is the school that takes responsibility for a module involving one or more guest teachers.
3.2 A Head of School may wish to include a 'guest teacher' in their evaluation programme.
3.3 Where there is a module run by more than one school, the 'lead school' takes responsibility for generating the questionnaires and evaluation of any teachers on it.
3.4 The head of the 'lead school' will receive the original SET summary report and is responsible for sending a copy to the 'guest teacher's' Head of School.
3.5 The administration of SET is conducted by the SETSEM coordinator(s) in a school or Student Services. If there is a change in the coordinator from the previous year then please ensure the Evaluate team are informed by emailing the helpline with the name, University username, email address and the School(s) that they are to create surveys for.
3.6 Only SETSEM coordinators will be able to access the selection, retrieval and storing facility for questions.
3.7 Administrative queries in the UK should be directed to the Evaluate Team.
4. When using Evaluate, teachers are asked to:
4.1 Ask students to read the questions carefully, giving special attention to the rating scale which ranges from Strongly Agree to Strongly Disagree. Strongly Agree is a positive response and Strongly Disagree a negative response to the question.
4.2 Assure students that their views are confidential and point out that the data are processed automatically.
4.3 Inform students that the teacher will read their comments.
4.4 Make clear that while they may make comments, favourable or unfavourable, responses containing personally offensive comments are unacceptable and will be deleted. Teachers in receipt of such comments should discuss this with their Head of School. Teachers are advised that they can also seek support from a University Dignity Advisor. For more information, please see the Dignity at Nottingham page.
5. Analysis of responses from SET / SEM Surveys
5.1 Student responses to the questionnaire are collected electronically and processed centrally. Schools can also add their own specific questions. The results of the school-specific questions will not form part of the University SET / SEM analysis but will be retained within the database for the School to view, download or publish.
5.2 Teachers and Heads of School are able to access a report summarising the analysis of responses for each teacher / module combination as soon as the survey has closed. These summaries can be obtained from (and will be retained within) the Evaluate (Blue Castle) system and can be used for promotion, appraisal and management assessment.
5.3 Where a teacher is 'guest teaching' and subject to the SET / SEM process outside of their school, the Head in the 'lead school' will receive the survey results. The guest teacher is responsible for ensuring that a copy is sent to their own Head of School.
5.4 Heads of Schools also have access to additional reports summarising all the evaluations that have taken place in their school in a particular academic year.
6. Student Evaluation of Modules (SEM)
6.1 SEM is required to gather feedback from students about modules for curriculum development. The completed questionnaires, and the analysis of module evaluations, must be made available to QAA Subject Reviewers. SEM should be implemented as part of a school's 'Course Review' strategy and further information about this can be found in the Quality Manual.
6.2 Schools may, if they choose, collect information about module evaluation at the same time as information concerning the performance of individual teachers (SET).
7. Student Evaluation of course (SEC)
7.1 Course evaluation enables schools to gather the views of students on each of their programmes as a whole, to identify strengths and weaknesses, overlaps and deficiencies. SEC can be used as part of a school's 'Course Review' strategy and further information about this can be found in the Quality Manual (see section 6, above). Gathering such information is optional and is not currently included as part of the Evaluate project.
8.1 Student Evaluation of Modules (SEM) results have to be published so that students can access SEM scores for modules they have attended and modules they are planning to register for. Schools can decide where they wish to publish their SEM results.
8.2 It is recommended that SEM evaluation is published in the context of a module or feedback report. This should be seen as an opportunity to engage with students who have completed the module as they are interested to see that their feedback has been read and perhaps acted upon. It is also an opportunity to engage with students who may be thinking of registering for the module and the report gives them information about whether the module will suit them. Evaluate produces a 'Moodle Report' for each module's SEM results. This report includes summary quantitative data and provides space for comments about the module. Once the module convenors comments have been added, the report can be saved and uploaded to Moodle.
8.3 Alongside individual results for each module, Schools / Departments might want to consider publishing an overall report on SEM evaluations. This could be published on the School content management system (CMS) or Moodle pages.
9.1 In September 2016, the SET questions were changed following consultation with staff and students. The changes were:
Changes to SET questions
Post Sept 2016 Questions | Pre Sept 2016 Questions |
X* is good at explaining things |
The teacher was an able communicator |
X's sessions are well structured |
The teacher retained my interest |
X is approachable |
The teacher was approachable |
X conveys enthusiasm for the subject |
Sessions were paced appropriately |
X's sessions encourage me to study |
Overall, this teacher assisted my learning |
Please provide any additional feedback on your tutor's teaching in this space |
Please comment on any positive or negative aspects of the teacher's delivery |
|
Please comment on any other positive or negative aspects of the lecturer's teaching |
* Where X is the teacher's [Given name] [Family name] e.g. Jane Doe
9.2. In September 2016, the rating scale was reversed. Previously a rating of 'Strongly agree' was a 1, so lower scores were more favourable. Following feedback from staff and students, and to keep Evaluate in line with the NSS and other surveys, this was reversed. Higher scores are now more favourable, with the 'best' score for a question being a 5 (Strongly agree) and the 'worst' score a 1 (Strongly disagree). Aggregate scores range from 25 to 5, where 25 is the best score and 5 is the worst.
9.3. In November 2017 the SEM questions were changed and increased from 5 to 7 questions. The changes are:
Changes to SEM questions
Post Nov 2017 Questions | Pre Nov 2017 Questions |
The module has provided me with opportunities to explore ideas or concepts in depth
|
The objectives of the module were made clear/available to me |
The module has challenged me to deliver my best work
|
The teaching methods used in the module help me to learn |
The module has been well organised and has been running smoothly
|
The method of assessment (exam, coursework, practical, etc) allows students to demonstrate what they have learned on the module
|
The resources in Moodle for this module have helped me to complete my work
|
The level of the module is about right
|
The criteria used in marking my work have been made clear to me
|
The workload on the module is about right
|
The workload on this module was reasonable for the number of credits
|
|
Overall, I am satisfied with this module
|
|
Please use the space below for any constructive comments about this module
|
Please use the space below for any constructive comments about this module
|
9.4. In November 2017 a new aggregate percentage was introduced.
For the 5 SET questions the aggregate percentage is calculated by:
(Aggregate score - 5) / 20 x 100% e.g. (24.50 - 5) / 20 x 100% = 97.5%
(Where aggregate score ranges from 5 to 25)
For the 7 SEM questions the aggregate percentage is calculated by:
(Aggregate score - 7) / 28 x 100% e.g. (29.15 - 7) / 28 x 100% = 79.1%
(Where aggregate score ranges from 7 to 35)
9.5 For 2020-21 and 2021-22, due to the Coronovirus pandemic, UoN UK stopped using SET surveys and the SEM questions were changed to:
1. The learning activities and resources are well organised and structured
2. The learning activities and resources are helping me to achieve the learning outcomes
3. The learning activities and resources have encouraged me to explore the topics further
4. The learning activities and resources are easily accessible
4a. You selected 'disagree' or 'strongly disagree' with this statement. Please tell us why.
5. I know how and where to get help and support for this module when I need it
6. The criteria for the assessments on this module are transparent and clearly explained
7. The workload associated with this module is manageable
7a. You selected 'disagree' or 'strongly disagree' with this statement. Please tell us why.
8. Please use the space below for any constructive comments about this module
SETSEM Coordinator Guides:
Student User Guides:
Contact Details
For guidance and further information, please contact the Evaluate team