Information for panel members
The following information is provided to assist individual teaching award panel members to interpret peer review of teaching reports.
Who are the applicants in the peer review process?
The applicants are RMIT teachers applying for individual teaching awards.
Who are the reviewers in the peer review process and what is their role?
Our reviewers come mainly from academic schools, but also from the Education Portfolio and college offices. All reviewers – those recommended by their colleges and those from the Education Portfolio chosen by the Deputy Vice Chancellor Education – are selected based on their teaching and education credentials, including teaching experience, knowledge of learning and teaching, key positions in learning and teaching within the University, and awards for teaching. All peer reviewers must participate in peer reviewer training facilitated by the Education Portfolio before they join the list of approved reviewers.
Each peer review requires two reviewers, preferably one from the applicant’s college and the other a learning and teaching expert. Both are chosen by the Education Portfolio to conduct the review. Peer review reports are completed by the two reviewers observing the same one hour session of an applicant’s teaching. This follows a pre-observation meeting where the applicant explains to the reviewers the context of the teaching that will be observed.
A peer review report template [DOCX, 37 KB, 7p] is used by reviewers to record their observations of the session. The reviewer uses the dimensions of teaching outlined in the template to indicate whether they observed evidence against the dimensions and whether this evidence appeared to be effective teaching in the context of the session.
It is important to understand that the reviewer is not making a judgement about whether the applicant should receive a teaching award.
What is your role as a panel member?
As a panel member deciding on individual teaching awards, you are expected to use peer review reports to triangulate evidence of claims by the peer review applicant in their teaching award.
The peer review reports provide you with independent third party evidence of the applicant’s effectiveness in demonstrating activity against the dimensions of teaching. The reviewer has not been involved in the award application process, and is providing peer feedback for you to complement the student feedback that you will have in your data set for the applicant.
To assist you in triangulating the peer review report with the claims of the teaching award applicant, on the left hand side of the report you will see that the reviewer provides some evidence to illustrate why they selected the quantity and quality of examples for each relevant teaching dimension.
The peer review reports are not likely to be identical, but would normally be consistent with each other. If there is a significant difference in the reviewer reports a third report may be in your data set to allow you to better interpret the session observed.
The reviewee is also offered the opportunity to write a one page rejoinder based on the peer review reports. In this case, you will have the review reports and the rejoinder (if submitted) to assist you in your deliberations on the application from the reviewee.
Because reviewers only observe a one hour session, you would not expect a teacher to necessarily use all dimensions equally in the observed time. You should consider the overall pattern in the report, and use individual dimensions in relation to the claims made in the application.
Examples of peer review reports
There are three examples of peer review reports [PDF, 85 KB, 6 pages] that illustrate aspects of the interpretation process.
In example 2, you will observe that the reviewer has written comments against all ten dimensions. The reviewee is encouraged to advise the reviewers of particular dimensions that will not be addressed in the session to be observed (as in example 3) because they are not relevant. You will notice in all the examples that the reviewers have marked certain boxes on the right hand side of the report. These boxes are not numeric scales and should not be used as numbers. The reviewer is indicating whether they observed no, some, many or extensive examples against the particular dimension. Some, many and extensive are all appropriate indicators, but the prevalence of examples must be considered alongside their apparent effectiveness.
Thus, some examples that are very effective or exceptionally effective are a good indication of thoughtful teaching. However, extensive examples with no apparent effectiveness indicates that the reviewee is trying, but missing the mark in terms of effective teaching.
All the examples illustrate how the comments space under each “dimension of learning and teaching activity” should be used. Comments should be very succinct, with the aim of supporting the choices made in identifying the quantity and quality of evidence.