Validation of assessment – Vocational Education instruction

Instruction statement

This instruction describes how validation of vocational education assessment practices and judgements are planned, completed and reported in a systematic manner.

Validation is a quality review and improvement process. It involves checking that the assessment tool produces valid, reliable, sufficient, current, and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the relevant aspects of the Training Package or Accredited Course have been met.

It includes reviewing and making recommendation for future improvements to the assessment tool, process, and/or outcomes.

Objectives

This instruction ensures:

an organizational approach to validation based on a five-year schedule

validation is completed and recorded in a consistent manner across the organization

centralized monitoring and reporting of validation schedules and outcomes occurs

a continuous improvement approach to assessment and assessment judgements

validation is responsive to the risk profile of each program and

compliance requirements are met as detailed under the Standard for RTOs 2015 – as detailed in the VE Validation Compliance Matrix.

Exclusions

Higher education programs

Research programs

Instruction steps and actions

Validation of assessment for Vocational Education (VE) comprises the following stages:

Development of the RMIT VE validation schedule

Determination of the statistically valid sample

Validator competence verification

Validation of assessment tools and judgements

Implementation of recommendations

Reporting of validation recommendations and completed actions.

1. Development of the RMIT VE validation schedule

Action

Responsibility

Timeline

1. The VE validation schedule (the Schedule) will include all training products listed on the Scope of Registration. The schedule will describe:

the national code and program name

who will be responsible for ensuring validation occurs (Lead Validator)

the year in which the training product will undergo validation

a risk rating and

the School/College the training product belongs to.

Office of Dean, Learning and Teaching (ODLT)

Annually

2. The Schedule will be reviewed and updated annually to reflect changes to the Scope of Registration.

The Academic Registrar’s Group (ARG) advises changes to the Scope of Registration (additions and removals) to the ODLT.

ARG

ODLT

Annually

3. The risk rating of each qualification will be supplied from each program area to the ODLT using the pro forma supplied to ensure consistency of information.

Information from Information from Global Quality, Regulation and Compliance (GQRC) will be used to assist in this process where possible.

Specific risk criteria to be evaluated at the Program level include:

Is the program delivered outside Australia?

Is it delivered to international students in Australia?

Does it have a licensed outcome?

It is regulated by industry?

Is it delivered over multiple sites?

Does it have large enrolment numbers ?

Is it delivered to apprentices or trainees?

Is it delivered to students under 18?

Is it delivered via third party / partnership arrangements?

Is it delivered using various modes of delivery for different cohorts? (i.e. online, face to face, workplace)

Has it been superseded or is it being taught out?

ODLT

Colleges/Schools

CQRC

Annually

4. The Lead Validator will be the Head of School or nominee for each program. This person will be identified on the Schedule.

Colleges/Schools; Head of School or nominee

Annually

5. The Schedule will be sorted to list programs with the highest risk to lowest risk. 50% of programs on the Schedule will be highlighted for validation within the next three years. High risk programs may be validated more regularly. This will be determined through consultation with Schools.

ODLT

Annually

6. The Schedule will be communicated to Colleges and Schools annually or when the risk profile of a program changes requiring the Schedule to be amended.

The Schedule will also be tabled at the VE Committee for approval and then Education Committee for noting.

ODLT

Annually

7. Colleges and Schools will then plan their validation activities in accordance with the Schedule on their program validation plan, i.e. all training products listed to be validated within that year.

It is the responsibility of the Colleges to ensure validation activities and resulting actions are completed before the next delivery of the training product.

Colleges/Schools

Annually

8. It is the responsibility of the Colleges to ensure the required records are generated and maintained for all validation activities including resulting actions.

Colleges/Schools

Annually

9. A report on outcomes and continuous improvement activities should be reported to the Course Assessment Committee / Program Assessment Board.

Colleges/Schools

Annually

A report to confirm the completion of validation activities, outcomes and follow up activities will be sent to the VE Committee for noting twice a year.

Colleges/Schools

Annually

2. Determining the statistically valid sample

Action

Responsibility

Timeline

1. At least two units from each qualification will be validated once every five years. More units may be validated if desired.

Enough assessments must be validated to ensure that the results of the validation process are accurate and are representative of the total completed assessments for the training product within a specific time frame. High risk qualifications may be validated more frequently.

Colleges/Schools

Lead Validator

As per the Schedule

2. In order to select these (at least) two units, consideration should be given to the risk profile of each individual unit. For instance, units that are prerequisites, have OH&S components or other high risk components should be given a high priority for validation.

The units to be validated for each program are detailed on the program validation plan. [Note: the program validation plan only needs to be completed for those qualifications selected for validation on an annual basis.]

Colleges/Schools

Lead Validator

As per the Schedule

3. For each unit to be validated, a statistically valid number of assessment decisions must be selected for validation. This is achieved as follows:

1. Identify the number of assessment decisions by using the class size and the number of assessment tasks: e.g., class size = 60, three tasks i.e. 3 x 60 = 180 assessment decisions.

2. Using the tool provided at: http://www.raosoft.com/samplesize.html with a confidence rate of 95% and an error rate of 5%, enter the number of assessment decisions to find out the sample size.

Colleges/Schools

Lead Validator

As per the Schedule

3. Validator competence verification

Action

Responsibility

Timeline

1. The Lead Validator is responsible for ensuring validation is completed by correctly qualified staff before the start of the validation process.

Validation can be completed by one person or by a team of people. Staff completing validation must collectively have:

Vocational competencies and current industry skills relevant to the assessment being validated

Current knowledge and skills in vocational teaching and learning and

The TAE40110 Certificate IV in Training and Assessment or the TAESS00001 Assessors Skills Set.

This information is collected and retained at RMIT on the staff qualifications matrix.

The VE validation checklist and report will include an acknowledgement that those participating in the validation activity hold the correct competencies.

Validators may be RMIT staff (ongoing or sessional) or from external organisations i.e. from industry. Collectively they must hold the qualifications listed above.

The trainer and assessor who delivered or assessed the training product being validated:

can participate in the validation process as part of a team

cannot conduct the validation on his/her own

cannot determine the validation outcome for any assessments judgements they made, and

cannot be the Lead Validator in the assessment team.

Given these rules, it is suggested that validation is completed in the following way:

in teams with a minimum of two staff, collectively holding the required competencies

in teams with a minimum of two staff, where one staff member may be the trainer/assessor involved in delivering/assessing the training product being validated

involves industry/employer representation where possible.

It is strongly encouraged that validation is completed using a broad cross section of teaching staff as possible.

Lead Validator

As per the Schedule

4. Validation of assessment tools and judgement

Action

Responsibility

Timeline

Validation must consider only the assessment evidence retained within the sample. While validators may discuss the assessment process with the assessor who conducted the assessment, only the evidence collected, retained and recorded will support a validation decision.

4.1 Prepare for the validation process

Complete VE validation checklist and report – part A

4.2 Validation of the assessment tool

Complete VE validation checklist and report – part B

4.3 Validation of assessment judgments

Complete VE validation checklist and report – part C

4.4 Report the outcomes of the Validation process

Complete VE validation checklist and report – part D

Lead Validator

Lead Validator

Lead Validator

Lead Validator

As per the Schedule

5. Implementing recommendations

Action

Responsibility

Timeline

Make changes to the assessment strategy and tools as per the recommendations from the validation process.

Document improvements in the program log and update course guides.

A record of the implementation of improvements must also be retained by the School. These may also include:

minutes of meetings and discussions recording actions taken in response to data from consultation with learners, enterprise clients, industry organisations and licensing bodies

revised assessment tools, strategies and related documentation.

Lead Validator

Following validation – before next offering.

6. Reporting of recommendations and actions completed

Action

Responsibility

Timeline

1. The completed VE validation checklist and report must be reported to the Course Assessment Committee and Program Assessment Board. It may also be reported as part of the Program Annual Review process.

This document should be available upon request.

Head of School

Upon completion of validation process.

[Next: Supporting documents and information]