Ammonite have been instrumental doing presentations around the country showcasing the design principles behind the platform, especially to stakeholders in the construction industry.
In particular we focus on how Ammonite's Validity and Authenticity Technology (VAT) is used to support clause 1.8 of the Australian Skills Quality Authority (ASQA) standards on effective assessments, namely:
Addressing the principles of assessment including fairness, flexibility, validity and reliability
Applying the rules of evidence including authenticity and validity
Ammonite applies principles of assessment and competency based training in the design of its Learning Management System
We have created a short video (1:15) on how the design of Ammonite's Learning Management System (LMS) has incorporated competency based training alongside the principles of assessment to ensure learners to obtain the required skills, abilities and knowledge:
The approach can allow the students to have an unlimited number of attempts at questions without reducing the validity. This unique design feature ensures assessment is flexible to the individual learner by reflecting the learner's needs, and also has two built in safeguards:
There is a penalty system, so when the student gets a question wrong, they are taken back to a previous question and presented with a different variation to the questions they have already answered. Course designers with over 20 years of experience designing and delivering Vocational Education and Training assessments in this manner, worked in the development of Ammonite's penalty system to ensure it is a very effective deterrent to guessing.
The Ammonite system provides trainers/assessors with a graphical indicator showing them the level to which student struggled with the assessment and allows the trainer/assessor to investigate student's learning history to see exactly where they had difficulties. This allows the trainer/assessor to conduct a validation exercise with the student to ensure they comprehend the assessment as well as providing personalised learning and assessing in the subsequent training, should that be required. This also allows course designers to refine training materials and/or questions over time with incremental improvements if a number of learners had difficulties with the material.
Applying the principles of assessment
Five principles to consider when designing assessment to ensure it is meaningful, useful and honest include understanding the reasoning behind the task, making sure the assessment is valid and reliable, providing a common measure, ensuring the difficulty level aligns with the average candidates and ensuring grades summarise abilities (McAlpine 2002, 4-5).
The Ammonite LMS was designed to align with the principles of assessment.
Fairness - Every student receives questions on the same topic as each question is constructed based upon a key concept. This concept is tested in various ways, however, every student is required to answer each question correctly.
Reliability - Concept based questions more reliably test student knowledge. If the students were tested with different questions pulled from a pool of questions (eg. of a bank of 20 questions give each student six random questions), then the assessment is not fair. Some students might get easier questions, others harder questions. This also is not a reliable assessment, as each attempt can result if vastly different outcomes.
Validity - Concept based questions ensure students knowledge is valid. When a student gets an answer incorrect, the question needs banks of variations in order to not dilute the validity of the question. If the learner were to receive the same question then they'll know which answer is incorrect weakening the question, and increasing the chance the student can answer it correctly by guessing.
Learners achieve the outcome against specific standards
Competency based training ensures the learner applies the “combination of skills, abilities, and knowledge needed to perform a specific task” (U.S. Department of Education, 2001, p. 1). It focuses on an outcome that is measured against specific standards within an industry rather than against specific students (Smith 2010).
The Vocational Education and Training industry is based on competency based training. The ASQA Standards state:
In a competency based training environment, which is centred on demonstrated competence against industry-defined standards of performance rather than strict course durations, students aren't required to study for a specified number of weeks or months
Ammonite's LMS assessment design principles requires the students to understand the material presented before applying their knowledge, skills and abilities to complete the task. In line with competency based training principles students are not compared to one another, instead their outcome is measured against specific standards (Smith 2010).
Technology: Validity and Authenticity Technology
Ammonite’s Validity and Authenticity Technology means a student cannot simply get through the learning through random clicking. The technology discourages learners guessing answers by taking them back a stage. This ensures the learning questions to fully engage the learner with the task by learning the content and answering the questions with confidence.
This short video (2:04) provides a summary of Ammonite's Validity and Authenticity Technology.
Help Ammonite help your learners
References
Australian Skills Quality Authority. (n.d.). Clauses 1.8 to 1.12-Conduct effective assessment. https://www.asqa.gov.au/standards/training-assessment/clauses-1.8-to-1.12.
Australian Skills Quality Authority. (n.d.). Clauses 1.1 to 1.4 and 2.2-Implementing, monitoring and evaluating training and assessment strategies and practices. https://www.asqa.gov.au/standards/training-assessment/clauses-1.1-to-1.4-2.2.
McAlpine, M. (2002). Principles of assessment. Luton: CAA Centre, University of Luton.
Smith, E. (2010). A review of twenty years of competency‐based training in the Australian vocational education and training system. International journal of training and development, 14(1), 54-64.
U.S. Department of Education, National Center for Education Statistics. (2002). Defining and Assessing Learning: Exploring Competency-Based Initiatives, NCES 2002-159. Prepared by Jones, E., Voorhees, R., & Paulson, K., for the Council of the National Postsecondary Education Cooperative Working Group on Competency-Based Initiatives. Washington, DC.