Managing Quality
Managing Quality and Standards
Managing teaching/learning and quality/standards to ensure there is no conflict of interest is something higher education in the UK has been faced with for many years. Doctoral College therefore follows the UK Quality Code for Higher Education advice and guidance established by the Quality Assurance Agency in November 2018 (www.qaa.ac.uk/quality-code).
External Expertise
The Expectations and Core practices of the UK Quality Code for Higher Education (Quality Code) are mandatory for higher education providers in all parts of the UK.
Terminology
External expertise: External experts are individuals who are not directly involved with a course and who can provide independent and impartial comment and input to a course design, its management, monitoring, evaluation and review. External experts will provide a level of independence that is important in decision-making and ensuring that quality and standards are met.
There are a variety of sources which a provider can access for external expertise. These include:
External examiners: Degree-awarding bodies engage external examiners to provide impartial and independent advice, as well as informative comment on the degree-awarding body’s standards and on student achievement in relation to those standards. External examiners confirm that the provider consistently and fairly implements their own policies and procedures to ensure the integrity and rigour of assessment practices. They also comment on the quality and standards of the courses in relation to the national standards and frameworks and comment on the reasonable comparability of standards achieved at other UK providers with whom the examiner has experience. External examiners also comment on good practice, and make recommendations for enhancement. External examiners will have sufficient standing, credibility and breadth of experience within the discipline to be able to command the respect of academic peers, and where appropriate, professional peers. External examiners do not contribute to delivery through teaching or any other direct capacity.
External advisers: Used to provide academic and professional expertise during the development and validation of new courses and at other relevant times. They can be called upon to provide academic, professional and industry/employer/business expertise to inform course design and to contribute to lecturing or teaching at the provider or in a professional setting, for example, workplace supervisors/ mentors for education, nursing, apprenticeships, and students on placement.
-
Professional, statutory and regulatory bodies (PSRBs) – PSRBs are a varied group of bodies, regulators and those with statutory authority over a profession or group of professionals. PSRBs may provide membership services and accredit or approve courses as confirmation that the courses meet their standards and expectations. PSRBs are recognised by employers; achievement of a PSRB-recognised course can be an essential requirement for entry to a particular role or occupation.
-
Employers and other external stakeholders – Providers are encouraged to engage with employers and other external stakeholders at all stages of course design, approval and review in order to ensure that their courses continue to be relevant and fit for purpose in line with employer needs. There may be mandatory external requirements to involve employers and other external stakeholders in course design, approval and review; providers need to ensure they meet all relevant requirements for external input into their courses. There may be specific groups with whom education providers are required to engage, for example, user groups for courses in, and related to, health and employers for apprenticeships.
-
Guest speakers and visiting academics – Providers can promote a wider engagement with guest speakers and visiting academics to support and enhance the overall student learning experience. This type of external expertise can provide students and staff with first-hand experience in a specialist area and facilitate students’ motivation. It can also help to promote opportunities for networking, and improve community relations and connections between the higher education sector, industry and businesses.
-
Students and alumni – Students and alumni from similar, and different, courses can provide useful input to course development, evaluation and review. They can also be invited to meet with applicants and new students to share their experiences and manage expectations.
Conflicts of interest: Conflicts may arise during engagement or tenure, where providers reasonably believe the interests of one party affect the motivations or impartiality of another. This might include close personal or professional relationships with staff, students, or previous experts involved in their area of responsibility, an excessive influence due to their standing in other roles, or because their inclusion in a team of experts creates an unbalanced view. A conflict of interest might build up over time because of an excessive engagement period or re-engagement by the same provider.
Reciprocal arrangements: These may arise during the engagement or tenure of an external expert, if a reasonable person would interpret one service in exchange for another. For example, if one provider engages an expert from another provider and then provides an expert to the same or closely-related provision in return.
Expectations and Practices
The Quality Code requires providers to seek external, impartial and independent academic and/or professional expertise ensuring that the standards and quality of a providers’ courses cohere with the relevant national qualifications framework, Subject Benchmark Statements, Characteristics Statements and any relevant professional or other requirements. Providers can also use external expertise to identify good practice in learning, teaching and assessment, areas for enhancement and to inform the continuous improvement of their courses.
Expectations for Standards
The academic standards of courses meet the requirements of the relevant national qualifications framework. External experts, including external advisers and external examiners, contribute to course design and approval, course review, setting and maintaining academic standards and alignment with the relevant national qualifications framework, Characteristics Statements and their alignment with Subject Benchmark Statements, and any other requirements, such as those from PSRBs or funding bodies.
The value of qualifications awarded to students at the point of qualification and over time is in line with sector-recognised standards. External examiners comment on the consistent application of the provider’s standards and confirm their alignment with UK-recognised standards in determining awards.
Core Practices:
-
The provider ensures that the threshold standards for its qualifications are consistent with the relevant national qualifications frameworks. In practice, this means that external experts contribute to course design, delivery and review and pay due attention to the requirements of the relevant national qualifications frameworks, Characteristics Statements and Subject Benchmark Statements. External examiners comment on the maintenance and application of academic standards through internal marking practices.
-
The provider ensures that students who are awarded qualifications have the opportunity to achieve standards beyond the threshold level that are reasonably comparable with those achieved in other UK providers. In practice, this means that external examiners, and other external experts, comment on whether students have the opportunity to achieve standards beyond the threshold level.
-
Where a provider works in partnership with other organisations, it has in place effective arrangements to ensure that the standards of its awards are credible and secure irrespective of where or how courses are delivered or who delivers them. In practice, this means that the external examiner role for provision in partnership with other organisations is consistent with the degree-awarding body’s approved practices and, where appropriate, consideration is given to comparison of cohorts across location and provider.
-
The provider uses external expertise, assessment and classification processes that are reliable, fair and transparent. In practice, this means that providers ensure that appropriate criteria are applied in the engagement of external experts, paying due attention to the relevant expertise of each and the avoidance of conflicts of interest. External examiners comment on the setting, maintenance and application of academic standards through rigorous assessment processes, to the best of their professional knowledge.
Common Practices:
-
The provider reviews its core practices for standards regularly and uses the outcomes to drive improvement and enhancement. In practice, this means that external experts are used to contribute to reviews within a provider and to comment on areas of good practice, innovation and enhancement.
Expectation for Quality
Courses are well-designed, provide a high-quality academic experience for all students and enable a student’s achievement to be reliably assessed. External experts, including relevant PSRBs, employers and other external stakeholders, contribute to course design and review. External examiners play an important role in identifying good practice and making recommendations for enhancement of assessment policies and procedures.
Key Practices:
-
The provider designs and/or delivers high-quality courses. In practice, this means that external experts such as advisers and, where relevant, PSRB and employer requirements, inform course design and approval, and course review. Course design and review involves consideration of all elements contributing to the learning journey including staff and resources to deliver a high-quality academic experience.
-
Where a provider works in partnership with other organisations, it has in place effective arrangements to ensure that the academic experience is high-quality irrespective of where or how courses are delivered or who delivers them. In practice, this means that at course design, approval and review, and as part of ongoing monitoring, external expertise forms part of the evidence by which providers assure the quality of the student learning experience when working in partnership with other organisations.
Common Practice:
-
The provider’s approach to managing quality takes account of external expertise. In practice, this means that at course design, approval and review, and as part of ongoing monitoring, external expertise forms part of the evidence by which providers manage the quality of the student academic experience. The provider ensures external advisers are engaged to advise on new and revised aspects of provision which may have a substantial impact on the quality of student learning opportunities. This includes new policies, or major changes to these, for example, on student support or access to learning resources.
Guiding principles
The guiding principles given here are not mandatory for any provider. They are a concise expression of the fundamental practices of the higher education sector, based on the experience of a wide range of providers. They are intended as a framework for providers to consider when establishing new or looking at existing higher education provision. They are not exhaustive and there will be other ways for providers to meet their requirements.
-
Providers use one or more external experts as advisers to provide impartial and independent scrutiny on the approval and review of all provision that leads to the award of credit or a qualification. Providers should make use of a range of external experts appropriate to the provision that is being developed or that is under review. These advisers may include PSRBs, employers, subject experts, past students, students and academic staff from other courses, subject librarians, care users, and so on. To ensure independence of decision-making, those involved with the submission of a proposal should not be the same as those who approve it.
-
Degree-awarding bodies engage independent external examiners to comment impartially and informatively on academic standards, student achievement and assessment processes for all provision that leads to the award of credit or a qualification. External examiners need to be experts on the subject matter and the intended mode(s) of delivery and they will be familiar with quality and standards and quality assurance in a higher education context. Where this cannot be achieved by one external examiner, or the workload is extensive, additional engagements may need to be made, for example, where a programme is accredited by a professional body there may be a requirement for one external examiner with academic expertise and another from professional practice.
-
Degree-awarding bodies have processes for the nomination, approval and engagement of external examiners and other independent external experts. Degree-awarding bodies provide clear guidance and procedures that set out the nomination process for the external examiners and other external advisers that need to be engaged to provide an appropriate range of expertise to inform the design and review of all of its provision. The approval and engagement processes are also clear and transparent, sufficiently robust and avoid conflicts of interest including reciprocal arrangements.
-
Providers ensure that the roles of those providing external expertise are clear to students, staff and other stakeholders. All providers will provide clear and transparent information on the roles of external experts (examiners and advisers) involved in the assurance of standards, maintenance and enhancement of the quality of the student learning experience. This information will be accessible to students, staff and other stakeholders.
-
Providers ensure that external experts are given sufficient and timely evidence and training to enable them to carry out their responsibilities. External examiners and external advisers will need to be supported by the provider to fully understand and appropriately fulfill the role that they are asked to undertake. This may include a range of training and guidance and may vary depending on any previous experience in the role. External experts will need access to a range of evidence to enable them to fulfill their role; providers should be clear on what evidence will be provided and ensure that it is made available in a timely manner in line with the requirements of the experts’ responsibilities.
-
Providers have effective mechanisms in place to provide a response to input from external examiners and external advisers. External examiners and external advisers offer an important service to providers to ensure that all provision can meet a range of requirements including the achievement of quality standards and quality assurance, alignment with external frameworks, standards and benchmarks and a range of other stakeholder needs. Providers will have effective measures in place to ensure that input from external experts is considered, and where appropriate is actioned, a timely and reasoned response is made to the experts on actions taken or not taken as a consequence.
Practical advice
This section provides practical, contextualised advice to providers on external expertise. Where relevant, we have indicated which guiding principles the advice will help you achieve. Please bear in mind that this guidance is illustrative, and intended to inform the approaches you consider and ultimately implement.
In line with the Core practice, ‘The provider actively engages students, individually and collectively, in the quality of their educational experience, providers can include student representation within their practices and procedures related to partnerships. Appropriate training and support will need to be given to enable students to fully engage with the activities.
This practical advice is aligned to the three key stages identified in Figure 1, which shows a course life cycle and where and how external expertise can be used in course design, quality and standards, and monitoring and review.
Design, development and approval (Guiding principles 1, 3, 4, 5, 6)
Course development and design benefits from the informal input of a range of external experts. This can include academic and professional services staff from other departments of the provider and from other providers. It may also include relevant expertise from employers, community organisations, students and alumni. Providers may choose to specify the nature and extent of informal external expertise required in their processes of course development and design.
Formal course approval requires the input of impartial, independent external expertise in the form of one or more academic external advisers and, where appropriate/required, one or more industry/ professional external advisers. External advisers must have sufficient knowledge and experience to contribute effectively to the approval of a new course. In particular, the academic external adviser must be able to provide meaningful comment on the academic standards of the course and their alignment with national qualification frameworks. The provider may ensure that relevant input from the external advisers is included in the formal record of course approval.
Experts who are ‘external’ to the course but ‘internal’ to the provider should also be included in the approval process. These will include those who are familiar with the quality assurance processes, student support services, learning resources, student representatives and any other relevant aspects for the course and the student learning experience associated with it.
Providers must ensure that all involved in the course approval procedures understand the role and remit of the external advisers. Providers must also assure themselves that any relevant additional requirements for external expertise in course design, development and approval procedures are satisfied, for example, requirements of PSRBs, that there are no conflicts of interest nor breaches of employment legislation in the engagement of external advisers.
Delivery and monitoring (Guiding principles 3, 5, 6)
External examiners
A degree-awarding body is responsible for the standards of its qualifications. The engagement of an external examiner with the relevant professional expertise and experience in higher education will provide assurance to the provider and other stakeholders that the academic standards and quality achieved are in accordance with national qualification frameworks and other requirements such as Characteristics Statements and Subject Benchmark Statements. An external examiner can also provide impartial and independent confirmation that the processes of the provider have been followed and that the assessment and classification processes are fair, reliable and transparent (See also Guiding principle 2.)
Engagement
The selection and engagement of an external examiner is explicitly the responsibility of the degree-awarding body, irrespective of whether all or part of the course is delivered by a partner, delivery organisation or support provider. Where a course is delivered in a number of different locations or modes of delivery (for example, full-time, part-time, block or off-site), the external examiners can be asked to comment on the consistency of quality, standards and achievements across the different cohorts. Policies managing the engagement of external examiners ensure that a degree-awarding body can demonstrate that their external examiners:
-
have a high degree of competence and experience in the fields covered by the course of study, or parts thereof, and have a good understanding of the UK higher education sector
-
are appropriately experienced in course design and student assessment at the level of the award
-
have the necessary academic experience and subject knowledge to assess standards in an effective manner, identify good practice and recommend enhancements to enable informed course development
-
are experienced in acting as an external examiner or are supported by the provider in undertaking their duties, for example, through training and mentoring
-
are impartial in judgement and wholly independent of the provider and its staff (including the governing body), and any relevant partners
-
do not personally benefit from any student outcomes, nor have any connection to any student being assessed
-
have had sufficient experience in quality assurance to enable them to discharge their role effectively
-
are drawn from a relevant variety of institutional or professional contexts and traditions in order that the course benefits from wide-ranging external scrutiny
-
comply with all relevant employment legislation, including safeguarding, as appropriate
-
normally hold a limited number of concurrent external examining engagements (for example either one or two).
Degree-awarding bodies have clear procedures for the early termination of external examiner contracts, on either side. Degree-awarding bodies have a systematic approach to managing external examining data records, including information about the provider’s staff acting as external examiners elsewhere. This will assist in assuring the independence and impartiality of the external examiner and the avoidance of conflicts of interest or reciprocal arrangements. At engagement, degree-awarding bodies make clear the process for external examiners to raise serious concerns about academic standards directly with the head of the degree-awarding body. The degree-awarding body also ensures that all external examiners are aware that, as a last resort and where the concern is systemic and not a one-off case of ineffective practice, the external examiner can raise the matter externally with the Quality Assurance Agency for Higher Education (QAA).
Guidance on when and how to raise such a concern can be found on these organisations’ websites.
Role:
External examiners are an important element in the ongoing monitoring of programmes. A key aspect of their role is the assurance of standards and processes. The role also includes analysis of data and reporting. Degree-awarding bodies also need to consider the feedback provided by external examiners and report on it. Providers and degree-awarding bodies should respond to this peer feedback, as well as identify and action any areas for enhancement. The information below provides more detail about the role and responsibilities of both the external examiner and the degree-awarding body.
A provider should develop policies and processes to enable external examiners to review and comment on:
-
the degree-awarding body’s standards and student performance in relation to those standards
-
the consistent and fair application of policies and procedures ensuring the integrity and rigour of academic practices
-
good practice and possible enhancements.
External examiners are full members of examination boards. External examiners submit a report at least annually to the degree-awarding body which provides clear and informative feedback to the provider on those areas defined for the role. In addition, their reports:
-
confirm that sufficient evidence was received to enable the role to be fulfilled (where evidence was insufficient, they give details)
-
state whether issues raised in previous reports have been, or are being, addressed to their satisfaction
-
address any issues as specifically required by any relevant professional body
-
give an overview of their term of office (when concluded).
The degree-awarding body ensures that external examiner reports are made available to students in full. External examiners receive formal responses from the degree-awarding body to their reports.
Providers demonstrate that feedback from external examiners has been considered and acted upon. At a senior committee level, it is established practice for degree-awarding bodies to consider a summary of external examiners’ responses annually, and this is reported to the appropriate academic authority. This enables them to draw out any themes or recurring recommendations, and ensure that these are fully addressed. Providers consider external examiner reports at an appropriately senior level. Where relevant, action plans may be written in response to the advice and contain a timeframe for their implementation.
External examiners for research degrees (Guiding principles 3, 4, 5, 6)
External examiners for research degrees are an essential part of the assessment of the award. They are engaged for their specialist knowledge, and research degree-awarding bodies may have additional engagement criteria to ensure that the external examiners have relevant qualifications and experience. Where the student is also a member of staff, additional external examiners may be engaged; the degree-awarding body should include requirements in their processes and guidelines.
Research degree-awarding bodies give the following careful consideration and provide clear guidance on the following areas for staff, students and research degree external examiners:
-
the criteria to be used in engaging external examiners
-
the criteria to be used for selecting external examiners when they have had previous affiliations with the research degree-awarding body or the provider
-
the format and submission of the external examiner report, whether there is a joint report from all the examiners and if this is made available to the student.
Guest speakers and visiting academics (Guiding principles 1, 4, 5, 6)
Guest speakers and visiting academics may be used by providers to deliver course content and may stem from a range of backgrounds. They may be involved in aspects of assessment processes where appropriate. Providers should ensure that their roles and responsibilities are clearly defined and agreed upon engagement/invitation. Relevant support and induction should be provided.
Employers and other external stakeholders (Guiding principles 1, 3, 4, 5, 6)
Employers and other external stakeholders might be invited to be guest speakers and workplace mentors or otherwise contribute to specific aspects of a course. Whatever their involvement, the degree-awarding body should ensure that respective roles and responsibilities are clear and that they provide adequate training and/or supervision. Employers and other external stakeholders are a valuable source for feedback and critical commentary; opportunities should be made available to seek and act on this data and show how it has been implemented, where relevant, in course design and delivery.
Professional, statutory and regulatory bodies (PSRBs) (Guiding principles 1, 3, 4, 5, 6)
PSRBs have a varied and wide-reaching involvement in the delivery and monitoring of provision. They may be involved with the delivery as guest speakers and review assessment and threshold standards regularly and via different mechanisms in line with the PSRB’s specific processes and requirements. PSRBs may also analyse data and report evidence. Degree-awarding bodies/providers should give a response to this peer feedback, as well as report on the evidence submitted and identify areas for enhancement. Providers should demonstrate that results have been considered and acted upon as part of a strategic approach to the enhancement of student learning opportunities and academic standards. Providers maintain records of courses with PSRB status, including a likely schedule of accreditation events. Providers ensure students are aware of the accreditation status and any conditions related to the attainment of professional recognitions.
Periodic/partner review (Guiding principles 1, 2, 3, 4, 5, 6)
Degree-awarding bodies undertake substantive reviews of their provision on a periodic basis. Where degree-awarding bodies work with another provider, the periodic review may be combined with a review of the partnership as a whole. Degree-awarding bodies may also combine their periodic review of provision with the review of a professional body, if appropriate. Degree-awarding bodies engage one or more experts who are external to the provider to provide impartial and independent advice for the periodic review of all provision that leads to the award of credit or a qualification of the degree-awarding body.
Examples include:
-
academic staff with expertise in appropriate subjects
-
experts from relevant sector networks, such as those concerned with developments in pedagogy, quality assurance or technology-enhanced learning
-
representatives of PSRBs.
Periodic review will also benefit from the input of experts who are external to the course development team, but who may be internal or have close ties to the provider. The aim is to involve all relevant stakeholders who can inform the provision and its development and contribute to enhancement by providing additional perspectives. Examples include:
-
staff from the provider who are external to the course/subject to be reviewed
-
employers, for example, through employer advisory groups or organisations in the communities with which the provider works
-
developers, library and learning resources staff, learning technologists, disability practitioners, and equality and diversity practitioners
-
former students, students studying in similar areas, or students’ union officers.
Policies and procedures relating to periodic review are clear on when external expertise is required, the nature of external expertise required, and how it is used. Providers apply policies consistently, explain how the provider engages with external expertise and how the engagement leads to: (a) confidence in the maintenance of academic standards and (b) the enhancement of student learning opportunities.
Providers and degree-awarding bodies have processes for the engagement of external experts that are clear, robust and support providers to demonstrate that where appropriate:
-
external experts are sufficiently impartial and independent, and have the necessary professional experience and knowledge to effectively contribute to the periodic review of courses, including identifying good practice and recommending enhancements within their area of expertise
-
external experts understand the context of UK higher education
-
an appropriate range of external experts are involved in quality assurance processes, including expertise in working with partners for those involved in partner review
-
external experts comply with all relevant employment legislation, including safeguarding as appropriate.
Providers develop role descriptors for external experts engaged for periodic and partner review processes. Appropriate training and guidance provided through, for example, workshops, guidance notes and briefings, will assist them in understanding and performing their role.
Periodic reviews include consideration of sufficient evidence to allow the objectives of the review to be met. The evidence required will vary depending on the provision being reviewed but may include:
-
course and curriculum documentation
-
statistics demonstrating student progression and achievement data since the last review
-
external examiner reports
-
student handbooks
-
access to virtual learning environment or other digital platforms
-
meetings with a range of students and staff
-
academic regulations
-
student feedback and student survey outcomes
-
assessment criteria
-
monitoring reports.
Reporting mechanisms, to the appropriate academic authority, demonstrate that the relevant external expertise has been considered and acted upon in periodic/partner review. Providers encourage and support their own staff to act as external experts for other providers.
Monitoring and evaluation
In addition to the comments noted within the individual sections, providers should also see the Monitoring and Evaluation Theme.
Key considerations for this Theme include:
-
systematic use of feedback from a range of external experts, including external examiners, PSRBs, employers and other sector groups to drive enhancement
-
external examiner verification that awards are sound and the process safe
-
robust policies and procedures governing the engagement and contribution to quality assurance of external experts
-
clear information accessible to all stakeholders.