© 2018 selection and editorial matter, David Boud, Rola Ajjawi, Phillip Dawson and Joanna Tai. A key skill to be mastered by graduates today is the ability to assess the quality of their own work, and the work of others. This book demonstrates how the higher education system might move away from a culture of unhelpful grades and rigid marking schemes, to focus instead on forms of feedback and assessment that develop the critical skills of its students. Tracing the historical and sociocultural development of evaluative judgement, and bringing together evidence and practice design from a range of disciplines, this book demystifies the concept of evaluative judgement and shows how it might be integrated and encouraged in a range of pedagogical contexts. Contributors develop various understandings of this often poorly understood concept and draw on their experience to showcase a toolbox of strategies including peer learning, self-regulated learning, self-assessment and the use of technologies. A key text for those working with students in the higher education system, Developing Evaluative Judgement in Higher Education will give readers the knowledge and confidence required to promote these much-needed skills when working with individual students and groups.

© David Boud, Ruch Cohen, Jane Sampson aud individual contributors. All rights reserved. While peer learning is often used informally by students - and for many can form an essential part of their HE experience - this book discusses methods of developing more effective learning through the systematic implementation of peer learning approaches.

© D Boud, G I Feletti and named contributors, 1997. All rights reserved. Problem-based learning is a way of constructing and teaching courses using problems as the stimulus and focus for student activity. This edition looks at the topic in the light of changes since the first edition (1991). There are new chapters on the impact of PBL, and inquiry and action learning.

© 1981, 1988 David Boud and named contributors. Although the focus will not be on teaching methods per se, but on strategies for fostering learning within the context of particular courses, various methods will be described. These include the use of learning contracts in which students prepare a formal plan for their learning and its evaluation which is validated by a staff member; one-to-one learning in which students work in pairs to facilitate each other’s learning; student-planned courses in which students work on their own and in groups to initiate their own projects and put them into practice; peer support systems in which newly arrived students are assisted in problems of personal and academic adjustment by students with longer experience of the institution; and collaborative assessment in which staff and students cooperate in establishing criteria for student assessment and make judgements on the basis of these. Each of these approaches is described in a particular context from the point of view of the teacher using the approach.

© 1999 David Boud and John Garrick and the contributors. Work now invariably requires a continual focus on learning: to improve productivity, to enhance the flexibility of employees and to develop and transform organizations. This volume brings together leading experts from the United States, Britain, Australia and New Zealand to critically evaluate the current debates on workplace learning and to propose directions for future developments in both research and practice. Topics covered include: * expectations of learning at work into the twenty-first century * learning theories, practice and performance implications * the relationship between workplace learning and other forms of lifelong learning * the international developments in competency-based approaches to learning and assessment * the influence of language, power, culture and gender upon the 'construction' of learning. Topical and informative, this volume will be an invaluable resource for students and researchers of training, HRD, continuing and adult education.

© 2013 D. Boud & E. Molloy. Learners complain that they do not get enough feedback, and educators resent that although they put considerable time into generating feedback, students take little notice of it. Both parties agree that it is very important.Feedback in Higher and Professional Education explores what needs to be done to make feedback more effective. It examines the problem of feedback and suggests that there is a lack of clarity and shared meaning about what it is and what constitutes doing it well. It argues that new ways of thinking about feedback are needed.There has been considerable development in research on feedback in recent years, but surprisingly little awareness of what needs to be done to improve it and good ideas are not translated into action. The book provides a multi-disciplinary and international account of the role of feedback in higher and professional education. It challenges three conventional assumptions about feedback in learning:◦ That feedback constitutes one-way flow of information from a knowledgeable person to a less knowledgeable person.◦ That the job of feedback is complete with the imparting of performance-related information.◦ That a generic model of best-practice feedback can be applied to all learners and all learning situationsIt seeking a new approach to feedback, it proposes that it is necessary to recognise that learners need to be much more actively involved in seeking, generating and using feedback. Rather than it being something they are subjected to, it must be an activity that they drive.

© 2019, © 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. Academic failure is an important and personal event in the lives of university students, and the ways they make sense of experiences of failure matters for their persistence and future success. Academic failure contributes to attrition, yet the extent of this contribution and precipitating factors of failure are not well understood. To illuminate this world-wide problem, we analysed institutional data at a large, comprehensive Australian university and surveyed 186 undergraduate students who had failed at least one unit of study in 2016, but were still enrolled in 2017. Academic failure increased the likelihood of course attrition by 4.2 times. The students who failed and persisted attributed academic failure to a confluence of dispositional, situational, and institutional factors. There was a compounded effect of academic failure on already-vulnerable students resulting in strong negative emotions. Viewing persistence as an interaction between individuals and their sociocultural milieu opens up different avenues for research and considerations for support.

© 2019, © 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. Work-integrated learning (WIL) is a feature of university courses, both in professional areas, where it is commonplace, but also across many different disciplines. Assessment of WIL can be complex as it involves parties and settings external to the university, and it can be problematic because of difficulties in aligning learning activities during placements with what is or can be assessed by the university. This paper explores the relationship between students’ placement experiences and accompanying assessments in contexts where activities are tightly coupled with the curriculum, and in those where it is not. It draws on a qualitative analysis of student interviews and drawings by the interviewees of their WIL experiences, supplemented with analysis of unit guides. Our findings highlight that students’ perceptions of authenticity of assessment were undermined by misalignments between the student, university and industry. Assessment authenticity was perceived by students as based on alignment between their current and future selves in the assessment process, involvement of industry supervisors and relevance of placement activities to assessment activities. The paper discusses the complexity of coordination of educational activities with external partners, especially when one party drives assessment. It then suggests a reframing of WIL assessment to promote alignment and authenticity.

© 2019, © 2019 Informa UK Limited, trading as Taylor & Francis Group. Tests and examinations are widely used internationally. Despite their pervasiveness, they tend to measure lower order thinking skills in a decontextualized manner at a time when the literature frequently argues for the benefits of a richer, authentic approach to assessment. The focus of this paper is to improve authenticity in test assessment methods through promoting realism, cognitive challenge and evaluative judgement during the planning, administering and following up of assessment tasks. The article builds on a systematic literature review, in which the main principles of authentic assessment were outlined. In this paper, we posit how these principles can be implemented through the three chronological phases of the assessment process: before, during and after the act of assessment.

Ensuring student success has long been on the research agenda in higher education. In this study, we seek to understand if the changes students make in light of academic failure are consistent with this literature. Little is known about students who fail but subsequently persist in their studies. Through an online survey with students who had failed and persisted, we identified drivers for persistence and how students adapted in response to academic failure. Thematic analysis showed that the majority of students did not seek institutional support following academic failure but they did seek support from peers, family and friends. These adaptations occurred at multiple levels: dispositional, situational and institutional. Drivers reported were internal (desire to complete) and external (desire to meet expectations). Although the majority of our students showed positive adaptations following academic failure, a significant portion reported no changes to their academic strategies. The paper poses the question of how students who fail can be better supported to continue successfully.

© 2018, © 2018 Informa UK Limited, trading as Taylor & Francis Group. Since the early 2010s the literature has shifted to view feedback as a process that students do where they make sense of information about work they have done, and use it to improve the quality of their subsequent work. In this view, effective feedback needs to demonstrate effects. However, it is unclear if educators and students share this understanding of feedback. This paper reports a qualitative investigation of what educators and students think the purpose of feedback is, and what they think makes feedback effective. We administered a survey on feedback that was completed by 406 staff and 4514 students from two Australian universities. Inductive thematic analysis was conducted on data from a sample of 323 staff with assessment responsibilities and 400 students. Staff and students largely thought the purpose of feedback was improvement. With respect to what makes feedback effective, staff mostly discussed feedback design matters like timing, modalities and connected tasks. In contrast, students mostly wrote that high-quality feedback comments make feedback effective–especially comments that are usable, detailed, considerate of affect and personalised to the student’s own work. This study may assist researchers, educators and academic developers in refocusing their efforts in improving feedback.

© 2019, © 2019 HERDSA. Despite an increasing focus on assessment feedback, educators continue to find that simply replicating an effective feedback practice from one context does not guarantee success in the next. There is a growing recognition that the contextual factors surrounding successful practices need to be considered. This article reports on a large-scale mixed methods project and proposes 12 conditions that enable successful feedback in higher education. The conditions were distilled from seven rich case studies through multiple stages of thematic analysis, case comparison and reliability checking. The conditions were also evaluated by surveying senior leaders of Australian universities. These conditions highlight the importance of carefully designing feedback processes, along with the need for addressing capacity and culture for feedback. This helps to explain why there are such variances in effective feedback across contexts, and offers insight into how it may be achieved.

© 2018, © 2018 HERDSA. Students’ capacity for making evaluative judgements of their own work is widely acknowledged as central to their learning within programmes as well as being vital to their subsequent professional practice. In higher education literature, the act of evaluative judgement is usually portrayed as a process of deliberative, analytical reasoning requiring student agency and objectivity, typically scaffolded by points of reference such as explicit criteria, rubrics or exemplars. This article challenges this common portrayal of judgement by drawing attention to research from outside higher education on the role of unconscious factors in judgement and decision-making. Drawing from the field of heuristics and bias studies, the article outlines six unconscious factors that have the potential to distort students’ analytical judgement of their work. A recent challenge to the heuristics and bias approach that radically repositions the place of reasoning in judgement is also considered. Since these unconscious factors have received scant attention in higher education literature, the purpose of this article is to draw attention to them, to identify the challenges they pose to current understandings of evaluative judgement, and to outline their implications for enhancing assessment practice.

© 2018, Instituto Superior de Psicologia Aplicada, Lisboa, Portugal and Springer Nature B.V. Recently, the concept of evaluative judgement has gained attention as a pedagogical approach to classroom formative assessment practices. Evaluative judgement is the capacity to be able to judge the work of oneself and that of others, which implies developing knowledge about one’s own assessment capability. A focus on evaluative judgement helps us to better understand what is the influence of assessment practices in the regulation of learning. In this paper, we link evaluative judgement to two self-regulated learning models (Zimmerman and Winne) and present a model on the effects on co-regulation of learning. The models help us to understand how students can be self-regulated through developing their evaluative judgement. The co-regulation model visualises how the learner can become more strategic in this process through teacher and peer assessment in which assessment knowledge and regulation strategies are shared with the learner. The connections we make here are crucial to strengthening our understanding of the influence of assessment practices on students’ learning.

© 2019, The Author(s). The article “Toward a Pedagogy for Professional Noticing: Learning through Observation”, written by Donna Rooney and David Boud, was originally published Online First without Open Access. After publication in volume 12, issue 3, page 441–457 the author decided to opt for Open Choice and to make the article an Open Access publication. Therefore, the copyright of the article has been changed to © The Author(s) 2019 and the article is forthwith distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The original article has been corrected.

A necessary skill that underpins all professional practice is noticing that which is salient. Noticing can be learned directly and indirectly through a variety of campus-based and placement activities. This paper suggests that developing a capacity for noticing is under conceptualised and underdeveloped in courses preparing students for the professions. It discusses three aspects of noticing: noticing in context, noticing of significance and noticing learning, and explores the use of these through a case study of simulation in nursing education. The case study points to the importance of close attention to the circumstances in which noticing can be fostered and, in doing so, points toward the potential of developing a pedagogy of professional noticing.

© 2018, Springer Nature B.V. The accuracy and consistency of peer marking, particularly when students have the power to reward (or penalise) during formative and summative assessment regimes, is largely unknown. The objective of this study is to evaluate students’ ability and behaviour in marking their peers’ teamwork performance in a collaborative group assessment context both when the mark is counted and not counted towards their final grade. Formative and summative assessment data were obtained from 98 participants in anonymous self and peer assessment of team members’ contributions to a group assessment in business courses. The findings indicate that students are capable of accurately and consistently judging their peers’ performance to a large extent, especially in the formative evaluation of the process component of group work. However, the findings suggest significant peer grading bias when peer marks contribute to final grades. Overall, findings suggest that students are reluctant to honestly assess their peers when they realise that their actions can penalise non-contributing students. This raises questions about the appropriateness of using peer marks for summative assessment purposes. To overcome the problems identified, this paper proposes a number of measures to guide educators in effectively embedding summative peer assessment in a group assessment context.

© 2018, © 2018 HERDSA. In recent years, there have been calls in the literature for the dominant model of feedback to shift away from the transmission of comments from marker to student, towards a more dialogic focus on student engagement and the impact of feedback on student learning. In the present study, we sought to gain insight into the extent to which such a shift is evident in practice, and how practice is shaped by national and disciplinary cultures. A total of 688 higher education staff from the UK and Australia completed a survey, in which we collected data pertaining to key influences on the design of feedback, and the extent to which emphasis is placed on student action following feedback. Our respondents reported that formal learning and development opportunities have less influence on feedback practice than informal learning and development, and prior experience. Australian respondents placed greater emphasis on student action following feedback than their counterparts in the UK, and were also more likely than UK respondents to judge the effectiveness of feedback by seeking evidence of its impact on student learning. We contextualise these findings within the context of disciplinary and career stage differences in our data. By demonstrating international differences in the adoption of learning-focused feedback practices, the findings indicate directions for the advancement of feedback research and practice in contemporary higher education.

© 2017 The Author(s). A ready source of data to investigate learning gain is that generated normally through student assessment. That such data cannot currently be used for this purpose needs explanation and the refreshment of assessment thinking to bring it in line with thinking about standards. This opinion piece discusses what is required for this to occur.

© 2016 Society for Research into Higher EducationAssessment as a field of investigation has been influenced by a limited number of perspectives. These have focused assessment research in particular ways that have emphasised measurement, or student learning or institutional policies. The aim of this paper is to view the phenomenon of assessment from a practice perspective drawing upon ideas from practice theory. Such a view places assessment practices as central. This perspective is illustrated using data from an empirical study of assessment decision-making and uses as an exemplar the identified practice of ‘bringing a new assessment task into being’. It is suggested that a practice perspective can position assessment as integral to curriculum practices and end separations of assessment from teaching and learning. It enables research on assessment to de-centre measurement and take account of the wider range of people, phenomena and things that constitute it.

© 2017 Society for Research into Higher Education Findings from interviews with mid-career academics in English and Australian universities elucidate how academics interpret and navigate complex institutional contexts in shaping academic jobs. The paper argues that how they do this is a function of what they notice and respond to as well as the mode of reflexivity they employ. Three core areas are seen to affect academics sense of agency as they shape their own jobs: how they orient themselves to the world around them including the academic institution and department; their underlying goals and purposes as they seek to have a fulfilling role; and how they relate to structural conditions of the workplace. The paper argues that understanding academics’ differing foci of awareness in these areas is helpful to institutional policies and strategies.

© 2017 Springer Science+Business Media B.V. In the changing context of universities, organisational structures for teaching and research problematise academic roles. This paper draws on a critical realist analysis of surveys and interviews with academics from universities in England and Australia. It identifies important academic work, not captured simply in descriptions of teaching or research. It shows that many academics, who are not research high flyers nor award-winning teachers, carry out this essential work which contributes to the effective functioning of their universities. That work is referred to as academic artisanal work and the people who do it as academic artisans. Characteristics and examples of academic artisans are presented, and the nature of artisanal work is explored. Implications for higher education management and for future studies are discussed. The paper points to an urgent need to better understand the complex nature of academic work.

© 2017 Informa UK Limited, trading as Taylor & Francis Group Teaching a large class can present real challenges in design, management and standardisation of assessment practices. One of the main dilemmas for university teachers is how to implement effective formative assessment practices with accompanying high-quality feedback consistently over time with large classroom groups. This article reports on how elements of formative practices can be implemented as part of summative assessment in very large undergraduate cohorts (n = 1500 in one semester), studying in different modes (on- and off-campus), with multiple markers, and under common cost and time constraints. Design features implemented include the use of exemplars, rubrics and audio feedback. The article draws on the reflections of the leading teacher, and argues that, for summative assessment to benefit learners, it should contain formative assessment elements. The teaching practices utilised in the case study provide some means to resolve the tensions between formative assessment and summative assessment that may be more generally applicable.

© 2016 Informa UK Limited, trading as Taylor & Francis Group Portfolios are embraced extensively in higher professional education as effective tools for students to represent their learning and help prepare them for future practice. They are very diverse, used for both formative and summative purposes; however, concerns are raised that the current emphasis on academic standards and/or the focus on employability may lead to the perception of portfolios simply as means to portray achievements. This paper argues that contemporary portfolios in digital environments can readily facilitate both purposes. It conceptualises a whole-of-programme approach to the use of portfolios in which consideration is given to the need to bring curation skills and feedback processes to the forefront of portfolio practices. For teachers considering these issues, a planning framework for the design of programme-wide portfolios is proposed.

© 2017 The Author(s) Evaluative judgement is the capability to make decisions about the quality of work of oneself and others. In this paper, we propose that developing students’ evaluative judgement should be a goal of higher education, to enable students to improve their work and to meet their future learning needs: a necessary capability of graduates. We explore evaluative judgement within a discourse of pedagogy rather than primarily within an assessment discourse, as a way of encompassing and integrating a range of pedagogical practices. We trace the origins and development of the term ‘evaluative judgement’ to form a concise definition then recommend refinements to existing higher education practices of self-assessment, peer assessment, feedback, rubrics, and use of exemplars to contribute to the development of evaluative judgement. Considering pedagogical practices in light of evaluative judgement may lead to fruitful methods of engendering the skills learners require both within and beyond higher education settings.

Writing practices are seen to be essential for professional engineers, yet many engineering students and academics struggle with written communication, despite years of interventions to improve student writing. Much has been written about the importance of getting engineering students to write, but there has been a little investigation of engineering academics’ perceptions of writing practices in the curriculum, and the extent to which these practices are visible to their students and to the academics. This paper draws on research from an ongoing study into the invisibility of writing practices in the engineering curriculum using a practice architectures lens. The paper uses examples from the sites of practice of two participants in the study to argue that prevailing practices in engineering education constrain more than enable the development and practice of writing in the engineering curriculum.

© 2016 Springer Science+Business Media DordrechtThere are dissonances between educators’ aspirations for assessment design and actual assessment implementation in higher education. Understanding how assessment is designed ‘on the ground’ can assist in resolving this tension. Thirty-three Australian university educators from a mix of disciplines and institutions were interviewed. A thematic analysis of the transcripts indicated that assessment design begins as a response to an impetus for change. The design process itself was shaped by environmental influences, which are the circumstances surrounding the assessment design, and professional influences, which are those factors that the educators themselves bring to the process. A range of activities or tasks were undertaken, including those which were essential to all assessment design, those more selective activities which educators chose to optimise the assessment process in particular ways and meta-design processes which educators used to dynamically respond to environmental influences. The qualitative description indicates the complex social nature of interwoven personal and environmental influences on assessment design and the value of an explicit and strategic ways of thinking within the constraints and affordances of a local environment. This suggests that focussing on relational forms of professional development that develops strategic approaches to assessment may be beneficial. The role of disciplinary approaches may be significant and remains an area for future research.

© 2016 British Educational Research Association A wide range of technologies has been developed to enhance assessment, but adoption has been inconsistent. This is despite assessment being critical to student learning and certification. To understand why this is the case and how it can be addressed, we need to explore the perspectives of academics responsible for designing and implementing technology-supported assessment strategies. This paper reports on the experience of designing technology-supported assessment based on interviews with 33 Australian university teachers. The findings reveal the desire to achieve greater efficiencies and to be contemporary and innovative as key drivers of technology adoption for assessment. Participants sought to shape student behaviors through their designs and made adaptations in response to positive feedback and undesirable outcomes. Many designs required modification because of a lack of appropriate support, leading to compromise and, in some cases, abandonment. These findings highlight the challenges to effective technology-supported assessment design and demonstrate the difficulties university teachers face when attempting to negotiate mixed messages within institutions and the demands of design work. We use these findings to suggest opportunities to improve support by offering pedagogical guidance and technical help at critical stages of the design process and encouraging an iterative approach to design.

© 2016 Informa UK Limited, trading as Taylor & Francis Group. Despite widespread recognition of the need to improve assessment in higher education, assessment tasks in individual courses are too often dominated by conventional methods. While changing assessment depends on many factors, improvements to assessment ultimately depend on the decisions and actions of individual educators. This paper considers research within the ‘heuristics and biases’ tradition in the field of decision-making and judgement which has identified unconscious factors with the potential to limit capacity for such change. The paper focuses on issues that may compromise the process of improving assessment by supporting a reluctance to change existing tasks, by limiting the time allocated to develop alternative assessment tasks, by underestimating the degree of change needed or by an unwarranted overconfidence in assessment design decisions. The paper proposes countering these unconscious limitations to change by requiring justification for changing, or not changing, assessment tasks, and by informal and formal peer review of assessment task design. Finally, an agenda for research on heuristics and biases in assessment design is suggested in order to establish their presence and help counter their influence.

© 2015 Taylor & Francis Sustainable assessment has been proposed as an idea that focused on the contribution of assessment to learning beyond the timescale of a given course. It was identified as an assessment that meets the needs of the present in terms of the demands of formative and summative assessment, but which also prepares students to meet their own future learning needs. This paper reviews the value of such a notion for assessment; how it has been taken up over the past 15 years in higher education and why it might still be needed. It identifies how it has been a successful intervention in assessment discourse. It explores what more is needed to locate assessment as an intervention to focus on learning for the longer term. It shows how sustainable assessment can help bridge the gap between assessment and learning, and link to ideas such as self-regulation, students’ making judgements about their own work and course-wide assessment.

This paper asks the question: do people with different levels of research productivity and identification as a researcher think of research differently? It discusses a study that differentiated levels of research productivity among English and Australian academics working in research-intensive environments in three broad discipline areas: science, engineering and technology; social science and humanities; and medicine and health sciences. The paper explores the different conceptions of research held by these academics in terms of their levels of research productivity, their levels of research training, whether they considered themselves an active researcher and a member of a research team, and their disciplinary differences.

Health professions education is characterised by work-based learning and relies on effective verbal feedback. However the literature reports problems in feedback practice, including lack of both learner engagement and explicit strategies for improving performance. It is not clear what constitutes high quality, learner-centred feedback or how educators can promote it. We hoped to enhance feedback in clinical practice by distinguishing the elements of an educator's role in feedback considered to influence learner outcomes, then develop descriptions of observable educator behaviours that exemplify them.An extensive literature review was conducted to identify i) information substantiating specific components of an educator's role in feedback asserted to have an important influence on learner outcomes and ii) verbal feedback instruments in health professions education, that may describe important educator activities in effective feedback. This information was used to construct a list of elements thought to be important in effective feedback. Based on these elements, descriptions of observable educator behaviours that represent effective feedback were developed and refined during three rounds of a Delphi process and a face-to-face meeting with experts across the health professions and education.The review identified more than 170 relevant articles (involving health professions, education, psychology and business literature) and ten verbal feedback instruments in health professions education (plus modified versions). Eighteen distinct elements of an educator's role in effective feedback were delineated. Twenty five descriptions of educator behaviours that align with the elements were ratified by the expert panel.This research clarifies the distinct elements of an educator's role in feedback considered to enhance learner outcomes. The corresponding set of observable educator behaviours aim to describe how an educator could engage, motivate and enable a learner to improve...

As the field of health care simulation matures, new questions about appropriate pedagogy are emerging which present challenges to research and practices. This has implications for how we investigate and deliver effective simulations, how we conceive effectiveness, and how we make decisions about investment in simulation infrastructure. In this article, we explore two linked challenges that speak to these wider concerns: student diversity and large cohorts. We frame these within contemporary simulation practices and offer recommendations for research and practice that will account for students' varying cultural expectations about learning and clinical practice in the Australian context.

Can extended opportunities for self-assessment over time help students develop the capacity to make better judgements about their work? Using evidence gathered through students’ voluntary self-assessment of their performance with respect to assessment tasks in two different disciplines at two Australian universities, the paper focuses on the effects of sequences of units of study and the use of different types of assessment task (written, oral, analysis, and project) in the development of student judgement. Convergence between student criteria-based gradings of their own performance in units of study and those allocated by tutors was analysed to explore the calibration of students’ judgement over time. First, it seeks to replicate analyses from an earlier smaller-scale study to confirm that students’ judgements can be calibrated through continuing opportunities for self- assessment and feedback. Second, it extends the analysis to coherently designed sequences of units of study and explores the effects of different types of assessment. It finds that disruptive patterns of assessment within a sequence of subjects can reduce convergence between student and tutor judgements.

This paper explores a dilemma in continuing professional learning: the way learning is typically inscribed in continuing professional education (CPE) frameworks differs from that elicited from practice. It examines these differences in relation to both different underlying assumptions about learning and varying epistemological perspectives as well as the different purposes of CPE frameworks of professional bodies and organisations. It suggests that the dominant adoption of narrower conceptions of learning in professional organisations' frameworks ignores understandings about work and learning emerging from recent research in the field of workplace learning and focuses on a view that may privilege formal provision. The paper concludes with suggestions for future research directions in developing alternative continuing professional learning frameworks.

The preparation of future professionals for practice is a key focus of higher education institutions. Among a range of approaches is the use of simulation peda- gogies. While simulation is often justified as a direct bridge between higher education and professional practice, this paper questions this easy assumption. It develops a conceptually driven argument to cast new light on simulation and its unarticulated potential in professional formation. The argument unfolds in, and is illustrated via, three accounts of a simulation event in an Australian undergraduate nursing program. This begins with a familiar approach, moves to one that problematizes this through a focus on disruption, culminating in a third that draws on socio-material theorisations. Here, simulation is conceived as emergent, challenging stable notions of fidelity, common in simulation literature. New possibilities of simulation in the production of agile practitioners and learners in practice are surfaced. This paper extends and enriches thinking by providing distinctive new ways of understanding simulation and the relationship it affords between education and professional practice, and by illuminating the untapped potential of simulation for producing agile practitioners.

With the increasing challenges facing professional engineers working in more complex, global and interdisciplinary contexts, different approaches to understanding how engineers practice and learn are necessary. This paper draws on recent research in the social sciences from the field of workplace learning, to suggest that a practice-theory perspective on engineers' professional learning is fruitful. It shifts the focus from the attributes of the individual learner (knowledge, skills and attitudes) to the attributes of the practice (interactions, materiality, opportunities and challenges). Learning is thus more than the technical acquisition and transfer of knowledge, but a complex bundle of activities, that is, social, material, embodied and emerging. The paper is illustrated with examples from a research study of the learning of experienced engineers in the construction industry to demonstrate common practices – site walks and design review meetings – in which learning takes place.

This paper presents the preliminary results of a multi-phased qualitative investigation of continuing professional learning. The study focused on the identification of common engineering practices that contribute to learning. This paper examines a particular practice, that of the site-walk. It draws on practice theory, an emerging set of conceptual resources used in workplace learning research. Data was elicited via qualitative methods such as interviews, focus groups and site visits with experienced engineers employed in a large Australian engineering company. It was analysed using the lens of practice theory. The findings suggest that site-walks, while an everyday practice for engineers, are also highly learning-rich. This understanding has implications for continual professional learning, and for educators of novice engineers.

Changes in expectations of research education worldwide have seen the rise of new demands beyond supervision and have highlighted the need for academic leadership in research education at a local level. Based on an interview study of those who have taken up local leadership roles in four Australian universities, this paper maps and analyses different dimensions of the emerging leadership role of research education coordination. It argues that while there is increasing clarity of what is required, there are considerable tensions in the nature of the coordination role and how coordination is to be executed. In particular, what leadership roles are appropriate and how can they be positioned effectively within universities? The paper draws on the Integrated Competing Values Framework to focus on the activities of coordination and on ideas of distributed leadership to discuss the leadership that characterises coordination. It is argued that without acknowledgement of the influences that coordinators need to exert and the positioning and support needed to achieve this, the contemporary agenda for research education will not be realised. © 2014 © 2014 Association for Tertiary Education Management and the LH Martin Institute for Tertiary Education Leadership and Management.

Despite increasing research and scholarship in the area of academic development in recent years, it remains an under-theorised field of endeavour. The paper proposes that academic developers take a view on what constitutes academic work and see it as a form of professional practice. It discusses the features of practice theory that illuminate professional practice and identifies three foci for the application of these ideas within academic development: practice development, fostering learning-conducive work and deliberately locating activity within practice. It also suggests that academic development be viewed as a practice and points to features within its own traditions on which to build.

Student feedback is a contentious and confusing issue throughout higher education institutions. This paper develops and analyses two models of feedback: the first is based on the origins of the term in the disciplines of engineering and biology. It posit

One of the implicit aims of higher education is to enable students to become better judges of their own work. This paper examines whether students who voluntarily engage in self-assessment improve in their capacity to make those judgements. The study utilises data from a web-based marking system that provides students with the opportunity to assess themselves on each criterion for each assessment task throughout a programme of study. Student marks were compared with those from tutors to plot changes over time. The findings suggest that overall students judgements do converge with those of tutors, but that there is considerable variation across achievement levels, with weaker students showing little improvement. Whilst the study is limited by the exigencies of voluntary participation and thus consequential gaps in the data set, it shows how judgement over time can be demonstrated and points to the potential for more systematic interventions to improve students judgements. It also illustrates the use of the web-based marking and feedback software (ReView) that has considerable utility in aiding self-assessment research.

International research collaboration raises questions about how groups from different national and institutional contexts can work together for common ends. This paper uses issues that have arisen in carrying out the first stage of an international resea

Copyright © 2013 The International Society for the Scholarship of Teaching and Learning Almost all tertiary educators make assessment choices, for example, when they create an assessment task, design a rubric, or write multiple-choice items. Edu¬ cators potentially have access to a variety of evidence and materials regarding good assessment practice but may not choose to consult them or be success¬ ful in translating these into practice. In this article, we propose a new challenge for the Scholarship of Teaching and Learning: the need to study the disjunction between proposals for assessment "best practice" and assessment in practice by examining the assessment decision-making of teachers. We suggest that assessment decision-making involves almost all university teachers, occurs at multiple levels, and is influenced by expertise, trust, culture, and policy. Assessment may dictate the curriculum from the student's perspective, and we argue that assess¬ ment decision-making dictates assessment.

Seeking a different angle on feedback in clinical education: the learner as seeker, judge and user of performance information Seeking a different angle on feedback in clinical education: the learner as seeker, judge and user of performance information

Many professions have requirements for professional development activities to ensure continuing registration or membership. These commonly focus on participation in a limited range of activities. This paper questions the assumptions behind such approache

Although there have been increasing attempts to involve undergraduate students in conducting research, a pivotal moment when students engage in knowledge production is during honours programmes. Honours programmes, particularly those in Australia, seek to develop students capacity to engage in higher order thinking that may lead to knowledge production. This transition is facilitated through advanced disciplinary knowledge, research training and a research project. However, there is a pedagogical tension between requiring students to engage in this deeper level of inquiry at the same time as they complete a heavy knowledge acquisition load. This paper explores how a number of disciplines in Australia balance these elements of the honours curricula. It argues that the combination of these curriculum goals can make it difficult for students to apply the knowledge they have gained in advanced disciplinary and research training courses to their research project work. This has serious implications for honours programmes.

Challenges conventional theories underpinning HRD that typically focus on the objects of learning - individuals, jobs, training. Discusses two case studies (public utility and winery) of learning across boundaries through, for example, the organisational practice of 'acting up' and identifies implications for HRD practice and practitioners.

The paper examines influences of doctoral study and structured early career interventions on academic work. It uses evidence from a survey in six Australian universities (n1158) to explore academics engagement with research and teaching development. It regards universities as social situations that are ambiguous, presenting conflicting opportunities for growth and development and the pursuit of personal objectives. Academics emphases on different aspects of academic work involve complex responses that are subject to many influences. It shows that doctoral work may avoid developing key skills that academics need and that the extent to which doctoral study prepares academics for different aspects of their role is different in different disciplines and institutions. It also shows that the conduct of academics in their major roles is little influenced by the training and development they engage in. The paper discusses the survey findings in terms of what constrains and what enables academics to take up particular development opportunities. It argues that to understand academic practice, analysis of how organisational influences are interpreted is needed.

The Honours undergraduate degree in Australia is unlike that in most other countries. It has taken on a particular significance as a qualification, as a pathway to and a pre-requisite for direct entry into doctoral programs. This paper explores the outco

A new approach to assessment design is considered through the process of developing a set of capabilities for osteopathic practice that starts from an understanding of a concept of 'practice' applicable to real, situated osteopathic healthcare. Appropria

The purpose of this paper is to examine the use of "learning" through what we have termed "integrated development practices". These are common organisational practices that both enhance organisational effectiveness and contribute to organisational and employee learning. Design/methodology/approach - The paper analyses the ways in which learning and being a learner were talked about and enacted with regard to one of the integrated development practices identified in a study of four different organisations - safety practices, and how learning and being a learner regarding safety were legitimate in one of the organisations. Data are drawn from semi-structured interviews with members of a variety of workgroups in one major division of the organisation. Findings - Interviewees' responses reflected that learning was fully embedded as an accepted part of a necessary function of the organisation. This use of a learning discourse is discussed in the light of findings from an earlier study on informal learning at work that suggested that learning and the identity of being a learner were sometimes resisted in the everyday culture of work.Originality/value - Using the theorisations of practice of Schatzki and the lifelong education framework of Delors the paper discusses the implications of these findings to examine when it is acceptable to articulate learning as part of work and be identified as a learner at work.

Noting the ever-increasing encroachment of discourses and practices from the private sector on public education providers, this paper argues that such organizations exist within competing sets of differences that seek to define and fix the meaning of 'education' and 'business'. We report on fieldwork conducted in an adult education college in Sydney. In the Australian context these colleges are referred to as community colleges and their history is one based in a strong liberal tradition. Utilising Judith Butler's idea of 'drag' we consider the effects of changing modes of governance in the college with specific reference to the stories told to us about it. Our discussion suggests that the organisation was caught between identifying itself with a masculinised discourse of business and a discourse of community cast as its feminised other. In navigating between these, the college was seen to perform as a 'drag king' an organisation performing the masculine but in so doing, undoing its gendered status. This leads us to suggest that the incorporation of business and market-based discourse into the management of community education is something that is actively resisted and undermined through such forms of gendered transgression. We conclude by proposing that this organization's capacity to perform drag is a contributing factor to its overall success, and particularly in an economic climate where many not-for-profit organisations are floundering.

The purpose of this paper is to challenge models of workplace learning that seek to isolate or manipulate a limited set of features to increase the probability of learning. Such models typically attribute learning (or its absence) to individual engagement, manager expectations or organizational affordances and are therefore at least implicitly causative. In contrast, we discuss the contributions of complexity theory principles such as emergence and novelty that suggest that learning work is more a creative and opportunistic process that emerges from contextualized interactional understandings among actors. Using qualitative case study methods, we discuss the experiences of workers in two organizations asked to 'act up' in their managers' role to ensure work continuity. We believe the differences in how workers take up these opportunities result from a complex combination of situational factors that generate invitational patterns signalled from and by various understandings and interactions among actors doing collective work.

Learning in workplaces is always mediated through talk. It is tempting for management to seek to utilise everyday talk as part of learning and therefore enhance productivity. This paper examines the responses of workers to interventions that aim to formalise informal conversations at work as part of an explicit workplace learning strategy. It draws on interviews with managers and workers in a public sector organisation to examine their experience of these practices. The paper raises questions about whether interventions in the name of fostering informal learning may well be hindering what they seek to promote.

There is growing interest in the application of citizen participation within all areas of public sector service development, where it is increasingly promoted as a significant strand of post-neoliberal policy concerned with re-imagining citizenship and more participatory forms of citizen/consumer engagement. The application of such a perspective within health services, via co-production, has both beneficial, but also problematic implications for the organisation of such services, for professional practice and education. Given the disappointing results in increasing consumer involvement in health services via 'choice' and 'voice' participation strategies, the question of how the more challenging approach of co-production will fare needs to be addressed. The article discusses the possibilities and challenges of system-wide co-production for health. It identifies the discourse and practice contours of co-production, differentiating co-production from other health consumer-led approaches. Finally, it identifies issues critically related to the successful implementation of co-production where additional theorisation and research are required.

This paper takes up understandings of organisations where practices constitute and frame past and present work, as well as future work practice possibilities. Within this view, work practices, and thus organisations,are both perpetuated and varied through employees' enactments of work. Using a practice lens, we are particularly interested in the ways workers simultaneously maintain and alter practices in their workplace' we characterise this as re-making one's job. This perspective challenges ways in which managers often depict jobs and everyday work' as rational, linear and easily describable. We suggest that workers at various levels of responsibility contribute more to the formation of organisational practices than is often assumed. The processes of re-making jobs and remaking organisational practices create tensions that we posit as sites for learning.

Project work has been a common feature of undergraduate degree programmes for many years. While it has been named in a variety of ways, it typically involves students undertaking a substantial learning activity that is partly self-initiated and managed. More recently, programmes organised around the idea of work-based learning partnerships have emerged. These can be regarded as programmes that rely on significant amounts of work-based project work. This paper examines the implications of practices in these new programmes for project advising more generally. It argues that the conception of the role of academics in project work needs to change from one focused on project supervision to one of learning adviser. It identifies key features of this practice and discusses differences in advising from one context to another. It suggests that the activities in which academics engage need to be reappraised and that the skills and knowledge of those acting in the role of adviser be extended.

Research education has been dominated in recent years by policy- driven preoccupations with doctoral completions, funding and contributions to the economy. This has led universities to focus on enhanced institutional support for research degrees, with an

Higher education institutions are responding to globalisation in various ways. This study describes and analyses challenges encountered in a recent case of global collaboration between four universities on different continents in developing a web-based m

Can the quality of students' work be improved through training in self- assessment practices? This paper considers the impact of training high school students on their performance in external examinations. Teachers were selected from a sample of high schools and trained in how to develop students' self-assessment skills. Ten high schools representative of the top, middle and bottom levels of academic achievement in national examinations were chosen and students trained in self-assessment by their normal class teachers as part of their final year curriculum. An experimental group comprising 256 participants received formal training in self-assessment skills for the entire three terms of the academic year. A control group was selected from matched classes not receiving such training. A significant difference favouring those trained in self-assessment was found overall and in each curriculum area. While it is demonstrated that self-assessment training can have an impact on student performance the paper considers the circumstances of the study and whether similar outcomes might be possible in less favourable conditions. © 2003, Taylor & Francis Group, LLC.

Research in workplace learning needs to take into account the reflexive nature of researchers’ learning. Explores how members of a research team examined their own learning and collaboration through a study of transcripts of interactions between them during planning meetings and reflections upon them. Identifies implications for collaboration and learning about workplace learning. A key finding was that experienced researchers and adult educators had difficulty legitimising a focus on their own workplace learning. This points to the problems likely to arise in getting others, who do not have a discourse of learning readily available to them, to take informal workplace learning seriously. © 2001, MCB UP Limited

Assessment practices in higher education institutions tend not to equip students well for the processes of effective learning in a learning society. The purposes of assessment should be extended to include the preparation of students for sustainable assessment. Sustainable assessment encompasses the abilities required to undertake those activities that necessarily accompany learning throughout life in formal and informal settings. Characteristics of effective formative assessment identified by recent research are used to illustrate features of sustainable assessment. Assessment acts need both to meet the specific and immediate goals of a course as well as establishing a basis for students to undertake their own assessment activities in the future. To draw attention to the importance of this, the idea that assessment always has to do double duty is introduced.

Universities are at a pivotal point in their history and are undergoing dramatic changes. One of the more significant of these changes is the move towards instrumental programmes of learning, as manifest for instance in workplace and work-based learning. This paper argues that this trend threatens the existence of the liberal university, where knowledge is pursued predominantly for its own sake. The paper identifies four dominant discourses in higher education and suggests that these discourses co-exist with one another, and are sometimes dominant, at other times recessive. It argues that the trend to a post-industrialised labour market has seen the emergence of a vocationalised discourse in higher education, which stresses the instrumental at the expense of the liberal.

Ideas about self assessment and reflection are widespread in professional courses. These terms are often used uncritically, with the assumption that they are necessarily worthwhile additions to the curriculum. The aim of this paper is to provide an overview of issues that need to be confronted in the use of self assessment and reflection in professional education. It identifies some features of good teaching and learning practice. An emphasis is placed on the importance of context, locating learning in both educational and professional practice. Questions are raised about how courses can respond to the challenges of context, and how the professionalism of teaching and learning in universities can be enhanced. © 1999, The Board of Social Work Education.

Various forms of peer, collaborative or cooperative learning, particularly small group activities, are increasingly used within university courses to assist students meet a variety of learning outcomes. These include working collaboratively with others, taking responsibility for their own learning and deepening their understanding of specific course content. The potential benefits of peer learning have long been recognised and are especially relevant today. However, many existing assessment practices act to undermine the goals of peer learning and lead students to reject learning cooperatively. If assessment gives students the message that only individual achievement is valued, and that collaborative effort is akin to cheating, then the potential of peer learning will not be realised. Inappropriate assessment practices may also lead to unhelpful forms of competition within and between groups that prevent groups functioning effectively. This paper examines some of the main assessment issues in connection with peer learning and suggests ways in which the benefits of this approach can be maintained while still meeting the formal assessment requirements of the course. It discusses the use of group assessment, peer feedback and self-assessment, assessment of participation and negotiated assessment and concludes with the identification of a number of issues which remain to be addressed. © 1999 Taylor & Francis Ltd.

Reflection and the promotion of reflective practice have become popular features of the design of educational programmes. This has often led to learning being more effectively facilitated. However, alongside these positive initiatives have grown more dis

The negotiated learning contract is potentially one of the most useful tools available to those interested in promoting flexible approaches to learning. A learning contract is able to address the diverse learning needs of different students and may be designed to suit a variety of purposes both on course and in the workplace. However, contracts need to be carefully introduced and supported and problems can arise if they are used in inflexible ways. This paper draws upon research conducted into the use of learning contracts to identify key issues which need to be considered before using them, and to suggest ways in which they may be developed. The authors make several recommendations for introducing contract learning and identify different styles of contracting in academic courses.

The negotiated learning contract is potentially one of the most useful tools available to those interested in promoting flexible approaches to learning. A learning contract is able to address the diverse learning needs of different students and may be de

Much time and effort has gone into trying to demonstrate an empirical link between research activity and teaching performance. In general, the correlations between these factors have been shown to be low. This paper argues that the attempt to find such a

This paper starts with the experience of the author and presents a perspective on teaching and learning in higher education which regards the experience of the learner as primary. The experience which is to be taken into account includes both that prior to the current learning event and that which is prompted by it. Emphasis is given to learning of all kinds in the cognitive, affective and conative realms. In the paper some propositions about learning from experience are outlined, a model of learning from experience is discussed and implications are drawn for the design of courses and subjects. Major areas for development are identified as those of recognising and responding to prior learning, developing reflective practice and self-assessment skills, and using problem-based, holistic and contextualised learning approaches. Copyright © 1993 by Taylor & Francis Group, LLC. All rights reserved.

The paper describes an approach to assessment in which students create a comprehensive and analytical summary of their learning in a given subject. The self-assessment schedule, as it is called, has been used in contexts in which there is an emphasis on

A frequently stated aim of physiotherapy educational programs is to produce graduates capable of, and wishing to pursue, lifelong learning. To help achieve this goal, curricula commonly include the promotion of independent or self-directed learning. This paper supports the need for developing self-directed learning, orientation and skills and argues that this can be best achieved by bringing these skills into the mainstream of tertiary education programs such as physiotherapy, rather than isolating them to specific subjects. The challenge of mainstreaming is to appreciate how the basic ideas of self-directed learning can be incorporated into all areas of the curriculum. This paper presents strategies, issues and problems associated with mainstreaming of self-directed learning. © 1991, Australian Physiotherapy Association. All rights reserved.

Increasing interest is now being focused on how adults learn from experience, but there is no adequate framework to assist learners and those who facilitate learning-promote learning in the midst of experience. This paper provides a way of conceptualising experience-based learning which considers the personal foundation of experience of learners, their intent and their interaction with a learning milieu. Two elements of the learning experience-noticing and intervening-are discussed in detail. The implications for the facilitation of learning are explored. © 1990, Taylor & Francis Group, LLC. All rights reserved.

NA

While the educational benefits of student self-assessment are being increasingly recognised and self-assessment procedures introduced into post-secondary courses of many different kinds, the use of self-assessment for grading purposes is a more controversial matter. Is there a role for student self-assessment in formal assessment proceedings? If there is to be a role, what should it be? This paper focuses on these questions and examines why a marking role for self-assessment should be considered and what evidence is available on the reliability of student-generated marks. The implications of these findings are considered and strategies are proposed to improve student markers reliability and to incorporate self-assessment indirectly into the formal assessment process. © 1989, Taylor & Francis Group, LLC. All rights reserved.

NA

NA

NA

NA

NA

NA

A method of incorporating self and peer marking in an undergraduate electronics subject is described and discussed. Similar methods could probably be used in other technical subjects. Its educational benefits are: greatly improved feedback to the students, additional reinforcement, and practice at self assessment. The method described has a high degree of student acceptance, and appears to produce reliable and valid marks. It can be incorporated into conventionally taught courses without disrupting teaching. The use of self and peer assessment in an isolated subject requires a great deal of attention to the details of the procedure, especially if it is to be part of the formal assessment in the subject. The procedure developed here requires the preparation of detailed model solutions and an extensive clerical procedure, but alternatives with a simpler clerical process are also possible. The nature and amount of the examination load on faculty is different when self and peer marking is used. For small classes it involves more work for faculty, but for large classes it reduces the work load. It is one of the few educational innovations that can do this, and at the same time have educational benefits.

No matter how participative a course might be in its conduct, it is normal for the aims to be specified by staff for students, the programme to be planned by staff for students and particularly for the assessment to be one way—of students by staff. It is possible for courses to be established on a different basis, and it is especially necessary when one is dealing with professional education. This paper focuses on an example of a course which is designed jointly by staff and students: the science education component of a post‐experience Graduate Diploma in Science Education at the Western Australian Institute of Technology. In keeping with the philosophy of the course this paper has been prepared by the staff member concerned (D J Boud) and one of the students (M T Prosser). A collaboratively designed course cannot be undertaken lightly for not only does it violate some of the norms of teaching in higher education as we know it, but it also places peculiar demands on both staff and students. These are both of an interpersonal, group interaction kind and of a strucutural, organizational type. The aim of this paper is to describe the main features of the course, to examine some of the demands that it makes, and to explore the applicability of its concepts to other areas of higher education. Copyright © 1980, Wiley Blackwell. All rights reserved

This paper offers an account of the development of a project entitled ‘Educational Objectives in Applied Science’ at the Universities of Strathclyde and Heriot-Watt. The first part focuses on the project's choice of style and approach to the replanning of engineering courses within an established system. Although every such project faces unique hazards, some issues are common to any activity which aims to influence teaching and learning within universities. The second part of the paper examines how course development might best be encouraged within science and engineering faculties. It points to some of the resulting problems and to ways of overcoming them. © 1979, Taylor & Francis Group. All rights reserved.

This paper aims to describe the original formulation and some of the subsequent interpretations, applications and evaluations of the self‐paced method of instruction known as the Keller plan, or the Personalised System of Instruction. Emphasis is placed upon bringing together evidence from the authors' own evaluations of such courses in the U.K., and from literature in this field. It is intended that this should give a framework for discussion of the method and indicate its advantages and limitations to potential users. Copyright © 1975, Wiley Blackwell. All rights reserved

A simple method of diagnosing areas for course improvement in university science and engineering laboratory courses is described. A first year physics laboratory course is examined in terms of a set of aims. A questionnaire was used for staff and students to rate the importance of these aims in the cases (i) of each group's conception of what an ideal course in the subject should be, and (ii) each group's rating of the "traditional" and "non-traditional" components of the particular course under investigation. One means of identifying specific areas of improvement is out-lined. The potentialities of the technique are discussed. © 1973 Elsevier Scientific Publishing Company.

This chapter discusses researching feedback inputs and processes to examine effects. Specifically, we promote a research agenda that contributes an understanding of how feedback works, for particular learners, in particular circumstances through research designs that take account of theory, occur in naturalistic settings and focus on students’ sense-making and actions. We draw attention to categories of research on effects of feedback: a) task-related performance/work; b) meta-learning processes such as self-regulation; and c) identity effects such as orienting students to the professionals they wish to become. We also discuss the difficulties in eliciting effects, attributing effects to particular feedback practices and the importance of exploring how effects are achieved and at what points in time, rather than simply looking for outcomes

This chapter has a particular focus on the observers’ role in simulation-based learning activities. Simulation-based learning is often organised so that participants rotates between active participation in the scenario and participation as observers. The research examples provided show that the conditions for learning are related to the locations where and the ways the observers are situated, and to how the instructions to the observers are formulated. Arguments are put forward that the observers’ role in simulation has unexploited potential for developing skills of noticing.

This chapter offers new insight regarding the theoretical, methodological and practical concerns relating to feedback in higher education. It begins with the construction of a new definition of feedback. We explain how feedback is a learner-centred process in which impact is a core feature. The chapter then explores the reasons why identifying, let alone measuring, impact is problematic. We briefly revisit the contingent nature of educational research into cause and effect and question the implications for feedback processes that are likely to be experienced by individuals in different ways with different effects over different timescales. It is here we then discuss some ways we conceive the various forms of feedback effect including the intentional and unintentional, immediate and delayed, cognitive, affective, motivational, relational and social.

This chapter focuses on influences, affordances and challenges for teachers in designing for (and identifying) feedback impact. We propose four key questions that need to be asked: Do learners know the purpose of feedback and their role(s) in it? Can learners make sense of the information? Can learners take action? What effects should we be looking for? We then explore strategies that have been shown to be valuable in designing feedback that makes a difference. These are organised according to three important considerations: creating opportunities for effective feedback; developing learner and teacher capacities; and looking for effects. We finish the chapter by taking a step back and considering the implications at the programme and institutional levels in cultivating feedback that makes a difference.

© 2018 selection and editorial matter, David Boud, Rola Ajjawi, Phillip Dawson and Joanna Tai. This chapter introduces a framework for evaluative judgement that underpins discussion in subsequent chapters. We start by identifying what we mean by evaluative judgement and examining why it is a necessary idea for organising courses and designing assessments. We suggest that the development of students’ evaluative judgement is an important and underexplored aspect of assessment which enables us to consider the longer-term influence of assessment activities on learning. We conclude by examining how common assessment practices - such as rubrics, exemplars, self-assessment, peer assessment and feedback - can be better tailored towards developing students’ evaluative judgement.

Through reflection on a lengthy series of research studies of diverse workplace learning conducted in Australia over the past decade, this chapter seeks to position discussion of informal learning as part of everyday working life. It uses a practice theory perspective to show how learning can be understood as a key feature of working and how it is implicated in the normal ebb and flow of work practices. It elucidates some of the tensions that such a view generates and points to the paradox in how promoting informal learning can effectively inhibit it

© 2018 selection and editorial matter, David Boud, Rola Ajjawi, Phillip Dawson and Joanna Tai. What practical directions can be taken to develop students’ evaluative judgement? Such processes involve more than adding activities into the curriculum. They require repositioning how students become graduates, how courses are structured, and what constitutes progression from one level of sophistication to another. This chapter examines the role of assessment in supporting the development of evaluative judgement; how it can contribute to students judging their own work and that of others; and how peers, educators and others can contribute to this process.

© 1985 Pinchas Tamir. In Western Australia, the education system is structured so that, of a child’s possible twelve years of formal schooling, seven are spent in the primary school and the remaining five in the secondary school. In fact, the need for a less specialized, more practically oriented curriculum for the year 11 and 12 physical sciences was identified as far back as 1975, when the Education Department carried out a needs analysis. In addition, a full time Research Assistant’s position was advertized, as a consequence of which Lesley Parker, experienced in science teaching and research in Western Australia and elsewhere, joined the evaluation team. Dave Boud, however, viewed the act of feedback collection as a major departure from the original illuminative model and a movement towards a bureaucratic model of evaluation. Although pure science is the predominant theme, continuous emphasis is given to the supporting themes, by explanation and discussion on technological applications of science and their concomitant cultural impact.

© 2018 selection and editorial matter, David Boud, Rola Ajjawi, Phillip Dawson and Joanna Tai. Henry is a new graduate working in finance. He graduated with excellent grades, and he is delighted to be in his dream job. Henry attributes his success to his skills at negotiating university assessment processes: by the end of his studies he was able to interrogate a rubric and really give his markers what they wanted. But now Henry has been asked to lead a big new report, and his manager hasn’t given him anything that looks remotely like a criteria sheet or marking guide. Henry’s manager is starting to seem frustrated with Henry’s requests for formative feedback on his work in progress.

The global refocusing of higher education on outcomes is prompting considerable curriculum innovation and the rethinking of teaching and learning practices. It has changed the discourse around degree programmes to foreground explicit learning outcomes, the development of programme-wide attributes and threshold standards. However, with a few exceptions it has had less impact on assessment whether for certification or learning purposes. This is surprising as the implications for assessment are profound. In a standards-based approach, what is central is that standards can be assured and appropriate criteria addressed by all students. Such an
approach implies that it needs to be made clear what students should be able to do as a result of particular episodes of study (the learning outcomes) and that assessment

© 2016, Springer International Publishing Switzerland. Much rhetoric is deployed on arguing that university courses should prepare students for the world of work. Indeed, the main rationale for courses for the professions is that they contribute to preparing students to become effective practitioners. Some professions recognise that there is a transitional period following graduation needed in this process. There is a basic assumption though that, whatever additional elements may also be needed to aid transition, the course itself is the main foundation. There is no shortage of features of courses claimed to prepare students for practice: various kinds of work-integrated learning, placements and practical work, authentic tasks and assessment activities, and indeed entire approaches to the curriculum that focus students’ attention on the kinds of issues that practitioners deal with (e.g. problem-based learning). But can it be reasonably claimed that such approaches recognise the nature of practice? This chapter suggests that courses tend to be poor exemplars of good educational practice for the professions. They have a poorly conceptualised view of what it is that professionals do. They are governed by what is involved in teaching within academic disciplines. They trap students in current knowledge without the capacity to move beyond it. And they do not have a strong sense that courses need to be actively designed and redesigned to produce graduates that will be deliberate professionals. The chapter provides, not a prescription of what is needed for a new curriculum, but an argument for how it might be developed and applied. That is, what educators themselves need to do to become deliberate professionals.

Abstract: This chapter examines how insights from workplace learning research might be used to problematize some common understandings of higher education practice and lead to new ones. It begins by outlining features of the changing higher education context that have implications for how we might think differently about traditional higher education. Following this it contrasts practices in higher education with several themes drawn from two decades of workplace learning research (e.g. learning as embedded, situated; social, and formed through practice). It focuses on specific higher education practices (both existing and possible) that have potential to meet new and different learning needs of the changing student population. Implications centre around four ideas. The first is that being a ‘learner’ is not as powerful as being a producer. The second is that the tasks students engage in should not be seen as isolated from the broader context they will be entering on graduation. The third idea is that assessment needs to be appropriated by learners, and finally, students must more actively construct their own learning.

© David Boud, Ruch Cohen, Jane Sampson aud individual contributors. All rights reserved. Previous chapters have introduced the ideas behind peer learning in university courses, have considered the ways in which it can be used, and have given examples of many different peer learning practices. The experiences of the various practitioners who have implemented these practices demonstrate that there are difficult issues to be addressed and that there are important considerations to be worked through for anyone contemplating using such approaches.

© David Boud, Ruch Cohen, Jane Sampson aud individual contributors. All rights reserved. This chapter describes our use of team-based learning to teach organizational behaviour to MBA classes. One of the guiding contexts of the subject is the trend in contemporary management towards the widespread adoption of new organizational forms and structures that focus on empowerment and the creation of more egalitarian work environments. Our adoption of peer learning to supplement tutor centred teaching was an attempt to promote autonomy in learning as a logical precursor of autonomy at work. Team-based learning programmes provide the learning environment that is most similar to the evolving work environments that students are likely to experience. Finally, it concludes that team-based peer learning programmes can contribute significantly to the development of knowledge and skills required for people-management roles, but the design of the peer learning programme must reflect the local context in which it is conducted. Thus, a peer learning programme in a business faculty is likely to be very different from one in an education faculty.

© David Boud, Ruch Cohen, Jane Sampson aud individual contributors. All rights reserved. This chapter describes a range of peer learning approaches that have been used in face to face and online classes to encourage higher quality student learning in a large first year undergraduate business subject. Initial experiments relied on teams submitting a sheet with the final part of their case study where the team gave an aggregate weighting of each individual's effort to the team assessment task. Some colleagues have sought to use SPARK for managing self and peer assessment to encourage student contributions to assessed team tasks in their subjects but found it did not work. As the chapter has illustrated, peer learning has considerable potential for engaging students in effective learning in very large classes but, even more than usual, its success depends on convincing other staff and students of its value. The development of the online confidential self and peer assessment system is producing positive responses in Australia and Europe.

Offers a historical overview of how assessment practices have changed over the past four decades. Boud observes that despite some drastic changes and shifts in positive directions, the emphasis on formative assessment has lessened today in that now much assessment ‘counts’ towards a final mark. He calls for a new agenda in assessment practices, one that focuses explicitly on developing students’ capacity for judgement. This capacity for judgement is associated with the emergent agenda of ‘sustainable assessment’, which suggests that the purpose of assessment is not just to generate grades, or to influence present learning, but importantly to build students’ capacity to learn and assess themselves beyond the current task, for example as they are engaged in lifelong learning in professional practice.

© John Stephenson, Mantz Yorke and contributors 1998. Introduction Learning contracts provide a way of structuring learning and assessment which allows students significantly to direct their own learning within the overall goals of a course. While they are becoming an increasingly common feature of higher education courses which promote capability, there has been little investigation of the expectations academic supervisors have of either the initial proposal or the completed work.

© John Holford, Peter Jarvis, Colin Griffin and named contributors, 1998. There now exist a number of major examples of this radical kind of work-based learning course. While there is a different emphasis in each institution, and arrangements for the management of learning vary, all permit the completion of a Bachelors or Masters degree primarily through work-based study. Moves to work-based learning are being prompted by the need for rapid diversification of university funding, as well as by the educational needs of members of the workforce and employers who have not benefited sufficiently from the older, front-end loaded models of higher education.

© D Boud, G I Feletti and named contributors, 1997. All rights reserved. Berkson, L (1993) Problem-based learning: have the expectations been met? Invited Review, Academic Medicine, 10, S79-88. Bond, D (1985) Problem-based learning in perspective, in D Bond (ed.) Problem-Based Learning in Education for the Professions. Sydney: Higher Education Research and Development Society of Australia. Boud, D (1995) Enhancing Learning through Self Assessment. London: Kogan Page. Bouhuijs, F; Schmidt, H and van Berkel H (eds) (1993) Problem-Based Learning as an Educational Strategy. Maastricht: Network Publications.

As the earlier chapters have shown, there is now considerable research-based knowledge about feedback for learning in a wide variety of modes and contexts. There are also a vast number of different ways of undertaking feedback that have been used. Many of these authors challenge common taken-for-granted assumptions about what feedback is and how it should be conducted. However, even if the challenge is accepted and Qllr notion of feedback reframed, there is the problem of choice of particular strategies and approaches. On what basis should feedback strategies be selected for any given purpose in any given situation? At what stage is it appropriate that such decisions are made? Who should be involved in them? And, should these decisions change as learners advance along the novice to expert spectrum in their field of study? This chapter brings together the themes of the book to focus on the design and choice of feedback approaches. It seeks to provide a summary resource to aid decision-making. It starts by revie"Wing some of the main messages that have arisen so far and mOves to what needs to be done to establish a program climate conducive to healthy feedback practices and produce a particular feedback episode. Rather than provide a reference-rich account, we have used our o\\tTI judgement about what has been presented earlier to generate key decision po~ts in planning for learning and the issues that need to be considered at each pomt. We do not seek to be prescriptive but to raise questions about what might be considered at each stage of thinking about a program. The chapter ends by reflecting on how organisational and dispositional changes might be made to move from a conventional view of feedback to one that has a serious influence on learning.

We all experience the influence of feedback in our lives and in our work. We are told that we can't park our car in a particular space, and we choose to go elsewhere. Our students tell us that they don't understand a point we have made in class and we find another way of explaining it. We get referees' comments on a paper submitted to a journal, we make revisions and resubmit it. These are familiar examples of everyday feedback. Feedback is a normal part of our lives; it is ubiquitous. If it seems to work so normally and so regularly, why then does it appear to be so troublesome in higher and professional education? Why is it that students complain more about feedback than almost any others parts of their courses? Is what we are doing so wrong, or are there other explanations of what is rapidly becoming a crisis of concern?

Almost all students and teachers spend a vast amount of time writing. Paradoxically, in the age of electronic media, \VIi.ring, emails or 'texting' has replaced many other forms of communication, so when the opportunity arises to give or receive feedback there is a high likelihood that it will be delivered in written form. There are great differences in what and how we write. For example, as young researchers, we might spend a week or tw-o blitzing out a long rambling article. Then, in the struggle to communicate to others, over the following many months, we hone the paper into something that somebody else would like to read, using many self-feedback loops. This type of auto-feedback forms the major part of most academics' experience of feedback. Finally when we ran out of our own resources, or when our writing showed visible improvement, we sent it to someone else for judgement or formal feedback. Sometimes that person was the editor of the intended journal, and sometimes it was just a colleague, Either way, there was usually a long wait, and the results were altogether unpredictable. 'iI1ith all this writing being done, and with the amount of self-feedback that we generate, one might think that: a) a lot of feedback to others gets delivered via writing; b) guidance about how to give written feedback is pretty well formulated; c) this guidance is underpinned by substantive research on written feedback.

Feedback has proved problematic for individual learners, for teachers and for institutions. The lack and availability of it is criticised by students. Teachers bemoan the burden of marking. And leaders of educational institutions wonder why, of all things they have to deal with, feedback creates so much difficulty. There is no shortage of proposals and recipes for action. Is it just a matter of seriously attending to these and ensuring that they are put into practice? If only it were clear what feedback was and how it could be implemented well, then the problems should severely diminish. The fact that so much has been written about the topic and so much energy has been expended without resolving the problem suggests more of the same is not enough. So much has been invested in the idea that it can't be wished away; it has to be confronted. New ways of thinking about feedback are needed. A clear view of current assumptions and practice is needed as a starting point, but it is also important to step back and examine feedback in its wider context to see what it promises and what it might be reasonably be expected to do.

© 1999 David Boud and John Garrick. Learning at work has become one of the most exciting areas of development in the dual fields of management and education. It has moved to become a central concern of corporations and universities; it is no longer the preoccupation of a small band of vocational training specialists. A new focus on learning is changing the way businesses see themselves. At the same time, educational institutions are realising that they need to engage with the world of work in a more sophisticated manner than ever before.

Considerations of employee-driven innovation generally posit innovation as an advance in the substantive products, services and/or processes of an organisation. More broadly, innovation can also refer to anything that seeks to do something new, or address a concern that would not otherwise be met. Employees contribute to innovation in many ways: they can generate and/or implement a product or service; they can generate and/or implement new technologies; however, they can also influence the ways in which an organisation adapts and evolves over time in more subtle ways through instigating work practice changes. Although these more subtle changes may not appear under the banner of organisational innovation they nevertheless contribute to the creation and application of new organisational processes, practices and outputs. They may also never be part of the conscious and explicit agenda of the organisation or be something that managers have a strong role in initiating. However, their effects can be cumulative and substantial.

In this chapter, we connect and challenge two conventional assumptions in workplace learning research and organizational change research: that learning can be understood isolated from its interrelationship with work and that managing change is a process to revert organizations back to desirable forms of stability. We believe the nexus of learning and change in organizational work lies in unpacking the apparent paradox between how work practices regularly get carried forward (persist and perpetuate), yet also adapted (change) by workers to achieve the purposes of work. We draw significantly from the theoretical writings of Schatzki and argue that practice theory has much to contribute in conceptualizing more dynamic views of organizing, work and learning. We illustrate our use of Schatzkian concepts by discussing how workers at an Australian utility company using safety practices to learn how to become new kinds of safe workers.

This chapter provides a snapshot of research in workplace learning in Australia from the perspective of some of its key players. Workplace learning as an interdisciplinary field covers a wide range of research and programme interests. This is exemplified in the diversity of journals that publish research on workplace learning (Fenwick, 2006). The range and complexity of the field contributes to its richness, but at the same time it presents interesting challenges in terms of making decisions on what is and what is not workplace learning. In order to identify and summarise the theoretical frameworks, conceptions and contributions to understandings of Australian workplace learning knowledge and practices we used an approach that focused on the researchers’ own representation of their workplace learning research. At the same time we also recognise that as authors of this chapter we are making our own selection of what should be included within the scope of workplace learning research and we are re-presenting the researchers’ representations of their work.

A brief picture of the broad socio-political context in Australia is provided in order to draw attention to features that influence research in workplace learning. We believe that the way research in workplace learning is undertaken and the nature and shape of its contributions to knowledge are a consequence of a dynamic relationship between academic researchers, institutional settings and government policies. The policies and settings provide both boundaries and opportunities, and the way these are enacted upon is influenced by a number of factors to do with the researchers — their disciplinary histories and professional trajectories, together with their institutional, national and international networks.

We begin the chapter by explaining our methodological approach. This is followed by a description of some of the features of the Australian research policy context of workplace learning. This leads into a secti...

In this chapter I argue that assessment should be oriented towards developing practice, even when the particularities of practice and the specific contexts for practice are not known. Indeed, a future-orientation is especially required when the specific demands of practice are unknown and unknowable. Assessment needs to do more than cel1iry; it also has an essential educational role that, if neglected, means that students are not prepared for the uncertainties they face. The emphasis needs to be on the processes of acquisition of knowledge and skills and how learners can develop their own capabilities and those of others. Such an orientation has profound implications for the way assessment in higher education is conducted, both in initial and continuing professional education.

Notions of reflection and reflective practice have hecome well established in professional education since the late 1980s. While some applications of these 1deas m courses have distorted their original intentions and taken an excessively instrumental. approach to their use, they have nevertheless provided useful frammg deVICes to help conceptualise some important processes in profeSSIOnal learnmg. One of the reasons why they were readily accepted is because they sh:u:ed an individualised view of learning with the very program?, e~ In whIch they were used. In the 2000s we are, however, seeing a q,:,esrlOO!ng ?f an overly individualistic view of learning previously associated with reflectIOn, a f~us on the nature of professional practice and an exploratlOn of alternatIve conceptions chat view reflection within the context of serrings wh~ch necessarily have more of a group- or team-based work oneoratlOn. ThIS chapter questions whether we should reject earlier views of reflection, rehabilitate them to capture their previous potential or move to new ways of tegarding reflection that are more in keeping with what we know abour the context of practice

Assessment in higher education is being challenged by a multiplicity of demands. The activities predominantly used examinations, assignments and other kinds of tests have emerged from within an educational tradition lightly influenced by ideas from psychological measurement, but mostly influenced by longstanding cultural practices in the academic disciplines. Assessment in higher education has for a long time been a process influenced more from within the university rather than externally. It has typically been judged in terms of how well it meets the needs of educational institutions for selection and allocation of places in later courses or research study, and whether it satisfies the expectations of the almost totally exclusive academic membership of examination committees. Within courses, it has been judged by how well it meets the needs of those teaching. In more recent times it is judged in terms of how well it addresses the learning outcomes for a course

NA

NA

Conference proceedings

BACKGROUND Recent national and international research has identified a number of gaps in the development of engineering graduate capabilities: one is the real-world problem-solving ability, which is linked to a lack of integration of theoretical and practical knowledge (ASEE 2009; King,2008; Royal Academy of Engineering, 2006; Sheppard, Macatanga, Colby & Sullivan 2009; Male, Bush & Chapman 2009; Walther & Radcliffe 2007). Another is written (and spoken) communication (King, 2008; Male, Bush & Chapman 2011). There is strong evidence to indicate that these gaps occur in part as a result of a predominance of engineering curricula in universities which emphasise knowledge acquisition, and the prevailing assessment tasks that focus learning on atomised pieces of knowledge. Such an approach encourages surface learning approaches, resulting in graduates who may lack the integrated knowledge required for engineering practice and who have limited communication capabilities. PURPOSE There is, however, a body of research that suggests deep approaches to learning in the disciplines can be achieved through particular kinds of writing that provide the opportunity to explore concepts which link theory and practice, thus developing both writing ability and integrated understanding. This paper presents the preliminary phase of a study to investigate the strategic use of discursive writing to foster both a deeper approach to learning and enhanced written communication skills in the engineering curriculum. The study focuses on discursive writing as a means of providing students with the opportunity to explore the theories and concepts that they are learning, in order to integrate knowledge from different parts of the curriculum and to link the theories to engineering practice. DESIGN/METHOD In order to investigate how writing is currently practised and assessed in Australian engineering curricula, a preliminary analysis of written assessment tasks in a unit of study in the mechani...

This paper reports on the first phase of an Australian inter-disciplinary partnership study concerned with professional learning of experienced engineers. It is a theoretically motivated, qualitative paper that aims to produce detailed descriptions of professional learning that arise within professional engineering work. The paper uses practice theory to conceptualise professional learning. By using `practices as the units of analysis, professional learning is understood as an integral part of everyday work practices that is embodied, relational and material rather than an individual attribute. The paper concludes by suggesting that practice theory may provide organisations with an alternative perspective of workplace learning, inviting them to reconsider how professional learning is acknowledged, rewarded and fostered in organisations

It is well recognised that changes in the contemporary world demand that professionals continuously learn. Often this continual professional learning takes the shape of formal courses, seminars and other obviously educational events. The starting point of this paper is an acknowledgement that people also learn in the day-to-day practices that constitute their work. Work can be understood as a bundle of practices that are typically shared by most people employed in that profession. For engineers, and experienced engineers in particular, an example might be attending design review meetings, toolbox talks and or carrying out site walks. In this paper we posit that these practices afford important opportunities for professional learning.

The point we explore in this paper, is how can we conceptualise our role in adult education when =the field` itself is so diverse? Moreover, it is =curious and curiouser to think about our role in =the field` before we can actually articulate what the field is? Is it like the grin on Lewis Carroll`s Cheshire cat, or do we still have a tangible practice to pursue?

Opportunities for expanding what is known about workplace learning can be missed when the focus of research remains on learners alone. Rather, accounts of learning at work need to acknowledge not only those who learn but who else may be implicated. Such accounts would require a wider lens that notices the work practices and relationships that contribute to learning, and in particular those involving the counterparties to those learning. We cannot talk about learning practices without also naming those who enact the practices. Hence, in this paper we intend `talking up the role of these others, or as others suggest `word (Richardson, 1994: 923) the practices of learning at work and those enacting them into existence. To do this requires looking beyond (or perhaps more accurately, amongst) the learners, as well as toward the workplace relationships and practices that provide opportunities for learning.

In tune with lifelong learning discourses, adult learning is now understood to be anywhere and everywhere in classrooms, in workplaces, in community settings and indeed in everyday life. This distribution of learning sites is accompanied by changing understandings of the relationship of formal and informal learning and of the role of adult education and adult educators. These changes in turn are reshaping learning theories and practices together with redirecting the research focus of adult educators. We suggest that this redirection can be a useful one. By researching learning across contexts, researchers are better able to engage with various kinds of pedagogic practices and therefore can contribute better to the debates and critiques of the changing relationship between education, training and learning.

The paper reports on the initial findings of the ARC project Beyond Training & Learning Integrated Development Projects in Organisations

This paper draws empirical data to explore how workers in one organization are 'inventing' and continue to develop what constitutes their work, ie making up their jobs. Doing so troubles accounts of common work-practices that pre-suppose stable organizational contexts, pre-designed jobs and agent- less workers.

The growth of knowledge work and the construction of the knowledge worker (Drucker 1993) have activated widespread interest in learning in organisations. In this contemporary context, investigations and characterisations of learning (at) work and work as learning, formal versus informal learning and so on, have drawn learning out of its educational, training or institutionally-oriented 'home' to situate it within business practices. New manifestations of learning can be linked to organisational practices such as, for example, coaching and mentoring, and perhaps less obviously to practices of performance management, teamwork, career development, and the like. These organisational practices are creating new meanings and understandings for learning at work and are therefore producing different learning experiences, and workers, from those of the past. We are interested in how and if these organisational practices promote learning while they simultaneously enact organisational functions.

An important rationale for higher education is that it equips students for learning beyond the point of graduation. This paper considers the role that assessment plays in this. It suggests we need to take a new perspective on assessment: assessment to promote learning throughout life. It focuses on ideas that can be used to contribute to the construction of assessment practices and on wider implications for course design. It concludes by exploring barriers to acceptance of this perspective and how they might be addressed.

The development of the research potential of university staff has been given less attention than many other aspects of professional development, particularly teaching development. Yet there is an important need for the development of staff in the research role in the light of growth over the past ten years of higher education and changes to the organisation of the sector in both Australia and the UK. This paper examines one strategy for research development: the use of writing groups. It argues that writing is best seen as a starting point, rather than an endpoint, of the research process and hence that fostering academic writing is a useful place to do research development work. The paper provides details of the use of a number of writing groups over three years in a Faculty and explores the responses of leaders and participants. It identifies factors which appear to be important in the successful use of this strategy and focuses on the contextual conditions required for initiatives of this kind to be effectively implemented.

`Building Research Supervision and Training in Australian Universities was undertaken with the aims of identifying existing higher degree research supervisor training provisions; identifying current and future needs of supervisors and making recommendations that assist universities in their ongoing development of effective higher degree research supervisor training.