UTS site search

Dr Keith Willey

Biography

Keith's research interest include Satellite Communications, Biomedical Applications, Professional Skill Development and the Formal and Informal processes through which Professionals and in particular, Engineers learn.
Keith's is also researching the learning mechanisms associated with collaborative learning oriented assessment activities. He has developed many innovative assessment processes and has lead the ongoing development of the Self and Peer assessment software tool SPARKPLUS.
Keith has received several awards related to his innovative teaching and learning practices and associated research including an ALTC Teaching Fellowship (2010), The Australasian Engineering Education Teaching Excellence Award (2010), UTS Team Teaching Award (2009) and a UNSW Innovative Teaching and Educational Technology Fellowship (ITET) (2004).
Before joining UTS, he was a Senior Lecturer at UNSW prior to which he worked for 20 years in the Telecommunications and Broadcasting fields.

Professional

Core Member Centre for Real-Time Information Networks
Associate member Centre for Health Technologies
Associate member Centre for Learning and Change
Member of the Technology & Education Design & Development Group (TEDD).
Organising committee Australasian Association for Engineering Education (AaEe) Conference 2010.

Image of Keith Willey
Associate of the Faculty, Associate Dean (Teaching & Learning)
Core Member, CRIN - Centre for Realtime Information Networks
BE (UTS), PhD (UTS)
Member, Institution of Electrical and Electronic Engineers
 
Phone
+61 2 9514 7605

Research Interests

Engineering Education, Peer Review and Academic Standards.

Current research projects

  • Professional Skill Development and the Informal processes incorporated in Practice through which Professionals and in particular, Engineers learn
  • The learning mechanisms associated with collaborative and interactive learning oriented assessment activities
  • The impact of Peer Review on research quality
  • Learning associated with Self and Peer Assessment
Can supervise: Yes

Communication and Transmission Systems, Telecommunications, Engineering Design.

Chapters

Rooney, D.L., Willey, K., Gardner, A.P., Boud, D.J., Reich, A.J. & Fitzgerald, T. 2014, 'Engineers' professional learning: through the lens of practice' in Williams, B., Figeiredo, J. & Trevelyan, J. (eds), Engineering practice in a global context: understanding the technical and the social, CRC Press/Balkema, Leiden, The Netherlands, pp. 265-280.
View/Download from: UTS OPUS

Conferences

Willey, K. & Gardner, A.P. 2015, 'Learning activity design and scaffolding to promote sustainable changes in students' goal orientation', Research in Engineering Education Symposium 2015, Dublin Institute of Technology, Dublin, Ireland.
View/Download from: UTS OPUS
While previous research has shown that assessments impact how students view the curriculum and influences what they learn and how they learn, the way that students approach their learning is also affected by aspects of the teaching and learning environment. Two different approaches adopted by students are mastery and performance goal orientation. Students with a performance goal orientation plan their approach to achieve a certain grade. This might be a high distinction or simply a pass but they do what has to be done to achieve this grade sometimes with little evaluation of what they have actually learnt. Conversely, mastery students seek to learn as much as possible, they strive to deeply understand the subject material and be able to apply it in different contexts. In this paper we explore the impact of a flipped instruction design and scaffolding to promote sustainable changes towards a mastery goal orientation.
Gardner, A.P. & Willey, K. 2015, 'Engineering academics' identity transitions in becoming established engineering education researchers', Research in Engineering Education Symposium 2015, Dublin Institute of Technology, Dublin, Ireland.
View/Download from: UTS OPUS
Becoming a particular type of scholar or researcher and developing a higher level of expertise in a field of academic activity involves a transformation of identity. The purpose of this research is to help members of the engineering education community better understand the transition to becoming an engineering education researcher, and through discussion, evaluate their own progress and support their peers in this process. Our research approach is interpretive using the identity-trajectory concept as a framework with which to view the data collected through semi-structured interviews of engineering academics who are active participants in the Australian engineering education community. Our analysis demonstrates how the intellectual, networking and institutional strands can be characterised for different levels of expertise. This allows individual researchers to self-assess their development and for those managing these researchers to plan activities for their continuing development.
Willey, K., Meng, Q. & Gardner, A.P. 2015, 'Insights from using a subject specific Facebook group for student engagement and learning', Research in Engineering Education Symposium 2015, Dublin, Ireland.
View/Download from: UTS OPUS
Although discussion boards have been available in the Learning Management System (LMS) for several years, they have not served well as a means of extending student engagement outside class time. The social media site Facebook was incorporated into an Engineering Mechanics class with the aim of increasing subject specific student engagement. This paper reports a small preliminary study exploring the effect of the introduction of the Facebook group. These students found the Facebook group increased the frequency of their engagement with the subject material compared to other subjects, and they considered it valuable because almost all students and the instructor were involved. However, students emphasised that the Facebook group was a supplement to, and not a substitute for, the face-to-face lecture and tutorial sessions. This study confirmed the value of undertaking focus groups with students to assist interpretation of data collected by more objective methods such as social network analysis.
Gardner, A.P., Willey, K. & Figueroa, E. 2015, 'Supporting students learning despite difficult workplace interactions', SEFI Annual Conference, Orleans, France.
View/Download from: UTS OPUS
Students report the opportunities for learning provided by an internship or work experience include gaining an understanding of engineering practice, developing competencies, networking, awareness of the relevance of engineering studies, awareness of possible future roles and future employers. However, these learning opportunities can be negatively affected by undesirable workplace practices such as bullying and discrimination which may be due to gender, culture, disability, sexual orientation or age. We can't protect students from these types of interactions but we can help them deal with them when they occur. This paper describes a workshop run for undergraduate students to increase their awareness of issues often encountered by students that may result from workplace culture such as discrimination, differing expectations and stereotyping. Participants in the workshop were sensitised to the importance of visual cues in generating perceptions of people that we haven't met and the impact this has on stereotyping. Through a series of role plays participants explored how to interpret and respond to situations as they arise from their own and other student's experiences. Participants reported that the workshop increased their capacity to recognise even subtle instances of workplace bullying and discrimination and hence increased their resolve to not participate in, support or promote such behaviour. This awareness also improved their confidence to deal with the negative behaviours themselves and support others that may be experiencing them. The results suggest that all students would benefit from incorporating a series of such workshops in professional development activities.
Goldsmith, R. & Willey, K. 2015, 'Activity theory analysis of the visibility of writing practicesin the engineering curriculum', Research in Engineering Education Symposium, Dublin, Ireland.
View/Download from: UTS OPUS
Gardner, A.P. & Willey, K. 2014, 'Authors' perceptions of peer review of conference papers and how they characterise a 'good' one', SEFI 2014 Educating Engineers for Global Competitiveness, Birmingham, UK.
View/Download from: UTS OPUS
Peer review has been the focus of an ongoing study at the 2010, 2011 and 2012 conferences of the Australasian Association for Engineering Education (AAEE) [1 – 3]. We found that the opportunity to use the peer review process to induct people into the field and improve research methods and practice was being missed with almost half of the reviews being rated as 'ineffectual'. The results also highlight the lack of a shared understanding in our community of what constitutes quality research. The study has been extended to explore the AAEE authors' perspective/s of the potential of peer review to support their development as researchers. This is particularly relevant to our community as engineering education research is still emerging as a recognised research area in Australian universities [4, 5]. Another complicating issue is that most scholars who identify with this emerging field are engineering academics [6] who may hold research qualifications and expertise in their own stereotypical engineering field but are faced with developing new perspectives and expertise when moving into educationally related research [7]. As a result of engineering education research being both emerging and interdisciplinary there is a wide variety of views as to what quality research looks like [6, 8]. The implication for authors is that their work can generate divergent opinions which can be difficult to interpret and/or reconcile for the final version of their paper. A broad objective of this research is to help members of the AAEE community to better understand themselves and their peers as they struggle with the new ideas, methods etc involved in social/educational research compared to positivist perspective of most engineering research. In better understanding themselves and their peers this transition can hopefully be better supported. More specifically the findings of this project can inform future strategies of professional associations such as AAEE and SEFI in regards to both...
Gardner, A.P. & Willey, K. 2014, 'Mapping the landscape of engineering education research: an Australian perspective', SEFI 2014 Educating Engineers for Global Competitiveness, Birmingham, UK.
View/Download from: UTS OPUS
Engineering education research is still consolidating as a recognised research area in Australian universities [1, 2]. A current project funded by the US National Science Foundation is attempting to develop a taxonomy for engineering education as a research area [3]. Our project takes a different perspective. Finding out what topics members of the AAEE community are researching will enable us to view engineering education as a knowledge domain that includes a variety of areas of endeavour. Our intention is to assist engineering education researchers to appreciate the differences in methods, frameworks and theories typically used in different parts of the landscape. Our intention is for the landscape to be used as the foundation for conversations to facilitate the social construction and subsequent understanding of the community standards and norms used to judge research quality. This will help both the community and individuals to articulate and understand observed changes in their and their peers' research as expertise is developed, as well as provide a language for researchers, particularly those new to the field to plan, discuss and evaluate this development if they so choose.
Willey, K. & Gardner, A.P. 2014, 'Combining flipped instruction and multiple perspectives to develop cognitive and affective processes.', SEFI 2014 Educating Engineers for Global Competitiveness, Birmingham, UK.
View/Download from: UTS OPUS
While the phrase 'flipped learning' may be relatively new it has been practised by some academics and teachers for decades. Flipped learning or as we prefer flipped instruction (as the learning should ideally occur at all stages of the process) is a form of blended learning that replaces transmission-based lectures with more participative, interactive and collaborative learning opportunities. Activities are typically undertaken before, during and after class, freeing in class time to participate in activities and engage with concepts at a higher level. Flipped activities should require students to engage in dialogue and include assessment (typically formative) to allow them to evaluate their understanding and progress in meeting the desired learning outcomes. Flipping creates an opportunity for academics to provide more dynamic and thus specific feedback to students, and to receive feedback from students about both the activities they are undertaking and what they don't yet understand. Hence, the learning environment is socially constructed as academics and students combine to influence the nature, focus, complexity and timing of subject activities. Social cognitive theory provides a way to frame our thinking about this learning context by foregrounding aspects such as the environment created for learning, as well as considering development of student self-efficacy and how to scaffold the processes for this development. This paper reports part of an ongoing study investigating relationships between engagement, goal orientation, affective outcomes and professional identity development in the context of flipped instruction. This study supported modification of our collaborative learning model [1] to explicitly provide multiple perspectives to assist students to overcome learning thresholds, develop disciplinary literacy, professional identity and expertise. In addition, it highlighted the impact of scaffolding and learning activity design on affective outcomes such...
Willey, K., Gardner, A.P. & Kadi, A. 2014, 'Flipped learning: comparing the student experience from 1st year to postgraduate', SEFI 2014 Educating Engineers for Global Competitiveness, Birmingham, UK.
View/Download from: UTS OPUS
While flipped instruction may be the standard practice in many social science programs it is often promoted as a recent innovation in learning design within science and technology classrooms. Flipped instruction is a form of blended learning that replaces transmission-based lectures with more participative, interactive and collaborative learning opportunities. Activities are typically undertaken before, during and after class, freeing in class time to participate in activities that often engage concepts at a higher level. Flipped activities typically require students to undertake out of class preparatory work, engage in in-class dialogue and collaborative learning and we suggest be more independent in their approach to learning. Typically instructors focus on their students' cognitive development in relation to the subject material. However, students' expectations of how a subject can be delivered and their level of metacognitive skill development will also affect how they engage with the content of a subject and hence impact on their learning. Our investigation is concerned with how these factors impact students' response to flipped instruction. This paper reports the exploratory phase of an ongoing study investigating the impact of flipped instruction on first year, third year and postgraduate students. We found that most students irrespective of their stage of study preferred flipped instruction compared to the more traditional lecture style approach agreeing that it had a positive impact on their learning experience. There was also evidence that the different learning expectations and focus of the undergraduate students in particular those in first-year meant that many may not be ready for the responsibility and independence demanded to engage with flipped instruction without scaffolding and support.
Gardner, A.P., Willey, K., Vessalas, K. & Li, J. 2014, 'Experiences with flipped learning in subjects in consecutive stages of aCivil Engineering programme', Australasian Association for Engineering Education Annual Conference 2014, School of Engineering & Advanced Technology, Massey University, Turitea Campus, Palmerston North 4442, Wellington, NZ.
View/Download from: UTS OPUS
CONTEXT Flipped learning is an instructional approach which allows instructors the opportunity to use a blending of online and face to face learning activities. The main affordance of flipped learning is the opportunity to free up class time to allow students and instructors to engage in collaborative learning activities designed to consolidate and deepen conceptual understanding of the subject material. Research has shown that participating in flipped instruction can change the way students approach their studies and improve motivation. PURPOSE OR GOAL We are interested in students' experience of the flipped environment, particularly their expectations of a learning environment, and the level of responsibility they take for their own learning. The purpose of this analysis is to create a baseline against which future instances of flipped learning can be compared. APPROACH Student perceptions of flipped instruction were investigated through survey responses and observations of students in a first year subject (Engineering Mechanics) a second year subject (Engineering Computations) and a third year subject (Construction Materials) in a Civil Engineering degree programme. In particular, students were asked to explain the impact of the flipped activities on their learning experience including any changes in how they approach their studies or managed their time. OUTCOMES Most students in each of the three subjects agreed that they 'liked' flipped instruction compared to the traditional lecture format. The majority of students in each subject also agreed that it is reasonable to expect students to prepare before attending a face to face session. However, some students made strong negative comments demonstrating how the flipped environment did not meet their expectations of how learning should be organised. This study suggests that students in the second and third year subjects were not necessarily showing signs of better self-regulation and time management sk...
Willey, K. & Gardner, A.P. 2014, 'Impact of student's goal orientation in a flipped learning environment', Australasian Association for Engineering Education Annual Conference 2014, School of Engineering & Advanced Technology, Massey University, Turitea Campus, Palmerston North 4442, Wellington, NZ.
View/Download from: UTS OPUS
BACKGROUND Flipped instruction is a form of blended learning that typically encompasses the use of technology to move instruction and preparation outside the classroom. This facilitates the use of 'in class' time for more participative learning activities. These activities should require students to interact and collaborate to improve both their learning and their learning experience. Many researchers have highlighted issues of importance to designing learning activities including student behaviour, assessment, student self-efficacy and goal orientation and the importance of dialogue and feedback for learning. PURPOSE In this paper we explore these themes in the context of a flipped instruction environment for different types of students: those with a learning mastery orientation, those focussed on grade achievement and a subset of grade achievement students - those who struggled to pass. APPROACH Student perceptions of flipped instruction were investigated through survey responses, observations and focus group discussion. In particular, students were asked to explain the impact of the flipped activities on their learning experience including how they approached their studies or managed their time. OUTCOMES Students reported liking flipped instruction compared to the more traditional lecture style delivery format. Most students believe that it had a positive impact on their learning experience and promoted them to become more independent and responsible learners. The main finding that emerged from the interviews and focus group was how the student's goal orientation affected their engagement with the learning opportunities provided. CONCLUSIONS The authors present a model showing potential pathways for a change in goal orientation prompted by the quality of assessment and learning design. We found evidence to support our theory that a student's orientation can be changed temporarily by the quality of the learning opportunity provided and in particular...
Goldsmith, R.J. & Willey, K. 2014, 'Invisible writing practices in the engineering curriculum', Proceedings of the AAEE2014 Conference Wellington, New Zealand, Australasian Association of Engineering Education, Wellington, NZ.
View/Download from: UTS OPUS
Gardner, A.P. & Willey, K. 2013, 'Exploring the impact of peer review on the development of engineering education researchers', Proceedings of the 24th Annual Conference of the Australasian Association for Engineering Education AAEE2013, 24th Annual Conference of the Australasian Association for Engineering Education AAEE2013, Griffith School of Engineering, Griffith University, Queensland, Australia, Surfers Paradise, Queensland, Australia, pp. 1-9.
View/Download from: UTS OPUS
Willey, K. & Gardner, A.P. 2013, 'Flipping your classroom without flipping out', SEFI Annual Conference 2013, SEFI Annual Conference, SEFI, Belgium, pp. 1-9.
View/Download from: UTS OPUS
There is some contention as to who are considered to be the pioneers of flipped learning. Within the secondary school system Bergman and Sams, who used live video recordings and screencast software in 2007, are frequently mentioned [1- 3]; while within the tertiary sector, Mazurs work on peer instruction is often highlighted [4, 5]. While the phrase `flipped learning may be relatively new it has been practised by numerous academics and teachers for decades, and is the disciplinary norm in some contexts, for example, it is extensively used in social science classes. To find a popular accepted definition of flipped learning we consulted Wikipedia, which describes it as ...a form of blended learning that encompasses any use of technology to leverage the learning in a classroom, so a teacher can spend more time interacting with students instead of lecturing [6]. We would modify this definition to omit the need for the use of technology, while it is common practice to replace in-class lectures with online video or audio files, out of class readings from text, notes or inquiry-based activities using non-online resources may also be used. Hence the requirement for flipped learning is that didactic transmission-based lectures are at least in part replaced with out of class tasks allowing class time for participative learning activities. Additionally, we would suggest that it should be referred to as flipped instruction as the learning should occur at all stages of the process. Flipped activities should preferably require students to engage in dialogue and include assessment (typically formative) to allow students to evaluate their understanding or progress. Furthermore, flipped instruction should not merely create an opportunity for academics to provide more personal feedback and assistance to students, but also to receive feedback from their students about the activities that they are undertaking and what they dont yet understand. In this way the learning environment is...
Gardner, A.P. & Willey, K. 2013, 'Mapping the engineering education research landscape in Australia', Proceedings of the 24th Annual Conference of the Australasian Association for Engineering Education - AAEE2013, Annual Conference of the Australasian Association for Engineering Education AAEE2013, Australasian Association for Engineering Education, Gold Coast, Queensland, Australia, pp. 1-10.
View/Download from: UTS OPUS
BACKGROUND Engineering education research is still consolidating as a recognised research area in Australian universities. A current project funded by the US National Science Foundation is attempting to develop a taxonomy for engineering education as a research area. Our project takes a slightly different perspective by using a landscape model to describe engineering education as a knowledge domain that includes a variety of areas of endeavour. PURPOSE This paper is motivated by questions around the range of topics being addressed in the AAEE community and as a means of initiating a discussion about how we define, evaluate, understand and move within our research domain. APPROACH This paper reports data collected as part of a wider project examining the peer review process for the Australasian Association for Engineering Education (AAEE) annual conference. During semistructured interviews nineteen participants used one or two coloured adhesive stars to locate their paper on a model of the engineering education research landscape presented in this paper. The location of the stars was then analysed in relation to various elements of the model and the explanations were coded in NVivo 10 for themes relating to the star location. OUTCOMES All participants could locate the topic of their conference paper on the presented model, and articulate clearly why their star belonged in the selected location demonstrating an individual understanding of the focus and outcomes of their research. Not surprisingly most stars were clustered in the `teaching and learning of engineering element or on one of the trajectories leading to it. This reflects that for many participants, their educational publications are inextricably linked to their practice of teaching engineering. Interestingly, there were strong voices from participants across all expertise levels and university types against a perceived move to make the annual AAEE conference focus on theoretical research. This was seen a...
Gardner, A.P. & Willey, K. 2013, 'The response of emerging engineering education researchers to peer review of conference papers', Proceedings of the Research in Engineering Education Symposium 2013 - Malaysia, Research in Engineering Education Symposium 2013 (REES 2013), Curran Associates, Inc., Putrajaya, Malaysia, pp. 1-8.
View/Download from: UTS OPUS
Our research, an element which is reported in this paper, investigates the effectiveness of peer review of conference papers in enabling peer learning within the Australasian Association for Engineering Education (AAEE) community. This paper reports the responses of six new/emerging? authors from three types of Australian universities to the peer reviews of their AAEE 2012 conference papers. The findings and discussion focus on those aspects of the reviews and the authors? circumstances that appear to either enable or constrain their development as engineering education researchers. The study finds that authors belonging to a discipline-based educational research group made substantial changes to their papers before final submission and we argue that these research groups support these authors in developing their academic identity as an engineering education researcher.
Willey, K. & Gardner, A.P. 2013, 'A Framework to Develop Academic Standards and Improve Feedback Quality', Proceedings of the Research in Engineering Education Symposium 2013 - Malaysia, Research in Engineering Education Symposium 2013 (REES 2013), Curran Associates, Inc., Putrajaya, Malaysia, pp. 1-7.
View/Download from: UTS OPUS
This paper describes the benefits of co-constructed understandings of academic standards using a framework that includes collaborative conversations about assessment artefacts supported by online self and peer assessment technology. In particular in this paper we describe this process in the context of addressing the variations in understanding, grading and feedback between multiple tutors in large classes. The described implementation of the framework not only improved understanding and reduced grading variation, it also improved student feedback by helping tutors to convert tacit understandings into explicit explanations and resulted in improved student satisfaction with the assessment process.
Willey, K., Hancock, P. & Dale-Jones, G. 2013, 'Impact of collaborative peer and self assessment on students' judgment and written communication', TL Forum (2013). Design, develop, evaluate: The core of the learning environment. Proceedings of the 22nd Annual Teaching Learning Forum, 7-8 February 2013., Teaching and Learning Forum, Murdoch University 7-8 February 2013, Murdoch University, Perth Western Australia Murdoch University, pp. 1-12.
View/Download from: UTS OPUS
https://otl.curtin.edu.au/professional_development/conferences/tlf/tlf2013/refereed/contents-refereed.html
Gardner, A. & Willey, K. 2013, 'The response of emerging engineering education researchers to peer review of conference papers', Research in Engineering Education Symposium, REES 2013, pp. 223-230.
Our research, an element which is reported in this paper, investigates the effectiveness of peer review of conference papers in enabling peer learning within the Australasian Association for Engineering Education (AAEE) community. This paper reports the responses of six 'new/emerging' authors from three types of Australian universities to the peer reviews of their AAEE 2012 conference papers. The findings and discussion focus on those aspects of the reviews and the authors' circumstances that appear to either enable or constrain their development as engineering education researchers. The study finds that authors belonging to a discipline-based educational research group made substantial changes to their papers before final submission and we argue that these research groups support these authors in developing their academic identity as an engineering education researcher.
Willey, K. & Gardner, A. 2013, 'A framework to develop academic standards and improve feedback quality', Research in Engineering Education Symposium, REES 2013, pp. 335-341.
This paper describes the benefits of co-constructed understandings of academic standards using a framework that includes collaborative conversations about assessment artefacts supported by online self and peer assessment technology. In particular in this paper we describe this process in the context of addressing the variations in understanding, grading and feedback between multiple tutors in large classes. The described implementation of the framework not only improved understanding and reduced grading variation, it also improved student feedback by helping tutors to convert tacit understandings into explicit explanations and resulted in improved student satisfaction with the assessment process. Copyright © 2013 Wiley & Gardner.
Rooney, D.L., Reich, A.J., Willey, K., Gardner, A.P. & Boud, D.J. 2012, 'Site walks as a learning practice for professional engineers', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, 23rd AAEE Conference, Swinburne University of Technology, Melbourne, Victoria, pp. 1-9.
View/Download from: UTS OPUS
It is well recognised that changes in the contemporary world demand that professionals continuously learn. Often this continual professional learning takes the shape of formal courses, seminars and other obviously educational events. The starting point of this paper is an acknowledgement that people also learn in the day-to-day practices that constitute their work. Work can be understood as a bundle of practices that are typically shared by most people employed in that profession. For engineers, and experienced engineers in particular, an example might be attending design review meetings, toolbox talks and or carrying out site walks. In this paper we posit that these practices afford important opportunities for professional learning.
Gardner, A.P. & Willey, K. 2012, 'Student participation in and perceptions of regular formative assessment activities', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, Swinburne University of Technology, Melbourne, Australia, pp. 1-10.
View/Download from: UTS OPUS
BACKGROUND The benefits to student learning from participation in formative assessments have been reported by many educational researchers and scholars. This literature reports improved engagement when students see formative activities as being highly relevant and valuable for their learning. However, many academics still report that students are reluctant to participate in a learning activity unless it contributes some marks to their final subject grade. PURPOSE The purpose of this study was to investigate the impact on learning of undertaking a series of out-of-class formative learning activities. In addition, for the students that undertook these activities, we are interested in why they chose to participate and what, if any, learning benefits they experienced from participating. In regards to the students that chose not to undertake the activities, we are interested in their reasons for not participating. We are also interested in investigating the impact on students of scaffolding the learning activities used throughout the semester. DESIGN/METHOD For the Autumn 2012 offering of Engineering Mechanics, out-of-class formative learning activities were designed for four topics in the syllabus. These activities required the students to read the relevant section of the textbook and subsequently answer multiple-choice questions provided online including entering comments to explain their chosen answer. While there were no marks allocated to these activities, the instructor did allocate more time in lectures to the material related to questions with the largest variation in responses. Students who answered the online questions could log back on at the end of the submission period to compare their responses to the instructorâs answers and reasoning. At the conclusion of the semester students were asked to complete a survey consisting of both closed and open-ended questions to investigate their perceptions of the effects of the online activities. Descriptive statistic...
Goldsmith, R., Willey, K. & Boud, D.J. 2012, 'How can writing develop students' deep approaches to learning in the engineering curriculum?', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, Swinburne University of Technology, Melbourne, Australia, pp. 1-8.
View/Download from: UTS OPUS
BACKGROUND Recent national and international research has identified a number of gaps in the development of engineering graduate capabilities: one is the real-world problem-solving ability, which is linked to a lack of integration of theoretical and practical knowledge (ASEE 2009; King,2008; Royal Academy of Engineering, 2006; Sheppard, Macatanga, Colby & Sullivan 2009; Male, Bush & Chapman 2009; Walther & Radcliffe 2007). Another is written (and spoken) communication (King, 2008; Male, Bush & Chapman 2011). There is strong evidence to indicate that these gaps occur in part as a result of a predominance of engineering curricula in universities which emphasise knowledge acquisition, and the prevailing assessment tasks that focus learning on atomised pieces of knowledge. Such an approach encourages surface learning approaches, resulting in graduates who may lack the integrated knowledge required for engineering practice and who have limited communication capabilities. PURPOSE There is, however, a body of research that suggests deep approaches to learning in the disciplines can be achieved through particular kinds of writing that provide the opportunity to explore concepts which link theory and practice, thus developing both writing ability and integrated understanding. This paper presents the preliminary phase of a study to investigate the strategic use of discursive writing to foster both a deeper approach to learning and enhanced written communication skills in the engineering curriculum. The study focuses on discursive writing as a means of providing students with the opportunity to explore the theories and concepts that they are learning, in order to integrate knowledge from different parts of the curriculum and to link the theories to engineering practice. DESIGN/METHOD In order to investigate how writing is currently practised and assessed in Australian engineering curricula, a preliminary analysis of written assessment tasks in a unit of study in the mechani...
Jolly, L., Willey, K. & Gardner, A.P. 2012, 'Quality in Engineering Education Research: arriving at consensus', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, Swinburne University of Technology, Melbourne, Australia, pp. 1-8.
View/Download from: UTS OPUS
BACKGROUND Arguably the most important opportunity to acquire the standards and norms of any discipline and develop researchers' judgement is the peer review process â and this is probably particularly true in an emerging discipline such as engineering education. Ironically, research in many disciplines has established that the review process is deeply flawed in conception as well as (often) in operation, with the American Medical Association asserting that if peer review were a drug it would never be allowed on to the market. And yet university ranking systems for published research, on which all of our careers depend, rely on this flawed instrument. With this in mind we have been examining how members of our community (AAEE) give and respond to reviews with a view to making the process more useful. PURPOSE Reviewing is an inexact and subjective process so it would be misguided to think that somehow inter-rater reliability or some notion of objective âtruthâ may be attained. Instead, we ask what reviewers need to do to provide helpful advice that can help shape norms and standards in the field. DESIGN/METHOD In previous work (Willey et. al. 2011; Jolly et.al. 2011) there appeared to be a need for well-expressed criteria that would guide authors on what a publication should contain and guide reviewers in how judgements should be made. With the help of a Delphi panel made up of 12 international researchers in the field a set of criteria were developed. Volunteers were then sought to apply the criteria to sample texts in an online tool (SPARKplus). Individual interviews with some respondents were then used to clarify participantâs understandings and goals. RESULTS The criteria developed by the Delphi panel are those being used for this conference. The members of the panel particularly approved the âcommentsâ accompanying the criteria per se which were intended primarily as guidance to authors about acceptable practice. Anecdotal evidence to date suggests that autho...
Wandel, A. & Willey, K. 2012, 'Student Perceptions of Teammatesâ Performance: Influence of Team Formation Method', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, Swinburne University of Technology, Melbourne, Australia, pp. 1-9.
View/Download from: UTS OPUS
BACKGROUND This study investigates the group dynamics within the third problem-solving course in a series of four studied by on-campus and distance-education students. The cohorts were organised into teams of on-campus and distance-education students. The distance-education students were further organised into teams comprised of members from the same city or region (where possible). The motivation for this practice was to provide the opportunity for some social interaction for students who otherwise can feel isolated. Research has indicated that collocated teams tend to behave in a more socially-oriented fashion, while distributed teams tend to behave in a more task-oriented fashion. This paper is interested in how the team formations affect peer assessments to obtain individual marks. PURPOSE The hypothesis is that distance-education students placed in collocated teams will tend to behave more like the on-campus students when completing and reviewing their peer assessments. DESIGN/METHOD Where possible, distance-education students were grouped together in the same city or otherwise in the same region; the remaining students were randomly allocated to teams. A survey was conducted at the end of semester to ascertain their experiences of the peer assessment; trends from the survey are compared across two years to provide evidence to support the hypothesis. RESULTS We found that to some extent the social aspects of being in a collocated team resulted in these distance-education students to treat the peer assessment in a similar manner to the on-campus students, that is they had a heightened level of interest in the outcomes and how that impacted their team. A major contributor to the collocated distance-education studentsâ behaviour change was the introduction in the latest offering of an initial practice (formative) peer assessment that encouraged them to critically assess at an early stage whether their team was working effectively. A low response rate for the on...
Willey, K. & Gardner, A.P. 2012, 'Collaborative Learning Frameworks to Promote a Positive Learning Culture', 2012 Frontiers in Education Conference Proceedings - Soaring to New Heights in Engineering Education, 2012 Frontiers in Education Conference: Soaring to New Heights in Engineering Education, IEEE, Seattle, Washington, USA, pp. 638-643.
View/Download from: UTS OPUS
Engineers are often required to make critical judgments involving decisions that extend beyond traditional discipline boundaries. This requires professional engineers to undertake ongoing learning. Much of this learning is informal, learnt on the job from peers. Hence, to prepare students for professional practice they require opportunities to experience, practise, reflect and improve their ability to work in collaborative learning environments. While few would argue the benefits of collaborative learning these benefits are not automatic. Thoughtful design including scaffolding to motivate desired approaches and behavior is required. In this paper we report the results of several studies investigating the components of successful collaborative learning activities. We found that assessment scaffolding directed at promoting a culture of learning rather than a focus on passing a series of assessments was effective in engaging students, that formative activities allowed students to focus on learning and that learning from collaborative activities improved if the activities included variation for learning and a confirmation task. Using the results of these studies we developed two frameworks characterizing the elements of collaborative learning activities. In this paper we report investigating the capacity of these frameworks to develop an effective and integrated learning experience for students.
Gardner, A.P., Willey, K., Jolly, L. & Tibbits, G. 2012, 'Peering at the peer review process for conference submissions', 2012 Frontiers in Education Conference Proceedings - Soaring to New Heights in Engineering Education, 2012 Frontiers in Education Conference: Soaring to New Heights in Engineering Education, IEEE, Seattle, Washington, USA, pp. 852-857.
View/Download from: UTS OPUS
For many scholars conference papers are a stepping stone to submitting a journal article. However with increasing time pressures for presentation at conferences, peer review may in practice be the only developmental opportunity from conference attendance. Hence it could be argued that the most important opportunity to acquire the standards and norms of the discipline and develop researchers' judgement is the peer review process - but this depends on the quality of the reviews. In this paper we report the findings of an ongoing study into the peer review process of the Australasian Association for Engineering Education (AAEE) annual conference. We began by examining the effectiveness of reviews of papers submitted to the 2010 conference in helping authors to improve and/or address issues in their research. Authors were also given the chance to rate their reviews and we subsequently analysed both the nature of the reviews and authors' responses. Findings suggest that the opportunity to use the peer review process to induct people into the field and improve research methods and practice was being missed with almost half of the reviews being rated as 'ineffectual'. Authors at the 2011 AAEE conference confirmed the findings from the 2010 data. The results demonstrate the lack of a shared understanding in our community of what constitutes quality research. In this paper in addition to the results of the abovementioned studies we report the framework being adopted by the AAEE community to develop criteria to be applied at future conferences and describe the reviewer activity aimed at increasing understanding of standards and developing judgement to improve research quality within our engineering education community.
Rooney, D.L., Willey, K., Gardner, A.P., Boud, D.J., Reich, A.J. & Fitzgerald, T. 2012, 'Using practice theory to investigate professional engineers workplace learning', 2012 Frontiers in Education Conference Proceedings - Soaring to New Heights in Engineering Education, 2012 Frontiers in Education Conference: Soaring to New Heights in Engineering Education, IEEE, Seattle, Washington, USA, pp. 1031-1036.
View/Download from: UTS OPUS
This paper reports on the first phase of an Australian inter-disciplinary partnership study concerned with professional learning of experienced engineers. It is a theoretically motivated, qualitative paper that aims to produce detailed descriptions of professional learning that arise within professional engineering work. The paper uses practice theory to conceptualise professional learning. By using âpracticesâ as the units of analysis, professional learning is understood as an integral part of everyday work practices that is embodied, relational and material rather than an individual attribute. The paper concludes by suggesting that practice theory may provide organisations with an alternative perspective of workplace learning, inviting them to reconsider how professional learning is acknowledged, rewarded and fostered in organisations.
Freeman, M., Willey, K., Hancock, P., Howieson, B., Watty, K., Abraham, A., O'Connell, B. & Delange, P. 2012, 'Using technology to improve peer review and collaborative conversations to benchmark academic standards', 2012 Frontiers in Education Conference Proceedings - Soaring to New Heights in Engineering Education, 2012 Frontiers in Education Conference: Soaring to New Heights in Engineering Education, IEEE, Seattle, Washington, USA, pp. 1283-1288.
View/Download from: UTS OPUS
In 2010 the Australian government commissioned the Australian Learning and Teaching Council (ALTC) to undertake a national project to facilitate disciplinary development of threshold learning standards. The aim was to lay the foundation for all higher education providers to demonstrate to the new national higher education regulator, the Tertiary Education Quality and Standards Agency (TEQSA), that graduates achieved or exceeded minimum academic standards. Through a yearlong consultative process, representatives of employers, professional bodies, academics and students, developed learning standards applying to any Australian higher education provider. Willey and Gardner reported using a software tool, SPARKPLUS, in calibrating academic standards amongst teaching staff in large classes. In this paper, we investigate the effectiveness of this technology to promote calibrated understandings with the national accounting learning standards. We found that integrating the software with a purposely designed activity provided significant efficiencies in calibrating understandings about learning standards, developed expertise and a better understanding of what is required to meet these standards and how best to demonstrate them. The software and supporting calibration and assessment process can be adopted by other disciplines, including engineering, seeking to provide direct evidence about performance against learning standards.
Mujkanovic, A., Lowe, D.B., Willey, K. & Guetl, C. 2012, 'Unsupervised learning algorithm for adaptive group formation: Collaborative learning support in remotely accessible laboratories', International Conference on Information Society (i-Society 2012), International Conference on Information Society (i-Society 2012), IEEE, London, United Kingdom, pp. 50-57.
View/Download from: UTS OPUS
Skills and knowledge that can be gained by groups of individuals will be affected by the characteristics of those groups. Systematic formation of the groups could therefore potentially lead to significantly improved learning outcomes. This research explores a framework for group formation that continuously adapts rules used for the grouping process in order to optimize the selected performance criteria of the group. We demonstrate an implementation of this approach within the context of groups of students undertaking remote laboratory experiments. The implementation uses multiple linear regression analysis to adaptively update the rules used for creating the groups. In order to address specific learning outcomes, certain behaviors of the group might be desired to achieve this learning outcome. We can show that by using a set of individual/group characteristics and group behavior we can dynamically create rules and hence optimize the selected performance criteria. The selected performance is in reality the group behaviour, which might lead to improved learning outcomes.
Willey, K. & Gardner, A.P. 2012, 'Threshold exams to promote learning and assurance of learning', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, Swinburne University of Technology, Melbourne, Australia, pp. 1-10.
View/Download from: UTS OPUS
BACKGROUND Formal examinations are often used in engineering classes as the tool to evaluate student learning. These exams are often high stakes assessment tasks and provide no opportunity for feed-forward. Despite academic claims that all topics in their subject are requisite material, students are regularly able to pass these assessment tasks with unsatisfactory, and perhaps even no capacity to demonstrate learning in some topics. Furthermore, while undertaking the exam often highlights to students their learning deficiencies, it typically has no impact on their learning as they rarely receive feedback other than a mark or grade and there is no further opportunity to address these learning gaps. This paper reports on the impact of a two-staged examination process on both student learning and assurance of that learning. PURPOSE The aim of the staged examination process was to improve confidence that students had satisfactory knowledge in all requisite subject topics and to test its capacity to be learning-oriented in that it provides improved opportunities for students to learn while simultaneously increasing the level of learning assurance. DESIGN/METHOD The first stage of the process was an exam that covered all requisite subject topics. This exam consisted of multiple choice questions set at or just above the level of threshold learning outcomes. Students were required to score 80% on this exam to qualify to undertake the second part of the assessment process at a later date. Students used IFAT (Immediate Feedback Assessment Technique) cards for this stage to facilitate immediate feedback as to their strengths and weaknesses. The time between exams allowed students to review identified areas of weakness before attempting the second stage of the exam. Note: while not contributing to their final grade students who failed the first exam were also permitted to undertake the second exam as an opportunity to learn and as a means of evaluating the impact of the proc...
Mujkanovic, A., Lowe, D.B. & Willey, K. 2012, 'Adaptive group formation to promote desired behaviours', Proceedings of the 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, 23rd Annual Conference for the Australasian Association for Engineering Education - The Profession of Engineering Education: Advancing Teaching, Research and Careers, Swinburne University of Technology, Melbourne, Australia, pp. 1-9.
View/Download from: UTS OPUS
BACKGROUND There is substantial literature that shows the benefits of collaborative work, though these benefits vary enormously with circumstances. Irrespective of their structure and composition, groups usually exist for a particular reason and implicitly or explicitly target one or more outcomes. The achievements of group outcomes depend on many factors, including the individual behaviour of each group member. These behaviours are, in turn, affected by the individual characteristics, the context and the group composition. Constructing groups in a way that maximises the achievement of a specific outcome is complex with the optimal group composition depending on the attributes of the group members. Previous work has in most cases considered group formation based on one particular attribute, such as learning style, gender, personality, etc. Less common are instances of group formation rules being adjusted systematically to accommodate changes in an individualâs attributes or disposition. PURPOSE This paper considers how the multi-factorial nature of group performance and the variations in desired behaviour across different circumstances can be addressed within a consistent framework. DESIGN/METHOD The methodology consisted of two main stages. In the first stage, a simulation was encoded in MatLab to assess the conceptual approach of progressively updating rules for group formation. The method uses an unsupervised learning algorithm and correlation factors between quantifiable group characteristics (average age, degree of motivation, etc.) and resultant behaviours of the groups that are actually formed (level of dialogue, interface interactions, etc.) to update the rules used for group formation, and hence progressively construct groups that are more likely to behave in desired ways. The second stage involved an evaluation of this approach in a real world scenario using remotely accessible laboratories where engineering students voluntarily participated in a study ...
Rooney, D.L., Boud, D.J., Reich, A.J., Willey, K., Fitzgerald, T. & Gardner, A.P. 2012, 'Using practice theory to investigate professional engineers' workplace learning', Frontiers in Education Conference, IEEE, Oklahoma City, Seattle, pp. 1031-1036.
This paper reports on the first phase of an Australian inter-disciplinary partnership study concerned with professional learning of experienced engineers. It is a theoretically motivated, qualitative paper that aims to produce detailed descriptions of professional learning that arise within professional engineering work. The paper uses practice theory to conceptualise professional learning. By using `practices as the units of analysis, professional learning is understood as an integral part of everyday work practices that is embodied, relational and material rather than an individual attribute. The paper concludes by suggesting that practice theory may provide organisations with an alternative perspective of workplace learning, inviting them to reconsider how professional learning is acknowledged, rewarded and fostered in organisations
Hancock, P., Dale-Jones, G. & Willey, K. 2012, 'Impact of collaborative peer and self-assessment on students judgment and written communication', Proceedings of the The RMIT Accounting Educators Conference, 2013, RMIT, RMIT Melbourne Australia.
Willey, K. & Gardner, A.P. 2011, 'Want to change learning culture: provide the opportunity', Proceedings of Research in Engineering Education Symposium 2011, Research in Engineering Education Symposium 2011, Universidad Politecnica de Madrid, Spain, Madrid, Spain, pp. 259-267.
View/Download from: UTS OPUS
Many students resist having to take responsibility for their own learning rather expecting this to be the responsibility of their teaching academics. This resistance is often associated with Asian cultures where there is a perception of a reliance on rote learning and passively being taught Furthermore, undertaking collaborative activities may be more difficult when students are not being taught in their primary language. While teaching an undergraduate engineering science program in Hong Kong the authors had initially found it difficult to motivate students to actively participate in their learning. ln response, learning activities were redesigned to promote a culture of learning rather than a focus on passing a series of assessments. We found that despite some initial apprehension students enthusiastically engaged in collaborative learning when given the opportunity. Furthermore, formative activities freed students from the burden of strategically collecting marks, allowing them to focus on learning, enjoy the activities and take responsibility for their own progress.
Jolly, L., Willey, K., Tibbits, G. & Gardner, A.P. 2011, 'Conferences, Reviews and conversations about improving engineering education', Research in Engineering Education Symposium Madrid 2011, Research in Engineering Education Symposium Madrid 2011, Universidad Politecnica de Madrid, Spain, Madrid, Spain, pp. 834-840.
View/Download from: UTS OPUS
Peer reviews are supposed to ensure the quality of published work and are also applied to confer=ence papers with the same aim. But numerous studies have demonstrated that reviews cannot be considered objective or reliable. Even if they were they do not provide the opportunity to refine and develop ideas that conferences such as REES promote. We began by examining how well reviews of papers submitted to the 2010 conference of the Australasian Association for Engineering Education helped authors to improve and found them to be often inadequate. The literature reveals that this is true for peer review generally. We conclude with some suggestions for how ideas might be shared, developed and disseminated through scholarly conversation while avoiding most of the pitfalls of the review process.
Gardner, A.P. & Willey, K. 2011, 'Investigating the characteristics of successful collaborative activities', Proceedings of Research in Engineering Education Symposium Madrid 2011, Research in Engineering Education Symposium Madrid 2011, Universidad Politecnica de Madrid, Spain, Madrid, Spain, pp. 332-339.
View/Download from: UTS OPUS
Given that engineering is a practice-based profession, one of the benefits of collaborative learning is that it provides the opportunity to simulate this practice within classroom activities. While not replacing the benefits of actual practice, thoughtfully designed collaborative learning activities provide opportunities for students to construct and test their knowledge while developing their professional judgement. Hence, it is important to identify the common characteristics of collaborative activities that improve student learning. Based on the results of our research we hypothesised that with the correct scaffolding, activities that include integrated collaborative conversations improved the learning within small group activities. In this paper we report the first step in a research project to determine the characteristics of successful collaborative learning activities that include integrated peer conversations to assist academics in designing their own successful collaborative activities. To test our hypothesis we first examined a series of studies that report the effect of collaborative activity on student learning to identify any common characteristics that seemed to have a positive impact.
Willey, K., Jolly, L., Tibbits, G. & Gardner, A.P. 2011, 'Investigating research quality and the review process: some suggestions for improvement', Proceedings of SEFI 2011 Annual Conference: Global Engineering Recognition, Sustainability, Mobility, SEFI 2011 Annual Conference: Global Engineering Recognition, Sustainability, Mobility, Instituto Superior de Engenharia de Lisboa, Lisbon, Portugal, pp. 178-184.
View/Download from: UTS OPUS
Arguably, the most important opportunity to acquire the standards and norms of the discipline and develop researchers' judgement is the peer review process â but this depends on the quality of the reviews. 'Good' feedback has been identified as being timely, specific and relevant. Yet often reviews lack these basic qualities. In this paper we report an investigation of the peer review process at a recent engineering education conference. Authors at the conference were given the chance to rate their reviews, we subsequently analysed both the nature of the reviews and authors' responses. We found that reviewers generally do a poor job of applying criteria, leaving some authors feeling that the review process does not offer them enough help in improving their papers. On the other hand, authors showed some of the same tendencies we see in students to take criticism personally and hence reject it. We conclude by discussing some strategies that might be implemented to help both parties.
Gardner, A.P. & Willey, K. 2011, 'Peer feedback - what are students telling each other?', Proceedings of SEFI 2011 Annual Conference: Global Engineering Recognition, Sustainability, Mobility, European Society for Engineering Education, Instituto Superior de Engenharia de Lisboa, Lisbon, Portugal, pp. 437-444.
View/Download from: UTS OPUS
Accreditation bodies in various countries, universities and industry call for engineering students to develop professional skills associated with working in a group. We questioned whether students value the same skills and attributes in each other when working in a group. This paper reports on the analysis of the text of peer feedback comments provided by students in courses at two stages (second year and fourth year) of a four year undergraduate engineering degree program. Analysis of the peer feedback is based on the framework used by Miller [10] and investigates the amount of feedback provided, the type of feedback (ie whether it is positive, negative or neutral), the topic of the feedback and whether it could be classed as specific, since this is one of the identified characteristics of 'good' feedback. The topics covered by the feedback were determined by first coding the comments into themes and then grouping related themes into a category, rather than grouping the data into predetermined categories. This process resulted in the following six categories: general evaluation, team attributes, generic professional skills, subject knowledge, ideas and problem solving, and reliability. While the feedback comments provided by students to each other covered a range of issues, the topics most commented on include: team related attributes such as whether the peer was helpful, made an effort and participated in group activities; generic professional skills such as communication, leadership and time management; and reliability including timeliness of peer submissions, and the value of the peer's contribution to completing the task. These characteristics align strongly with what employers are looking for in engineering graduates, and what accreditation bodies list as essential competencies of a graduate engineer, showing that students themselves recognise the value of these skills when they have to work with others.
Willey, K. & Wandel, A. 2011, 'Peer Review of Teamwork for Encouraging Equal Commitment to the Group Effort', Proceedings of the 22nd Annual Conference for the Australasian Association for Engineering Education (AAEE2011) - Developing Engineers for Social Justice: Community Involvement, Ethics & Sustainability, AAEE - Annual Conference of Australasian Association for Engineering Education, Engineers Australia, Fremantle, WA, pp. 557-562.
View/Download from: UTS OPUS
An important graduate attribute is the ability to work in teams, so many university courses incorporate this as part of the learning experience. However, it is inevitable that in some teams there will be members who do not contribute as much to the overall effort as others, leading to frustration in those members who carry the majority of the burden. When there are students enrolled in distance-education mode, this can be exacerbated because many of the teams cannot meet face-to-face, so it can be difficult to exert sufficient influence to force problematic individuals to amend their behaviour. In an effort to mitigate against this problem, self and peer assessment was used for both team assignments in a Problem-Based Learning (PBL) course and the results of the peer assessment were used to scale the team mark for the corresponding assignment to obtain individual grades. After submitting their final assignment, a survey instrument was used to investigate the success of this process. The students overwhelmingly supported the idea of distributing marks based on the value of the individualâs contribution because in many teams it had the desired effect of motivating underperforming members to involve themselves more in the second assignment. There was some dissatisfaction about the process used to distribute marks, which the authors will attempt to address by providing better scaffolding in subsequent uses of the software tool. Regardless of these difficulties, we found that a transparent mechanism for distributing team marks to individual grades is beneficial for encouraging equal commitment to the team effort by all team members.
Willey, K. & Gardner, A.P. 2011, 'Scratch that itch to learn: a comparative study', Proceedings of the 22nd Annual Conference for the Australasian Association for Engineering Education (AAEE2011) - Developing Engineers for Social Justice: Community Involvement, Ethics & Sustainability, AAEE - Annual Conference of Australasian Association for Engineering Education, Engineers Australia, Fremantle, WA, pp. 601-606.
View/Download from: UTS OPUS
Engineers today are required to make critical judgements involving decisions that often extend beyond traditional discipline boundaries. This requires professional engineers to undertake ongoing learning. Much of this learning is informal, learnt on the job from peers from different disciplines. To enable students to develop the skills required for professional practice they need opportunities to experience, practise, reflect and improve their ability to work in a collaborative environment. One method used at the University of Technology, Sydney to develop these skills is collaborative activities incorporating immediate feedback. Subject topics are tested through quizzes that are initially undertaken individually and then collaboratively using immediate feedback assessment technique (IF-AT) cards. These activities allow students to first identify and subsequently have gaps in their learning addressed initially by their peers within the one activity. This paper reports on a comparative evaluation of the collaborative use of IF-AT quizzes in four subjects taught by the authors. We found that these methods not only consistently improved student engagement, learning and developed skills required for life-long learning, but also promoted changes in their learning culture by having them take more responsibility for their own learning.
Willey, K. & Gardner, A.P. 2011, 'Getting tutors on the same page', Proceedings of the 22nd Annual Conference for the Australasian Association for Engineering Education (AAEE2011) - Developing Engineers for Social Justice: Community Involvement, Ethics & Sustainability, AAEE - Annual Conference of Australasian Association for Engineering Education, Engineers Australia, Fremantle, WA, pp. 454-459.
View/Download from: UTS OPUS
In large engineering subjects, it is common to have multiple tutors where each tutor is responsible for grading the assessment tasks for students in their tutorial. An issue regularly faced by subject coordinators is how to achieve a consistent standard of marking and feedback quality amongst different tutors. To address this issue the authors initially used a number of methods including double blind marking to support consistent grading. However, with increasing demands on academics these time-consuming activities became an unrealistic option. This process was improved by using a software tool to compare both the marking and feedback provided by different tutors for a number of randomly selected project tasks. In this paper, we report using new software features developed as a result of this previous research to quickly establish and build a community of assessment practice amongst subject tutors. The reported process promotes inclusiveness by using a software tool to anonymously record and report tutor assessments allowing all opinions to be considered during a subsequent discussion activity. Even though this pilot exercise was undertaken by experienced tutors it significantly influenced their feedback skills and to a lesser extent their marking standards.
Willey, K., Jolly, L., Tibbits, G. & Gardner, A.P. 2011, 'Gatekeeping or filtering?: Investigating the connection between peer review & research quality', Proceedings of the 22nd Annual Conference for the Australasian Association for Engineering Education (AAEE2011) - Developing Engineers for Social Justice: Community Involvement, Ethics & Sustainability, AAEE - Annual Conference of Australasian Association for Engineering Education, Engineers Australia, Fremantle, WA, pp. 241-247.
View/Download from: UTS OPUS
Arguably, the most important opportunity to acquire the standards and norms of the discipline and develop researchers' judgement is the peer review process - but this depends on the quality of the reviews. 'Good' feedback - which we take to mean feedback that has the capacity to improve subsequent practice - has been identified as being timely, specific and relevant. Yet often reviews lack these basic qualities. In this paper we report an investigation of the peer review process at the 2010 Australasian Association of Engineering Education (AAEE) conference. Authors at the conference were given the chance to rate their reviews and we subsequently analysed both the nature of the reviews and authors' responses. Findings suggest that the opportunity to use the peer review process to induct people into the field and improve practice is being missed. As in other disciplines there is also ample evidence that the review process does little or nothing to ensure the standard and relevance of conference presentations. It is therefore legitimate to ask whether there may not be better processes to attain these ends and we conclude with some discussion of how the review process may be made more helpful for everyone involved.
Willey, K. & Gardner, A.P. 2011, 'Building a community of practice to improve inter marker standardisation and consistency', Proceedings of SEFI 2011 Annual Conference: Global Engineering Recognition, Sustainability, Mobility, SEFI 2011 Annual Conference: Global Engineering Recognition, Sustainability, Mobility, Instituto Superior de Engenharia de Lisboa, Lisbon, Portugal, pp. 666-671.
View/Download from: UTS OPUS
Over several years the authors have coordinated engineering subjects, with large cohorts of up to 300+ students. In each case, lectures were supported by tutorials. In the larger subjects it was not uncommon to have in excess of 10 tutors, where each tutor is responsible for grading the assessment tasks for students in their tutorial. A common issue faced by lecturers of large multiple tutor subjects is how to achieve a consistent standard of marking between different tutors. To address this issue the authors initially used a number of methods including double-blind marking and remarking. This process was improved by using the benchmarking tool in SPARKPLUS [1] to compare both the grading and feedback provided by different tutors for a number of randomly selected project tasks. In these studies we found that while students' perception of difference in grading was not unfounded, the problem was exacerbated by inconsistencies in the language tutors use when providing feedback. In this paper, we report using new SPARKPLUS features developed as a result of this previous research to quickly establish and build a community of practice amongst subject tutors. We found that in just one session these processes assisted tutors to reach a higher level of shared understanding of the concepts and practices pertinent to the subject assessment activities. In addition, it enabled tutors to gain an appreciation of the grading issues frequently reported by students. This resulted in not only improving both the understanding and skills of tutors but changing the way they both marked and provided feedback.
Willey, K. & Gardner, A.P. 2011, 'Change Learning Culture with Collaboration', Global Engineering Recognition, Sustainability, Mobility, European Society for Engineering Education, Instituto Superior de Engenharia de Lisboa, Lisbon, Portugal, pp. 93-98.
View/Download from: UTS OPUS
Engineers are often required to make critical judgments involving decisions that extend beyond traditional discipline boundaries, requiring professional engineers to undertake ongoing learning. Much of this learning is informal, learnt on the job from peers from different disciplines. To develop the skills required for professional practice students need opportunities to work in a collaborative learning environment. Many students resist having to take responsibility for their own learning rather expecting this to be the responsibility of their teaching academics. This resistance is often associated with Asian cultures where there is a perception of reliance on rote learning and an expectation of being passively taught. Furthermore, undertaking collaborative activities may be more difficult when students are not being taught in their primary language. While teaching an undergraduate engineering science program in Hong Kong the authors had initially found it difficult to motivate students to participate in learning activities that involved them using their judgement or critical analysis. In response, learning activities were redesigned to integrate collaborative peer learning and promote a culture of learning rather than a focus on passing a series of assessments. We found that students whose previous learning experiences were mainly passive despite some initial apprehension not only adjusted, but enthusiastically engaged in collaborative learning when given the opportunity. Furthermore, the formative nature of the activities freed students from the burden of strategically collecting marks, allowing them to focus on learning, take responsibility for their own progress and encouraged active participation in the learning process.
Gardner, A.P. & Willey, K. 2010, 'Does pre-feedback self reflection improve student engagement, learning outcomes and tutor facilitation of group feedback sessions?', Engineering Education 2010 (EE2010): Inspiring the next generation of engineers, Engineering Education Conference, The Higher Education Academy Engineering Subject Centre, Aston University, UK, pp. 1-10.
View/Download from: UTS OPUS
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes by providing opportunities to practise, assess and provide feedback on students' learning and development. Despite this work and the research of others, we observed some students felt they had nothing to learn from feedback sessions. Hence they missed the opportunity for reflection and to receive feedback to complete the learning cycle. This behaviour suggested that students needed more guidance to facilitate deeper engagement. We hypothesised that student engagement would increase if they were provided with guiding 'feedback catalyst questions' to initiate reflection and facilitate effective feedback on learning outcomes. In this paper we report testing whether this approach assisted students to gain more benefit from the self and peer assessment feedback sessions. In our investigation both students and tutors were asked to evaluate the effectiveness of the feedback catalyst questions in improving student engagement and learning. We found that the pre-feedback self reflection exercise improved learning outcomes and student engagement with more than 80% of students reporting multiple benefits. Furthermore tutors reported that the exercise assisted them to facilitate their sessions. However, not surprisingly the degree of success was related in part to the attitude of the tutor to the exercise. This suggests that while the feedback catalyst questions were extremely effective there is no substitute for enthusiastic and engaging tutorial staff.
Willey, K. & Gardner, A.P. 2010, 'Collaborative Peer Learning to Change Learning Culture and Develop the Skills for Lifelong Professional Practice', Proceedings of the 21st Annual Conference for the Australasian Association for Engineering Education, Annual Conference of Australasian Association for Engineering Education, University of Technology, Sydney, Sydney, Australia, pp. 222-229.
View/Download from: UTS OPUS
Engineers today are often required to make critical judgements involving decisions that extend beyond traditional discipline boundaries requiring ongoing learning, much of which is informal, learnt on the job from peers who are often from different disciplines. To prepare students for professional practice, they need opportunities to experience, practise, reflect and improve their ability to work in a collaborative environment. The University of Technology, Sydney teaches an undergraduate engineering science programme in Hong Kong. The authors have found it initially difficult to get students to participate in collaborative learning activities and in particular those that involved students in using their own judgement or critical analysis. In response, the authors redesigned their course to integrate collaborative peer learning activities into all areas of the curriculum including collaborative problem solving exercises that are subsequently assessed through a series of first individual then collaborative quizzes (using the immediate feedback assessment techniques (IF-AT)) and exams. Initial results from students overwhelmingly showed that the collaborative activities improved their understanding, ability to think through and resolve problems, and the identification and addressing of gaps in their learning. This approach has potential to benefit all engineering students as it prepares students to make the most of the informal collaborative learning opportunities provided in professional practice while simultaneously enhancing their ability to undertake lifelong learning.
Willey, K. & Gardner, A.P. 2010, 'Improving the standard and consistency of multi-tutor grading in large classes', Assessment: Sustainability, Diversity and Innovation. A conference on assessment in higher education 2010, ATN Assessment Conference, Institute for Interactive Media and Learning, University of Technology Sydney, UTS, Sydney, Australia, pp. 88-98.
View/Download from: UTS OPUS
For several years the authors have coordinated a large engineering design subject, having a typical cohort of more than 300 students per semester. Lectures are supported by tutorials of approximately 32 students that incorporate a combination of collaborative team and project-based learning activities. Each tutor is responsible for grading the assessment tasks for students in their tutorial. A common issue is how to achieve a consistent standard of marking and student feedback between different tutors. To address this issue the authors have used a number of methods including double-blind marking and/or random re-marking to support consistent grading. However, even when only small variations between the overall grading of different tutors were found, students still complained about a perceived lack of consistency. In this paper we report on an investigation into the use of a collaborative peer learning process among tutors to improve mark standardisation, and marker consistency, and to build tutorsâ expertise and capacity in the provision of quality feedback. We found that studentsâ perceptions of differences in grading were exacerbated by inconsistencies in the language tutors use when providing feedback, and by differences in tutorsâ perceptions of how well individual criterion were met.
Gardner, A.P. & Willey, K. 2010, 'Critical Conversations: How Collaborative Learning Activities Can Prepare Students for Structural Engineering Practice', Proceedings of the 21st Annual Conference for the Australasian Association for Engineering Education, Annual Conference of Australasian Association for Engineering Education, University of Technology, Sydney, Sydney, Australia, pp. 468-476.
View/Download from: UTS OPUS
Junior structural engineers rarely work alone. Their design calculations are usually peer reviewed, and they may be involved in reviewing other engineersâ designs. They are also likely to have to articulate their design decisions to their supervisor or the project team, if not the client. At the University of Technology, Sydney the authors redesigned the asssessment tasks in the subject Concrete Design to be collaborative learning-oriented tasks that provide an opportunity for students to develop and practice the skills they will need to interact with other professionals in the workplace and continue learning during their career. We theorised that allowing students to collaborate during quizzes and a project would make these activities more learning-oriented, in that students would actively learn from each other while completing their assessment. Data from various sources were collected to examine the impact of this collaborative assessment on student learning. These sources included instructor observation, analysis of student responses to a reflection activity, student surveys and student results. Students reported that not only were these activities enjoyable but they also significantly improved their learning.
Willey, K., Gardner, A. & IEEE 2010, 'Perceived Differences in Tutor Grading in Large Classes: Fact or Fiction?', 2010 IEEE FRONTIERS IN EDUCATION CONFERENCE (FIE).
Watt, D.J., Kayis, B. & Willey, K. 2009, 'Supplier Selection Criteria in Different Project Environments: An Empirical Study', Proceedings of the 14th International Symposium on Logistics (14th ISL): Global supply chains and inter-firm networks, International Symposium on Logistics, Nottingham University Business School, UK, Istanbul, Turkey, pp. 495-501.
View/Download from: UTS OPUS
Research in identifying the relative importance of criteria used to select a preferred supplier has, for the most part, relied on subjective lists of criteria being presented to respondents. This paper is a summary of the research conducted by the authors to quantify the importance of nine common criteria used in an actual evaluation and selection of a contractor/supplier. Unique choice sets were constructed, each comprising 3 tender evaluation outcomes (alternatives) described in terms of all criteria, but with varying levels. Respondents simultaneously evaluated all three alternatives within each choice set and selected the most preferred. Utility estimates for each criterion level were determined as was the overall contribution made by the individual criterion. Results indicate past project performance, technical expertise and cost are the most important criteria in an actual choice of contractor with organisational experience, workload, and reputation being the least important.
Willey, K. & Gardner, A.P. 2009, 'Changing Student's Perceptions of Self and Peer Assessment', Proceedings of the Research in Engineering Education Symposium 2009, Research in Engineering Education Symposium, University of Melbourne School of Engineering, Palm Cove Queensland, pp. 1-9.
View/Download from: UTS OPUS
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes by providing opportunities to practise, assess and provide feedback on students learning and development. Despite this work and the research of others, we found a significant number of students perceive self and peer assessment to be an instrument to facilitate fairness, focusing on its free-rider deterrent capacity, rather than providing opportunities for reflection and feedback to complete the learning cycle. We assumed that these perceptions were enforced by the fact that the main use of self and peer assessment was to moderate marks and provide feedback to individuals on their contribution to team tasks. We hypothesised that these perceptions would change if students were provided with opportunities to use self and peer assessment for different purposes. In this paper we report testing this hypothesis by using self and peer assessment multiple times a semester to not only assess team contributions but to assess individual student assignments and in benchmarking exercises. Our aim was to test whether this approach would assist students to gain more benefit from self and peer assessment processes while simultaneously breaking down their narrow focus on fairness.
Willey, K. & Gardner, A.P. 2009, 'Self and Peer Assessment: A Necessary Ingredient in Developing and Tracking Students' Graduate Attributes', Proceedings of the Research in Engineering Education Symposium, Research in Engineering Education Symposium, University of Melbourne School of Engineering, Palm Cove Queensland, pp. 1-9.
View/Download from: UTS OPUS
Recently there has been a shift to focus on assessing students learning outcomes in terms of graduate attributes which they should develop and demonstrate during the course of their degree. A number of universities have tried to address these issues for example by using software tools such as ReView to track attribute development or by producing both academic and professional skill development transcripts. However, many attributes such as teamwork and the ability to give and receive feedback are typically practised in collaborative peer exercises. Furthermore these exercises are often conducted outside of regular class sessions, hence thorough assessment of these attributes should include input from both individual students and their peers. Hence we propose that any method of developing and tracking students graduate attributes should include self and peer assessment.
Willey, K. & Gardner, A.P. 2009, 'Investigating the capacity of self and peer assessment to engage students and increase their desire to learn', Proceedings of the Annual SEFI Conference 2009: Attracting young people to engineering, Annual Conference of European Society for Engineering Education, Delft University of Technology, Rotterdam, The Netherlands, pp. 1-11.
View/Download from: UTS OPUS
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes by providing opportunities to practise, assess and provide feedback on studentsâ attribute development. Despite this work and the research of others, we have found a significant number of students perceive self and peer assessment to be an instrument to facilitate fairness focusing on its freerider deterrent capacity, rather than providing opportunities for reflection and feedback to complete the learning cycle. It is the authorsâ intention that all students would benefit, both from the reflective nature of self and peer assessment and the feedback it provides, and for these benefits to be seen as valuable and desirable so that students are eager to participate. In this paper we report investigating the use of self and peer assessment multiple times for different purposes within a single subject. In particular, we examine whether providing students with multiple opportunities to practise and receive feedback in different contexts encourages peer learning, increases engagement and studentsâ desire to learn.
Trieu, T., Willey, K. & Nguyen, H.T. 2009, 'Adaptive shared control strategies based on the Bayesian recursive technique for an intelligent wheelchair', Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE Engineering in Medicine and Biology Society Annual Conference, IEEE, Minneapolis, Minnesota, USA, pp. 7118-7121.
View/Download from: UTS OPUS
In this paper we present an adaptive shared control method for an intelligent wheelchair based on the Bayesian recursive technique to assist a disable user in performing obstacle avoidance tasks. Three autonomous tasks have been developed for different types of environments to improve the performance of the overall system. The system combines local environmental information gathered using a laser range finder sensor with the userâs intentions to select the most suitable autonomous task in different situations. The evidences of these tasks are estimated by the Bayesian recursive technique during movements of the wheelchair. The most appropriate task is chosen to be the with the highest evidence value. Experimental results show significant performance improvements compared to our previously reported shared control methods.
Beamish, B., Kizil, M., Willey, K. & Gardner, A.P. 2009, 'Monitoring mining engineering undergraduate perceptions of contribution to group project work', Proceedings of the 20th Australasian Association for Engineering Education Conference, Annual Conference of Australasian Association for Engineering Education, The School of Mechanical Engineering, The University of Adelaide, Adelaide, Australia, pp. 318-325.
View/Download from: UTS OPUS
A combination of self and peer assessment is a powerful and rich teaching and learning management tool that can be used to monitor and evaluate group performance in project work. An on-line system (SPARKPLUS â Self and Peer Assessment Resource Kit) has been developed to simplify this process for the academic. This system generates factors for both the peer assessment weighting to be applied for individual contribution and the studentâs perception of their contribution compared to their peers by using key assessment criteria and a rigorous algorithm that is applied to the student evaluations of themselves and their peers. This paper describes and evaluates the introduction of SPARKPLUS to assess the performance of Mining Engineering Undergraduates in Year 3 at The University of Queensland in the first semester of 2008 in two of their core undergraduate courses that required group project work to be completed for assessment. The results obtained from this initial trial show the potential for improving student behaviour in group work through a structured approach to monitoring and feedback of their performance. It was found that male students with GPAs ⤠5 tend to overestimate their contribution to group work more frequently than their peers whereas the opposite applies for male students with GPAs > 5.
Willey, K. & Gardner, A.P. 2009, 'Using Benchmarking to Improve Students' Learning and Make Assessment More Student Centred', Proceedings of the 20th Australasian Association for Engineering Education Conference, Annual Conference of Australasian Association for Engineering Education, The School of Mechanical Engineering, The University of Adelaide, Adelaide, Australia, pp. 726-734.
View/Download from: UTS OPUS
If universities are to prepare students for professional practise then they should also prepare them for the type of learning that occurs in the workplace. Graduatesâ workplace learning often differs considerably from their university experience. Firstly there is typically no lecturer or tutor to instruct them, and workplace learning is often collaborative. Hence studentsâ preparation for entering this workplace environment should include opportunities to practise collaborative learning with their peers. While designing collaborative learning tasks that involve students having to make and reflect on their judgements are extremely beneficial, these participatory exercises can result in an intolerable administrative burden especially for large classes. The authors have recently developed a specific benchmarking tool to assist academics to produce regular student centred assessments to improve studentsâ judgement and learning with a manageable academic effort. In this paper we reflectively deconstruct a purposely designed collaborative learning activity, investigating the effect of each of its components on student learning. Furthermore we explore how effectively the educational technology used reduced both the marking and administrative burden of running these exercises particularly in large classes.
Willey, K. & Gardner, A.P. 2009, 'Investigating the Potential of Self and Peer Assessment to Develop Learning Oriented Assessment Tasks', Proceedings of the 20th Australasian Association for Engineering Education Conference, Annual Conference of Australasian Association for Engineering Education, The School of Mechanical Engineering, The University of Adelaide, Adelaide, Australia, pp. 943-952.
View/Download from: UTS OPUS
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes in groupwork by providing opportunities to practise, assess and provide feedback on studentsâ attribute development. Combining this research and that reported in the literature on learning-oriented assessment we theorised that self and peer assessment would be an ideal tool to develop and facilitate assessments specifically designed to promote learning. In this paper we report testing this theory by integrating self and peer assessment into different learning oriented assessment tasks within a single subject. These tasks use self and peer assessment to not only assess a studentâs contribution to a team project but also assess individual student assignments, their understanding and judgement. The results demonstrate that self and peer assessment is an extremely effective tool in facilitating learning orientated assessments.
Willey, K. & Gardner, A.P. 2009, 'Assessment for learning: using minor assessment to promote major learning', Conference Proceedings ATN Assessment Conference 2009: Assessment in Different Dimensions, ATN Assessment Conference, RMIT University, Melbourne, pp. 352-362.
View/Download from: UTS OPUS
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes in groupwork by providing opportunities to practise, assess and provide feedback on studentsâ attribute development. Combining this research and that reported in the literature regarding learning-oriented assessment we theorised that self and peer assessment would be an ideal tool to develop and efficiently facilitate activities specifically designed to be student centred and promote learning. In this paper we report the effectiveness of a self and peer assessment activity specifically designed to promote collaborative peer learning, require students to take responsibility for their learning and improve their judgement, while at the same time only imposing a small assessment load on academics.
Willey, K. & Gardner, A.P. 2008, 'Improvements in the self and peer assessment tool SPARK: do they improve learning outcomes?', ATN Assessment Conference 2008: Engaging Students in Assessment, ATN Assessment Conference, University of South Australia, Adelaide, South Australia, pp. 1-9.
View/Download from: UTS OPUS
Willey, K. & Gardner, A.P. 2008, 'The effectiveness of using self and peer assessment in short courses: Does it improve learning?', 19th Annual Conference Australasian Association for Engineering Education: Program and Proceedings, Annual Conference of Australasian Association for Engineering Education, Engineers Australia, Yeppoon, Queensland, pp. 1-6.
View/Download from: UTS OPUS
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes. Its thoughtful use provides opportunities to practise, assess and provide feedback on students graduate attribute development. Our previous research has shown that it is beneficial for teams to use self and peer assessment multiple times a semester to produce regular feedback allowing students to reflect on their performance, then put into practice what they have learned to improve their future performance and overall grade. In this paper we report our investigation as to whether the use of self and peer assessment produces similar benefits in a short-course where teams rather than staying the same, change for different assessment tasks.
Gardner, A.P. & Willey, K. 2008, 'Developing Teamwork and other professional skills while teaching reinforced concrete design', 17th Congress of IABSE Chicago 2008 Creating and Renewing Urban Structures, International Association for Bridge and Structural Engineers, IABSE, Chicago, USA, pp. 334-341.
View/Download from: UTS OPUS
Willey, K. & Gardner, A.P. 2008, 'Using self and peer assessment for professional and team skill development: do well functioning teams experience all the benefits?', ATN Assessment Conference 2008: Engaging Students in Assessment, ATN Assessment Conference, University of South Australia, Adelaide, pp. 1-9.
View/Download from: UTS OPUS
Trieu, T., Nguyen, H.T. & Willey, K. 2008, 'Advanced Obstacle Avoidance for a Laser-based Wheelchair using Optimised Bayesian Neural Networks', Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE Engineering in Medicine and Biology Society Annual Conference, IEEE, Vancouver, Canada, pp. 3463-3466.
View/Download from: UTS OPUS or Publisher's site
In this paper we present an advanced method of obstacle avoidance for a laser based intelligent wheelchair using optimized Bayesian neural networks. Three neural networks are designed for three separate sub-tasks: passing through a door way, corridor and wall following and general obstacle avoidance. The accurate usable accessible space is determined by including the actual wheelchair dimensions in a real-time map used as inputs to each networks. Data acquisitions are performed separately to collect the patterns required for specified sub-tasks. Bayesian frame work is used to determine the optimal neural network structure in each case. Then these networks are trained under the supervision of Bayesian rule. Experiment results showed that compare to the VFH algorithm our neural networks navigated a smoother path following a near optimum trajectory.
Willey, K. & Gardner, A.P. 2008, 'Using self assessment to integrate graduate attribute development with discipline content delivery', Proceedings of the 36th Annual Conference of SEFI, Annual Conference of European Society for Engineering Education, Sense Publishers, Aalborg, Denmark, pp. 1-9.
View/Download from: UTS OPUS
Professionals, in addition to being technically competent, require skills of collaboration, communication and the ability to work in teams [1,2]. There is a reported competency gap between these skills required by employers and those developed by students during their undergraduate courses [3,4]. In response to this gap Universities have introduced graduate attributes which their students should develop during the course of their degree. Some of these attributes are discipline specific, others are generic to all professions. Generic attributes include teamwork skills, being able to think both critically and independently, being able to critically appraise their work and the work of others, and an appreciation of the need and value of critical reflection in one's academic, personal, and professional life. The development of all these attributes can be promoted by employing self and peer assessment. Thoughtful use provides opportunities to practise, develop, assess and provide feedback on these attributes and develop students judgement [5] even within subjects where traditional discipline content is taught. Our research involves using two assessment metrics produced from confidential student self and peer evaluations. These metrics are shared between all group members in structured feedback sessions several times a semester. This allows students to identify their individual strengths and weaknesses and address any competency gaps in their development. These metrics also allow progress to be assessed not only within a single subject but throughout an entire degree program.
Trieu, T., Nguyen, H.T. & Willey, K. 2008, 'Shared Control Strategies for Obstacle Avoidance Tasks in an Intelligent Wheelchair', Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE Engineering in Medicine and Biology Society Annual Conference, IEEE, Vancouver, Canada, pp. 3463-3466.
View/Download from: UTS OPUS or Publisher's site
In this paper we present a method of shared control strategy for an intelligent wheelchair to assist a disable user in performing obstacle avoidance tasks. The system detects obstacles in front of the wheelchair using a laser range finder sensor. As the wheelchair moves the information from the laser range finder is combined with data from the encoders mounted in its driving wheels to build a 360º real-time map. The accuracy of the map is improved by eliminating the systematic error that would result from both the uncertainty of effective wheelbase and unequal driving wheel diameters. The usable wheelchair accessible space is determined by including the actual wheelchair dimensions in producing the real-time map. In making a decision the shared control method considers the user's intentions via the head-movement interface, accessible space of the environment and user safety. The experiments show promising results in the intelligent wheelchair system.
Willey, K., Jarman, R. & Gardner, A.P. 2008, 'Redeveloping Capstone Projects in UTS Faculty of Engineering: Has integrating Engineers Australia competencies into the process improved learning?', Proceedings of the 2008 AaeE conference, Annual Conference of Australasian Association for Engineering Education, Engineers australia, Yeppoon, Queensland, pp. 1-6.
View/Download from: UTS OPUS
Willey, K., Jacobs, B.J. & Walmsley, M. 2007, 'Self and peer assessment to promote professional skill development: moving from ad-hoc to planned integration', Proceedings of the 18th Conference of the Australasian Association of Engineering Education, AAEE - Annual Conference of Australasian Association for Engineering Education, Australasian Association for Engineering Education, Melbourne, Australia, pp. 1-8.
Trieu, T., Nguyen, H.T. & Willey, K. 2007, 'Obstacle avoidance for power wheelchair using Bayesian neural network', Proceedings of the 29th International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE Engineering in Medicine and Biology Society Annual Conference, IEEE, Lyon, France, pp. 4771-4774.
View/Download from: UTS OPUS or Publisher's site
In this paper we present a real-time obstacle avoidance algorithm using a Bayesian neural network for a laser based wheelchair system. The raw laser data is modified to accommodate the wheelchair dimensions, allowing the freespace to be determined accurately in real-time. Data acquisition is performed to collect the patterns required for training the neural network. A Bayesian frame work is applied to determine the optimal neural network structure for the training data. This neural network is trained under the supervision of the Bayesian rule and the obstacle avoidance task is then implemented for the wheelchair system. Initial results suggest this approach provides an effective solution for autonomous tasks, suggesting Bayesian neural networks may be useful for wider assistive technology applications.
Jarman, R. & Willey, K. 2007, 'Benchmarking Capstone Projects in UTS Faculty of Engineering', ATN EVALUATION AND ASSESSMENT Conference, ATN Assessment Conference, Queensland University of Technology, Brisbane, Australia, pp. 45-57.
View/Download from: UTS OPUS
UTS:Engineering Capstone Projects are undertaken in the final semester(s) of study and provide an opportunity for students to demonstrate a capacity to perform at the levels expected of a professional engineer. In many cases, students undertake projects in their workplace environment affording a rich context to integrate real-world problems and solutions with academic rigour. The existing subject requirements have remained relatively unchanged for over 10 years; there is negligible evidence of the impact of its introduction and only anecdotal evidence of its effectiveness as a âcapstoneâ subject. Further, project supervisors have expressed their concern regarding a drop in quality of project work â and its assessment. Some students opt for âsofterâ project topics, moderation of assessment criteria across Faculty staff is problematic, and there is a lack of adequate evaluation (feedback) data from students to inform improvement strategy. As changes to subject requirements are being introduced in Spring 2007, a benchmarking exercise is timely. This paper will present the pertinent outcomes from a detailed evaluation undertaken by 85 capstone project students in Autumn semester 2007. Students were asked to self-evaluate on a 0-5 scale the âapplicabilityâ and their âcapacity to demonstrate in their project work each of 61 competency indicators chosen from Engineers Australia Stage 1 Competency Standards. The outcomes will serve as a valuable guide for areas of improvement as well as a benchmark against which future change can be measured.
Willey, K. & Gardner, A.P. 2007, 'Building Better Teams at Work using Self and Peer Assessment Practices', Proceedings of the 18th Conference of the Australasian Association of Engineering Education, Annual Conference of Australasian Association for Engineering Education, Australasian Association for Engineering Education, Melbourne, Australia, pp. 1-8.
View/Download from: UTS OPUS
There is a reported competency gap between the teamwork skills required by employers and those developed by engineering students during their undergraduate courses. The University of Technology, Sydney is addressing this issue by combining project-based learning with self and peer assessment to determine an individualâs team performance. A confidential online tool is used to collect and collate the student self and peer assessment ratings used both for formative feedback as well as assessment purposes. We found this approach improved student teamwork, engagement and satisfaction, while requiring only a small commitment of academic resources. We propose that using self and peer assessment would also be beneficial in industry to change workplace cultures, promote teamwork, individual skill development, engagement and productivity. Our implementation produces two assessment factors. The performance factor may be used by managers to coach staff to improve their performance, while the formative feedback factor will assist individuals to identify their strengths and weaknesses as perceived by their peers, facilitating ongoing skill development.
Willey, K. & Gardner, A.P. 2007, 'USING SELF AND PEER ASSESSMENT TO CHANGE WORKPLACE CULTURE TO PROMOTE TEAMWORK AND SELF DEVELOPMENT IN ENGINEERS', Proceedings of the International Conference Institute of Public Works Engineering - Australia, Cairns International Public Works Conference, Institute of Public Works Engineers Australia, Cairns Australia, pp. 1-7.
There is a reported competency gap between the teamwork skills required by employers and those developed by engineering students during their undergraduate courses. The University of Technology, Sydney is addressing this issue by combining project-based learning with self and peer assessment to determine an individualâs team performance. A confidential online tool is used to collect and collate the student self and peer assessment ratings used both for formative feedback and improvement as well as assessment purposes. Assessment ratings guided by predetermined criteria are automatically converted to two assessment factors. These factors are used to both measure performance and provide feedback to facilitate ongoing skill development. We found this approach improved student teamwork, engagement and satisfaction. In addition, it facilitated students supporting each other to develop their teamwork skills in an engineering context while requiring only a small commitment of academic resources. We propose that using self and peer assessment would also be beneficial in industry to change workplace cultures, to promote teamwork and individual skill development. By assessing both team work and project outcomes this system would help teams to self manage, perform and promote the ongoing development of team work skills. The performance factor may be used by managers to coach staff to improve their performance, while the formative feedback factor will assist individuals to identify their strengths and weaknesses as perceived by their peers, facilitating ongoing skill development.
Trieu, H.T., Nguyen, H.T. & Willey, K. 2007, 'Obstacle avoidance for power wheelchair using Bayesian neural network', Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings, pp. 4771-4774.
View/Download from: Publisher's site
In this paper we present a real-time obstacle avoidance algorithm using a Bayesian neural network for a laser based wheelchair system. The raw laser data is modified to accommodate the wheelchair dimensions, allowing the freespace to be determined accurately in real-time. Data acquisition is performed to collect the patterns required for training the neural network. A Bayesian frame work is applied to determine the optimal neural network structure for the training data. This neural network is trained under the supervision of the Bayesian rule and the obstacle avoidance task is then implemented for the wheelchair system. Initial results suggest this approach provides an effective solution for autonomous tasks, suggesting Bayesian neural networks may be useful for wider assistive technology applications. © 2007 IEEE.
Watt, D.J. & Willey, K. 2006, 'A Preference Structuring Framework for Appraising Complex Engineering', 3rd International Conference on Project Management (ProMAC2006), Transformative Project and Program Management for the Advancement of Organisations, through Government Agencies and Communities, Australian Institute of Project Management, Sydney, Australia, pp. 1-8.
View/Download from: UTS OPUS
Willey, K. & Freeman, M. 2006, 'Completing the learning cycle: The role of formative feedback when using self and peer assessment to improve teamwork and engagement', Proceedings of the 17th Annual Conference of the Australasian Association for Engineering Education, AAEE - Annual Conference of Australasian Association for Engineering Education, School of Engineering, Auckland University of Technology, Auckland, New Zealand, Auckland, New Zealand, pp. 1-9.
View/Download from: UTS OPUS
Willey, K. & Freeman, M. 2006, 'Narrated Visual Answers to FAQs: leveraging student learning and academic productivity with accessible technology', Proceedings of the 17th Annual Conference of the Australasian Association for Engineering Education, AAEE - Annual Conference of Australasian Association for Engineering Education, School of Engineering, Auckland University of Technology, Auckalnd, Newzealand, Auckland, New Zealand, pp. 1-12.
View/Download from: UTS OPUS
Watt, D.J. & Willey, K. 2005, 'The complex, chaotic, and fractal nature of complex systems', Proceedings Systems, Man and Cybernetics, 2005 IEEE International Conference, IEEE Conference on Systems, Man and Cybernetics, IEEE, The Big Island, Hawaii, USA, pp. 3155-3160.
View/Download from: UTS OPUS or Publisher's site
The systems perspective has been established as a means to structure and understand complex phenomenon. Current approaches are premised on defining and establishing a hierarchical (reductionist) structure that, in effect represents a complex system as a system of systems. The underlying assumption of reductionism is that the resultant component behaviour and dynamics represent the overall system behaviour. In reality, most complex systems and management approaches are highly interconnected, dynamic and subject to non-linear behaviour. As such, they defy any rational description in terms of overall behaviour and effectiveness as prescribed through reductionism. This paper considers the management of major projects as a complex of systems and discusses the nature, characteristics and properties that lead to the difficulties associated with reductionism. The need for systems science, or synthesis, approaches is briefly discussed as an effective means to addressing the situation.
Watt, D.J. & Willey, K. 2004, 'Value Base Decision Models of Management for Complex Systems', Engineering Management Conference, 2004. Proceedings. 2004, International Conference on Engineering Management, IEEE International, Singapore, pp. 1278-1283.
View/Download from: UTS OPUS or Publisher's site
Large scale complex engineering systems (LSCES), or complex systems, and their development comprise many factors of influence, of which some are currently known, others not yet known, and some may never be fully understood. Given the very nature of, and operating difficulties in developing complex systems, often there are inconsistencies associated with providing an objective business case and in establishing a realistic (acquisition) program. The inconsistencies arise as a result of measurement difficulties in determining overall value in the presence of both "hard" and "soft" factors. The traditional approach and continued practice of allocating subjective weightings to each factor of interest also contributes to the problem of inconsistency. This paper provides a structural framework that establishes a measure of value through the application of discrete choice methods. The methods are widely used to identify the decision making behaviour of individuals and organizations. This is achieved through considering relevant qualitative and quantitative factors or attributes that as a whole provide a measure of utility, and therefore value. Whilst there has been considerable success in the use of these models within the marketing and transportation domains, there has been little application toward their use in evaluating and assessing competitively tendered contracts (CTC) for the acquisition of complex systems.
Watt, D.J. & Willey, K. 2003, 'The Project Management Systems Engineering Dichotomy', Engineering Management Conference, 2003. IEMC '03. Managing Technologically Driven Organizations: The Human Side of Innovation and Change, International Conference on Engineering Management, IEEE, Albany, USA, pp. 306-310.
View/Download from: UTS OPUS or Publisher's site
Considerable attention has been drawn by many researchers toward the System Development Life Cycle and how best to manage the development ofa Large Scale Systems within a competitive environment. The success in developing these systems is dependent upon many factors of influence both within the organisation delivering the system, and the market sector within which that organisation operates. These factors alone create an environment of extreme competition and uncertainty. Coupled to this is the uncertainty associated with the overall project objectives in understanding explicitly what is to be delivered and when. Technology may further complicate this situation, particularly when it is unprecedented.
Willey, K. & Aboutanios, E. 2003, 'Low Rate Data Acquisition Using Low Earth Orbit Satellites', Proceedings 3rd Australian Space Science Workshop, National Space Society of Australia, Sydney, Australia.
Willey, K. 2001, 'Rapid Pedestal Calibration For Transportable Earth Stations', Proceedings 7th Ka Band Utilization Conference, 7th Ka Band Utilization Conference, Instituto Internazionale Delle Comunicazioni, Santa Margherita Ligure, Italy.
Willey, K. 2000, 'Antenna Optimization to Minimize Pointing Loss', Proceedings 6th Ka Band Utilization Conference, 6th Ka Band Utilization Conference, Instituto Internazionale Delle Comunicazioni, Cleveland, Ohio, USA..
Willey, K., Osborn, T., Eckert, M.P., Liwanag, R. & Reisenfeld, S. 1999, 'The Suitability of Using NORAD TLE's to Track LEO Satellites with Ka Band Communications', Proceedings 5th Ka Band Utilization Conference, 5th Ka Band Utilization Conference, Instituto Internazionale Delle Comunicazioni, Taormina, Sicily Island, Italy..
Reisenfeld, S., Aboutanios, E., Willey, K., Eckert, M.P., Clout, R.B. & Thoms, A. 1999, 'The Design of the FedSat Ka Band Fast Tracking Earth Station', Proceedings 8th International Aerospace Congress, 8th International Aerospace Congress, Adelaide, South Australia.

Journal articles

Rooney, D.L., Reich, A.J., Boud, D.J., Willey, K., Gardner, A.P. & Fitzgerald, T. 2015, 'Reimagining site-walks: sites for rich learning', Australasian Journal of Engineering Education, vol. 20, no. 1, pp. 19-30.
View/Download from: UTS OPUS or Publisher's site
This paper presents the preliminary results of a multi-phased qualitative investigation of continuing professional learning. The study focused on the identification of common engineering practices that contribute to learning. This paper examines a particular practice, that of the site-walk. It draws on practice theory, an emerging set of conceptual resources used in workplace learning research. Data was elicited via qualitative methods such as interviews, focus groups and site visits with experienced engineers employed in a large Australian engineering company. It was analysed using the lens of practice theory. The findings suggest that site-walks, while an everyday practice for engineers, are also highly learning-rich. This understanding has implications for continual professional learning, and for educators of novice engineers.
Reich, A., Rooney, D., Gardner, A., Willey, K., Boud, D. & Fitzgerald, T. 2015, 'Engineers' professional learning: a practice-theory perspective', European Journal of Engineering Education, vol. 40, no. 4, pp. 366-379.
View/Download from: UTS OPUS or Publisher's site
© 2014 SEFI. With the increasing challenges facing professional engineers working in more complex, global and interdisciplinary contexts, different approaches to understanding how engineers practice and learn are necessary. This paper draws on recent research in the social sciences from the field of workplace learning, to suggest that a practice-theory perspective on engineers' professional learning is fruitful. It shifts the focus from the attributes of the individual learner (knowledge, skills and attitudes) to the attributes of the practice (interactions, materiality, opportunities and challenges). Learning is thus more than the technical acquisition and transfer of knowledge, but a complex bundle of activities, that is, social, material, embodied and emerging. The paper is illustrated with examples from a research study of the learning of experienced engineers in the construction industry to demonstrate common practices – site walks and design review meetings – in which learning takes place.
Reich, A., Rooney, D.L., Gardner, A., Willey, K., Boud, D. & Fitzgerald, T. 2014, 'Engineers' professional learning: a practice-theory perspective', European Journal of Engineering Education.
View/Download from: UTS OPUS or Publisher's site
Dale-Jones, G., Hancock, P. & Willey, K. 2013, 'Accounting Students in an Australian University Improve their Writing: But How Did it Happen?', Accounting Education, vol. 22, no. 6, pp. 544-562.
View/Download from: UTS OPUS or Publisher's site
The ability to communicate ' orally and in writing' is a graduate attribute that employers in many countries rank as number one in importance, aside from relevant qualifications. This paper reports the implementation and evaluation of a collaborative peer assessment and self-assessment learning and teaching (L&T) initiative, which was designed to improve postgraduate students' judgment of writing standards and to improve their own writing according to that standard. The initiative was embedded in an introductory financial accounting unit in an Australian university. In a mixed methods study, the matched pair design revealed improvements in the written communication skills of students as determined by an independent assessor. There was also statistically significant improvement in the ability of students to apply assessment standards to grammatical, structural and presentation components of written communication. Whereas it was not possible to attribute the improvements entirely to the collaborative peer assessment initiative, our observations and students' self-reporting comments suggest that the L&T initiative was effective
Gardner, A. & Willey, K. 2013, 'Editorial', European Journal of Engineering Education, vol. 38, no. 6, pp. 579-581.
View/Download from: Publisher's site
watt, D., Kayis, B. & Willey, K. 2010, 'The relative importance of tender evaluation and contractor selection criteria', International Journal of Project Management, vol. 28, no. 1, pp. 51-60.
View/Download from: UTS OPUS or Publisher's site
Research in identifying the relative importance of criteria used to select a preferred supplier has, for the most part, relied on subjective lists of criteria being presented to respondents. The research reported here uses an experimental design approach to quantify the importance of nine common criteria used in an actual evaluation and selection of a contractor/supplier. Unique choice sets were constructed, each comprising three tender evaluation outcomes (alternatives) described in terms of all criteria, but with varying levels. Respondents simultaneously evaluated all three alternatives within each choice set and selected the most preferred. Utility estimates for each criterion level were determined as was the overall contribution made by the individual criterion. Results indicate Past Project Performance, Technical Expertise and Cost are the most important criteria in an actual choice of contractor with Organisational Experience, Workload, and Reputation being the least important.
Willey, K. & Gardner, A.P. 2010, 'Investigating the capacity of self and peer assessment activities to engage students and promote learning', European Journal of Engineering Education, vol. 35, no. 4, pp. 429-443.
View/Download from: UTS OPUS or Publisher's site
The authors have previously reported the effectiveness of using self and peer assessment to improve learning outcomes by providing opportunities to practise, assess and provide feedback on students' attribute development. Despite this work and the research of others, a significant number of students and, indeed, many academics focus on the free-rider deterrent capability of self and peer assessment, rather than its capacity to provide opportunities for developing judgement and facilitating reflection and feedback to complete the learning cycle. The advent of web-based tools such as SPARKPLUS allows the frequent and efficient implementation of self and peer assessment activities even in large classes. This article reports the results of an investigation into whether the regular use of self and peer assessment in different contexts promoted effective peer learning, increased engagement and encouraged students to learn.
Willey, K. & Gardner, A.P. 2009, 'Developing team skills with self and peer assessment', Campus-Wide Information Systems, vol. 26, no. 5, pp. 365-378.
View/Download from: UTS OPUS or Publisher's site
Purpose - Self- and peer assessment has proved effective in promoting the development of teamwork and other professional skills in undergraduate students. However, in previous research approximately 30 percent of students reported that its use produced no perceived improvement in their teamwork experience. It was hypothesised that a significant number of these students were probably members of a team that would have functioned well without self- and peer assessment and hence the process did not improve their teamwork experience. This paper aims to report the testing of this hypothesis.
Watt, D.J., Kayis, B. & Willey, K. 2009, 'Identifying key factors in the evaluation of tenders for projects and services', International Journal of Project Management, vol. 27, no. 3, pp. 250-260.
View/Download from: UTS OPUS or Publisher's site
Clients select contractors on the basis of the relative importance of tender evaluation criteria, such as, experience, expertise, past performance, and cost. This paper presents the results of a recent study that sought to identify a suite of representative (principal) tender evaluation and contractor selection criteria for use in future research. The study included an examination of management literature on contractor selection and tender evaluation, and an exploratory survey. For each we developed a separate master list and mapped all identified and reported criteria into one of 16 defined categories. Threshold tests were applied to identify common and unique categories and an absolute difference test to determine the principal categories of criteria used in the evaluation of tenders. The study, using a pragmatic and heuristic approach, resulted in the identification of eight principal categories suitable for our research program. The findings indicate that the preferred criteria for evaluating tenders are those which provide a measure of contractors' ability in terms of their management and technical capability, past experience and performance, reputation, and the proposed method of delivery or technical solution.
Willey, K. & Gardner, A.P. 2009, 'Improving self and peer assessment processes with technology', Campus-Wide Information Systems, vol. 26, no. 5, pp. 379-399.
View/Download from: UTS OPUS
As a way of focusing curriculum development and learning outcomes universities have introduced graduate attributes, which their students should develop during their degree course. Some of these attributes are discipline-specific, others are generic to all professions. The development of these attributes can be promoted by the careful use of self- and peer assessment. The authors have previously reported using the self- and peer assessment software tool SPARK in various contexts to facilitate opportunities to practise, develop, assess and provide feedback on these attributes. This research and that of the other developers identified the need to extend the features of SPARK, to increase its flexibility and capacity to provide feedback. This paper seeks to report the results of the initial trials to investigate the potential of these new features to improve learning outcomes.
Willey, K. & Freeman, M. 2006, 'Improving teamwork and engagement: the case for self and peer assessment', Australasian Journal of Engineering Education, vol. 2006-02, pp. 1-19.
View/Download from: UTS OPUS
Willey, K. 2002, 'Antenna Selection to Minimize Pointing Requirements', Microwave Journal, vol. 45, no. 1, pp. 114-122.
Willey, K. 2000, 'Selecting a pedestal for tracking LEO satellites at Ka band', Microwave Journal, vol. 43, no. 4, pp. 118-126.