Simon Buckingham Shum is Professor of Learning Informatics, and Director of the Connected Intelligence Centre (CIC). CIC is an innovation centre for UTS, building the capacity of staff and students to gain insights from educational data science applications.
Prior to joining UTS (Aug 2014) Simon was Professor of Learning Informatics and Associate Director (Technology) at the the UK Open University's Knowledge Media Institute, a 70-strong lab researching the future internet for the knowledge society, and applications into the OU.
Simon has a background in Psychology (B.Sc., York), Ergonomics (M.Sc., UCL) and Human-Computer Interaction (Ph.D., York, in collaboration with Rank Xerox EuroPARC). This informs his human-centred computing perspective to the challenge of building analytics, collective intelligence and sensemaking tools. Simon is a co-founder of the Society for Learning Analytics Research which catalyses dialogue at the intersection of data science and education among researchers, educators, policymakers and vendors. He has served as a Vice-President, and continues as an Executive member. He is a Fellow of The RSA.
Simon is committed to developing research-informed tools that practitioners can use, and is a founding member of the Compendium Institute (global user forum for the Compendium knowledge cartography application) and Learning Emergence (advancing complex systems thinking about deep learning through leadership development and technology). He was Co-Founder and General Editor of the award-winning Journal of Interactive Media in Education, one of the earliest open access e-journals. He was one of the Investigators on the OpenLearn programme which published thousands of hours of OU courses as Creative Commons resources along with open sensemaking tools, and was Director of the OU’s SocialLearn platform.
In a schools (K-12) context he served as Chair of Governors at Bushfield School in the UK.
Can supervise: YES
Simon's research sits at the intersection of Learning Analytics, Sensemaking and Collective Intelligence: the design of human-centred computation to...
- make visible the dynamics of learning
- develop analytics that provide insight into 21st century graduate attributes
- scaffold social dialogue and rigorous argumentation with collaborative tools
- conduct reflective, collective deliberation on complex sociotechnical problems at a range of different scales
In the UK he secured >£1.8M continuous funding for this team 1997-2014 from the UK Research Councils, European Union, industry, public sector and philanthropic bodies. He is currently on three projects funded by the Australian Office for Learning & Teaching. He is currently funded by the UK's Engineering & Physical Sciences Research Council (EPSRC) on the Election Debate Visualization Project, and the European Union on the Catalyst Project.
Simon lectures at postgraduate and professional levels on Learning Analytics and Argument Visualization. In 2015 CIC launched the highly successful transdisciplinary Masters of Data Science & Innovation. At the UK Open University, he worked on the world's first MBA program in Managing Knowledge, writing online distance teaching materials on Human-Centred Knowledge Management Technologies.
© Springer-Verlag London 2014. In analyzing controversies and debates – which would include reviewing a literature in order to plan research, or assessing intelligence to formulate policy – there is no one worldview which can be mapped, for instance as a single, coherent concept map. The cartographic challenge is to show which facts are agreed and contested, and the different kinds of narrative links that use facts as evidence to define the nature of the problem, what to do about it, and why. We will use the debate around the invasion of Iraq to demonstrate the methodology of using a knowledge mapping tool to extract key ideas from source materials, in order to classify and connect them within and across a set of perspectives of interest to the analyst. We reflect on the value that this approach adds, and how it relates to other argument mapping approaches.
Lucas, C, Gibson, A & Shum, SB 2019, 'Pharmacy Students' Utilization of an Online Tool for Immediate Formative Feedback on Reflective Writing Tasks.', American journal of pharmaceutical education, vol. 83, no. 6.View/Download from: UTS OPUS or Publisher's site
Objective. To assess pharmacy students' perceptions of the benefits and utility of a novel online reflective-writing tool. Methods. After completing a required Academic Writing Analytics (AWA) workshop, Master of Pharmacy students submitted a reflective writing assignment in the AWA web application. A six-item survey was administered to students prior to and immediately after using AWA. Results. Sixty students volunteered to participate in the study; however, only 39 students provided a pseudonym that allowed their pretest and posttest to be matched. A comparison of students' responses on the pretest with those on the posttest, which was administered four weeks after the workshop, suggest a noticeable increase in agreement with AWA's benefits as an effective, online tool for improving their reflective learning skills. Conclusion. This novel online tool has the potential to assist pharmacy students with self-critiquing and improving their reflective writing assignments prior to submission. Furthermore, as the elements of reflection are transferable, this tool has the potential to be used in other educational contexts.
Martinez-Maldonado, R, Kay, J, Buckingham Shum, SJ & Yacef, K 2019, 'Collocated Collaboration Analytics: Principles and Dilemmas for Mining Multimodal Interaction Data', Human-Computer Interaction, vol. 34, no. 1, pp. 1-50.View/Download from: UTS OPUS or Publisher's site
Copyright © 2017 Taylor & Francis Group, LLC Learning to collaborate effectively requires practice, awareness of group dynamics, and reflection; often it benefits from coaching by an expert facilitator. However, in physical spaces it is not always easy to provide teams with evidence to support collaboration. Emerging technology provides a promising opportunity to make collocated collaboration visible by harnessing data about interactions and then mining and visualizing it. These collocated collaboration analytics can help researchers, designers, and users to understand the complexity of collaboration and to find ways they can support collaboration. This article introduces and motivates a set of principles for mining collocated collaboration data and draws attention to trade-offs that may need to be negotiated en route. We integrate Data Science principles and techniques with the advances in interactive surface devices and sensing technologies. We draw on a 7-year research program that has involved the analysis of six group situations in collocated settings with more than 500 users and a variety of surface technologies, tasks, grouping structures, and domains. The contribution of the article includes the key insights and themes that we have identified and summarized in a set of principles and dilemmas that can inform design of future collocated collaboration analytics innovations.
Reynolds, R, Chu, S, Ahn, J, Buckingham Shum, S, Hansen, P, Haythornthwaite, C, Huang, H, Meyers, EM & Rieh, SY 2019, 'Inaugural issue perspectives on Information and Learning Sciences as an integral scholarly Nexus', Information and Learning Science, vol. 120, no. 1-2, pp. 2-18.View/Download from: UTS OPUS or Publisher's site
© 2019, Emerald Publishing Limited. Purpose: The new journal, Information and Learning Sciences, aims to advance the understanding of human inquiry, learning and knowledge building in human design and uses of information systems, e-learning systems and socio-technical system contexts. Design/methodology/approach: Under the new editorial team, advisory board, and reviewer community, the authors aim to develop and provide an established and rigorous space for scholarly development that explores phenomena at the intersections of these two fields of inquiry. The editorial advisory board brings together over 70 leading international scholars who are evenly divided between and across these two disciplines. Findings: To chart the course for the journal’s scope and mission and set a standard for quality, the editorial team decided to launch with an inaugural issue of eight articles comprising mainly of theoretical syntheses and editorial essays, alongside three works that report empirical study research findings as exemplars. Authors were chosen by the editorial team based on their known thought leadership at this intersection, and the articles underwent a single-blind review process with a period of revisions. All articles and issues moving forward will be double-blind peer reviewed under both fields’ expected norms and standards of rigorous editorial excellence. The authors invite all interested experts and experts-in-development in our two disciplines, to submit work and volunteer as reviewers in these pursuits. The authors present a brief summary narrative discussing a selection of four recent areas of interdisciplinary research out of which the journal has emerged. The authors then describe each of the special issue articles contributed by our generous invited authors. Originality/value: The authors enthusiastically and warmly invite continued engagement along these lines in the journal’s pages, and also welcome related, and wholly contrary points of view, and point...
© 2019, UTS ePRESS. All rights reserved. The design of effective learning analytics (LA) extends beyond sound technical and pedagogical principles. If analytics are to be adopted and used successfully to support learning and teaching, their design process needs to take into account a range of human factors, including why and how they will be used. In this editorial, we introduce principles of human-centred design developed in other, related fields that can be adopted and adapted to support the development of human-centred learning analytics (HCLA). We draw on the papers in this special section, together with the wider literature, to define human-centred design in the field of LA and to identify the benefits and challenges that this approach offers. We conclude by suggesting that HCLA will enable the community to achieve more impact, more quickly, with tools that are fit for the purpose and a pleasure to use.
Shum, SJB & Luckin, R 2019, 'Learning analytics and AI: Politics, pedagogy and practices', BRITISH JOURNAL OF EDUCATIONAL TECHNOLOGY, vol. 50, no. 6, pp. 2785-2793.View/Download from: Publisher's site
Echeverria, V, Martinez-Maldonaldo, R, Buckingham Shum, S, Chiluiza, K, Granda, R & Conati, C 2018, 'Exploratory versus Explanatory Visual Learning Analytics: Driving Teachers’ Attention through Educational Data Storytelling', Journal of Learning Analytics, vol. 5, no. 3, pp. 72-97.View/Download from: Publisher's site
Knight, S, Buckingham Shum, S, Ryan, P, Sándor, Á & Wang, X 2018, 'Designing Academic Writing Analytics for Civil Law Student Self-Assessment', International Journal of Artificial Intelligence in Education, vol. 28, no. 1, pp. 1-28.View/Download from: UTS OPUS or Publisher's site
Research into the teaching and assessment of student writing shows that many students find academic writing a challenge to learn, with legal writing no exception. Improving the availability and quality of timely formative feedback is an important aim. However, the time-consuming nature of assessing writing makes it impractical for instructors to provide rapid, detailed feedback on hundreds of draft texts which might be improved prior to submission. This paper describes the design of a natural language processing (NLP) tool to provide such support. We report progress in the development of a web application called AWA (Academic Writing Analytics), which has been piloted in a Civil Law degree. We describe: the underlying NLP platform and the participatory design process through which the law academic and analytics team tested and refined an existing rhetorical parser for the discipline; the user interface design and evaluation process; and feedback from students, which was broadly positive, but also identifies important issues to address. We discuss how our approach is positioned in relation to concerns regarding automated essay grading, and ways in which AWA might provide more actionable feedback to students. We conclude by considering how this design process addresses the challenge of making explicit to learners and educators the underlying mode of action in analytic devices such as our rhetorical parser, which we term algorithmic accountability.
Buckingham Shum, S, Sándor, Á, Goldsmith, R, Bass, R & McWilliams, M 2017, 'Towards Reflective Writing Analytics: Rationale, Methodology and Preliminary Results', Journal of Learning Analytics, vol. 4, no. 1, pp. 58-84.View/Download from: UTS OPUS or Publisher's site
Iandoli, L, Quinto, I, De Liddo, A & Shum, SB 2016, 'On online collaboration and construction of shared knowledge: Assessing mediation capability in computer supported argument visualization tools', JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY, vol. 67, no. 5, pp. 1052-1067.View/Download from: UTS OPUS or Publisher's site
Iandoli, L, Quinto, I, De Liddo, A & Buckingham Shum, S 2014, 'Socially-augmented argumentation tools: rationale, design and evaluation of a debate dashboard', International Journal of Human-Computer Studies, vol. 72, no. 3, pp. 298-319.View/Download from: UTS OPUS or Publisher's site
Collaborative Computer-Supported Argument Visualization (CCSAV) is a technical methodology that offers support for online collective deliberation over complex dilemmas. As compared with more traditional conversational technologies, like wikis and forums, CCSAV is designed to promote more critical thinking and evidence-based reasoning, by using representations that highlight conceptual relationships between contributions, and through computational analytics that assess the structural integrity of the network. However, to date, CCSAV tools have achieved adoption primarily in small-scale educational contexts, and only to a limited degree in real world applications. We hypothesise that by reifying conversations as logical maps to address the shortcomings of chronological streams, CCSAV tools underestimate the importance of participation and interaction in enhancing collaborative knowledge-building. We argue, therefore, that CCSAV platforms should be socially augmented in order to improve their mediation capability. Drawing on Clark and Brennan influential Common Ground theory, we designed a Debate Dashboard, which augmented a CCSAV tool with a set of widgets that deliver meta-information about participants and the interaction process. An empirical study simulating a moderately sized collective deliberation scenario provides evidence that this experimental version outperformed the control version on a range of indicators, including usability, mutual understanding, quality of perceived collaboration, and accuracy of individual decisions. No evidence was found that the addition of the Debate Dashboard impeded the quality of the argumentation or the richness of content.
Knight, S, Buckingham Shum, S & Littleton, K 2014, 'Epistemology, assessment, pedagogy: where learning meets analytics in the middle space', Journal of Learning Analytics, vol. 1, no. 2, pp. 23-47.View/Download from: UTS OPUS or Publisher's site
Learning Analytics is an emerging research field and design discipline which occupies the ‘middle space’ between the learning sciences/educational research, and the use of computational techniques to capture and analyse data (Suthers and Verbert, 2013). We propose that the literature examining the triadic relationships between epistemology (the nature of knowledge), pedagogy (the nature of learning and teaching) and assessment provide critical considerations for bounding this middle space. We provide examples to illustrate the ways in which the understandings of particular analytics are informed by this triad. As a detailed worked example of how one might design analytics to scaffold a specific form of higher order learning, we focus on the construct of epistemic beliefs: beliefs about the nature of knowledge. We argue that analytics grounded in a pragmatic, sociocultural perspective are well placed to explore this construct using discourse-centric technologies. The examples provided throughout this paper, through emphasising the consideration of intentional design issues in the middle space, underscore the “interpretative flexibility” (Hamilton & Feenberg, 2005) of new technologies, including analytics.
Selvin, A & Buckingham Shum, S 2014, 'Constructing Knowldge Art: An Experiential Perspective on Crafting Participatory Representations', Synthesis Lectures on Human-Centered Informatics, vol. 7, no. 3, pp. 1-119.View/Download from: Publisher's site
Selvin, A & Buckingham Shum, S 2014, 'Constructing Knowledge Art: An Experiential Perspective on Crafting Participatory Representations', Synthesis Lectures on Human-Centered Informatics, vol. 7, no. 3, pp. 1-119.View/Download from: Publisher's site
Iandoli, L, Quinto, I, De Liddo, A & Shum, SB 2013, 'Improving Knowledge Management by Supporting Asynchronous Communications with a Debate Dashboard', International Journal of Information Systems and Social Change, vol. 4, no. 4, pp. 26-44.View/Download from: Publisher's site
Ikioda, F, Kendall, S, Brooks, F, De Liddo, A & Buckingham Shum, S 2013, 'Factors That Influence Healthcare Professionals’ Online Interaction in a Virtual Community of Practice', Social Networking, vol. 02, no. 04, pp. 174-184.View/Download from: UTS OPUS or Publisher's site
Brown, M, Downie, A & Buckingham Shum, S 2012, 'Mapping the matrix: using compendium as a tool for recording the analytic group', Group Analysis, vol. 45, no. 1, pp. 99-115.View/Download from: UTS OPUS or Publisher's site
This paper describes the application of Compendium, a knowledge cartography software tool, for the recording of group process. As a hypertext tool, it enables analysts to visualise connections between people, ideas and information, establishing an evidence base within and across contexts (such as group sessions). After customising its visual language, templates and keyword system, it has been piloted as a research tool for the measurement of group process. This would appear to hold the promise of providing a digital substrate for recording, discussing and analysing long term group dynamics in new ways. While the project is in its early stages, early indications are that it is a useful tool which can highlight group process and record change over time. In the longer term, it seems plausible that group processes such as multiple mirroring and identification, and such complex structures as the matrix, could be made visible and researchable through this methodology.
We propose that the design and implementation of effective Social Learning Analytics (SLA) present significant challenges and opportunities for both research and enterprise, in three important respects. The first is that the learning landscape is extraordinarily turbulent at present, in no small part due to technological drivers. Online social learning is emerging as a significant phenomenon for a variety of reasons, which we review, in order to motivate the concept of social learning. The second challenge is to identify different types of SLA and their associated technologies and uses. We discuss five categories of analytic in relation to online social learning; these analytics are either inherently social or can be socialised. This sets the scene for a third challenge, that of implementing analytics that have pedagogical and ethical integrity in a context where power and control over data are now of primary importance. We consider some of the concerns that learning analytics provoke, and suggest that Social Learning Analytics may provide ways forward. We conclude by revisiting the drivers and trends, and consider future scenarios that we may see unfold as SLA tools and services mature.
Buckingham Shum, S, Aberer, K, Schmidt, A, Bishop, S, Lukowicz, P, Anderson, S, Charalabidis, Y, Domingue, J, de Freitas, S, Dunwell, I, Edmonds, B, Grey, F, Haklay, M, Jelasity, M, Karpistsenko, A, Kohlhammer, J, Lewis, J, Pitt, J, Sumner, R & Helbing, D 2012, 'Towards a global participatory platform: Democratising open data, complexity science and collective intelligence', The European Physical Journal. Special Topics, vol. 214, no. 1, pp. 109-152.View/Download from: UTS OPUS or Publisher's site
The FuturICT project seeks to use the power of big data, analytic models grounded in complexity science, and the collective intelligence they yield for societal benefit. Accordingly, this paper argues that these new tools should not remain the preserve of restricted government, scientific or corporate élites, but be opened up for societal engagement and critique. To democratise such assets as a public good, requires a sustainable ecosystem enabling different kinds of stakeholder in society, including but not limited to, citizens and advocacy groups, school and university students, policy analysts, scientists, software developers, journalists and politicians. Our working name for envisioning a sociotechnical infrastructure capable of engaging such a wide constituency is the Global Participatory Platform (GPP). We consider what it means to develop a GPP at the different levels of data, models and deliberation, motivating a framework for different stakeholders to find their ecological niches at different levels within the system, serving the functions of (i) sensing the environment in order to pool data, (ii) mining the resulting data for patterns in order to model the past/present/future, and (iii) sharing and contesting possible interpretations of what those models might mean, and in a policy context, possible decisions. A research objective is also to apply the concepts and tools of complexity science and social science to the projects own work. We therefore conceive the global participatory platform as a resilient, epistemic ecosystem, whose design will make it capable of self-organization and adaptation to a dynamic environment, and whose structure and contributions are themselves networks of stakeholders, challenges, issues, ideas and arguments whose structure and dynamics can be modelled and analysed.
De Liddo, A, Sandor, A & Buckingham Shum, S 2012, 'Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study', Computer Supported Cooperative Work: the journal of collaborative computing, vol. 21, no. 4-5, pp. 417-448.View/Download from: UTS OPUS or Publisher's site
We propose the concept of Contested Collective Intelligence (CCI) as a distinctive subset of the broader Collective Intelligence design space. CCI is relevant to the many organizational contexts in which it is important to work with contested knowledge, for instance, due to different intellectual traditions, competing organizational objectives, information overload or ambiguous environmental signals. The CCI challenge is to design sociotechnical infrastructures to augment such organizational capability. Since documents are often the starting points for contested discourse, and discourse markers provide a powerful cue to the presence of claims, contrasting ideas and argumentation, discourse and rhetoric provide an annotation focus in our approach to CCI. Research in sensemaking, computer-supported discourse and rhetorical text analysis motivate a conceptual framework for the combined human and machine annotation of texts with this specific focus. This conception is explored through two tools: a social-semantic web application for human annotation and knowledge mapping (Cohere), plus the discourse analysis component in a textual analysis software tool (Xerox Incremental Parser: XIP). As a step towards an integrated platform, we report a case study in which a document corpus underwent independent human and machine analysis, providing quantitative and qualitative insight into their respective contributions. A promising finding is that significant contributions were signalled by authors via explicit rhetorical moves, which both human analysts and XIP could readily identify. Since working with contested knowledge is at the heart of CCI, the evidence that automatic detection of contrasting ideas in texts is possible through rhetorical discourse analysis is progress towards the effective use of automatic discourse analysis in the CCI framework.
Iandoli, L, Quinto, I, De Liddo, A & Buckingham Shum, S 2012, 'A Debate Dashboard to Enhance Online Knowledge Sharing.', VINE The Journal of Information and Knowledge Management Systems, vol. 42, no. 1, pp. 67-93.View/Download from: UTS OPUS or Publisher's site
Johnson, J, Buckingham Shum, S, Willis, A, Bishop, S, Zamenopoulos, T, Swithenby, S, MacKay, R, Merali, Y, Lorincz, A, Costea, C, Bourgine, P, Louca, J, Kapenieks, A, Kelly, P, Caird, S, Bromley, J, Crick, RD, Goldspink, C, Collet, P, Carbone, A & Helbing, D 2012, 'The FuturICT education accelerator', The European Physical Journal. Special Topics, vol. 214, pp. 215-243.View/Download from: UTS OPUS or Publisher's site
Education is a major force for economic and social wellbeing. Despite high aspirations, education at all levels can be expensive and ineffective. Three Grand Challenges are identified: (1) enable people to learn orders of magnitude more effectively, (2) enable people to learn at orders of magnitude less cost, and (3) demonstrate success by exemplary interdisciplinary education in complex systems science. A ten year `man-on-the-moon project is proposed in which FuturICTs unique combination of Complexity, Social and Computing Sciences could provide an urgently needed transdisciplinary language for making sense of educational systems. In close dialogue with educational theory and practice, and grounded in the emerging data science and learning analytics paradigms, this will translate into practical tools (both analytical and computational) for researchers, practitioners and leaders; generative principles for resilient educational ecosystems; and innovation for radically scalable, yet personalised, learner engagement and assessment. The proposed Education Accelerator will serve as a `wind tunnel for testing these ideas in the context of real educational programmes, with an international virtual campus delivering complex systems education exploiting the new understanding of complex, social, computationally enhanced organisational structure developed within FuturICT
Buckingham Shum, S, Cannavacciuolo, L, De Liddo, A, Iandoli, L & Quinto, I 2011, 'Using Social Network Analysis to Support Collective Decision-Making Process', International Journal of Decision Support System Technology, vol. 3, no. 2, pp. 15-31.View/Download from: UTS OPUS or Publisher's site
Current traditional technologies, while enabling effective knowledge sharing and accumulation, seem to be less supportive of knowledge organization, use and consensus formation, as well as of collaborative decision making process. To address these limitations and thus to better foster collective decision-making around complex and controversial problems, a new family of tools is emerging able to support more structured knowledge representations known as collaborative argument mapping tools. This paper argues that online collaborative argumentation has the rather unique feature of combining knowledge organization with social mapping and that such a combination can provide interesting insights on the social processes activated within a collaborative decision making initiative. In particular, the authors investigate how Social Network Analysis can be used for the analysis of the collective argumentation process to study the structural properties of the concepts and social networks emerging from users interaction. Using Cohere, an online platform designed to support collaborative argumentation, some empirical findings obtained from two use cases are presented.
Selvin, AM, Buckingham Shum, S & Aakhus, M 2010, 'The practice level in participatory design rationale: studying practitioner moves and choices', Human Technology, vol. 6, no. 1, pp. 71-105.View/Download from: UTS OPUS
Most research in design rationale focuses on specific tools, methods, models, or artifacts. There has been relatively little attention to the practice level of design rationale work: the human experience of working with the tools and methods to create rationale artifacts. This paper explores a particular juncture of creativity and design rationale that is found in the special case of helping groups of people construct representations of rationale in live meetings. Doing such work poses challenges and requires skills different than those of indi- viduals working alone. We describe the role of practitioners, who perform caretaking and facilitative functions in collaborative or participatory design rationale sessions, and present a set of analytical tools aimed at making the practice level more visible. We locate the analysis in a theoretical framework aimed at understanding the experiential dimensions of such practice, includ- ing sensemaking, narrative, aesthetics, ethics, and improvisation
Bailey, H, Bachler, M, Buckingham Shum, S, Le Blanc, A, Popat, S, Rowley, A & Turner, M 2009, 'Dancing on the Grid: Using e-Science Tools to Extend Choreographic Research', Royal Society of London. Philosophical Transactions A. Mathematical, Physical and Engineering Sciences, vol. 367, no. 1898, pp. 2793-2806.View/Download from: UTS OPUS or Publisher's site
This paper considers the role and impact of new and emerging e-Science tools on practice-led research in dance. Specifically it draws on findings from the e-Dance project. This two-year project brings together an interdisciplinary team combining research aspects of choreography, next generation of video conferencing, and Human-Computer Interaction analysis incorporating hypermedia and non-linear annotations for recording and documentation.
Okada, A & Buckingham Shum, S 2008, 'Evidence-Based Dialogue Maps as a Research Tool to Investigate the Quality of School Pupils Scientific Argumentation', International Journal of Research and Method in Education, vol. 31, no. 3, pp. 291-315.View/Download from: UTS OPUS or Publisher's site
This pilot study focuses on the potential of Evidence-based Dialogue Mapping as a participatory action research tool to investigate young teenagers scientific argumentation. Evidence-based Dialogue Mapping is a technique for representing graphically an argumentative dialogue through Questions, Ideas, Pros, Cons and Data. Our research objective is to better understand the usage of Compendium, a Dialogue Mapping software tool, as both (1) a learning strategy to scaffold school pupils argumentation and (2) as a method to investigate the quality of their argumentative essays. The participants were a science teacher-researcher, a knowledge mapping researcher and 20 pupils, 12-13 years old, in a summer science course for gifted and talented children in the UK. This study draws on multiple data sources: discussion forum, science teacher-researchers and pupils Dialogue Maps, pupil essays, and reflective comments about the uses of mapping for writing. Through qualitative analysis of two case studies, we examine the role of Evidence-based Dialogue Maps as a mediating tool in scientific reasoning: as conceptual bridges for linking and making knowledge intelligible; as support for the linearisation task of generating a coherent document outline; as a reflective aid to rethinking reasoning in response to teacher feedback; and as a visual language for making arguments tangible via cartographic conventions.
Okada, A, Tomadaki, E, Buckingham Shum, S & Scott, P 2008, 'Fostering Open Sensemaking Communities by Combining Knowledge Maps and Videoconferencing', Upgrade: the European journal for the informatics professional, vol. 9, no. 3, pp. 27-36.
Buckingham Shum, S, Uren, V, Li, G, Sereno, B & Mancini, C 2007, 'Modelling naturalistic argumentation in research literatures: representation and interaction design issues', International Journal of Intelligent Systems, vol. 22, no. 1, pp. 17-47.View/Download from: Publisher's site
Mancini, C & Buckingham Shum, S 2006, 'Modelling discourse in contested domains: A semiotic and cognitive framework', International Journal Of Human-computer Studies, vol. 64, no. 11, pp. 1154-1171.View/Download from: Publisher's site
Mancini, C, Scott, D & Buckingham Shum, S 2006, 'Visualising Discourse Coherence in Non-Linear Documents', TAL: Traitement Automatique des Langues, vol. 47, no. 2.
Uren, V, Buckingham Shum, S, Bachler, M & Li, G 2006, 'Sensemaking tools for understanding research literatures: design, implementation and user evaluation.', International Journal Of Human-computer Studies, vol. 64, no. 5, pp. 420-445.View/Download from: Publisher's site
Page, KR, Michaelides, DT, De Roure, DC, Shadbolt, NR, Chen-Burger, YH, Dalton, J, Potter, S, Tate, A, Shum, SJB, Eisenstadt, M, Bachler, M & Komzak, J 2005, 'Collaboration in the semantic grid: A basis for e-learning', Applied Artificial Intelligence, vol. 19, no. 9-10, pp. 881-904.View/Download from: Publisher's site
The CoAKTinG project aims to advance the state of the art in collaborative mediated spaces for the Semantic Grid. This paper presents an overview of the hypertext and knowledge-based tools that have been deployed to augment exiting collaborative environments, and the ontology that is used to exchange structure, promote enhanced process tracking, and aid navigation of resources before, after, and during a collaboration. While the primary focus of the project has been supporting e-Science, this paper also explores the similarities and application of CoAKTinG technologies as part of a human-centered design approach to e-Learning.
Selvin, AM & Buckingham Shum, S 2005, 'Hypermedia as a productivity tool for doctoral research', New Review of Hypermedia and Multimedia, vol. 11, no. 1, pp. 91-101.
Sierhuis, M, Clancey, WJ, Alena, RL, Berrios, D, Shum, SB, Dowding, J, Graham, J, Van Hoof, R, Kaskiris, C, Rupert, S & Tyree, KS 2005, 'NASA's Mobile Agents Architecture: A multi-agent workflow and communication system for planetary exploration', European Space Agency, (Special Publication) ESA SP, no. 603, pp. 471-479.
NASA Ames' Mobile Agents Architecture is a distributed agent-based architecture, which integrates diverse mobile entities in a wide-area wireless system for lunar and planetary surface operations. Software agents, implemented in the Brahms multiagent language, run in Brahms virtual machines onboard laptops for space suits, robots, and surface habitats. "Personal agents" support the habitat crew and surface astronauts, as well as the their robotic assistant. People communicate with their personal agents via a speech dialogue system and via a meeting-capture hyperlink database tool.
Buckingham Shum, S 2004, 'Contentious, dynamic, multimodal domains and ontologies?', IEEE Intelligent Systems, vol. 19, no. 1, pp. 80-81.
Sereno, B, Uren, V, Shum, SB & Motta, E 2004, 'Semantic annotation support in the absence of consensus', SEMANTIC WEB: RESEARCH AND APPLICATIONS, vol. 3053, pp. 357-371.
Selvin, AM & Buckingham Shum, S 2002, 'Rapid knowledge construction: a case study in corporate contingency planning using collaborative hypermedia', Knowledge and Process Management, vol. 9, no. 2, pp. 119-128.
How can new media positively transform scholarly practices? In this article, we describe the Journal of Interactive Media in Education (JIME). JIME's peer review process is designed to promote multidisciplinary dialogue through the use of a purpose-designed Web document-discussion interface. This innovative peer review model and the resulting enriched digital documents illustrate some of the possibilities for promoting knowledge construction and preserving intellectual products in digital scholarly publications. We present JIME's technical infrastructure, editorial policy, and peer review process, and discuss how these features are used to support the journal's goals. Finally, we conclude by considering what aspects of our approach might be suitable for e-journals in other disciplines. Copyright ©2001, First Monday.
Buckingham Shum, S, Motta, E & Domingue, J 2000, 'ScholOnto: An Ontology-Based Digital Library Server for Research Documents and Discourse', International Journal on Digital Libraries, vol. 3, no. 3, pp. 237-248.
Motta, E, Buckingham Shum, S & Domingue, J 2000, 'Ontology-Driven Document Enrichment: Principles, Tools and Applications', International Journal Of Human-computer Studies, vol. 52, no. 6, pp. 1071-1109.View/Download from: Publisher's site
Blandford, A & Buckingham Shum, S 1998, 'Training Software Engineers in a Novel Usability Evaluation Technique', International Journal Of Human-computer Studies, vol. 49, no. 3, pp. 245-279.View/Download from: Publisher's site
Buckingham Shum, S 1997, 'Negotiating the Construction and Reconstruction of Organisational Memories', Journal of Universal Computer Science, vol. 3, no. 8, pp. 899-928.
Buckingham Shum, S, MacLean, A, Bellotti, V & Hammond, N 1997, 'Graphical Argumentation and Design Cognition', Human-Computer Interaction, vol. 12, no. 3, pp. 267-300.View/Download from: Publisher's site
Buckingham Shum, SC 1997, 'World Wide Web Usability: Introduction to this Special Issue', International Journal Of Human-computer Studies, vol. 47, no. 1, pp. 1-4.
Buckingham Shum, S & Hammond, N 1994, 'Argumentation-Based Design Rationale: What Use at What Cost?', International Journal Of Human-computer Studies, vol. 40, no. 4, pp. 603-652.View/Download from: Publisher's site
Buckingham Shum, S & Hammond, N 1994, 'Delivering HCI Modelling to Designers: A Framework and Case Study of Cognitive Modelling', Interacting With Computers, vol. 6, no. 3, pp. 311-341.View/Download from: Publisher's site
Buckingham Shum, S & Hammond, N 1994, 'Delivering hci modelling to designers: A framework and case study of cognitive modelling', Interacting with Computers, vol. 6, no. 3, pp. 314-341.View/Download from: Publisher's site
The human-computer interaction (HCI) research community is generating a large number of usability-oriented models and design frameworks. However, a critical factor which will determine whether any of these achieve significant penetration into the real world of software design is the effort required by practitioners to understand and apply them. In short, analytic tools for usability design must themselves be usable. In response to this challenge, we present a framework which identifies four different ‘gulfs’ between user-centred modelling and design approaches, and their intended users. These gulfs are potential opportunities to support designers if a given analytic approach can be encapsulated in appropriate forms. We then illustrate the framework’s application with a concrete example. An evaluation is reported which investigates gulfs associated with an approach which uses an expert system to automate cognitive modelling for human factors designers. An early prototype was evaluated in order to assess the knowledge required to use it. The study demonstrates that whilst this tool does shield users from the complexities of the underlying modelling, they need to understand the way in which it builds its description of the task and user interface. Implications for bridging the different gulfs are then considered. © 1994 Butterworth-Heinemann Ltd.
Buckingham Shum, S 1990, 'Real and Virtual Spaces: Mapping from Spatial Cognition to Hypertext', Hypermedia, vol. 2, no. 2, pp. 133-158.
Buckingham Shum, S, 'Carol Tenopir and Donald King (1999) Towards Electronic Journals: Realities for Scientists, Librarians and Publishers . Book review by Simon Buckingham Shum', Journal of Interactive Media in Education, vol. 2000, no. 4, pp. 4-4.View/Download from: Publisher's site
Buckingham Shum, SJ, Sumner, T & Spohrer, J, 'Educational Authoring Tools and the Educational Object Economy: Introduction to this Special Issue from the East/West Group', Journal of Interactive Media in Education, vol. 1998, no. 2, pp. 10-10.View/Download from: Publisher's site
Martinez-Maldonaldo, R, Buckingham-Shum, S, Schneider, B, Charleer, S, Klerkx, J & Duval, E, 'Learning Analytics for Natural User Interfaces', Journal of Learning Analytics, vol. 4, no. 1, pp. 24-57.View/Download from: Publisher's site
Kitto, K, Cutrupi, J, Philips, M, Gardiner, G, Ghodrati, M & Buckingham Shum, S 2019, 'The connected university: Connectedness Learning across a lifetime' in Bridgstock, R & Tippett, N (eds), Higher Education and the Future of Graduate Employability, Edward Elgar Publishing, pp. 230-249.View/Download from: UTS OPUS
This book challenges the dominant ‘employability skills’ discourse by exploring socially connected and networked perspectives to learning and teaching in higher education.
Thompson, K, Alhadad, S, Buckingham Shum, S, Howard, S, Knight, S, Martinez-Maldonado, R & Pardo, A 2019, 'Connecting expert knowledge in the design of classroom learning experiences' in Lodge, J, Cooney Horvath, J & Corrin, L (eds), Learning analytics in the classroom: Translating learning analytics research for teachers, Routledge.View/Download from: UTS OPUS
Knight, S & Buckingham Shum, S 2016, 'Theory and Learning Analytics' in The Handbook of Learning Analytics and Educational Data Mining, SOCIETY for LEARNING ANALYTICS RESEARCH, UK, pp. 17-22.View/Download from: Publisher's site
The challenge of understanding how theory and analytics relate is to move “from clicks to
constructs” in a principled way. Learning analytics are a specific incarnation of the bigger
shift to an algorithmically pervaded society, and their wider impact on education needs
careful consideration. In this chapter, we argue that by design — or else by accident — the
use of a learning analytics tool is always aligned with assessment regimes, which are in
turn grounded in epistemological assumptions and pedagogical practices. Fundamentally
then, we argue that deploying a given learning analytics tool expresses a commitment to
a particular educational worldview, designed to nurture particular kinds of learners. We
outline some key provocations in the development of learning analytic techniques, key
questions to draw out the purpose and assumptions built into learning analytics. We suggest
that using “claims analysis” — analysis of the implicit or explicit stances taken in the design
and deploying of technologies — is a productive human-centred method to address these
key questions, and we offer some examples of the method applied to those provocations.
Sierhuis, M & Shum, SJB 2014, 'Human-agent knowledge cartography for e-science: NASA field trials at the mars desert research station' in Advanced Information and Knowledge Processing, pp. 381-399.View/Download from: Publisher's site
© Springer-Verlag London 2014. This chapter describes the sociotechnical embedding of a knowledge cartography approach (Conversational Modelling) within a prototype e-science work system. This was evaluated over two 2-week field trials, simulating collaborative Mars-Earth geological exploration. We believe this work is the first demonstration of a knowledge mapping tool embedded within a human/software multiagent work system, with humans and agents reading and writing structures amenable to agent understanding and autonomous agent execution, and human understanding, annotation and argumentation. Secondly, in terms of the applied problem, we have demonstrated how human and agent plans, data, multimedia documents, metadata, discussions, interpretations and arguments can be mapped in an integrated manner, and successfully deployed in field trials which simulated aspects of mission workload pressure.
Selvin, AM, Buckingham Shum, S & Aakhus, M 2013, 'The Practice Level in Participatory Design Rationale: Studying Practitioner Moves and Choices.' in Carroll, JM (ed), Creativity and Rationale, Springer, London, pp. 287-326.View/Download from: UTS OPUS or Publisher's site
Most research in design rationale focuses on specific tools, methods, models, or artifacts. There has been relatively little attention to the practice level of design rationale work: the human experience of working with the tools and methods to create rationale artifacts. This paper explores a particular juncture of creativity and design rationale that is found in the special case of helping groups of people construct representations of rationale within live meetings. Such work poses challenges and requires skills different from those of individuals working alone. We describe the role of practitioners who perform caretaking and facilitative functions in collaborative or participatory design rationale sessions, and present a set of analytical tools aimed at making the practice level more visible. We locate the analysis in a theoretical framework aimed at understanding the experiential dimensions of such practice, including sensemaking, narrative, aesthetics, ethics, and improvisation.
Ferguson, R & Buckingham Shum, S 2012, 'Towards a Social Learning Space for Open Educational Resources' in Okada, A, Connolly, T & Scott, P (eds), Collaborative Learning 2.0: Open Educational Resources, IGI Global, Hershey PA, pp. 309-327.View/Download from: UTS OPUS or Publisher's site
This chapter examines the meaning of open in terms of tools, resources, and education, and goes on to explore the association between open approaches to education and the development of online social learning. It considers why this form of learning is emerging so strongly at this point, what its underlying principles are, and how it can be defined. Openness is identified as one of the motivating rationales for a social media space tuned for learning, called SocialLearn, which is currently being trialed at The Open University in the UK. SocialLearn has been designed to support online social learning by helping users to clarify their intention, ground their learning and engage in learning conversations. The emerging design concept and implementation are described here, with a focus on what personalization means in this context, and on how learning analytics could be used to provide different types of recommendation that support learning.
Okada, A, Shum, SB, Bachler, M, Tomadaki, E, Scott, P, Little, A & Eisenstadt, M 2010, 'Knowledge Media Tools to Foster Social Learning' in Social Computing, IGI Global, pp. 864-884.View/Download from: Publisher's site
The aim of this chapter is to overview the ways in which knowledge media technologies create opportunities for social learning. The Open Content movement has been growing rapidly, opening up new opportunities for widening participation. One of the Open Educational Resources (OER) initiatives is the OpenLearn project, launched by the Open University, which integrates three knowledge media technologies: Compendium, FM and MSG. In this chapter, the authors analyse some examples, which show how these tools can be used to foster open sensemaking communities by mapping knowledge, location and virtual interactions. At the end, they present some questions and future horizons related to this research.
Okada, A, Buckingham Shum, S, Bachler, M, Tomadaki, E, Scott, P, Little, A & Eisenstadt, M 2009, 'Knowledge media tools to foster social learning' in Hatzipanagos, S & Warburton, S (eds), Handbook of Research on Social Software and Developing Community Ontologies, IGI Global, Hershey, PA, USA, pp. 357-380.View/Download from: UTS OPUS or Publisher's site
The aim of this chapter is to overview the ways in which knowledge media technologies create opportunities for social learning. The Open Content movement has been growing rapidly, opening up new opportunities for widening participation. One of the Open Educational Resources (OER) initiatives is the OpenLearn project, launched by the Open University, which integrates three knowledge media technologies: Compendium, FM and MSG. In this chapter, the authors analyse some examples, which show how these tools can be used to foster open sensemaking communities by mapping knowledge, location and virtual interactions. At the end, they present some questions and future horizons related to this research
Okada, A, Tomadaki, E, Bachler, M, Eisenstadt, M, Buckingham Shum, S & Scott, P 2009, 'Knowledge Media Tools to Foster Social Learning in Open Sensemaking Communities' in Hatzipanagos, S & Warburton, S (eds), Handbook of Research on Social Software and Developing Community Ontologies, IGI Global Snippet.
"This book explores how social software and developing community ontologies are challenging the way we operate in a performative space"--Provided by publisher.
Sierhuis, M & Buckingham Shum, S 2008, 'Human-agent knowledge cartography for e-science: NASA field trials at the Mars Desert Research Station' in Okada, A, Buckingham Shum, S & Sherborne, T (eds), Knowledge Cartography: Software Tools and Mapping Techniques, Springer, London, pp. 287-306.View/Download from: UTS OPUS
This chapter describes the sociotechnical embedding of a knowledge cartography approach (Conversational Modelling) within a prototype e-science work system. This was evaluated over two 2-week field trials, simulating collaborative Mars-Earth geological exploration. We believe this work is the first demonstration of a knowledge mapping tool embedded within a human/software multiagent work system, with humans and agents reading and writing structures amenable to agent understanding and autonomous agent execution, and human understanding, annotation and argumentation. Secondly, in terms of the applied problem, we have demonstrated how human and agent plans, data, multimedia documents, metadata, discussions, interpretations and arguments can be mapped in an integrated manner, and successfully deployed in field trials which simulated aspects of mission workload pressure.
Buckingham Shum, S 2007, 'Hypermedia Discourse: Contesting networks of ideas and arguments' in Priss, U, Polovina, S & Hill, R (eds), Conceptual Structures: Knowledge Architectures for Smart Applications, Springer, Berlin, pp. 29-44.View/Download from: UTS OPUS
This invited contribution motivates the Hypermedia Discourse research programme, investigating the reading, writing and contesting of ideas as hypermedia networks grounded in discourse schemes. We are striving for cognitively and computationally tractable conceptual structures: fluid enough to serve as augmentations to group working memory, yet structured enough to support long term memory. I will describe how such networks can be (i) mapped by multiple analysts to visualize and interrogate the claims and arguments in a literature, and (ii) mapped in real time to manage a team's information sources, competing interpretations, arguments and decisions, particularly in time- pressured scenarios where harnessing collective intelligence is a priority. Given the current geo-political and environmental context, the growth in distributed teamwork, and the need for multidisciplinary approaches to wicked problems, there has never been a greater need for sensemaking tools to help diverse stakeholders build common ground.
Buckingham Shum, S, Selvin, AM, Sierhuis, M, Conklin, J, Haley, CB & Nuseibeh, B 2006, 'Hypermedia support for argumentation-based rationale: 15 years on from gIBIS and QOC' in Dutoit, A, McCall, R, Mistrik, I & Paech, B (eds), Rationale Management in Software Engineering, Springer-Verlag, Berlin, pp. 111-132.View/Download from: UTS OPUS
Having developed, used and evaluated some of the early IBIS-based approaches to design rationale (DR) such as gIBIS and QOC in the late 1980s/mid-1990s, we describe the subsequent evolution of the argumentation-based paradigm through software support, and perspectives drawn from modeling and meeting facilitation. Particular attention is given to the challenge of negotiating the overheads of capturing this form of rationale. Our approach has maintained a strong emphasis on keeping the representational scheme as simple as possible to enable real time meeting mediation and capture, attending explicitly to the skills required to use the approach well, particularly for the sort of participatory, multi-stakeholder requirements analysis demanded by many design problems. However, we can then specialize the notation and the way in which the tool is used in the service of specific methodologies, supported by a customizable hypermedia environment, and interoperable with other software tools. After presenting this approach, called Compendium, we present examples to illustrate the capabilities for support security argumentation in requirements engineering, template driven modeling for document generation, and IBIS-based indexing of and navigation around video records of meetings.
Shum, SJB, Selvin, AM, Sierhuis, M, Conklin, J, Haley, CB & Nuseibeh, B 2006, 'Hypermedia support for argumentation-based rationale' in Rationale Management in Software Engineering, pp. 111-132.View/Download from: Publisher's site
Having developed, used and evaluated some of the early IBISbased approaches to design rationale (DR) such as gIBIS and QOC in the late 1980s/mid-1990s, we describe the subsequent evolution of the argumentation-based paradigm through software support, and perspectives drawn from modeling and meeting facilitation. Particular attention is given to the challenge of negotiating the overheads of capturing this form of rationale. Our approach has maintained a strong emphasis on keeping the representational scheme as simple as possible to enable real time meeting mediation and capture, attending explicitly to the skills required to use the approach well, particularly for the sort of participatory, multistakeholder requirements analysis demanded by many design problems. However, we can then specialize the notation and the way in which the tool is used in the service of specific methodologies, supported by r a customizable hypermedia environment, and interoperable with other software tools. After presenting this approach, called Compendium, we present examples to illustrate the capabilities for support security argumentation in requirements engineering, template driven modeling for document generation, and IBIS-based indexing of and navigation around video records of meetings. © Springer-Verlag Berlin Heidelberg 2006.
Buckingham Shum, S 2003, 'The roots of computer supported argument visualization' in Kirschner, PA, Buckingham Shum, SJ & Carr, CS (eds), Visualizing Argumentation: Software Tools for Collaborative and Educational Sense Making, Springer-Verlag, London, pp. 3-24.View/Download from: UTS OPUS
Buckingham Shum, S, Uren, V, Li, G, Domingue, J & Motta, E 2003, 'Visualizing internetworked argumentation' in Kirschner, PA & Buckingham Shum, SJA (eds), Visualizing Argumentation: Software Tools for Collaborative and Educational Sense Making, Springer-Verlag, London, pp. 185-204.View/Download from: UTS OPUS
Sumner, T, Buckingham Shum, S & Yates, S 2003, 'Interactive peer review in the Journal of Interactive Media in Education: processes, tools, and techniques for managing persistent discourse' in Jones Wayne (ed), E-serials: Publishers, Libraries, Users, and Standards, The Hawthorn Information, Binghamton, NY, pp. 197-216.
Buckingham Shum, S, Domingue, J & Motta, E 2001, 'Ontology-Driven Knowledge Management: Philosophical, Modelling and Organizational Issues' in Roy, R (ed), Industrial Knowledge Management. A Mirco Level Approach, Springer-Verlag, London, pp. 265-280.View/Download from: UTS OPUS or Publisher's site
Our approach to knowledge management, called Ontology-Driven Document Enrichment, aims to support knowledge creation and sharing by enriching documents through formal models. These reflect a particular viewpoint over a set of resources, which is shared by a community of practice. In the first part of this chapter we describe the basic tenets of the approach and we briefly illustrate the technologies which we have developed to support it and the application domains in which it has been applied. In the second part of this chapter we evaluate our approach in terms of a number of issues, which have been raised in the research community. These pose important research questions for designers of knowledge management technologies that make use of formal knowledge models. These questions address issues of epistemology (what happens to the knowledge as it is formalized?), socio-technology (how to integrate knowledge technologies into work practice?), and usability (do end-users understand, and use, the formal scheme?). In the chapter we show how these issues are tackled in our approach and we emphasize the multi-faceted nature of knowledge management solutions.
Buckingham Shum, S 1998, 'Negotiating the Construction of Organisational Memories' in Borghoff, UM & Pareschi, R (eds), Information Technology for Knowledge Management, Springer, Berlin, pp. 55-78.View/Download from: UTS OPUS or Publisher's site
This chapter describes an approach to capturing organisational memory, which serves to ground an analysis of human issues that knowledge management (KM) technologies raise. In the approach presented, teams construct graphical webs of the arguments and documents relating to key issues they are facing. This supports collaborative processes which are central to knowledge work, and provides a group memory of this intellectual investment. This approach emphasises the centrality of negotiation in making interdisciplinary decisions in a changing environment. Discussion in the chapter focuses on key human dimensions to KM technologies, including the cognitive and group dynamics set up by an approach, the general problem of preserving contextual cues, and the political dimensions to formalising knowledge processes and products. These analyses strongly motivate the adoption of participatory design processes for KM systems.
Buckingham Shum, S & Sumner, T 1998, 'New Scenarios in Scholarly Publishing and Debate' in Eisenstadt, M & Vincent, T (eds), The Knowledge Web: Learning and Collaborating on the Net, Kogan, London, pp. 135-151.View/Download from: UTS OPUS
This chapter describes how we are seeking to exploit the potential of interactive Web media through user-centred analysis of a mode of working that defines academic lifecritical, structured debate about documents. We are as interested in supporting school and university students critiquing course texts and publishing essays, as in professional scholars and researchers engaged in journal peer review and publishing. We question the replication of papyrocentric models which do not stop to question whether the new media make possible new modes of working, and new forms of document. We then describe the design principles underpinning the environments for reading and critiquing Web documents that the D3E Publishing Toolkit generates, and illustrate how things might be different by working through two scenarios of academic document-centred discourse, as supported by D3E. We conclude by pointing to directions in which future work could develop.
Buckingham Shum, S 1996, 'Analyzing the Usability of a Design Rationale Notation' in Moran, TP & Carroll, JM (eds), Design Rationale: Concepts, Techniques, and Use, Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 185-215.View/Download from: UTS OPUS
Semiformal, argumentation-based notations are one of the main classes of formalism currently being used to represent design rationale (DR). However, our understanding of the demands on designers of using such representations has to date been drawn largely from informal and anecdotal evidence. One way to tackle the fundamental challenge of reducing DRs representational overheads, is to understand the relationship between designing, and the idea structuring tasks introduced by a semiformal DR notation. Empirically based analyses of DR in use can therefore inform the design of the notations in order to turn the structuring effort to the designers advantage. This is the approach taken in this chapter, which examines how designers use a DR notation during design problem solving. Two empirical studies of DR-use are reported, in which designers used the QOC notation (MacLean et al., this volume) to express rationale for their designs. In the first study, a substantial and consistent body of evidence was gathered, describing the demands of the core representational tasks in using QOC, and the variety of strategies which designers adopt in externalising ideas. The second study suggests that an argumentation-based design model based around laying out discrete, competing Options is inappropriate during a depth-first, `evolutionary mode of working, centered around developing a single, complex Option. In addition, the data provide motivation for several extensions to the basic QOC notation. The chapter concludes by comparing the account of the QOCdesign relationship which emerges from these studies, with reports of other DR approaches in use.
Buckingham Shum, S 1996, 'Design Argumentation as Design Rationale' in Kent, A & Williams, JG (eds), The Encyclopedia of Computer Science and Technology, Vol 35, Supplement 20, Marcel Dekker Inc., New York, pp. 95-128.
A design rationale (DR) is a representation of the reasoning behind the design of an artifact. In recent years, the use of semiformal notations for organising arguments about design decisions has attracted much interest within the software engineering and human-computer interaction communities, leading to the development of a number of DR notations and tool environments. This article begins by reviewing the motivation for expressing DR as design argumentation, and then surveys evidence from design studies to show when and how it can be productive to construct explicit design argumentation during design. The article then discusses practical cognitive, organizational and technological factors which could facilitate the uptake of design rationale systems.
Shum, SB, Blandford, A, Duke, D, Good, J, May, J, Paterno’, F & Young, R 1996, 'Multidisciplinary Modelling for User-Centred System Design: An Air-traffic Control Case Study' in People and Computers XI, Springer London, pp. 201-219.View/Download from: Publisher's site
MacLean, A, Bellotti, V & Buckingham Shum, S 1993, 'Developing the Design Space with Design Space Analysis' in Byerley, PF, Barnard, PJ & May, J (eds), Computers, Communication and Usability: Design Issues, Research and Methods for Integrated Services, Elsevier, Amsterdam, pp. 197-219.View/Download from: UTS OPUS
Shum, SB, Hammond, N, Cockton, G, Draper, S & Weir, GRS, 'Transferring HCI Modelling and Design Techniques to Practitioners: A Framework and Empirical Work' in People and Computers IX, Cambridge University Press, pp. 21-36.View/Download from: Publisher's site
Buckingham Shum, S, Echeverria, V & Martinez-Maldonado, R 2019, 'The Multimodal Matrix as a Quantitative Ethnography Methodology', Communications in Computer and Information Science, pp. 26-40.View/Download from: Publisher's site
© Springer Nature Switzerland AG 2019. This paper seeks to contribute to the emerging field of Quantitative Ethnography (QE) by demonstrating its utility to solve a complex challenge in Learning Analytics: the provision of timely feedback to collocated teams and their coaches. We define two requirements that extend the QE concept in order to operationalise it such a design process, namely, the use of co-design methodologies, and the availability of automated analytics workflow to close the feedback loop. We introduce the Multimodal Matrix as a data modelling approach that can integrate theoretical concepts about teamwork with contextual insights about specific work practices, enabling the analyst to map between higher order codes and low-level sensor data, with the option add the results of manually performed analyses. This is implemented in software as a workflow for rapid data modelling, analysis and interactive visualisation, demonstrated in the context of nursing teamwork simulations. We propose that this exemplifies how a QE methodology can underpin collocated activity analytics, at scale, with in-principle applications to embodied, collocated activities beyond our case study.
Echeverria, V, Martinez-Maldonado, R & Shum, SB 2019, 'Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data', CHI 2019: PROCEEDINGS OF THE 2019 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, CHI Conference on Human Factors in Computing Systems (CHI), ASSOC COMPUTING MACHINERY, Glasgow, SCOTLAND.View/Download from: Publisher's site
Shibani, A, Knight, S & Shum, SB 2019, 'Contextualizable learning analytics design: A generic model and writing analytics evaluations', Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK'19), International Conference on Learning Analytics and Knowledge, ACM, USA, pp. 210-219.View/Download from: UTS OPUS or Publisher's site
© 2019 Copyright is held by the owner/author(s). Publication rights licensed to ACM. A major promise of learning analytics is that through the collection of large amounts of data we can derive insights from authentic learning environments, and impact many learners at scale. However, the context in which the learning occurs is important for educational innovations to impact student learning. In particular, for student-facing learning analytics systems like feedback tools to work effectively, they have to be integrated with pedagogical approaches and the learning design. This paper proposes a conceptual model to strike a balance between the concepts of generalizable scalable support and contextualized specific support by clarifying key elements that help to contextualize student-facing learning analytics tools. We demonstrate an implementation of the model using a writing analytics example, where the features, feedback and learning activities around the automated writing feedback tool are tuned for the pedagogical context and the assessment regime in hand, by co-designing them with the subject experts. The model can be employed for learning analytics to move from generalized support to meaningful contextualized support for enhancing learning.
Echeverria, V, Martinez-Maldonado, R, Granda, R, Chiluiza, K, Conati, C & Shum, SB 2018, 'Driving data storytelling from learning design', Proceedings of the 8th International Conference on Learning Analytics and Knowledge, International Conference on Learning Analytics and Knowledge, ACM, Sydney, New South Wales, Australia, pp. 131-140.View/Download from: UTS OPUS or Publisher's site
© 2018 Association for Computing Machinery. Data science is now impacting the education sector, with a growing number of commercial products and research prototypes providing learning dashboards. From a human-centred computing perspective, the end-user’s interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that ‘usable’ visualisations are not necessarily effective from a learning perspective. Since an educator’s interpretation of visualised data is essentially the construction of a narrative about student progress, we draw on the growing body of work on Data Storytelling (DS) as the inspiration for a set of enhancements that could be applied to data visualisations to improve their communicative power. We present a pilot study that explores the effectiveness of these DS elements based on educators’ responses to paper prototypes. The dual purpose is understanding the contribution of each visual element for data storytelling, and the effectiveness of the enhancements when combined.
Echeverria, V, Martinez-Maldonado, R, Power, T, Hayes, C & Shum, SB 2018, 'Where is the Nurse? Towards automatically visualising meaningful team movement in healthcare education', 19th International Conference, AIED 2018, London, UK, June 27–30, 2018, Proceedings, Part II (LNCS), International Conference on Artificial Intelligence in Education, Springer, London, United Kingdom, pp. 74-78.View/Download from: UTS OPUS or Publisher's site
© Springer International Publishing AG, part of Springer Nature 2018. Providing immediate, effective feedback on team and individual performance in healthcare simulations is a challenging task for educators, such is their complexity. Focusing on emergency procedures on patient manikins, our prior work has demonstrated the feasibility of using multimodal data capture and analysis to generate visualisations of student movement, talk and treatment actions. The limitation to date has been the need for manual steps in the analytic workflow. This paper documents how we have automated several key steps, using new technologies, which were piloted during a nursing simulation. Combining role-based nurses’ movement data with high fidelity manikin logs, we have implemented a zone-based classification model, and are able to automatically visualise movements within an emergency response team, providing the data needed to design near real-time feedback for both educators and students.
Kitto, K, Shum, SB & Gibson, A 2018, 'Embracing imperfection in learning analytics', Proceedings of the 8th International Conference on Learning Analytics and Knowledge, International Conference on Learning Analytics and Knowledge, ACM, Sydney, New South Wales, Australia, pp. 451-460.View/Download from: UTS OPUS or Publisher's site
© 2018 Copyright held by the owner/author(s). Learning Analytics (LA) sits at the confluence of many contributing disciplines, which brings the risk of hidden assumptions inherited from those fields. Here, we consider a hidden assumption derived from computer science, namely, that improving computational accuracy in classification is always a worthy goal. We demonstrate that this assumption is unlikely to hold in some important educational contexts, and argue that embracing computational “imperfection” can improve outcomes for those scenarios. Specifically, we show that learner-facing approaches aimed at “learning how to learn” require more holistic validation strategies. We consider what information must be provided in order to reasonably evaluate algorithmic tools in LA, to facilitate transparency and realistic performance comparisons.
Knight, S, Shibani, A & Buckingham Shum, S 2018, 'Augmenting Formative Writing Assessment with Learning Analytics: A Design Abstraction Approach', 13th International Conference of the Learning Sciences: Rethinking Learning in the Digital Age. Making the Learning Sciences Count, International Conference of the Learning Sciences, International Society of the Learning Sciences, London, UK, pp. 1783-1790.View/Download from: UTS OPUS
Martinez-Maldonado, R, Anderson, TD, Silva Feraud, I & Buckingham Shum, S 2018, 'Making learning journeys visible: Towards supporting collective reflection on graduate attributes', Proceedings of International Conference of the Learning Sciences, ICLS, International Society of the Learning Sciences, London, UK, pp. 941-944.View/Download from: UTS OPUS
© ISLS. Although some computer-based systems exist to facilitate the management of capstone projects, it is not really clear how learners can be supported to reflect on the connection between their (past and ongoing) learning experiences, and the graduate attributes (GAs) they are intended to develop. This paper proposes a technological infrastructure, and the epistemic and social scaffolding, for students to collectively reflect on how each of their learning products generated across different units of study contribute to the development of their GAs, in light of a final capstone project. We illustrate the feasibility of our approach through the authentic deployment of the toolset in an immersive environment for supporting teams of final year students to reflect on their GAs development.
Prieto-Alvarez, CG, Martinez-Maldonado, R & Shum, SB 2018, 'Mapping learner-data journeys: Evolution of a visual co-design tool', OzCHI '18 Proceedings of the 30th Australian Conference on Computer-Human Interaction, Australian Conference on Computer-Human Interaction, ACM, Australia, pp. 205-214.View/Download from: UTS OPUS or Publisher's site
© 2018 Copyright held by the owner/author(s). In this paper we present a three-phase process for crafting Learner-Data Journey maps and using them as communication tools to involve other stakeholders in the co-design of a data-intensive educational tool. The three phases in this process are i) scaffolding groups of learners to collaboratively co-create a Learner-Data Journey based on their own experience, ii) distilling key insights from these journey maps, and iii) providing the means for multiple stakeholders to integrate and synthesise key insights from these journey maps to suggest design requirements. We illustrate the process and the kind of tools that can support the co-creation of Learner-Data Journeys in two educational scenarios where learners have become partners of their own 'surveillance'.
Shibani, A, Knight, S & Shum, SB 2018, 'Understanding Revisions in Student Writing Through Revision Graphs', Artificial Intelligence in Education, Springer, pp. 332-336.View/Download from: UTS OPUS or Publisher's site
Text revision is regarded as an important process in improving written products. To study the process of revision activity from authentic classroom contexts, this paper introduces a novel visualization method called Revision Graph to aid detailed analysis of the writing process. This opens up the possibility of exploring the stages in students’ revision of drafts, which can lead to further automation of revision analysis for researchers, and formative feedback to students on their writing. The Revision Graph could also be applied to study the direct impact of automated feedback on students’ revisions and written outputs in stages of their revision, thus evaluating its effectiveness in pedagogic contexts.
Shum, SB 2018, 'Transitioning education’s knowledge infrastructure: Shaping design or shouting from the touchline?', Proceedings of International Conference of the Learning Sciences, ICLS, 13th International Conference of the Learning Sciences, London, UK, pp. 5-5.View/Download from: UTS OPUS
© ISLS. Bit by bit, a data-intensive substrate for education is being designed, plumbed in and switched on, powered by digital data from an expanding sensor array, data science and artificial intelligence. The configurations of educational institutions, technologies, scientific practices, ethics policies and companies can be usefully framed as the emergence of a new “knowledge infrastructure” (Paul Edwards). The idea that we may be transitioning into significantly new ways of knowing – about learning and learners – is both exciting and daunting, because new knowledge infrastructures redefine roles and redistribute power, raising many important questions. For instance, assuming that we want to shape this infrastructure, how do we engage with the teams designing the platforms our schools and universities may be using next year? Who owns the data and algorithms, and in what senses can an analytics/AI-powered learning system be ‘accountable’? How do we empower all stakeholders to engage in the design process? Since digital infrastructure fades quickly into the background, how can researchers, educators and learners engage with it mindfully? If we want to work in “Pasteur’s Quadrant” (Donald Stokes), we must go beyond learning analytics that answer research questions, to deliver valued services to frontline educational users: but how are universities accelerating the analytics innovation to infrastructure transition? Wrestling with these questions, the learning analytics community has evolved since its first international conference in 2011, at the intersection of learning and data science, and an explicit concern with those human factors, at many scales, that make or break the design and adoption of new educational tools. We are forging open source platforms, links with commercial providers, and collaborations with the diverse disciplines that feed into educational data science. In the context of ICLS, our dialogue with the learning sciences must continue to deepen t...
Shum, SB, Ferguson, R, Merceron, A & Ochoa, X 2018, 'LAK 2018 program chairs’ welcome', ACM International Conference Proceeding Series, pp. iii-iv.
Echeverria, V, Martinez-Maldonado, R & Shum, SB 2017, 'Towards data storytelling to support teaching and learning', ACM International Conference Proceeding Series, 29th Australian Conference on Computer-Human Interaction 2017, Brisbane, Queensland, Australia, pp. 347-351.View/Download from: UTS OPUS or Publisher's site
© 2017 Association for Computing Machinery. All rights reserved. Data science is now impacting the educational sector, with a growing number of commercial products and research prototypes providing learning dashboards as feedback for both educators and students. From a human-centred computing perspective, the end-user's interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that 'usable' visualisations are not necessarily effective from a teaching and learning perspective. Since an educator's interpretation of visualised data is essentially the construction of a narrative about that student's progress, we draw on the growing body of work on 'Data Storytelling' (DS) as the inspiration for a set of enhancements that could be applied to data visualisations to improve their communicative power. We present a pilot study that explores the effectiveness of these DS elements based on educators' responses to paper prototypes. The dual purpose is understanding the contribution of each visual element for data storytelling, and the effectiveness of the enhancements when combined. The results suggest that DS elements could add clarity, especially when there are multiple possible stories in a complex visualisation.
Echeverria, V, Martinez-Maldonado, R, Chiluiza, K & Shum, SB 2017, 'DBCollab: Automated Feedback for Face-to-Face Group Database Design', 25TH INTERNATIONAL CONFERENCE ON COMPUTERS IN EDUCATION (ICCE 2017), International Conference on Computers in Education, ASIA PACIFIC SOC COMPUTERS IN EDUCATION, Christchurch, New Zealand, pp. 156-165.View/Download from: UTS OPUS
Gibson, A, Shum, SB, Aitken, A, Tsingos-Lucas, C, Sándor, Á & Knight, S 2017, 'Reflective writing analytics for actionable feedback', Proceedings of the Seventh International Learning Analytics & Knowledge Conference, International Learning Analytics & Knowledge Conference, ACM, Vancouver, British Columbia, Canada, pp. 153-162.View/Download from: UTS OPUS or Publisher's site
© 2017 ACM. Reflective writing can provide a powerful way for students to integrate professional experience and academic learning. However, writing reflectively requires high quality actionable feedback, which is time-consuming to provide at scale. This paper reports progress on the design, implementation, and validation of a Reflective Writing Analytics platform to provide actionable feedback within a tertiary authentic assessment context. The contributions are: (1) a new conceptual framework for reflective writing; (2) a computational approach to modelling reflective writing, deriving analytics, and providing feedback; (3) the pedagogical and user experience rationale for platform design decisions; and (4) a pilot in a student learning context, with preliminary data on educator and student acceptance, and the extent to which we can evidence that the software provided actionable feedback for reflective writing.
Knight, S, Allen, L, Gibson, A, McNamara, D & Shum, SB 2017, 'Writing analytics literacy - Bridging from research to practice', ACM International Conference Proceeding Series, pp. 496-497.View/Download from: UTS OPUS or Publisher's site
© 2017 ACM. There is untapped potential in achieving the full impact of learning analytics through the integration of tools into practical pedagogic contexts. To meet this potential, more work must be conducted to support educators in developing learning analytics literacy. The proposed workshop addresses this need by building capacity in the learning analytics community and developing an approach to resourcing for building 'writing analytics literacy'.
Knight, S, Martinez-Maldonado, R, Gibson, A & Shum, SB 2017, 'Towards mining sequences and dispersion of rhetorical moves in student written texts', ACM International Conference Proceeding Series, International Learning Analytics & Knowledge Conference, ACM, Vancouver, British Columbia, Canada, pp. 228-232.View/Download from: UTS OPUS or Publisher's site
© 2017 ACM. There is an increasing interest in the analysis of both student's writing and the temporal aspects of learning data. The analysis of higher-level learning features in writing contexts requires analyses of data that could be characterised in terms of the sequences and processes of textual features present. This paper (1) discusses the extant literature on sequential and process analyses of writing; and, based on this and our own first-hand experience on sequential analysis, (2) proposes a number of approaches to both pre-process and analyse sequences in whole-texts. We illustrate how the approaches could be applied to examples drawn from our own datasets of 'rhetorical moves' in written texts, and the potential each approach holds for providing insight into that data. Work is in progress to apply this model to provide empirical insights. Although, similar sequence or process mining techniques have not yet been applied to student writing, techniques applied to event data could readily be operationalised to undercover patterns in texts.
Martinez-Maldonado, R, Buckingham-Shum, S, Pechenizkiy, M, Power, T, Hayes, C & Axisa, C 2017, 'Modelling embodied mobility teamwork strategies in a simulation-based healthcare classroom', UMAP 2017 - Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, Conference on User Modeling, Adaptation and Personalization, ACM, Bratislava, Slovakia, pp. 308-312.View/Download from: UTS OPUS or Publisher's site
©2017 ACM. In many situations, it remains critical for team members to develop strategies to effectively use the space and tools available to complete demanding tasks. However, despite the availability of sensors and analytics for instrumenting physical space, relatively little progress has been made in modelling the embodied dimensions of co-located teamwork. This paper explores an in-The-wild pilot study through which we explore a methodology to model embodied mobility teamwork strategies in the context of healthcare education. We developed the means for tracking, clustering and processing student-nurses' mobility data around a patient manikin. We illustrate the feasibility of our approach by discussing ways to make sense of these data to uncover meaningful trends, and the inherent challenges of applying physical space analytics in authentic settings.
Martinez-Maldonado, R, Power, T, Hayes, C, Abdiprano, A, Vo, T, Axisa, C & Shum, SB 2017, 'Analytics meet patient manikins: Challenges in an authentic small-group healthcare simulation classroom', LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, International Learning Analytics and Knowledge Conference, ACM, Vancouver, British Columbia, Canada, pp. 90-94.View/Download from: UTS OPUS or Publisher's site
© 2017 ACM. Healthcare simulations are hands-on learning experiences aimed at allowing students to practice essential skills that they may need when working with real patients in clinical workplaces. Some clinical classrooms are equipped with patient manikins that can respond to actions or that can be programmed to deteriorate over time. Students can perform assessments and interventions, and enhance their critical thinking and communication skills. There is an opportunity to exploit the students' digital traces that these manikins can pervasively capture to make key aspects of the learning process visible. The setting can be augmented with sensors to capture traces of group interaction. These multimodal data can be used to generate visualisations or feedback for students or teachers. This paper reports on an authentic classroom study using analytics to integrate multimodal data of students' interactions with the manikins and their peers in simulation scenarios. We report on the challenges encountered in deploying such analytics 'in the wild', using an analysis framework that considers the social, epistemic and physical dimensions of collocated group activity.
Martinez-Maldonado, R, Yacef, K, Santos, ADPD, Shum, SB, Echeverria, V, Santos, OC & Pechenizkiy, M 2017, 'Towards Proximity Tracking and Sensemaking for Supporting Teamwork and Learning', Proceedings - IEEE 17th International Conference on Advanced Learning Technologies, ICALT 2017, 2017 IEEE 17th International Conference on Advanced Learning Technologies, IEEE, Timisoara, Romania, pp. 89-91.View/Download from: UTS OPUS or Publisher's site
© 2017 IEEE. A large number of learning tools offering some sort of personalisation features rely mainly on the analysis of logged interactions between students and particular user interfaces. Much less attention has been given to the analysis of physical aspects so often present in 'traditional' intellectual tasks, although these are both important in the full development of a life-long learner. This paper (1) discusses existing literature focused on supporting learning using proximity and location analytics and sensors, and, based on this, (2) illustrates the feasibility and potential of these analytics for teaching and learning through an study in the context of proximity and location analytics in a team-based health simulation classroom.
Pardo, A, Martínez-Maldonado, R, Shum, SB, Schulte, J, McIntyre, S, Gašević, D, Gao, J & Siemens, G 2017, 'Connecting data with student support actions in a course: A hands-on tutorial', ACM International Conference Proceeding Series, pp. 522-523.View/Download from: UTS OPUS or Publisher's site
© 2017 ACM. The amount of data extracted from learning experiences has grown at an astonishing pace both in depth due to the increasing variety of data sources, and in breath with courses now being offered to massive student cohorts. However, in this emerging scenario instructors are now facing the challenge of connecting the knowledge emerging from data analysis with the provision of meaningful support actions to students within the context of an instructional design. The objective of this tutorial is to give attendees a set of hypothetical scenarios in which the knowledge extracted from a learning experience needs to be used to provide frequent personalized feedback to students.
Schulte, J, De Mendonca, PF, Martinez-Maldonado, R & Shum, SB 2017, 'Large scale predictive process mining and analytics of university degree course data', ACM International Conference Proceeding Seventh International Learning Analytics & Knowledge Conference, International Learning Analytics & Knowledge Conference, Vancouver, British Columbia, Canada, pp. 538-539.View/Download from: UTS OPUS or Publisher's site
© 2017 ACM. For students, in particular freshmen, the degree pathway from semester to semester is not that transparent, although students have a reasonable idea what courses are expected to be taken each semester. An often-pondered question by students is: "what can I expect in the next semester?" More precisely, given the commitment and engagement I presented in this particular course and the respective performance I achieved, can I expect a similar outcome in the next semester in the particular course I selected? Are the demands and expectations in this course much higher so that I need to adjust my commitment and engagement and overall workload if I expect a similar outcome? Is it better to drop a course to manage expectations rather than to (predictably) fail, and perhaps have to leave the degree altogether? Degree and course advisors and student support units find it challenging to provide evidence based advise to students. This paper presents research into educational process mining and student data analytics in a whole university scale approach with the aim of providing insight into the degree pathway questions raised above. The beta-version of our course level degree pathway tool has been used to shed light for university staff and students alike into our university's 1,300 degrees and associated 6 million course enrolments over the past 20 years.
Shibani, A, Knight, S, Buckingham Shum, S & Ryan, P 2017, 'Design and Implementation of a Pedagogic Intervention Using Writing Analytics', Proceedings of the 25th International Conference on Computers in Education, International Conference on Computers in Education, Asia-Pacific Society for Computers in Education, Christchurch, New Zealand.View/Download from: UTS OPUS
Academic writing is a key skill required for higher education students, which is often challenging to learn. A promising approach to help students develop this skill is the use of automated tools that provide formative feedback on writing. However, such tools are not widely adopted by students unless useful for their discipline-related writing, and embedded in the curriculum. This recognition motivates an increased emphasis in the field on aligning learning analytics applications with learning design, so that analytics-driven feedback is congruent with the pedagogy and assessment regime. This paper describes the design, implementation, and evaluation of a pedagogic intervention that was developed for law students to make use of an automated Academic Writing Analytics tool (AWA) for improving their academic writing. In exemplifying this pedagogically aligned learning analytic intervention, we describe the development of a learning analytics platform to support the pedagogic design, illustrating its potential through example analyses of data derived from the task.
Shum, SB, Sándor, Á, Goldsmith, R, Wang, X, Bass, R & Mcwilliams, M 2016, 'Reflecting on reflective writing analytics: Assessment challenges and iterative evaluation of a prototype tool', ACM International Conference Proceeding Series, International Learning Analytics & Knowledge Conference, ACM, Edinburgh, United Kingdom, pp. 213-222.View/Download from: UTS OPUS or Publisher's site
© 2016 ACM.When used effectively, reflective writing tasks can deepen learners' understanding of key concepts, help them critically appraise their developing professional identity, and build qualities for lifelong learning. As such, reflecting writing is attracting substantial interest from universities concerned with experiential learning, reflective practice, and developing a holistic conception of the learner. However, reflective writing is for many students a novel genre to compose in, and tutors may be inexperienced in its assessment. While these conditions set a challenging context for automated solutions, natural language processing may also help address the challenge of providing real time, formative feedback on draft writing. This paper reports progress in designing a writing analytics application, detailing the methodology by which informally expressed rubrics are modelled as formal rhetorical patterns, a capability delivered by a novel web application. This has been through iterative evaluation on an independently humanannotated corpus, showing improvements from the first to second version. We conclude by discussing the reasons why classifying reflective writing has proven complex, and reflect on the design processes enabling work across disciplinary boundaries to develop the prototype to its current state.
Kitto, K, Bakharia, A, Lupton, M, Mallet, D, Banks, J, Bruza, P, Pardo, A, Shum, SB, Dawsony, S, Gaševícy, D, Siemensg, G & Lynch, G 2016, 'The connected learning analytics toolkit', ACM International Conference Proceeding Series, International Learning Analytics & Knowledge Conference, Association of Computing Machinery, Edinburgh, United Kingdom, pp. 548-549.View/Download from: UTS OPUS or Publisher's site
© 2016 Copyright held by the owner/author(s).This demonstration introduces the Connected Learning An- Alytics (CLA) Toolkit. The CLA toolkit harvests data about student participation in specified learning activities across standard social media environments, and presents information about the nature and quality of the learning interactions.
Martinez-Maldonado, R, Anderson, T, Shum, SB & Knight, S 2016, 'Towards supporting awareness for content curation: The case of food literacy and behavioural change', CEUR Workshop Proceedings, Learning Analytics for Learners (LAK-LAL), Central Europe Workshop, Edinburgh, Scotland, pp. 42-46.
Copyright © 2016 for the individual papers by the papers' authors.This paper presents a theoretical grounding and a conceptual proposal aimed at providing support in the initial stages of sustained behavioural change. We explore the role that learning analytics and/or open learner models can have in supporting life-long learners to enhance their food literacy through a more informed curation process of relevant-content. This approach grounds on a behavioural change perspective that identifies i) knowledge, ii) attitudes, and iii) self-efficacy as key factors that will directly and indirectly affect future decisions and agency of life-long learners concerning their own health. The paper offers some possible avenues to start organising efforts towards the use of learning analytics to enhance awareness in terms of: knowledge curation, knowledge sharing and knowledge certainty. The paper aims at triggering discussion about the type of data and presentation mechanisms that may help life-long learners set a stronger basis for behavioural change in the subsequent stages.
Martinez-Maldonado, R, Schneider, B, Charleer, S, Buckingham Shum, S, Klerkx, J & Duval, E 2016, 'Interactive surfaces and learning analytics: data, orchestration aspects, pedagogical uses and challenges', LAK '16 Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, International Learning Analytics & Knowledge Conference, ACM, Edinburgh, United Kingdom, pp. 124-133.View/Download from: UTS OPUS or Publisher's site
The proliferation of varied types of multi-user interactive surfaces (such as digital whiteboards, tabletops and tangible interfaces) is opening a new range of applications in face-to-face (f2f) contexts. They offer unique opportunities for Learning Analytics (LA) by facilitating multi-user sensemaking of automatically captured digital footprints of students' f2f interactions. This paper presents an analysis of current research exploring learning analytics associated with the use of surface devices. We use a framework to analyse our first-hand experiences, and the small number of related deployments according to four dimensions: the orchestration aspects involved; the phases of the pedagogical practice that are supported; the target actors; and the levels of iteration of the LA process. The contribution of the paper is twofold: 1) a synthesis of conclusions that identify the degree of maturity, challenges and pedagogical opportunities of the existing applications of learning analytics and interactive surfaces; and 2) an analysis framework that can be used to characterise the design space of similar areas and LA applications.
Simsek, D, Sándor, Á, Buckingham Shum, S, Ferguson, R, De Liddo, A & Whitelock, D 2015, 'Correlations between automated rhetorical analysis and tutors' grades on student essays', http://dl.acm.org/citation.cfm?id=2723576&picked=prox&cfid=677454654&cf…, International Learning Analytics & Knowledge Conference, ACM, Poughkeepsie, New York, pp. 355-359.View/Download from: UTS OPUS or Publisher's site
When assessing student essays, educators look for the students' ability to present and pursue well-reasoned and strong arguments. Such scholarly argumentation is often articulated by rhetorical metadiscourse. Educators will be necessarily examining metadiscourse in students' writing as signals of the intellectual moves that make their reasoning visible. Therefore students and educators could benefit from available powerful automated textual analysis that is able to detect rhetorical metadiscourse. However, there is a need to validate such technologies in higher education contexts, since they were originally developed in non-educational applications. This paper describes an evaluation study of a particular language analysis tool, the Xerox Incremental Parser (XIP), on undergraduate social science student essays, using the mark awarded as a measure of the quality of the writing. As part of this exploration, the study presented in this paper seeks to assess the quality of the XIP through correlational studies and multiple regression analysis.
Arastoopour, G, Shum, SB, Collier, W, Kirschner, PA, Wise, AF, Knight, S & Shaffer, DW 2014, 'Analytics for learning and becoming in practice', Proceedings of International Conference of the Learning Sciences, ICLS, International Conference of the Learning Sciences (ICLS) 2014, pp. 1680-1683.
© ISLS. Learning Analytics sits at the intersection of the learning sciences and computational data capture and analysis. Analytics should be grounded in the existing literature with a view to data 'geology' or 'archeology' over 'mining'. This workshop explores how analytics may extend the common notion of activity trace data from learning processes to encompass learning practices, with a working distinction for discussion as (1) process: a series of related actions engaged in as part of learning activities; and (2) practice: a repertoire of processes organised around particular foci recognised within a social group. The workshop intersperses attendee presentations and demonstrations with relevant theme-based discussions.
De Liddo, A & Shum, SB 2014, 'New ways of deliberating online: An empirical comparison of network and threaded interfaces for online discussion', Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 90-101.
© IFIP International Federation for Information Processing 2014. One of the Web’s most phenomenal impacts has been its capacity to connect and harness the ideas of many people seeking to tackle a problem. Social media appear to have played specific and significant roles in helping communities form and mobilize, even to the level of political uprisings. Nevertheless the online dialogue spaces we see on the Web today are often repurposed social networks that offer no insight into the logical structure of the ideas, such as the coherence or evidential basis of an argument. This hampers both quality of citizen participation and effective assessment of the public debate. We report on an exploratory study in which we observed users interaction with a new tool for online deliberation and compared network and threaded visualizations of arguments. Results of the study suggest that network visualization of arguments can effectively improve online debate by facilitating higher-level inferences and making the debate more engaging and fun.
Ferguson, R, De Liddo, A, Whitelock, D, De Laat, M & Shum, SB 2014, 'DCLA14: Second international workshop on discourse-centric learning analytics', ACM International Conference Proceeding Series, Fourth International Conference on Learning Analytics And Knowledge, pp. 283-284.View/Download from: Publisher's site
The first international workshop on discourse-centric learning analytics (DCLA) took place at LAK13 in Leuven, Belgium. That workshop succeeded in its aim of catalysing ideas and building community connections between those working in this field of social learning analytics. It also proposed a mission statement for DCLA: To devise and validate analytics that look beyond surface measures in order to quantify linguistic proxies for deeper learning. This year, the focus of the second international DCLA workshop, like that of LAK14, is on the intersection of learning analytics research, theory and practice. Once researchers have developed and validated discourse-centric analytics, how can these be successfully deployed at scale to support learning?. Copyright © 2014 by the Association for Computing Machinery, Inc.
Knight, S, Arastoopour, G, Williamson Shaffer, D, Buckingham Shum, S & Littleton, K 2014, 'Epistemic Networks for Epistemic Commitments', Learning and Becoming in Practice The International Conference of the Learning Sciences (ICLS) 2014, International Conference of the Learning Sciences, International Society of the Learning Sciences, USA.View/Download from: UTS OPUS
Simsek, D, Shum, SB, De Liddo, A, Ferguson, R & Sándor, A 2014, 'Visual analytics of academic writing', ACM International Conference Proceeding Series, Fourth International Conference on Learning Analytics And Knowledge, pp. 265-266.View/Download from: UTS OPUS or Publisher's site
This paper describes a novel analytics dashboard which visualises the key features of scholarly documents. The Dashboard aggregates the salient sentences of scholarly papers, their rhetorical types and the key concepts mentioned within these sentences. These features are extracted from papers through a Natural Language Processing (NLP) technology, called Xerox Incremental Parser (XIP). The XIP Dashboard is a set of visual analytics modules based on the XIP output. In this paper, we briefly introduce the XIP technology and demonstrate an example visualisation of the XIP Dashboard. Copyright © 2014 by the Association for Computing Machinery, Inc.
Buckingham Shum, S, De Laat, M, De Liddo, A, Ferguson, R, Kirschner, P, Ravenscroft, A, Sándor, A & Whitelock, D 2013, 'DCLA13: 1st International Workshop on Discourse-Centric Learning Analytics', ACM International Conference Proceeding Series, ACM, pp. 282-282.View/Download from: UTS OPUS or Publisher's site
This workshop anticipates that an important class of learning analytic will emerge at the intersection of research into learning dynamics, online discussion platforms, and computational linguistics. Written discourse is arguably the primary class of data that can give us insights into deeper learning and higher order qualities such as critical thinking, argumentation, mastery of complex ideas, empathy, collaboration and interpersonal skills. Moreover, the ability to write in a scholarly manner is a core competence, often taking the form of discourse with oneself and the literature. Computational linguistics research has developed a rich array of tools for machine interpretation of human discourse, but work to develop these tools in the context of learning is at a relatively early stage. Moreover, there is a significant difference between designing tools to assist researchers in discourse analysis, and their deployment on platforms to provide meaningful analytics for the learners and educators who are conducting that discourse. This workshop aims to catalyse ideas and build community connections among those who want to shape this field. © 2013 Authors.
Buckingham Shum, S, Hawksey, M, Baker, RSJD, Jeffery, N, Behrens, JT & Pea, R 2013, 'Educational data scientists: A scarce breed', ACM International Conference Proceeding Series, International Conference on Learning Analytics and Knowledge, ACM, Leuven, Belgium, pp. 278-281.View/Download from: UTS OPUS or Publisher's site
The Educational Data Scientist is currently a poorly understood, rarely sighted breed. Reports vary: some are known to be largely nocturnal, solitary creatures, while others have been reported to display highly social behaviour in broad daylight. What are their primary habits? How do they see the world? What ecological niches do they occupy now, and will predicted seismic shifts transform the landscape in their favour? What survival skills do they need when running into other breeds? Will their numbers grow, and how might they evolve? In this panel, the conference will hear and debate not only broad perspectives on the terrain, but will have been exposed to some real life specimens, and caught glimpses of the future ecosystem. © 2013 Authors.
De Liddo, A & Buckingham Shum, S 2013, 'Improving online deliberation with argument network visualizatio', Digital Cities 8, Munich, Germany..
De Liddo, A & Buckingham Shum, S 2013, 'The Evidence Hub: Harnessing the Collective Intelligence of Communities to Build Evidence-Based Knowledge', Large Scale Ideation and Deliberation Workshop, 6th International Conference on Communities & Technologies, 6th International Conference on Communities and Technologies, Munich, Germany.
Conventional document and discussion websites provide users with no help in assessing the quality or quantity of evidence behind any given idea. Besides, the very meaning of what evidence is may not be unequivocally defined within a community, and may require deep understanding, common ground and debate. An Evidence Hub is a tool to pool the community collective intelligence on what is evidence for an idea. It provides an infrastructure for debating and building evidence-based knowledge and practice. An Evidence Hub is best thought of as a filter onto other websites a map that distills the most important issues, ideas and evidence from the noise by making clear why ideas and web resources may be worth further investigation. This paper describes the Evidence Hub concept and rationale, the breath of user engagement and the evolution of specific features, derived from our work with different community groups in the healthcare and educational sector.
Ferguson, R, Wei, Z, He, Y & Buckingham Shum, S 2013, 'An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions', Proc. 3rd International Conference on Learning Analytics & Knowledge., Third Conference on Learning Analytics and Knowledge (LAK 2013), Leuven, Belgium, pp. 85-93.
Social learning analytics are concerned with the process of knowledge construction as learners build knowledge together in their social and cultural environments. One of the most important tools employed during this process is language. In this paper we take exploratory dialogue, a joint form of co-reasoning, to be an external indicator that learning is taking place. Using techniques developed within the field of computational linguistics, we build on previous work using cue phrases to identify exploratory dialogue within online discussion. Automatic detection of this type of dialogue is framed as a binary classification task that labels each contribution to an online discussion as exploratory or non-exploratory. We describe the development of a self-training framework that employs discourse features and topical features for classification by integrating both cue-phrase matching and k-nearest neighbour classification. Experiments with a corpus constructed from the archive of a two-day online conference show that our proposed framework outperforms other approaches. A classifier developed using the self-training framework is able to make useful distinctions between the learning dialogue taking place at different times within an online conference as well as between the contributions of individual participants.
Knight, S, Buckingham Shum, S & Littleton, K 2013, 'Collaborative sensemaking in learning analytics', CSCW and Education Workshop (2013): Viewing education as a site of work practice, co-located with the 16th ACM Conference on Computer Support Cooperative Work and Social Computing (CSCW 2013),, San Antonio, Texas.
Knight, S, Buckingham Shum, S & Littleton, K 2013, 'Epistemology, Pedagogy, Assessment and Learning Analytics', Proc. 3rd International Conference on Learning Analytics & Knowledge, International Learning Analytics & Knowledge Conference, ACM, Leuven, Belgium, pp. 75-84.View/Download from: UTS OPUS or Publisher's site
Knight, S, Buckingham Shum, S & Littleton, K 2013, 'Tracking epistemic beliefs and sensemaking in collaborative information retrieval', Collaborative Information Seeking Workshop 2013 (CIS'13): Consolidating the Past, Creating the future, co-located with the 16th ACM Conference on Computer Support Cooperative Work and Social Computing (CSCW 2013), San Antonio, Texas.
Schreurs, B, Teplovs, C, Ferguson, R, De Laat, M & Buckingham Shum, S 2013, 'Visualizing Social Learning Ties by Type and Topic: Rationale and Concept Demonstrator', Proc. 3rd International Conference on Learning Analytics & Knowledge, Third Conference on Learning Analytics and Knowledge (LAK 2013), Leuven, Belgium, pp. 33-37.
Social Learning Analytics (SLA) are designed to support students learning through social networks, and reflective practitioners engage in informal learning through a community of practice. This short paper reports work in progress to develop SLA motivated specifically by Networked Learning Theory, drawing on the related concepts and tools of Social Network Analytics and Social Capital Theory, which provide complementary perspectives onto the structure and content of such networks. We propose that SLA based on these perspectives needs to devise models and visualizations capable of showing not only the usual SNA metrics, but the types of social tie forged between actors, and topic-specific subnetworks. We describe a technical implementation demonstrating this approach, which extends the Network Awareness Tool by automatically populating it with data from a social learning platform SocialLearn. The result is the ability to visualize relationships between people who interact around the same topics.
Taibi, D, Sandor, A, Simsek, D, Buckingham Shum, S, De Liddo, A & Ferguson, R 2013, 'Visualizing the LAK/EDM literature using combined concept and rhetorical sentence extraction', Proceedings of the LAK Data Challenge, 3rd Int. Conf. on Learning Analytics and Knowledge (LAK '13), Leuven, Belgium.
Buckingham Shum, S & Deakin Crick, R 2012, 'Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics', Proc. 2nd Int. Conf. Learning Analytics & Knowledge, ACM Press, Vancouver, BC.
De Liddo, A, Buckingham Shum, S, Convertino, G, Sándor, A & Klein, M 2012, 'Collective intelligence as community discourse and action', Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, pp. 5-6.View/Download from: Publisher's site
Collective Intelligence (CI) research investigates the design of infrastructures to enable collectives to think and act intelligently, and intriguingly, more intelligently than individuals. Technologies such as idea management or argumentation tools, blogs, wikis, chats, forums, Q&A sites, and social networks provide unprecedented opportunities for entire communities or organizations to express a discourse and act at a massive scale. This workshop seeks to understand the forms of CI that can be constructed through discourse and action, which enables advanced forms of collective sensemaking such as idea generation and prioritization, argumentation, and deliberation. When does effective discourse help a collective outperform individuals? What functions should the next generation of social platforms support? How can we allow communities to efficiently manage many diverse ideas, argument, and deliberate? What patterns in discourse and action can be modeled computationally? © 2012 Authors.
De Liddo, A, Buckingham Shum, S, McAndrew, P & Farrow, R 2012, 'The Open Education Evidence Hub: A Collective Intelligence Tool for Evidence Based Policy', Cambridge 2012: Joint OER12 and OpenCourseWare Consortium Global 2012 Conference, Cambridge, UK.
De Liddo, A, Sándor, A & Buckingham Shum, S 2012, 'Cohere and XIP: Human annotation harnessing machine annotation power', Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, pp. 49-50.View/Download from: Publisher's site
In this video we demonstrate the practical application of research on human and machine annotation of online documents to support reflective reading and collective sensemaking of online documents. We present an innovative research prototype which integrate a discourse analysis software (XIP) with an open source Web Annotation and Knowledge-Mapping tool (Cohere). We visualize an interactive scenario of use of the two integrated technologies in a unique user experience. This dynamic scenario will give an inspiring vision of future CSCW systems, which combine human annotation to harness machine analysis and reasoning power. © 2012 Authors.
Ferguson, R & Buckingham Shum, S 2012, 'Social Learning Analytics: Five Approaches', Proc. 2nd Int. Conf. Learning Analytics & Knowledge, ACM Press, Vancouver, BC.
De Liddo, A, Buckingham Shum, S, Quinto, I, Bachler, M & Cannavacciuolo, L 2011, 'Discourse-Centric Learning Analytics', Proc. 1st International Conference on Learning Analytics & Knowledge, ACM Press, Banff, Alberta.
Ferguson, R & Buckingham Shum, S 2011, 'Learning Analytics to Identify Exploratory Dialogue Within Synchronous Text Chat', LAK 2011: 1st International Conference on Learning Analytics & Knowledge, ACM Press, Banff, Alberta.
Buckingham Shum, S & De Liddo, A 2010, 'Collective intelligence for OER sustainability', Proceedings OpenEd2010: Seventh Annual Open Education Conference, Barcelona, Spain.
Buckingham Shum, S & Ferguson, R 2010, 'Towards a social learning space for open educational resources', OpenEd2010: Seventh Annual Open Education Conference, Barcelona, Spain.
Buckingham Shum, S, Sierhuis, M, Park, J & Brown, M 2010, 'Software agents in support of human argument mapping', Computational Models of Argument: Proceedings of COMMA 2010: Frontiers in Artificial Intelligence and Applications Series, IOS Press, Amsterdam, pp. 123-134.
De Liddo, A & Buckingham Shum, S 2010, 'Capturing and representing deliberation in participatory planning practices', Fourth International Conference on Online Deliberation (OD2010), Leeds, UK.
De Liddo, A & Buckingham Shum, S 2010, 'Cohere: A prototype for contested collective intelligence', ACM Computer Supported Cooperative Work (CSCW 2010) - Workshop: Collective Intelligence In Organizations - Toward a Research Agenda, Savannah, Georgia, USA.View/Download from: UTS OPUS
Quinto, I, Buckingham Shum, S, De Liddo, A & Iandoli, L 2010, 'A debate dashboard to enhance on-line knowledge sharing', IFKAD Conference - Intellectual Capital in a Complex Business Landscape, Matera, Italy.
De Waard, A, Shum, SB, Carusi, A, Park, J, Samwald, M & Sándor, A 2009, 'Hypotheses, evidence and relationships: The HypER approach for representing scientific knowledge claims', CEUR Workshop Proceedings.
Biological knowledge is increasingly represented as a collection of (entity-relationship-entity) triplets. These are queried, mined, appended to papers, and published. However, this representation ignores the argumentation contained within a paper and the relationships between hypotheses, claims and evidence put forth in the article. In this paper, we propose an alternate view of the research article as a network of 'hypotheses and evidence'. Our knowledge representation focuses on scientific discourse as a rhetorical activity, which leads to a different direction in the development of tools and processes for modeling this discourse. We propose to extract knowledge from the article to allow the construction of a system where a specific scientific claim is connected, through trails of meaningful relationships, to experimental evidence. We discuss some current efforts and future plans in this area.
Groza, T, Handschuh, S, Clark, T, Shum, SB & De Waard, A 2009, 'A short survey of discourse representation models', CEUR Workshop Proceedings.
With the advancement of technology and the wide adoption of ontologies as knowledge representation formats, in the last decade, a handful of models were proposed for the externalization of the rhetoric and argumentation captured within scientific publications. Conceptually, most of these models share a similar representation form of the scientific publication, i.e. as a series of interconnected elementary knowledge items. The main differences are given by the terminology used, the types of rhetorical and / or argumentation relations connecting the knowledge items and the foundational theories supporting these relations. This paper analyzes the state of the art and provides a concise comparative overview of the five most prominent discourse representation models, with the goal of sketching an unified model for discourse representation.
Benn, N, Shum, SB, Domingue, J & Mancini, C 1970, 'Ontological Foundations for Scholarly Debate Mapping Technology', COMPUTATIONAL MODELS OF ARGUMENT, PROCEEDINGS OF COMMA 2008, 2nd Conference on Computational Models of Argument, IOS PRESS, Inst Rech Informat Toulouse, Toulouse, FRANCE, pp. 61-72.
Shum, SB 1970, 'Cohere: Towards Web 2.0 Argumentation', COMPUTATIONAL MODELS OF ARGUMENT, PROCEEDINGS OF COMMA 2008, 2nd Conference on Computational Models of Argument, IOS PRESS, Inst Rech Informat Toulouse, Toulouse, FRANCE, pp. 97-108.
De Liddo, A, Concilio, G & Buckingham Shum, S 2007, 'Advancing knowledge management and exchange between collaborative environments: A tool integration perspective', CEUR Workshop Proceedings, pp. 58-69.
This paper focuses on supporting knowledge management and exchange between web-based and traditional collaborative environments. In particular we discuss the integration between a tool (CoPe-it!) supporting collaborative argumentation and learning in web-based Communities of Practices and a hypermedia and sense making tool (Compendium) acting as a personal and collective Knowledge Management (KM) system in traditional collaborative environments. We focus on the general applicability of this integration for supporting Communities of Practices and, more generally, collaborative works, and discuss the main objectives and challenges to be addressed.
Potter, S, Kalfoglou, Y, Alani, H, Bachler, M, Shum, SB, Carvalho, R, Chakravarthy, A, Chalmers, S, Chapman, S, Hu, B, Preece, A, Shadbolt, N, Tate, A & Tuffield, M 2007, 'The application of advanced knowledge technologies for emergency response', Intelligent Human Computer Systems for Crisis Response and Management, ISCRAM 2007 Academic Proceedings Papers, pp. 361-368.
Making sense of the current state of an emergency and of the response to it is vital if appropriate decisions are to be made. This task involves the acquisition, interpretation and management of information. In this paper we present an integrated system that applies recent ideas and technologies from the fields of Artificial Intelligence and semantic web research to support sense-and decision-making at the tactical response level, and demonstrate it with reference to a hypothetical large-scale emergency scenario. We offer no end-user evaluation of this system; rather, we intend that it should serve as a visionary demonstration of the potential of these technologies for emergency response.
Sereno, B, Shum, SB & Motta, E 2007, 'Formalization, user strategy and interaction design: Users' behaviour with discourse tagging semantics', CEUR Workshop Proceedings.
When authors publish their interpretations of the ideas, opinions, claims or rebuttals in the literature, they are drawing on a repertoire of well understood moves, contributing to an extended discourse. Readers also bring their own perspective to documents, interpreting them in the light of their own research interests, and initiating, for instance, new connections that may not have been intended by authors. Collaborative, social, tagging holds promise as an approach to mediating these processes via the Web, but may lack the discourse dimension that is fundamental to the articulation of interpretations. We therefore take a hybrid semiformal approach to add structure to freeform folksonomies. Our experience demonstrates that this particular brand of tagging requires tools designed specifically for this sensemaking task by providing enough support to initiate the annotation, while not overwhelming users with suggestions. We describe a tool called ClaimSpotter that aims at supporting this tradeoff, through a novel combination of system-initiated tag recommendations, Web interface design, and an expanded conception of how tags can be both expressed, and semantically linked. We then report a detailed study which analysed the tool's usability and the tag structures created, contributing to our understanding of the implications of adding structure to collaborative tagging.
Shum, SB 2007, 'Hypermedia discourse: Contesting networks of ideas and arguments', Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 29-44.
This invited contribution motivates the Hypermedia Discourse research programme, investigating the reading, writing and contesting of ideas as hypermedia networks grounded in discourse schemes. We are striving for cognitively and computationally tractable conceptual structures: fluid enough to serve as augmentations to group working memory, yet structured enough to support long term memory. I will describe how such networks can be (i) mapped by multiple analysts to visualize and interrogate the claims and arguments in a literature, and (ii) mapped in real time to manage a team's information sources, competing interpretations, arguments and decisions, particularly in time- pressured scenarios where harnessing collective intelligence is a priority. Given the current geo-political and environmental context, the growth in distributed teamwork, and the need for multidisciplinary approaches to wicked problems, there has never been a greater need for sensemaking tools to help diverse stakeholders build common ground. © Springer-Verlag Berlin Heidelberg 2007.
Weigand, H, Lind, M & Shum, SB 2007, 'Welcome to Tilburg, and to ICPW 2007!', ACM International Conference Proceeding Series.
Michaelides, D, Shum, SB, Juby, B, Mancini, C, Slack, R, Bachler, M, Procter, R, Daw, M, Rowley, A, Chown, T, De Roure, D & Hewitt, T 2006, 'Memetic: Semantic meeting memory', Proceedings of the Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises, WETICE, pp. 382-387.View/Download from: Publisher's site
This paper introduces the Memetic toolkit for recording meetings held over Internet-based video conferencing technologies, and making these navigable in linear and nonlinear ways. We introduce the tools and technologies that form the toolkit and discuss the semantics of the information they capture. © 2006 IEEE.
Clancey, WJ, Sierhuis, M, Alena, RL, Berrios, D, Dowding, J, Graham, JS, Tyree, KS, Hirsh, RL, Brent Garry, W, Semple, A, Buckingham Shum, SJ, Shadbolt, N & Rupert, SM 2005, 'Automating CapCom using mobile agents and robotic assistants', A Collection of Technical Papers - 1st Space Exploration Conference: Continuing the Voyage of Discovery, pp. 835-874.
We have developed and tested an advanced EVA communications and computing system to increase astronaut self-reliance and safety, reducing dependence on continuous monitoring and advising from mission control on Earth. This system, called Mobile Agents (MA), is voice controlled and provides information verbally to the astronauts through programs called "personal agents." The system partly automates the role of CapCom in Apollo-including monitoring and managing EVA navigation, scheduling, equipment deployment, telemetry, health tracking, and scientific data collection. EVA data are stored automatically in a shared database in the habitat/vehicle and mirrored to a site accessible by a remote science team. The program has been developed iteratively in the context of use, including six years of ethnographic observation of field geology. Our approach is to develop automation that supports the human work practices, allowing people to do what they do well, and to work in ways they are most familiar. Field experiments in Utah have enabled empirically discovering requirements and testing alternative technologies and protocols. This paper reports on the 2004 system configuration, experiments, and results, in which an EVA robotic assistant (ERA) Followed geologists approximately 150 m through a winding, narrow canyon. On voice command, the ERA took photographs and panoramas and was directed to move and wait in various locations to serve as a relay on the wireless network. The MA system is applicable to many space work situations that involve creating and navigating from maps (including configuring equipment for local topology), interacting with piloted and unpiloted rovers, adapting to environmental conditions, and remote team collaboration involving people and robots.
Sereno, B, Shum, SB & Motta, E 2005, 'ClaimSpotter: An environment to support sensemaking with knowledge triples', International Conference on Intelligent User Interfaces, Proceedings IUI, pp. 199-206.View/Download from: Publisher's site
Annotating a document with an interpretation of its contents raises a number of challenges that we are hoping to address via the creation of a supporting environment. We present these challenges and motivate an approach based on the notion of suggestions to support document annotation, hoping these suggestions would act as leads to follow for annotators, therefore reducing some of the difficulties inherent to the task. The environment resulting from this approach, ClaimSpotter, is presented. Aspects of its evaluation are also given, using the findings of a study involving a group of participants faced with a document annotation task. Copyright 2005 ACM.
Mancini, C & Shum, SB 2004, 'Towards 'Cinematic' hypertext', Proceedings of the ACM Conference on Hypertext, pp. 215-224.
This paper proposes the paradigm of 'Cinematic' Hypertext (CH), in which discourse form is represented following principles that underpin the expression of narrative patterns in cinema. Primarily tackling hypertext discourse coherence in the non-linear medium, CH is conceived as a way of thinking the hypertext medium that is consistent with its characteristics. CH envisages the consistent and concurrent use of the medium's formal features, grounded in structuring principles, in order to allow the emergence of a local language. Relational primitives based on Cognitive Coherence Relations are proposed as a structuring principle to define hypertext links, while the use the medium's graphic features is proposed to render these relational primitives as patterns that will take shape during navigation. Taking scholarly hypertext as a domain, this paper articulates the theoretical basis for cinematic hypertext, presents the elements of a prototype visual language to express a sub-set of CCR, provides experimental evidence of its significance, and finally envisages the realisation of a cinematic hypertext environment.
Sereno, B, Shum, SB & Motta, E 2004, 'SemiAutomatic annotation of contested knowledge on the world wide web', Proceedings of the 13th International World Wide Web Conference on Alternate Track, Papers and Posters, WWW Alt. 2004, pp. 276-277.View/Download from: Publisher's site
We describe a strategy to support the semantic annotation of contested knowledge, in the context of the Scholarly Ontologies project, which aims at building a network of interpretations enriching a corpus of scholarly papers. To model such knowledge, which does not have 'right' and 'wrong' values, we are building on the notion of active recommendations as a means to sparkle annotators' interest. We finally argue for a different approach to the evaluation of its impact.
Sereno, B, Shum, SB & Motta, E 2004, 'Semi-automatic annotation of contested knowledge on the world wide web', Thirteenth International World Wide Web Conference Proceedings, WWW2004, pp. 1008-1009.
We describe a strategy to support the semantic annotation of contested knowledge, in the context of the Scholarly Ontologies project, which aims at building a network of interpretations enriching a corpus of scholarly papers. To model such knowledge, which does not have 'right' and 'wrong' values, we are building on the notion of active recommendations as a means to sparkle annotators' interest. We finally argue for a different approach to the evaluation of its impact.
Uren, V, Shum, SB, Li, G, Domingue, J & Motta, E 2003, 'Scholarly publishing and argument in hyperspace', Proceedings of the 12th International Conference on World Wide Web, WWW 2003, pp. 244-250.View/Download from: Publisher's site
The World Wide Web is opening up access to documents and data for scholars. However it has not yet impacted on one of the primary activities in research: assessing new findings in the light of current knowledge and debating it with colleagues. The ClaiMaker system uses a directed graph model with similarities to hypertext, in which new ideas are published as nodes, which other contributors can build on or challenge in a variety of ways by linking to them. Nodes and links have semantic structure to facilitate the provision of specialist services for interrogating and visualizing the emerging network. By way of example, this paper is grounded in a ClaiMaker model to illustrate how new claims can be described in this structured way.
Uren, V, Shum, SB, Li, G, Domingue, J & Motta, E 2003, 'Scholarly publishing and argument in hyperspace', Proceedings of the twelfth international conference on World Wide Web - WWW '03, the twelfth international conference, ACM Press.View/Download from: Publisher's site
Li, GM, Uren, V, Motta, E, Shum, SB & Domingue, J 2002, 'ClaiMaker: Weaving a Semantic Web of research papers', SEMANTIC WEB - ISWC 2002, 1st International Semantic Web Conference (ISWC), SPRINGER-VERLAG BERLIN, SARDINIA, ITALY, pp. 436-441.
Conklin, J, Selvin, A, Shum, SB & Sierhuis, M 2001, 'Facilitated hypertext for collective sensemaking', Proceedings of the twelfth ACM conference on Hypertext and Hypermedia - HYPERTEXT '01, the twelfth ACM conference, ACM Press.View/Download from: Publisher's site
Conklin, J, Selvin, A, Shum, SB & Sierhuis, M 2001, 'Facilitated hypertext for collective sensemaking: 15 years on from gIBIS', Proceedings of the ACM Conference on Hypertext, pp. 123-124.
This paper outlines the technical and social dimensions to a hypertext tool that has been successfully used in organizational settings to improve meetings, and capture group memory in real time. The approach derives from hypertext research systems from the mid-1980s-90s which sought to manipulate conceptual structures as hypertextual concept maps. However, many did not receive sustained use due to issues of cognitive overheads and representational inflexibility. Many decided that such tools would never fulfill their promise. The gIBIS system exemplified this early work, but has since evolved into a broader approach to collective sensemaking called Compendium. We outline Compendium, which demonstrates the impact that a hypertext facilitator can have on the learning and adoption problems that often ambush hypertext sensemaking tools before they have the chance to establish roots in work practice.
Domingue, J, Motta, E, Buckingham Shum, S, Vargas-Vera, M, Kalfoglou, Y & Farnes, N 2001, 'Supporting ontology driven document enrichment within communities of practice', Proceedings of the First International Conference on Knowledge Capture, pp. 30-37.
Formative work by Lave and Wenger has articulated how practices emerge through the interplay of informal processes with symbolic codifications and artifacts. In this paper, we describe how ontologies can serve as symbolic tools within a community of practice supporting communication and knowledge sharing. We show that when a community's perspective on an issue is stable, it opens the possibility for introducing knowledge services, based on an ontology co-constructed by knowledge engineers with stakeholders. Using a case study we describe our approach, ontology driven document enrichment, looking at how ontology construction and population can be supported by web based technologies.
Domingue, J, Motta, E, Shum, SB, Vargas-Vera, M, Kalfoglou, Y & Farnes, N 2001, 'Supporting ontology driven document enrichment within communities of practice', Proceedings of the international conference on Knowledge capture - K-CAP 2001, the international conference, ACM Press.View/Download from: Publisher's site
Mancini, C & Buckingham Shum, S 2001, 'Cognitive coherence relations and hypertext: From cinematic patterns to scholarly discourse', Proceedings of the ACM Conference on Hypertext, pp. 165-174.
In previous work we argued that cinematic language may provide insights into the construction of narrative coherence in hypertext, and we identified in the shot juxtaposition of rhetorical patterns the source of coherence for cinematic discourse. Here we deepen our analysis, to show how the mechanisms that underpin cinematic rhetorical patterns are the same as those providing coherence in written text. We draw on computational and psycholinguistic analyses of texts which have derived a set of relationships that are termed Cognitive Coherence Relations (CCR). We validate this by re-expressing established cinematic patterns, and relations relevant to scholarly hypertext, in terms of CCR, and with this conceptual bridge in place, present examples to show how cinematic techniques could assist the presentation of scholarly discourse. This theoretical work also informs system design. We describe how an abstract relational layer based on CCR is being implemented as a semantic hypertext system to mediate scholarly discourse.
Mancini, C & Buckingham, SS 2001, 'Cognitive coherence relations and hypertext', Proceedings of the twelfth ACM conference on Hypertext and Hypermedia - HYPERTEXT '01, the twelfth ACM conference, ACM Press.View/Download from: Publisher's site
Mancini, C & Buckingham, SS 2001, 'Cognitive coherence relations and hypertext', Proceedings of the twelfth ACM conference on Hypertext and Hypermedia - HYPERTEXT '01, the twelfth ACM conference, ACM Press.View/Download from: Publisher's site
Shum, SB, Domingue, J & Motta, E 2000, 'Scholarly discourse as computable structure', Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 120-128.View/Download from: Publisher's site
© Springer-Verlag Berlin Heidelberg 2000. In their initial proposal for structural computing (SC), Nürnberg et al.  point to hypertext argumentation systems as an example of an application domain in which structure is of first-order importance. In this paper we summarise the goals and implementation of a knowledge based hypertext environment called ScholOnto (for Scholarly Ontologies), which aims to provide researchers with computational support in representing and analysing the structure of scholarly claims, argumentation and perspectives. A specialised web server will provide a medium for researchers to contest the significance of concepts and emergent structures. In so doing, participants construct an evolving structure that reflects a community's understandings of its field, and which can support computational services for scholars. Using structural analyses of scholarly argumentation, we consider the connections with structural computing, and propose a number of requirements for generic SC environments.
Shum, SB, Motta, E & Domingue, J 1999, 'Representing scholarly claims in Internet digital libraries: A knowledge modelling approach', RESEARCH AND ADVANCED TECHNOLOGY FOR DIGITAL LIBRARIES, PROCEEDINGS, 3rd European Conference on Research and Advanced Technology for Digital Libraries (ECDL 99), SPRINGER-VERLAG BERLIN, PARIS, FRANCE, pp. 423-442.
Sumner, T & Shum, SB 1998, 'A toolkit for publishing Web discussion documents: Design principles and case studies', Proceedings - 3rd Asia Pacific Computer Human Interaction, APCHI 1998, pp. 218-223.View/Download from: Publisher's site
© 1998 IEEE. We are looking at how new forms of document interface can be used to support new forms of scholarly discourse, and ultimately, new models of scholarly publishing. The vehicle we use to conduct this research is the Digital Document Discourse Environment (D3E). D3E is an experimental system supporting the publication of Web based documents with integrated discourse facilities and embedded interactive components. We report on three applications-an e-journal, a hybrid Web/face-to-face conference, and a national policy document-where we have used D3E to promote critical discourse between participants. These cases illustrate four principles guiding our design actions. We conclude by reflecting on interacting factors which motivate participation in the different sites.
Sumner, T & Shum, SB 1998, 'From documents to discourse', Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '98, the SIGCHI conference, ACM Press.View/Download from: Publisher's site
Sumner, T & Shum, SB 1998, 'From documents to discourse: Shifting conceptions of scholarly publishing', Conference on Human Factors in Computing Systems - Proceedings, pp. 95-102.
We are looking at how new forms of document interface can be used to support new forms of scholarly discourse, and ultimately, new models of scholarly publishing. The vehicle we use to conduct this research is the Digital Document Discourse Environment (D3E). D3E is an experimental system supporting the publication of web-based documents with integrated discourse facilities and embedded interactive components. We report here on two cases - an e-journal and a `new form' of conference - where we have used D3E to promote new forms of discourse between participants. We use these cases to illustrate four principles that guide our socio-technical design actions.
Bellotti, V, Shum, SB, MacLean, A & Hammond, N 1995, 'Multidisciplinary modelling in HCI design ... in theory and in practice', Conference on Human Factors in Computing Systems - Proceedings, pp. 146-153.
In one of the largest multidisciplinary projects in basic HCI research to date, multiple analytic HCI techniques were combined and applied within an innovative design context to problems identified by designers of an AV communication system, or media space. The problems were presented to user-, system- and design-analysts distributed across Europe. The results of analyses were integrated and passed back to the designers, and to other domain experts, for assessment. The aim of this paper is to illustrate some theory-based insights gained into key problems in media space design and to convey lessons learned about the process of contributing to design using multiple theoretical perspectives. We also describe some obstacles which must be overcome if such techniques are to be transferred successfully to practice.
Colvin, C, Rogers, T, Wade, A, Dawson, S, Gasevic, D, Buckingham Shum, S, Nelson, K, Alexander, S, Lockyer, L, Kennedy, G, Corrin, L & Fisher, J Australian Government Office for Learning and Teaching 2016, Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement, Canberra, Australia.View/Download from: UTS OPUS
Webinar for Transforming Assessment
The Open University UK, University of Bristol, University of Sydney, Queensland University of Technology, Curtin University, University of South Australia, RMIT, Naver Research Labs Europe, University of Leeds UK