Linking scripting and monitoring support in blended cscl scnarios

Resumen

Among the multiple concerns and tasks involved in the orchestration of Computer-Supported Collaborative Learning (CSCL) scenarios, many are oriented to foster effective collaboration. Several strategies may be adopted to support collaboration, assuming either a Learning Design or a Learning Analytics perspective. Before the interaction begins, scripting defines (within Learning Design approaches) the sequence of learning tasks, resources, and scaffolds that the students will need throughout the learning situation. From a Learning Analytics perspective, monitoring analyses students' interactions during the enactment of the learning scenario, and facilitates interventions to steer the situation towards a more productive state. Although these two strategies (scripting and monitoring) may help teachers in the orchestration of CSCL scenarios, their application is not without problems. Eventualities may emerge during the enactment that jeopardize the initial plan represented in the scripting of the learning scenario. Furthermore, even if the analysis of the students' interactions generates useful insights on how the learning process unfolds, the information that current monitoring proposals provide to the teachers is not always easy to interpret. Thus, teachers often lack relevant information to intervene and adapt their plans as the learning scenario evolves. Several researchers point out the potential synergies that may be derived from the alignment between scripting and monitoring, such as improving the monitorable evidence from the learning scenario, or providing teachers with data analyses connected to the pedagogical decisions described at design-time. In addition to these conceptual proposals, there exist few technological solutions that support such alignment, through ad-hoc integration into specific learning tools, such as scripted forums and collaborative canvases. However, despite the benefits of the alignment envisioned by the theoretical proposals and the positive results identified by the technological solutions, to the best of our knowledge, there is no research work dealing with the generic needs of blended CSCL scenarios (i.e., independent of specific tools or types of activities, and taking into account both face-to-face and computer-mediated learner interactions). In response to these needs, our main research objective is to help teachers monitor the accomplishment of their design decisions in the context of blended CSCL scenarios. Following the aforementioned alignment between scripting and monitoring, we propose to provide teachers with design and management support capable of linking their pedagogical intentions and awareness needs. Furthermore, in this dissertation we pay special attention to learning scenarios supported by Distributed Learning Environments (DLEs), which integrate existing learning platforms (e.g., mainstream Virtual Learning Environments such as Moodle or Sakai) with external tools (typically, Web 2.0 tools such as wikis, blogs or Google applications). These technological environments are becoming increasingly popular in current pedagogical practice but, at the same time, make the design and management of the learning scenario even more challenging. Our proposal aims to address several challenges that teachers face in trying to align these two strategies. The first challenge we encounter is the lack of attention to teachers' awareness needs in the process of designing the learning scenario, which may hinder the effectiveness of such monitoring. We propose a "monitoring-aware design process of CSCL scripts", to support teachers in identifying and including monitoring aspects throughout the design process of CSCL scenarios. This process describes the steps that teachers should follow during the design of CSCL scenarios, in order to reflect on their own monitoring needs and express their expectations about the students' interactions. The fact thatmonitoring solutions usually do not take into account teachers' design decisions is another challenge of this alignment. In this dissertation, we aim to inform teachers about the accomplishment of their design decisions, therefore contributing to the detection and regulation of emerging eventualities during the course of the learning situation. For that purpose we have formulated a "script-aware monitoring process of CSCL scenarios". This process defines how the design-time pedagogical decisions captured in the script may guide the data gathering and analysis in order to provide teachers with monitoring information connected to the concerns such design expresses. Furthermore, we have defined a "monitoring-aware model of CSCL scripts" to connect the design and monitoring processes. Aside from describing the connections between scripting and monitoring, the model also supports the data flow between the two aforementioned processes proposed in this dissertation. From the design point of view, the model represents the output of the "monitoring-aware design process"', providing a joint picture of the pedagogical and monitoring decisions made by the teacher. From the monitoring perspective, the model specifies the data to be gathered and the analysis criteria. The third challenge to be overcome deals with the problems that hinder the data gathering and integration in CSCL scenarios supported by DLEs, due to the increasingly heterogeneous and distributed nature of such technological environments. For example, the data to be gathered are decentralized and each tool follows its own formats and models. Moreover, sometimes the data relevant to the learning situation are not generated automatically through technological means (e.g., through log files), but rather in an ad-hoc manner by participants themselves (e.g., teacher's observations of the learning situation). To address these challenges we have proposed an "architecture for data gathering and integration in DLEs" that provides a conceptual solution for gathering and integrating participants' actions during the CSCL scenario, throughout the DLE. To achieve our objective of helping teachers monitor the accomplishment of their design decisions in blended CSCL scenarios over DLEs, we followed a Design-Based Research (DBR) approach involving three iterations and a total of seven studies in authentic CSCL scenarios. The first and second iterations were mainly exploratory and led us to the formulation of the four proposals presented in this dissertation. The third iteration focused on the evaluation of these proposals. The design of the third (evaluative) iteration has been supported by the CSCL-EREM framework (Computer Supported Collaborative Learning Evaluand-oriented Responsive Evaluation Model), which is especially appropriate for the evaluation of CSCL strategies and tools. In this case, our mixed-methods evaluation involved a variety of techniques for data gathering and analysis, during two authentic classroom experiments. The evaluation results suggest that the proposed model, processes and architecture can help teachers in the alignment between their pedagogical and monitoring concerns. Moreover, according to the teachers involved in the evaluation, the aforementioned proposals are likely to be used in real practice. Besides, this alignment has proven to be useful in supporting teachers in the orchestration of authentic CSCL scenarios. However, the evaluation also found shortcomings, hinting at paths for future research: providing teachers with technological support that enable them to design and monitor students in an autonomous way, investigating the possibilities of the provision of monitoring information to learners, or finding ways to scale up the innovations developed during the dissertation.