Category: News

At the end of December 2020, the DID-ACT project consortium welcomed two things: the holidays and the successful completion of Work Package 2. This post aims to provide an update on what that entailed, what we completed, as well as provide a short overview of what we are going to be developing in Work Package 3. To begin, we will provide a brief overview of what we learned using the age-old rhetorical question: “How do you eat an elephant?” To which the answer is, “not in one bite”. 

This rhetorical question is often used to illustrate how overcoming large and complex challenges is done by dividing them into smaller chunks. That when broken down into bite size pieces, these challenges are easier to manage. In the case of the DID-ACT project and beyond, every clinician, educator or researcher who has tried to describe the nature of teaching clinical reasoning, has realized this challenge. As a team, we most definitely learned this throughout work package 2 as we represent a collection of diverse professionals with the same ultimate goal, but with different ideas on how to get there.

When broken down further, we explored and learned how teaching clinical reasoning is a challenge that is inherently multifaceted. One facet, for example, is the complexity of a clinical situation, a second, is the need to grasp the nature of the varied competencies required to address the situation at hand, and third is how to support the learning of these competencies effectively.

Developing a clinical reasoning framework

Last fall the DID-ACT consortium developed a clinical reasoning curriculum framework that included clinical reasoning quality criteria. In addition to the above challenges, we emphasized having an interprofessional clinical reasoning curriculum. Our interprofessional framework was conceived with the input from different nations and educational cultures, all conducted online due to the current pandemic. So – similar to a complex clinical situation, we faced a plethora of challenges when producing and describing a clinical reasoning framework. This work led us to the development of two curricula: one directed to health professions students and one for teachers.

What makes a good clinical reasoning curriculum?

When we zoomed out to get a clear idea of the big picture, we noted a few crucial pedagogical aspects: a strong focus on student-centeredness; a perspective in which the student takes responsibility for their own learning process; as well as a strong connection to relevant clinical situations which means that knowledge and competencies were applied to context. We also noted that the philosophy of “constructive alignment” will be used when designing our clinical reasoning learning units. In practice, this means that the intended learning outcome should direct choices when designing learning activities; thereby creating a harmony between the clinical reasoning learning activities and how they are assessed. This means that the intended learning outcomes hold a central position when designing your clinical reasoning learning activities, assessments, and learning units overall. That is why we structured our interprofessional clinical reasoning framework according to ~ 50 learning objectives in an interdisciplinary consensus process.

DID-ACT’s Health Professions Education Framework

Did we eat the metaphorical elephant in this project phase? Yes! 

By using our various knowledge and skills collectively, then by dividing the bigger task into parts and iteratively working in small and greater teams, we put parts back together to form a much clearer picture. Building from Work Package 1, we had a framework that is grounded in an interdisciplinary needs analysis directed towards a breadth of European health professions schools to launch from. When taking our learning into WP2, our work entailed bringing forward and evaluating a large amount of open learning resources for clinical reasoning based on our desired learning outcomes. It was essential that these learning resources were accessible and of a strong quality for our online clinical reasoning curriculum. When we tied our learning objectives, outcomes, assessment ideas, and open resources together, we created a well-rounded, interprofessional framework and the beginnings of an actual online clinical reasoning course.

At this point, you are most welcome to look at our identified learning objectives, the framework and our recommendations to current national curricular descriptions on our reports page. Hopefully you and your school can benefit from them in order to support explicit learning of clinical reasoning. There is also a collection of existing open educational resources to support you, your students or colleagues that support clinical reasoning.

Every project needs evaluation. Even though it might sometimes be considered as cumbersome or stressful for those whose work is evaluated, it is important that the merits and limitations of any given project are clearly laid out. A well-conducted evaluation ideally goes beyond highlighting the particular achievements of a program by delivering ideas for improvement. Furthermore it justifies the need to continue the efforts surrounding the project and its aims. It is commonplace that evaluation and feedback are employed during the last stage of the curriculum development cycle. However, it is well-founded that initiating evaluations in program development should be started as early as possible. The benefits are many with the central reasoning being that evaluating early on maintains and ensures that the chosen tools align with the planned outcome(s).

In terms of evaluation for the DID-ACT project, the Evaluation Work Package is a shared effort of the consortium partners. Jagiellonian University in Kraków, Poland, is responsible for its coordination. Its first year of activities finished in December 2020 with a report published on the project’s website. During the first half of the year, the activities were focused on gauging the needs of potential users by developing a web survey to collect the specific expectations. From the data gathered, the DID-ACT project’s set of learning objectives and curricular framework were developed by another working group of the project. The goal of the second half of the year in terms of the evaluation work package was to propose a set of evaluation and learning analytic tools. Combined, these measure the planned outcome of the DID-ACT student curriculum and train-the-trainer course.

At the time of commencing our evaluation work, the specific set of learning objectives had not yet been set. Thus we first reviewed the literature in search of existing tools that measure participant satisfaction and perceived effectiveness of clinical reasoning training. This brought us the productive advantage and opportunity to reuse the outcomes of former projects. We experience this as an important point that demonstrates continuity and sustainability of research in this area. Our literature review identified a number of studies in which evaluation questionnaires of clinical reasoning learning activities were presented. Based on the analysis of the questions that aimed to measure student satisfaction, we were able to identify seven common themes of interest: course organisation, clear expectations, relevance, quality of group work, feedback, teaching competencies, and support for self-directed learning. We collected plenty of exemplar questions in each of the themes. Additionally, for the self-assessment questions we have assigned the gathered items to the DID-ACT learning goals and objectives.

Surprisingly our literature review did not yield any evaluation questions specific to clinical reasoning that could be used for our train-the-trainer courses. We resolved this challenge by broadening our goal. We adapted our search to include faculty development evaluation questionnaires that focused on honing teaching skills in general (not necessarily exclusively clinical reasoning). There was one evaluation tool from this group that caught our attention in particular: the Stanford Faculty Development Program Model (SFDP-26). We value its wide dissemination in many domains and clearly formulated set of 26 questions grouped in seven dimensions. An additional strength is that it has already been translated and validated in languages other than English, for example, in German. 

An interesting discovery for us was a tool that measures the impact of curricular innovation following the Concerns-Based Adoption Model (CBAM). This tool, developed at the University of Texas, proposes an imaginative way of measuring the progress of curriculum innovation. It does so by identifying the types of concerns teachers voice regarding new topics. These concerns  can range from disinterest, through concerns about efficiency of teaching of this element, and end with ideas for expanding the idea. 

The CBAM model is based on the assumption that certain types of statements are characteristic to particular developmental stages when introducing an innovation into a curriculum. The developmental stage of introducing the innovation is captured effectively by the Stage of Concern (SoC) questionnaire. When collecting the data from a particular school the outcome is a curve that displays the intensity of concerns found within the seven consecutive stages of innovation. The value this brings is that comparing the curves across several institutions can help us visualise any progress implementing the curriculum is having. We find this visualisation to be akin to following how waves traverse the ocean.

As the DID-ACT curriculum is planned to be a blended model of face-to-face and e-learning activities, we intend to use learning analytics in our curriculum evaluation. More specifically we will capture, process and interpret the digital footprints learners leave while using electronic learning environments. It is of course pivotal to be transparent about the purpose and to obtain consent regarding the collection of educational data. Upon receiving consent, computational power can be harnessed to optimise educational processes to the benefit of both learners and teachers. From the perspective of the curriculum developer, it is particularly important to know which activities attracted the most versus least engagement from students. 

This information, when triangulated with other data evaluation sources, e.g. from questionnaires or interviews, allows us to identify elements of the curriculum that are particularly challenging, attractive or in need of promotion or better alignment. The learning analytics dashboards are viewed for our purposes a bit like a car’s dashboard where our fuel, odometers, speedometer display key information; for DID-ACT, analytics present a clear range of visualised progress indicators in one place.

We selected then analysed two electronic tools that will be used to implement the technical side of the DID-ACT curriculum: “Moodle” (a learning management system) and “Casus” (a virtual patient platform). Our goal was to look for the relevant learner data that could be collected. In addition, we intended to determine how it is visualised when following learner progress and trajectories. To systematise the process, we have produced a table we dubbed the ‘Learning Analytic Matrix’ that shows how engagement in attaining individual DID-ACT learning goals and objectives is captured by these electronic tools. Logs of such activities, like the opening of learning resources, time spent on activities, number and quality of posts in discussion boards, or success rate in formative questions, will enable us to map what is happening in the learning units developed by the DID-ACT consortium. 

This is augmented by recording traces of some learning events which are characteristic to the clinical reasoning process. These events can be qualified as success rates in making the right diagnoses in virtual patient cases, student use of formal medical terminology in summary statements, or making reasonable connections in clinical reasoning concept maps. We also inspected the ways the captured data are presented graphically, identifying at the moment a predominance in tabular views. We foresee the possibility of extending the functionality of learning analytic tools available in the electronic learning environment by introducing a more diverse way of visualising evaluation results in learning clinical reasoning. 

The collection and interpretation of all that data related to the enactment of the DID-ACT curriculum using the described tools is something we are looking forward to pursuing in the two upcoming years of the DID-ACT project.