Evaluation results of piloting of student learning units

After we had tested the train-the-trainer courses of the DID-ACT curriculum, it was time to evaluate the quality of learning units for students. We have conducted a series of pilot studies that validated five different learning units in eight evaluation events across all partner institutions including also associate partners. We have recorded student activities in the virtual patient collection connected with the DID-ACT curriculum available for deliberate practice. In addition, we evaluated the usability of the project’s learning management system in several test scenarios.

Overview about student activities in piloted learning units


Overall, students agreed to a large extent that the piloted DID-ACT learning units improved their clinical reasoning skills (average 5.75 in 7-point Likert scale). As a special strength of
the curriculum students frequently named the benefit of virtual patients integrated with the learning units. Another highlight were small-group group discussions, often conducted in multinational teams which broadened their views on clinical reasoning. However, a challenge in the tested version of the curriculum implementation was navigation in the learning management system (Moodle). As a consequence, we have further analyzed these data and, furthermore, conducted a series of usability tests. These analyses and tests led to a process to address the issues wherever it is possible. We have also received several requests for modifications of the developed learning material that we will address in the next deliverable, in which we refine courses based on pilot implementation.

You can find the full report, that was published iend of March 2022 here: Evaluation and analysis of the pilot implementations of the student curriculum

Reflections from the Medical Education Forum 2021

The 2nd Medical Education Forum (MEF) hosted from 4 to 6 May 2021 as a virtual meeting was an opportunity to review and summarise current research outcomes in medical education. It was organised by Jagiellonian University Medical College, McMaster University and Polish Institute for Evidence-Based Medicine. The live event had five speakers from the DID-ACT project (Samuel Edelbring, Inga Hege, Sören Huwendiek, Małgorzata Sudacka & me) and had 110 participants from 24 countries, most of them from Canada, Poland and Ukraine.

During the MEF conference, I took on the task of reviewing the most recent systematic reviews of virtual patients effectiveness. A review of reviews is called an umbrella review. Effectiveness of virtual patients is an important topic for the DID-ACT project because we use this type of education resources as a vehicle to deliver interactive exercises to practice clinical reasoning in the designed DID-ACT curriculum. To see how effectiveness is measured of clinical reasoning outcomes is also important to inform the DID-ACT project pilot evaluations. 

I have identified in the recent three years five systematic reviews of virtual patients effectiveness. This included a systematic review I completed with my colleagues from the Digital Health Education collaboration in 2019. For me personally, preparation of the MEF presentation was an interesting exercise that gave an opportunity to see how much the results obtained in our former review align with the outcomes reported in other reviews published afterwards. To check it makes sense as systematic reviews often have unique scopes defined by the selected inclusion criteria, data extraction and synthesis methods and therefore may differ. 

The reviews published after 2019 were carried out by international teams from France, New Zealand, South Korea, UK and USA. Only one, similar as we, included all health professions; the remaining focused on particular health professions: nursing, medicine, pharmacy. The studies either included all possible outcomes or selected a particular skill. It was interesting to see that the skill that was in particular in the scope of interest in syntheses in the recent years were communication skills. The conclusions of the studies were consistent across the different professions and topics. The studies reported benefits of application of virtual patients in education with hardly any exceptions. As Lee and colleagues (Med Educ, 54(9), 2020) concluded in their systematic review, the effectiveness of virtual patients can be even more improved when their use is preceded or followed by reflection exercises and human-teacher provided feedback. The technological features of virtual patient platforms were less important. 

You may learn more about the result of my umbrella review, presentation of the other DID-ACT project speakers and the follow-up Question & Answers sessions as video recording.

Ocean waves, footprints and dashboards: the selection of DID-ACT evaluation and learning analytics tools

Every project needs evaluation. Even though it might sometimes be considered as cumbersome or stressful for those whose work is evaluated, it is important that the merits and limitations of any given project are clearly laid out. A well-conducted evaluation ideally goes beyond highlighting the particular achievements of a program by delivering ideas for improvement. Furthermore it justifies the need to continue the efforts surrounding the project and its aims. It is commonplace that evaluation and feedback are employed during the last stage of the curriculum development cycle. However, it is well-founded that initiating evaluations in program development should be started as early as possible. The benefits are many with the central reasoning being that evaluating early on maintains and ensures that the chosen tools align with the planned outcome(s).

In terms of evaluation for the DID-ACT project, the Evaluation Work Package is a shared effort of the consortium partners. Jagiellonian University in Kraków, Poland, is responsible for its coordination. Its first year of activities finished in December 2020 with a report published on the project’s website. During the first half of the year, the activities were focused on gauging the needs of potential users by developing a web survey to collect the specific expectations. From the data gathered, the DID-ACT project’s set of learning objectives and curricular framework were developed by another working group of the project. The goal of the second half of the year in terms of the evaluation work package was to propose a set of evaluation and learning analytic tools. Combined, these measure the planned outcome of the DID-ACT student curriculum and train-the-trainer course.

At the time of commencing our evaluation work, the specific set of learning objectives had not yet been set. Thus we first reviewed the literature in search of existing tools that measure participant satisfaction and perceived effectiveness of clinical reasoning training. This brought us the productive advantage and opportunity to reuse the outcomes of former projects. We experience this as an important point that demonstrates continuity and sustainability of research in this area. Our literature review identified a number of studies in which evaluation questionnaires of clinical reasoning learning activities were presented. Based on the analysis of the questions that aimed to measure student satisfaction, we were able to identify seven common themes of interest: course organisation, clear expectations, relevance, quality of group work, feedback, teaching competencies, and support for self-directed learning. We collected plenty of exemplar questions in each of the themes. Additionally, for the self-assessment questions we have assigned the gathered items to the DID-ACT learning goals and objectives.

Surprisingly our literature review did not yield any evaluation questions specific to clinical reasoning that could be used for our train-the-trainer courses. We resolved this challenge by broadening our goal. We adapted our search to include faculty development evaluation questionnaires that focused on honing teaching skills in general (not necessarily exclusively clinical reasoning). There was one evaluation tool from this group that caught our attention in particular: the Stanford Faculty Development Program Model (SFDP-26). We value its wide dissemination in many domains and clearly formulated set of 26 questions grouped in seven dimensions. An additional strength is that it has already been translated and validated in languages other than English, for example, in German. 

An interesting discovery for us was a tool that measures the impact of curricular innovation following the Concerns-Based Adoption Model (CBAM). This tool, developed at the University of Texas, proposes an imaginative way of measuring the progress of curriculum innovation. It does so by identifying the types of concerns teachers voice regarding new topics. These concerns  can range from disinterest, through concerns about efficiency of teaching of this element, and end with ideas for expanding the idea. 

The CBAM model is based on the assumption that certain types of statements are characteristic to particular developmental stages when introducing an innovation into a curriculum. The developmental stage of introducing the innovation is captured effectively by the Stage of Concern (SoC) questionnaire. When collecting the data from a particular school the outcome is a curve that displays the intensity of concerns found within the seven consecutive stages of innovation. The value this brings is that comparing the curves across several institutions can help us visualise any progress implementing the curriculum is having. We find this visualisation to be akin to following how waves traverse the ocean.

As the DID-ACT curriculum is planned to be a blended model of face-to-face and e-learning activities, we intend to use learning analytics in our curriculum evaluation. More specifically we will capture, process and interpret the digital footprints learners leave while using electronic learning environments. It is of course pivotal to be transparent about the purpose and to obtain consent regarding the collection of educational data. Upon receiving consent, computational power can be harnessed to optimise educational processes to the benefit of both learners and teachers. From the perspective of the curriculum developer, it is particularly important to know which activities attracted the most versus least engagement from students. 

This information, when triangulated with other data evaluation sources, e.g. from questionnaires or interviews, allows us to identify elements of the curriculum that are particularly challenging, attractive or in need of promotion or better alignment. The learning analytics dashboards are viewed for our purposes a bit like a car’s dashboard where our fuel, odometers, speedometer display key information; for DID-ACT, analytics present a clear range of visualised progress indicators in one place.

We selected then analysed two electronic tools that will be used to implement the technical side of the DID-ACT curriculum: “Moodle” (a learning management system) and “Casus” (a virtual patient platform). Our goal was to look for the relevant learner data that could be collected. In addition, we intended to determine how it is visualised when following learner progress and trajectories. To systematise the process, we have produced a table we dubbed the ‘Learning Analytic Matrix’ that shows how engagement in attaining individual DID-ACT learning goals and objectives is captured by these electronic tools. Logs of such activities, like the opening of learning resources, time spent on activities, number and quality of posts in discussion boards, or success rate in formative questions, will enable us to map what is happening in the learning units developed by the DID-ACT consortium. 

This is augmented by recording traces of some learning events which are characteristic to the clinical reasoning process. These events can be qualified as success rates in making the right diagnoses in virtual patient cases, student use of formal medical terminology in summary statements, or making reasonable connections in clinical reasoning concept maps. We also inspected the ways the captured data are presented graphically, identifying at the moment a predominance in tabular views. We foresee the possibility of extending the functionality of learning analytic tools available in the electronic learning environment by introducing a more diverse way of visualising evaluation results in learning clinical reasoning. 

The collection and interpretation of all that data related to the enactment of the DID-ACT curriculum using the described tools is something we are looking forward to pursuing in the two upcoming years of the DID-ACT project. 

DID-ACT at AMEE 2020

The Association for Medical Education in Europe (AMEE) is one of the biggest organisations focused on excellence and research in health professions education. It has been organising annual conferences for scholars engaged in this topic for close to 50 years.  The interest in these meetings is rising and has reached the level of around 4000 participants last year. The DID-ACT consortium decided to disseminate its outcomes at AMEE by submitting an abstract informing about the results of the project’s needs analysis.

This year’s conference was originally planned to be held in Glasgow, United Kingdom, however, changes had to be made due to the ongoing COVID-19 pandemic; AMEE’s traditionally face-to-face format was adapted to be a virtual conference that  rose to the challenge and exceeded expectations. Rather than following suit to mainstream reliance on primarily traditional audio-video teleconferencing tools, AMEE took on the challenge to host the conference in a virtual world. The virtual venue encompassed a group of interconnected locations with different purposes. A palm tree grown lobby with information booths led to several lecture theatres, exhibition halls, networking areas and poster rooms. The participants, prior to joining the conference, designed their own avatar and then navigated it through the locations meeting on the way avatars of other participants. The meetings enabled interactions either by typing in a chat window or an audio conversation. Participation in the events held in parallel conference communications could be interactive as well, enabling the audience to applaud, raise hands, and talk to the next-sited neighbour.

The DID-ACT submission was accepted for AMEE 2020 as a virtual poster. This presentation format involves constructing a digital “stack” of multimedia resources which could be presented either in a smartphone app or in a web browser. The content is organised in nested sections depicted as rectangular tiles, each containing resources as text entries, images, web links. Each conference presenter was encouraged to incorporate in the poster a short video showing a voice-over PowerPoint presentation giving an overview of the most important content. In addition it was required to prepare a one page digital print-out of the poster including a QR-code for easy access by smartphones from the real world. The DID-ACT poster was prepared by Andrzej Kononowicz, Małgorzata Sudacka, Felicitas L. Wagner,  Samuel Edelbring, Inga Hege and Sören Huwendiek on behalf of the consortium. In the image below we present the poster print-out. The content is available via this link https://api.ltb.io/show/BWPMF.

The virtual conference was held from 7th until 9th September. Several DID-ACT members participated in the conference events and networked with fellow researchers. In particular there were several conference presentations around the topic of clinical reasoning. By the end of conference the participants form DID-ACT project decided to gather virtually in one of the exhibition hall for a virtual group selfie:

Standing from the left are: Desiree Wiegleb Edstöm, Živa Ledinek, Małgorzata Sudacka, Maria Elvén, Andrzej Kononowicz and Inga Hege

The conference contributions presented at the virtual AMEE conference will be available at least throughout the next year and by that enable playback of the presentations and sustainable project dissemination. Participation in the conference was a memorable event, impressive by its innovation and showing how far virtualisation of education and research can nowadays go. Despite the many benefits of the virtual conference, and thankful it was possible to be held in these troubled pandemic times, we hope we will be able to meet up at the face-to-face conference next year at AMEE 2021 in real world Glasgow to present the community more news around the DID-ACT project.

Participants: Medical educators

Learning units: Person-centered approach and the role of patients
Level: Teacher

Description: Offered as part of the faculty development to faculty educators.
Mode: Part to the faculty development program with emphasize on the educational potential of clinical reasoning teaching techniques

Technical Integration: Access to the DID-ACT Moodle, synchronous sessions were held via the virtual platform NewRow.

Tips & Tricks:

Participants: Multi-professional educators (nurses, physicians, paramedics)

Learning units: What is Clinical Reasoning and Models
Level: Teacher

Description: Offered as additional faculty development opportunity to staff members of the Medical Education Department.
Mode: Option / additional part to the faculty development program with emphasize on the educational potential of clinical reasoning teaching techniques

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face.

Tips & Tricks: Give participants time to familiarize themselves with the learning material between the sessions. Do not put too much in one day - better to meet twice for shorter sessions. Face-to-face sessions lead to more productive discussions than Zoom meetings. Focus on discussion with the audience and on examples from practical teaching to illustrate the learning objectives.

Participants: Multi-professional educators

Learning units: Differences and similarities in clinical reasoning among health professions
Level: Teacher

Description: Offered as part of the faculty development program, it was held as a blended learning course with the synchronous phase online.
Mode: Part of the faculty development program with certificate

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held online via Zoom.

Tips & Tricks: Give good practical information how Moodle works and how the participants can find and work with the different assigments of the learning unit. Emphasize the importance to the participants to work with the different individual tasks, as the discussions will be more interesting and fruitful. Also important that all the participants from different occupations feel comfortable to meet and that they all are a part of the discussion. This is an important and maybe the primary task for the facilitator!

Participants: Multi-professional and international educators

Learning units: Clinical Reasoning teaching and assessment & What is Clinical Reasoning and Models
Level: Teacher

Description: Offered as optional learning unit for participants of the Master of Medical Education (MME) program in Bern/Switzerland.
Mode: Part of the faculty development program with certificate

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face.

Tips & Tricks: To foster a valuable learning experience it is indispensible to provide good case examples that are tailored to the needs and experiences of the participants, so that they can relate to their prior knowledge. Thus, the provided examples in this learning units might need some adaptations for your target group of educators.

Participants: Multi-professional educators across German-speaking coutries

Learning units: Differences and similarities in clinical reasoning among health professions
Level: Teacher

Description: Offered as part of the faculty development program at the University of Augsburg, but open to participants from Germany, Austria, and Switzerland. It was held as a blended learning course with the synchronous phase online. Participants were eager to exchange their experience and disucss their views across institutions and professions.
Mode: Part of the faculty development program with certificate

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held online via Zoom.

Tips & Tricks: To foster a valuable interprofessional experience the participants should represent a balanced mix of professions and also ideally the facilitators should at least represent two different professions. This allows a good discussion and also guarantees that the small groups can work interprofessionally. Our experience with a less-balanced group composition was that the over-represented profession dominates the discussions and it was quite difficult to counter-balance this.

Target group: Medical students in year 2

Learning units: Person-centered approach to clinical reasoning
Level: Novice

Description:The Learning Unit was run as an extra-curricular session as a virtual class. The facilitators were trained on this topic by attending the Train-the-trainer learning unit on "Person-centered approach and the role of patients".
Mode: Extracurricular activity

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held online via NewRow.

Tips & Tricks: Learning objective cross-referencing with the existing curriculum helps identify the level of integration. Encouraging and helping students to get familiar with Moodle before the session facilitates a smooth integration of the asynchronous phases.

Target group: Medical students in year 4 and 5

Learning units: Generating differential diagnoses and deciding about final diagnoses
Level: Novice

Description: The learning unit was integrated into a pediatric emergency department clerkship with a relation to virtual patients already used in this clerkship. Facilitators were trained by attending the train-the-trainer unit on "Information gathering, Generating differential diagnoses, Decision making, and Treatment planning".
Mode: Part of a regular curricular activity

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face.

Tips & Tricks: Integration into clerkships works well, especially with a relation to already used content.

Target group: Medical students in year 1-6

Learning units: All 25 learning units of the DID-ACT curriculum
Level: Novice - Advanced

Description:
Longitudinal integration of the DID-ACT learning units into a clinical skills & communication course with the following suggested distribution across years:
Year 1: What is Clinical Reasoning, Person-centered approach to clinical reasoning, Health profession roles in clinical reasoning, and Biomedical Knowledge & Clinical Reasoning. (Required time in curriculum ca. 5 hours / semester)
Year 2: Dual Process Theory, Illness scripts, Collect and prioritize key clinical findings/problems, and What is clinical reasoning and How can theories be put into practice (Intermediate). (Required time in curriculum: ca. 4 1/2 hours/semester)
Year 3: Generating differential diagnoses and deciding about final diagnosis, Biases and cognitive errors - an Introduction, Analyzing and avoiding errors. (Required time in curriculum: ca. 4 1/2 hours/semester)
Year 4: Using the Outcome Present State Test Model, Developing a treatment plan, Metacognition, reflection and models for reflection, Collaboration of health professions in Clinical Reasoning (Intermediate). (Required time in curriculum: ca. 6 hours/semester)
Year 5: All remaining intermediate learning units: Shared Decision Making in Clinical Reasoning, Decision Support Systems, Ethical aspects - patient management and treatment, Uncertainty. (Required time in curriculum: ca. 5 hours/semester)
Year 6: All 6 advanced learning units: Collaborate with others in clinical reasoning, Decision Support Systems, Biases and cognitive errors, Uncertainty, Metacognition, reflection and models for reflection, Analyzing and avoiding errors.
(Required time: ca. 6 hours / semester. Final year students often have a day/week off from clinical work so these days could be used for DID-ACT learning units and optional participation in the train-the-trainer units)
Years 1-5: Virtual Patients (VPs) as additional deliberate practice activities in increasing number and complexity (e.g. starting with 5 VPs (=ca. 1.5 hours)/semester) in Year 1 and increasing to 10 VPs/semester) in Year 3-5.
Mode: Integrated into a clinical longitudinal course that runs from year 1 to 6 resulting in a total of 2.4 ECTS.

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions face-to-face or online.

Tips & Tricks:
Alining a longitudinal curriculum with curricula of other health professions remains a challenge and requires careful and early planning, but the asynchronous phases might be a good starting point for interprofessional teaching sessions. The VPs can be quite easily aligned with other curricular courses e.g., based on key symptoms.

Target group: Medical students in year 3

Learning units: Person-centered approach to clinical reasoning
Level: Novice

Description: The learning unit was integrated into the Laboratory Training of Clinical Skills. This course consists of six face-to-face meetings for groups of around 10 students. Within the meetings the students discuss various topics related to communication skills and person-centered approach. The fifth of the six meetings of the course was replaced by the DID-ACT learning unit. Students could then refer back to this learning unit during final session of the course.
Mode: Part of a regular curricular activity

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face. If possible, a uniform technology to provide access to the online resources in the class (using university tablets in our case) and support of the technical staff on-site was helpful to lower the technical barrier.

Tips & Tricks: Changing of the standard format of classes is interesting and motivating for the students. Virtual patients are a tool to present authentic clinical scenarios which are appreciated by the students. A blend of role-play and virtual patients allows students to take advantages of the two methods to reach the learning objectives.

Target group: Medical students in year 2 and nursing students in year 3 across Europe

Learning units: Collaboration of Health Professions in Clinical Reasoning
Level: Intermediate

Description:
This learning unit was implemented with , aside from clinical reasoning, tow additional objectives: 1) providing internationalization experience for students without traveling. 2) Deepening the knowledge of one's own professional roles and responsibilities and knowledge of another profession .
Mode: Extra-curricular interprofessional learning session with international participants

Technical Integration: Self-registration on DID-ACT Moodle, synchronous sessions via zoom.

Tips & Tricks: Facilitators running this course and supporting the discussions should be careful about supporting and encouraging all the participating professions' perspectives.

Target group: Medical students in year 1 and 2 (preclinical) across Europe

Learning units: Introduction into Clinical Reasoning & Health profession roles in clinical reasoning
Level: Novice

Description: The course was offered as a blended learning module to students from different medical schools in Europe as an international elective. Synchronous phases were held online via Zoom. The international aspect was very motivating for students and they learnt a lot from each other by exchanging their perspectives and how they are taught clinical reasoning vaspects. This teaching mode could also be implemented as an activity to welcome or prepare new Erasmus students. Facilitairs were trained by attending the train-the-trainer learning unit on "Differences and similarities in clinical reasoning among health professions".
Mode: Elective course

Technical Integration: Self-registration on DID-ACT Moodle, synchronous sessions via zoom and use of Padlet for interactivities.

Tips & Tricks: The organization with the registration of students was a bit complex, so, we suggest just setting dates and let students book. Allow enough (more time) for discussion and introduction rounds as participants do not know each other and are eager to hear and learn from peers at other schools and countries. Ideally, this session could also be held interprofessionally, however, it makes the finding of suitable dates even more complex.

How to teach synchronously in a virtual setting

  • You need a reliable camera, microphone, and virtual platform and be familiar with its features, such as whiteboard, chat, polling, breakout rooms, etc.
  • At the beginning establish communication rules, e.g. whether participants should raise their (virtual) hand, use the chat, and/or just speak. Also, we recommend asking participants to turn on their camera
  • For small group work break out rooms work very well, just be clear about the tasks the groups should work on prior to dividing them into the groups.
  • For collaboration the use of integrated virtual whiteboards or other platforms such as Padlet are very useful. Just make sure prior to the session that you have everything setup and the links at hand, e.g. to post them in the chat.
  • Allow a bit more time for starting the session and the group works as there might be participants who are not familiar with the platform or technical problems might occur.

How to motivate unprepared participants

  • Make clear that the asynchronous assignments are a core part of the course and that its content will not be repeated. Even if it is difficult, stick to that when starting the synchronous teaching session.
  • If you expect unprepared participants, you can start the session with a student-centered group exercise mixing prepared and unprepared students to increase peer-pressure and make them realize that being unprepared does not feel good.  
  • Use the introductory or closing quizzes / tests so that participants can self- assess whether they have the required knowledge and you as a facilitator can see the level of knowledge and preparation of your participants.

Further recommended reading:

How to involve participants with different levels of experience

  • To account for such different levels, we recommend making use of the asynchronous preparatory phases which also include introductory quizzes in which participants can self-assess their prior knowledge and you as a facilitator can assess the differences within your group. Participants with less prior experience can also be guided to additional preparatory resources.
  • Encourage participants to work in pairs or small groups when preparing so that they can help and learn from each other. You could even facilitate this by dividing them into groups with different levels of experience.
  • Similarly, during the synchronous phases, we recommend forming groups with participants different levels of experience and emphasize the peer support aspects of such group activities.
  • We also recommend starting with rather smaller groups and allow more time than stated in the course outlines, if you expect a heterogenous level of experience. This way you can better manage this challenge.
  • Encourage your participants to ask questions, emphasizing that nobody knows everything and that it is important for learning to ask questions.  
  • Especially in the train-the-trainer course you might have to deal with over-confident participants, who especially in an interprofessional setting can dominate the group. This is a complex cultural challenge, but you could try to establish (and follow) communication rules at the beginning of a session.  

How to address potential overlaps or redundancies

  • Identify what is already included and what is missing in your curriculum related to clinical reasoning outcomes and compare it to the DID-ACT blueprint. Prioritize learning outcomes that are not yet covered but regarded as important.
  • Identify activities, resources, or teaching sessions with similar learning outcomes that might be in need for change anyway because of low evaluation results, teachers or students struggle with it. These could be suitable for adding or replacing parts with DID-ACT activities.
  • Ask teachers and students about overlaps and gaps they see in their teaching / learning of clinical reasoning and where they struggle. This could also be done by a reflection round after related teaching activities in the curriculum
  • Although ideally a longitudinal integration is aimed at, we recommend to starting small with a pilot implementation to gain experience and develop a show case.

How to teach in an interprofessional setting

  • Allow for enough time prior to the teaching for the organization and motivation / encouragement of stakeholders and participants
  • Allow for enough time and guidance during the course so that the participants from the different professions can get to know each other and their professions and discuss their different perspectives. This might mean that you need to calculate some extra time in addition to the suggested duration of the learning unit.
  • There may be a different understanding of clinical reasoning in the different health professions, so we recommend making participants aware of this. You could for example use and adapt activities from the learning units on the health profession roles to facilitate this.
  • Courses in an interprofessional setting should not come too early in the curriculum (not before professions have formed their own professional identity - however, this also depends on the aim of the course). 
  • Make sure you have enough participants from different professions. If possible, the facilitator could divide the participants in smaller groups with an equal distribution of professions. 
  • Similarly, you need an equal distribution of facilitators / facilitators from different professions.
  • Develop customized learning materials considering the different professions. If needed you can adapt the material and activities provided in the DID-ACT curriculum.

Further recommended reading:

van Diggele, C., Roberts, C., Burgess, A. et al. Interprofessional education: tips for design and implementation. BMC Med Educ 20, 455 (2020). (Link)

Theoretical / Background Knowledge

These resources and activities summarize all topics from the student learning units to introduce educators to these concepts. These resources are part of the train-the-trainer courses and marked as "optional" . Thus, they can be used optionally by course facilitators if participants are not yet familiar with basic concepts.

Theme(s): All basic concepts of clinical reasoning
Level: Educators
Format: Additional resources and material provided for each train-the-trainer learning unit that can be integrated if needed, e.g. if participants are quite new to the topic. These resources cover the basic concepts of a topic without going into the teaching aspects, so they can be used as preparatory steps.

Metacognition, reflection and models for reflection

Similar to the novice learning unit learners are asked to complete a reflective diary for five days. However, in this learning unit they should focus on critical or difficult situations in the clinical context.

Theme(s): Errors & biases
Level: Advanced
Format: Asynchronous online preparatory phase (ca. 60 min) with a synchronous follow-up meeting (ca. 60 min)
Recommended ECTS: 0.07
Links: Student course - Facilitator resources

Ethical aspects - patient management and treatment

his learning unit provides an introduction into bioethical principles, consent, capacity, and ethical clinical reasoning.

Theme(s): Ethical Aspects
Level: Intermediate
Format: Asynchronous online preparatory phase (ca. 60 min) with a synchronous follow-up meeting (ca. 90 min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Decision Support Systems

The learning unit includes the generation of a decision tree based on a breast cancer data set from radiology department using the RapidMiner software package and an elaboration of the concepts of sensitivity and specificity. Furthermore, we will apply Bayesian reasoning and give an opportunity to discuss the base rate fallacy problem and the use of electronic calculators to judge the risk. The learning unit is finished with a discussion of the barriers/facilitators of using computers/AI in hospitals to support clinical reasoning.

Theme(s): Gathering, interpreting, and synthesizing information, Decision making
Level: Advanced
Format: Synchronous meeting (ca. 90 min) followed by a synchronous meeting (ca. 90 min)
Recommended ECTS: 0.1
Links: Student course - Facilitator resources

Uncertainty

In this learning unit, the approach of practice inquiry will be introduced and applied.

Theme(s): Biases & errors
Level: Advanced
Format: Asynchronous online preparation (ca. 60 min) followed by a synchronous meeting (ca. 60 min)
Recommended ECTS: 0.07
Links: Student course - Facilitator resources

Analyzing and avoiding errors

This learning unit will provide general and specific aspects of a morbidity and mortality conference and apply the knowlege by working through a case report.

Theme(s): Biases & errors
Level: Advanced
Format: Asynchronous online preparation (ca. 60 min) followed by a synchronous meeting (ca. 90 min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Biases and cognitive errors

This learning unit introduces additional errors and biases and your will have the opportunity to work on virtual patients to identify error-prone situations.

Theme(s): Biases & errors, Ethical aspects, Theories of clinical reasoning
Level: Advanced
Format: Asynchronous online preparataion (ca. 60 min) for a follow-up synchronous meeting (ca. 90 min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Shared Decision Making in Clinical Reasoning

In this learning unit, learners will be familiarized or re-familiarized with the basic concept of shared decision-making (SDM) in a way that serves as a steping stone for how to implement key concepts and models into practice. A job aid on shared decision-making will be a takeaway from this learning unit.

Theme(s): Patient Perspective, Decision Making
Level: Intermediate
Format: Asynchronous online preparation (ca. 60 min) with a follow-up synchronous meeting (ca. 90 min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Collaboration of health professions in clinical reasoning

In this learning unit you will be able to apply interprofessional aspects of clinical reasoning and understand similarities and differences between the clinical reasoning of health professions. This facilitates a better communicate across professions in the clinical reasoning process to meet the needs of the patient.

Theme(s): (Interprofessional) Collaboration
Level: Intermediate
Format: Starting with two synchronous meetings (each 60 min or combined) followed by an asynchronous follow-up (ca. 60 min)
Recommended ECTS: 0.1
Links: Student course - Facilitator resources

Decision Support Systems

The aim of this learning unit is to facilitate a discussion about which aspects of clinical reasoning can be supported by artificial intelligence and what the limitations of machines in clinical reasoning are.

Theme(s): Gathering, interpreting, and synthesizing information, Decision making
Level: Intermediate
Format: Synchronous meeting (ca. 90 min) with a follow-up asynchronous phase (ca. 90 min)
Recommended ECTS: 0.1
Links: Student course - Facilitator resources

Evaluation of Clinical Reasoning

This learning unit provides an overview about surveys and questions suitable to evaluate clinical reasoning teaching. It also introduces the relevance of learning analytics.

Theme(s): Teaching clinical reasoning
Level: Educators
Format: Self-guided on-demand course with different material on clinical reasoning evaluation (ca. 60 min)
Recommended ECTS: 0.03
Links: Course

Discussing and teaching about cognitive errors and biases

In this learning unit participating educators and clinicians will have the opportunity to share ideas on how a culture for discussing errors should look like. They will also learn more about most common errors and biases in clinical resoning and (teaching) strategies on how to avoid these. This learning unit supports educators in teaching the student courses on the novice level: Biases and cognitive errors, Uncertainty, and Analyzing & avoiding errors

Theme(s): Teaching clinical reasoning, Errors & biases, Theories of clinical reasoning, Ethical aspects
Level: Educators
Format: Asynchronous preparation (ca. 60 min) followed by a synchronous meeting (ca. 90 min). Optional additional activities are provided for beginners (ca. 80 min).
Recommended ECTS: 0.08 (including optional phase: 0.13)
Links: Participant course - Facilitator resources

Information gathering, Generating differential diagonses, Decision making, and Treatment planning

This learning unit is designed to support educators in implementing the student courses on Collect and prioritize key clinical findings/problems, Generating differential diagnoses and deciding about final diagnosis, Developing a treatment plan, and Biomedical Knowledge and Clinical Reasoning

Theme(s): Teaching clinical reasoning, Gathering, interpreting & synthesizing information, Generating differential diagnoses, Developing a treatment / management plan, Decision making, Ethical aspects
Level: Educators
Format: Asynchronous preparation (ca. 60 min) followed by a synchronous meeting (ca. 60 min). Optional additional activities for beginners are available (ca. 60 min).
Recommended ECTS: 0.07 (including optional phase: 0.1)
Links: Participant course - Facilitator resources

Person-centred approach and the role of patients

This learning unit prepares educators to comprehensively and confidently teach the the learning unit on Person-centred approach to clinical reasoning. This unit will take educators through the learning content provided in the learning unit, as well as supports them in familiarizing themselves with the resources and exercises. Educators will have the opportunity to create their own teaching notes as part of this learning unit.

Theme(s): Teaching clinical reasoning, Patient perspective
Level: Educators
Format: Asynchronous preparation (ca. 60 min), followed by a synchronous meeting (ca. 90 min). Optional additional activites are available for beginners (ca. 100 min).
Recommended ECTS: 0.08 (including optional phases: 0.14)
Links: Participant course - Facilitator resources

Differences and similarities in clinical reasoning among health professions

This learning unit introduces teaching methods for clinical reasoning in different healthcare professions and prepares educators to teach the learning units on Health profession roles in clinical reasoning

Theme(s): Teaching clinical reasoning, (Interprofessional) collaboration
Level: Educators
Format: Asynchronous preparation (ca. 60 min) followed by a synchronous meeting (ca. 70 min). Optional additional activities provided for beginners (ca. 120 min).
Recommended ECTS: 0.07 (including optional phases: 0.14)
Links: Course,Facilitator resources

What is Clinical Reasoning and Models

This learning unit familiarizes healthcare profession educators on teaching aspects related to the clinical reasoning process and terminology of the different health professions. This includes how to explain the importance of clinical reasoning in the different health professions to students and how to support students in reflecting on clinical reasoning theories. The learning unit prepares you for teaching the novice courses on What is clinical reasoning, Dual Process Theory, Outcome Present State model, and Illness scripts.

Theme(s): Teaching clinical reasoning, Theories of clinical reasoning
Level: Educators
Format: Two synchronous meetings (ca. 60 and 45 min) with asynchronous phase (ca. 45 min) in between. Optional additional phase with ca. 45 min.
Recommended ECTS: 0.08 (including optional phase: 0.11)
Links: Course, Facilitator resources

Clinical Reasoning teaching and assessment

This learning unit provides and overview about teaching and assessment methods for clinical reasoning. It also highlights some general apsects, such as the importance of constructive alignment or how to organize group discussions.

Theme(s): Teaching clinical reasoning
Level: Educators
Format: Self-guided on-demand course with different material on clinical reasoning teaching and assessment.
Recommended ECTS: 0.03
Links: Course

DID-ACT clinical reasoning curriculum

This learning unit provides an overview about the DID-ACT student curriculum including all course outlines and resources neede to implement these learning units. It also introduces a tutorial on how to read the course outlines and use the provided resources.

Theme(s): Teaching clinical reasoning
Level: Educators
Format: Self-guided on-demand course with different material on our DID-ACT curriculum including all course outlines for student learning units and material needed.
Links: Course

Analyzing and avoiding errors

Along a case report this learning unit introduces the root cause analysis (RCA) to analyze errors and start to find ways for preventing / avoiding errors.

Theme(s): Errors & Biases
Level: Novice
Format: Asynchronous preparation (ca. 90 min) followed by a synchronous meeting (ca. 70 min)
Recommended ECTS: 0.09
Links: Student course - Facilitator resources

Metacognition, reflection and models for reflection

In this learning unit reflection models will be introduced and self-reflection will be applied on form of a reflective diary.

Theme(s): Errors & biases
Level: Novice
Format: Synchronous meeting (ca. 60 min), followed by an asynchronous online phase (ca. 80 min) and a concluding synchronous meeting (ca. 80 min)
Recommended ECTS: 0.12
Links: Student course - Facilitator resources

Uncertainty

The aim of this learning unit is to introduce situations of uncertainty and strategies that can be applied in such situations to avoid errors.

Theme(s): Errors & biases
Level: Intermediate
Format: Asynchronous preparation (ca. 60 min) followed by a synchronous session (ca. 90 min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Biases and cognitive errors - an Introduction

This learning unit provides a basic introduction into the topic of biases and cognitive errors. It introduces a selection of common biases, such as premature closure or confirmation bias with providing the opportunity to elaborate on these biases with case vignettes.

Theme(s): Errors & biases, Theories of clinical reasoning, Ethical aspects
Level: Novice
Format: Asynchronous preparation (ca. 70 min) with a follow-up synchronous meeting (ca. 90 min)
Recommended ECTS: 0.09
Links: Student course - Facilitator resources

Developing a treatment plan

This learning unit provides an introduction into the topics "EBM in the context of clinical reasoning" and "developing a treatment/management plan" for students with no or some prior clinical experience.

Theme(s): Developing a treatment / management plan, Ethical aspects, Patient perspective
Level: Novice
Format: Asynchronous preparation (ca. 45 min) followed by a synchronous meeting (ca. 80 min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Generating differential diagnoses and deciding about final diagnosis

This learning unit introduces different methods of creating and organizing differential diagnoses. There is also opportunity to practice the finding of differential diagnoses as well as discriminating and confining features on a prototypical case.

Theme(s): Generating differential diagnoses, Decision Making
Level: Novice
Format: Two synchronous meetings (ca. 90 min each) with an asynchronous phase in between (ca. 60 min)
Recommended ECTS: 0.13
Links: Student course - Facilitator resources

Collect and prioritize key clinical findings/problems

This learning unit highlights how to collect and prioritize key clinical findings using case examples.

Theme(s): Gathering, interpreting & synthesizing information
Level: Novice
Format: Asynchronous preparatory phase (ca. 45 min) followed by a synchronous meeting (ca. 60 min)
Recommended ECTS: 0.06
Links: Student course - Facilitator resources

Biomedical Knowledge and Clinical Reasoning

This learning unit explains the interconnection of biomedical knowledge and differential diagnoses formulation and explores different techniques to visualize encapsulated knowledge.

Theme(s): Generating differential diagnoses
Level: Novice
Format: Two synchronous sessions (ca. 60 min each) with an asynchronous learning phase in between (ca. 45 min)
Recommended ECTS: 0.09
Links: Student course - Facilitator resources

Person-centered approach to clinical reasoning

The goal of this learning unit is to define what a 'person perspective' is in the context of healthcare provision and highlight why it is important when providing a quality healthcare experience. Learners will also visit the definitions of biomedical information as well as recite the terminology "diagnostic and analysis" in a way that helps patients and their families understand this stage in healthcare provision. Lastly, learners will combine the above into practical ability using questions that promote support for families and patients using terminology that facilitates mutual understanding.

Theme(s): Patient perspective
Level: Novice
Format: A synchronous meetings (ca. 90 min), with a preparatory and a follow-up asynchronous phase (ca. 90 and 45 min)
Recommended ECTS: 0.13
Links: Student course - Facilitator resources

Collaborate with others in clinical reasoning

This learning unit provides basic knowledge within different healthcare professions or across medical specialisations e.g. surgery, internal medicine and their collaboration with others in clinical reasoning.

Theme(s): (Interprofessional) Collaboration
Level: Advanced
Format: Asynchronous online preparation (ca. 60mins) for a follow-up synchronous meeting (ca. 90 mins)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Health profession roles in clinical reasoning

This learning unit provides an introduction to the various health professions involved in health care on the subject of clinical reasoning. The unit has been developed for beginner and novice learners, and is appropriate for those who have and have not yet had extensive clinical experience due to the team aspect of the assessments. The learning unit will highlight varied professions ranging between physiotherapy, medicine, nursing, and occupational therapy and learners will be able to compare and contrast the definitions of clinical reasoning within said professions, as well as relate how this team-understanding fits into the broader picture of healthcare, with a goal to establish a common understanding and definition of 'clinical reasoning'.

Theme(s): (Interprofessional) Collaboration
Level: Novice
Format: Asynchronous preparatory phase (ca 45 min) followed by a synchronous meeting (ca. 90min)
Recommended ECTS: 0.08
Links: Student course - Facilitator resources

Using the Outcome Present State Test Model

This learning unit provides an introduction into the the Outcome-Presenter-State model for clinical reasoning, which is applied especially in nursing. The learning unit is designed for novices of all health professions who are at the beginning of their education.

Theme(s): Theories of Clinical Reasoning
Level: Novice
Format: Asynchronous preparation phase (ca. 80 min) with a follow-up synchronous meeting (ca. 90 min)
Recommended ECTS: 0.09
Links: Student course - Facilitator resources

Illness Scripts

This learning unit provides an introduction into scripts in general and more specifically into llness scripts for novices who are at the beginning of their education and do not have any prior knowledge or experience with illness scripts.

Theme(s): Theories of Clinical Reasoning
Level: Novice
Format: Two synchronous sessions (ca. 160 min) with an asynchronous learning phase (ca. 60 min) in between
Recommended ECTS: 0.12
Links: Student course - Facilitator resources

Dual Process Theory

This learning unit provides an introduction into the dual processing theory amd highlights the differences between system 1 and system 2 reasoning.

Theme(s): Theories of Clinical Reasoning
Level: Novice
Format: Asynchronous preparation phase (ca. 90 min) followed by a synchronous meeting (ca. 90 min)
Recommended ECTS: 0.1
Links: Student course - Facilitator resources

What is clinical reasoning and how can theories be put into practice

This learning unit covers how clinical reasoning theories can be used/applied during beside teaching, internships or other patient-centered situations and why it is important to know these theories. It deepens the differences and similarities of clinical reasoning in the health professions, terminology used and importance of clinical reasoning.

Theme(s): Theories of Clinical Reasoning
Level: Intermediate
Format: Asynchronous preparation (ca. 45 min) for a follow-up synchronous meeting (ca. 60 min)
Recommended ECTS: 0.07
Links: Student course - Facilitator resources

What is Clinical Reasoning - An Introduction

This learning unit provides an introduction into the topic for novices who are at the beginning of their education and do not have any prior knowledge or experiences with clinical reasoning.

Theme(s): Theories of Clinical Reasoning
Level: Novice
Format: Synchronous meeting (60 min) followed by asynchronous follow-up (60 min)
Recommended ECTS: 0.07
Links: Student course - Facilitator resources

Skip to content