DID-ACT Integration Guide

With Q1 rapidly coming to a close, we are racing through our final year of the DID-ACT Project with excitement and fervor. We have created so much over the past 2 years and have solidified a place to clearly consolidate the knowledge we have created and amassed. We therefore created the DID-ACT integration guide. We will continue to update it in an ongoing manner as we develop our integration guideline. 

DID-ACT’s Integration Guide

The DID-ACT Integration Guide is found under our Curriculum heading and is broken into 5 categories. The goal of this resource is to provide an overview of the curriculum in its entirety, a guide to getting started for both educators and students, as well as our additional resources and FAQ about the curriculum. 

Curricular blueprint 

The DID-ACT curriculum is incredibly succinct and provides content for both educators and students. In order to effectively maneuver through the learning units, we put together the curricular blueprint earlier in the project. This page provides this blueprint in the form of an interactive table where you can view all of the learning units, broken down by level, audience, and theme. Using this table will help you best organize your learning and facilitate finding specific learning opportunities either as a student or educator.

Clinical reasoning learning outcomes

Building from the blueprint, learners and facilitators are able to explore the themes and overarching learning objectives we have defined as a basis for our curriculum. There are 14 themes in our curriculum, ranging from topics like decision-making and attitudes toward clinical reasoning, to the basic theories around clinical reasoning. 

How to use the DID-ACT curriculum

For students, there are 25 learning units to navigate from novice to advanced-level students in health professions. This page holds many functions, one of them being the provision of the who, why, what and how around making this curriculum be all it can be. 

For educators, there are 8 train-the-trainer learning units. The content includes multiple aspects of how to teach clinical reasoning to students including models and theories, cognitive errors and biases, as well as differences and similarities within different health professions. To support all of this learning are virtual patients, group work, and facilitator guides. 

Clinical reasoning integration guideline

Our Integration Guide is still currently growing. The integral guideline provides insight on how institutions and educators can easily integrate content from the curriculum into their pre-existing learning structure. Coupled with examples from our pilots but also from interactions with associate partners and external stakeholders  that highlight real-world experience and application, this guideline is a key to the successful adoption of the DID-ACT clinical reasoning curriculum. 

If you have used our resources successfully, we would love to hear from you as your input and experience would be very valuable. Just leave a comment!

DID-ACT’s evaluation process for pilot implementations of the train-the-trainer courses

We are kicking off 2022 by building from our “Pilot Implementations on Clinical Reasoning” held from June 2021 until December 2022. This will provide added applicable information gathered from our recently published “Evaluation and analysis of learner activities of the pilot implementations of the train-the-trainer course“.

Evaluation for Quality Control

To ensure the quality of our curriculum’s development, our pilots accompanied a questionnaire for participants and facilitators. We are using this feedback to create necessary emphasis and/ or create a clearer final product for our learners. These responses were coupled with monitoring our chosen learning management system (LMS), Moodle, and virtual patient system, CASUS. DID-ACT’s six institutional partners took part in the evaluation by facilitating 9 pilot courses across Europe. In brief, approximately 100 teachers participated in the 5 clinical reasoning teaching topics in the train-the-trainer course pilots. Approximately half of the participants returned their evaluation questionnaires alongside 12 responses from facilitators. The results, discussed further here, coding was double-checked and disagreements were solved by consensus.  

Survey tools for clinical reasoning curriculum assessment

In our pilots we made the decision to use survey-based tools for measuring the train-the-trainer (TTT) and students courses. Our goal was to capture responses using fewer questions in a way that allowed for comparison between piloted units. In the end, we used Evaluation of Technology-Enhanced Learning Materials (ETELM). This tool, developed by David Cook and Rachel Ellaway, gave us our launch pad to have questionnaires be our standard evaluation tool. This was attractive for many reasons, including facilitating implementation into our learning unit template within our LMS.

Lessons learned around evaluations for our pilot implementation

Through iteration and collaboration with a psychologist, experienced educators, and researchers, we found the following pertinent for our project and beyond: 

  • Ensure you are using consistent language; i.e use either ‘course’ or ‘learning unit’, as pertinent to your project
  • Be mindful of using the word ‘practice’ as it can be interpreted many ways; i.e in DID-ACT’s case, we changed, “This learning unit will change my practice” to “This learning unit will improve my clinical reasoning” 
  • Providing participants an option to write free-text in their native language, as a project allows 
  • Avoid too many questions that may lead to overloading participants 
  • Asking about years in a profession versus age provides more succinct answers for what we needed.

 TTT Pilot Implementation Survey Results

We set up our questionnaires using a scale of 1 (definitely agree) to 7 (definitely disagree). The average score of responses was 5.8 when prompted with the question about whether the courses would improve the teaching of clinical reasoning. The pilots excelled  in the areas of selection of topics, small group discussions, the facilitators, and inter-professional learning opportunities. Growth was suggested in the areas of technical navigation of the LMS, assessment and feedback on process, and content that was tailored more to professions other than physicians. 

Analysis of Pilot Implementations

The survey questionnaires were analyzed on Microsoft Excel where, using quantitative methodology, we calculated the descriptive statistics. In contrast, for open-ended responses, we performed a content analysis. Participant utterances were coded with the categories proposed in D3.2 (Didactical, Content, Technical, Interaction/ Collaboration, Implementation/ Time, Implementation /Facilitators). As well, we extended by adding three more categories (Content/Assessment, Overall and Others). All data was processed anonymously with each statement being set as positive, negative, or neutral. 

Overall, the TTT pilot implementations were a success as well as were our efforts in evaluating them. We will implement constructive feedback applicable to other learning units as we continue to develop them. Alongside this, we will return to the original pilot implementations and amend what needs to be improved. You can read a more detailed overview of D5.2 Evaluation and analysis of learner activities during these TTT pilot implementations here.

A Review of Reviewing Itself: Improvements on DID-ACT’s Learning Unit Review Process

The DID-ACT project’s in-person September meeting in Bern, Switzerland, brought forward many interesting insights and opportunities for streamlining tasks. Aspects of effective project management in our development of a clinical reasoning curriculum were brought up a few times. Some key takeaways were small, like how to more clearly manage our folders using the feedback from the interim report. Larger topics, like tools for effectively writing blog posts and reports, were also brought up. These tools help to ensure the language of posts are at an appropriate audience level. One of our largest takeaways was how to streamline the review process for our learning units (LU) in a way that was more time-efficient and thorough.

Streamlining the curriculum review process

Our previous process for reviewing the learning units developed was to set up a small working group. Groups would be given a week or two for review, then come together to discuss our thoughts using a standardized review template. The team who developed the learning unit would then implement the necessary implementations based on feedback. Following this, there was a final review opened to the group.  We noticed a few downfalls to this method: 

  1. It took many weeks to get the review done due to requiring a sync between the entire team;
  2. Things slipped through the cracks upon the more scrutinized review that the Moodle implementation required; 
  3. By not being as high a stakeholder as, for example, someone who was testing the learning unit themselves, reviewers were not as engaged as needed for proper scrutiny.

This third point was the experience of one of the EDU teammates when preparing to implement the Person-Centred Care learning unit for trainers.

New review process for clinical reasoning learning units

omething many of us know about preparing anything is that running through it in detail, as close to how it will be used in real life, is a key part of ensuring you are producing a quality item. This is exactly the circumstance Jennifer and Daniel found themselves in when preparing their PCC learning unit pilot. Despite having made it through the pre-described learning unit review process, tiny errors slipped through. Ideas around how to more effectively use time, adequate prep for an activity, and Moodle implementation itself were all aspects of the curriculum we could streamline before the actual pilot. The EDU team brought this experience forward to the consortium at the Bern meeting and from this fruitful discussion came the following modified review process: 

  1. LU’s are to be completed in batches
  2. When a working group has their LU prepared for review, they email the consortium and a review group of 2-3 people, including 2 of the authors, is formed using a Google doc sign up sheet
  3. A review date is agreed on
  4. At this point, the team is to review the learning unit asynchronously
  5. Upon meeting synchronously, one of the authors is to proceed to run through the learning unit as if they were piloting it. The second is to take notes of their own as well as marking the reflections of the ‘learners’
  6. Amendments are to be made and when completed go through a final review before being added to Moodle. 

This modified review process helps to ensure that there is a stronger stakeholder in the review, the person who has to actually teach it to their teammates. We will keep you posted on how it goes!

For any questions regarding this process, please contact us!

Curricular Framework for DID-ACT

We published our most recent deliverable this week: Developing, implementing, and disseminating an adaptive clinical reasoning curriculum for healthcare students and educators.

Team meeting to discuss the framework and work in virtual small groups

The goal of this deliverable was to provide our curricular framework with teaching/ assessment methods for the student curriculum and the train-the-trainer course.

Having already established our initial needs assessment and definition of goals and objectives (Deliverable 2.1), we have reached the exciting point of providing educational strategies in terms of a curriculum framework for clinical reasoning. We followed the constructive alignment theory to ensure an optimal alignment of learning objectives, teaching, and assessment. We have employed a theme-based approach. We plan to continue using a blended-learning format to help ensure flexibility for our learners while also utilizing an optimal match of teaching and assessment. 

Blended learning combines online activities, such as virtual patients and interactive videos, with face-to-face methods such as bedside teaching. We aim for our courses to have the learner at the centre, meaning that the student is actively engaged in their learning. In this set up, the teacher is more to support and facilitate learning. 

Some of our biggest wins in this work package have been:

  • Defining 35 general learning objectives in D2.1 and aligned them in 14 themes/ categories to describe the DID-ACT student curriculum and the train-the-trainer course.
  • We have defined four different learner levels: Novice, Intermediate, Advanced, and Teacher. 
  • Our list of suitable learning and assessment methods that align with our previously defined categories.
  • A breakdown of our teaching and learning assessment strategies for clinical reasoning clearly defined. 
  • Overarching curricular outline for the categories, theories, errors, and aspects of patient participation related to the clinical reasoning process. These outlines include the specific learning objectives, teaching and learning activities, as well as assessments, both summative and formative, for our courses. 

Our most recent deliverable is a big step as it establishes the framework for the next steps in our curriculum development process. Our team is both multi-professional international; thereby reflecting the needs of the different health profession curricula and curricular formats of the partner schools. Due to the current COVID-19 pandemic, we could not organize as originally planned in a face-to-face meeting to discuss the framework. However, we were able to organize the work in small groups across professions and contexts who worked asynchronously and met online according to their needs. In addition, we held a series of online meetings to discuss specific aspects and make decisions in consensus. 

Read more about the deliverable in detail here.

What’s next? Coming later in December 2020 will be our “Collection of available Open Educational Resources (OER)”, “Publication of recommendations for learning objectives of a clinical reasoning curriculum”, and our “Set of evaluation and analysis tools”

You can keep track of what is upcoming in the project on our Results page, or by clicking here. 

Participants: Medical educators

Learning units: Person-centered approach and the role of patients
Level: Teacher

Description: Offered as part of the faculty development to faculty educators.
Mode: Part to the faculty development program with emphasize on the educational potential of clinical reasoning teaching techniques

Technical Integration: Access to the DID-ACT Moodle, synchronous sessions were held via the virtual platform NewRow.

Tips & Tricks:

Participants: Multi-professional educators (nurses, physicians, paramedics)

Learning units: What is Clinical Reasoning and Models
Level: Teacher

Description: Offered as additional faculty development opportunity to staff members of the Medical Education Department.
Mode: Option / additional part to the faculty development program with emphasize on the educational potential of clinical reasoning teaching techniques

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face.

Tips & Tricks: Give participants time to familiarize themselves with the learning material between the sessions. Do not put too much in one day - better to meet twice for shorter sessions. Face-to-face sessions lead to more productive discussions than Zoom meetings. Focus on discussion with the audience and on examples from practical teaching to illustrate the learning objectives.

Participants: Multi-professional educators

Learning units: Differences and similarities in clinical reasoning among health professions
Level: Teacher

Description: Offered as part of the faculty development program, it was held as a blended learning course with the synchronous phase online.
Mode: Part of the faculty development program with certificate

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held online via Zoom.

Tips & Tricks: Give good practical information how Moodle works and how the participants can find and work with the different assigments of the learning unit. Emphasize the importance to the participants to work with the different individual tasks, as the discussions will be more interesting and fruitful. Also important that all the participants from different occupations feel comfortable to meet and that they all are a part of the discussion. This is an important and maybe the primary task for the facilitator!

Participants: Multi-professional and international educators

Learning units: Clinical Reasoning teaching and assessment & What is Clinical Reasoning and Models
Level: Teacher

Description: Offered as optional learning unit for participants of the Master of Medical Education (MME) program in Bern/Switzerland.
Mode: Part of the faculty development program with certificate

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face.

Tips & Tricks: To foster a valuable learning experience it is indispensible to provide good case examples that are tailored to the needs and experiences of the participants, so that they can relate to their prior knowledge. Thus, the provided examples in this learning units might need some adaptations for your target group of educators.

Participants: Multi-professional educators across German-speaking coutries

Learning units: Differences and similarities in clinical reasoning among health professions
Level: Teacher

Description: Offered as part of the faculty development program at the University of Augsburg, but open to participants from Germany, Austria, and Switzerland. It was held as a blended learning course with the synchronous phase online. Participants were eager to exchange their experience and disucss their views across institutions and professions.
Mode: Part of the faculty development program with certificate

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held online via Zoom.

Tips & Tricks: To foster a valuable interprofessional experience the participants should represent a balanced mix of professions and also ideally the facilitators should at least represent two different professions. This allows a good discussion and also guarantees that the small groups can work interprofessionally. Our experience with a less-balanced group composition was that the over-represented profession dominates the discussions and it was quite difficult to counter-balance this.

Target group: Medical students in year 2

Learning units: Person-centered approach to clinical reasoning
Level: Novice

Description:The Learning Unit was run as an extra-curricular session as a virtual class. The facilitators were trained on this topic by attending the Train-the-trainer learning unit on "Person-centered approach and the role of patients".
Mode: Extracurricular activity

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held online via NewRow.

Tips & Tricks: Learning objective cross-referencing with the existing curriculum helps identify the level of integration. Encouraging and helping students to get familiar with Moodle before the session facilitates a smooth integration of the asynchronous phases.

Target group: Medical students in year 4 and 5

Learning units: Generating differential diagnoses and deciding about final diagnoses
Level: Novice

Description: The learning unit was integrated into a pediatric emergency department clerkship with a relation to virtual patients already used in this clerkship. Facilitators were trained by attending the train-the-trainer unit on "Information gathering, Generating differential diagnoses, Decision making, and Treatment planning".
Mode: Part of a regular curricular activity

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face.

Tips & Tricks: Integration into clerkships works well, especially with a relation to already used content.

Target group: Medical students in year 1-6

Learning units: All 25 learning units of the DID-ACT curriculum
Level: Novice - Advanced

Description:
Longitudinal integration of the DID-ACT learning units into a clinical skills & communication course with the following suggested distribution across years:
Year 1: What is Clinical Reasoning, Person-centered approach to clinical reasoning, Health profession roles in clinical reasoning, and Biomedical Knowledge & Clinical Reasoning. (Required time in curriculum ca. 5 hours / semester)
Year 2: Dual Process Theory, Illness scripts, Collect and prioritize key clinical findings/problems, and What is clinical reasoning and How can theories be put into practice (Intermediate). (Required time in curriculum: ca. 4 1/2 hours/semester)
Year 3: Generating differential diagnoses and deciding about final diagnosis, Biases and cognitive errors - an Introduction, Analyzing and avoiding errors. (Required time in curriculum: ca. 4 1/2 hours/semester)
Year 4: Using the Outcome Present State Test Model, Developing a treatment plan, Metacognition, reflection and models for reflection, Collaboration of health professions in Clinical Reasoning (Intermediate). (Required time in curriculum: ca. 6 hours/semester)
Year 5: All remaining intermediate learning units: Shared Decision Making in Clinical Reasoning, Decision Support Systems, Ethical aspects - patient management and treatment, Uncertainty. (Required time in curriculum: ca. 5 hours/semester)
Year 6: All 6 advanced learning units: Collaborate with others in clinical reasoning, Decision Support Systems, Biases and cognitive errors, Uncertainty, Metacognition, reflection and models for reflection, Analyzing and avoiding errors.
(Required time: ca. 6 hours / semester. Final year students often have a day/week off from clinical work so these days could be used for DID-ACT learning units and optional participation in the train-the-trainer units)
Years 1-5: Virtual Patients (VPs) as additional deliberate practice activities in increasing number and complexity (e.g. starting with 5 VPs (=ca. 1.5 hours)/semester) in Year 1 and increasing to 10 VPs/semester) in Year 3-5.
Mode: Integrated into a clinical longitudinal course that runs from year 1 to 6 resulting in a total of 2.4 ECTS.

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions face-to-face or online.

Tips & Tricks:
Alining a longitudinal curriculum with curricula of other health professions remains a challenge and requires careful and early planning, but the asynchronous phases might be a good starting point for interprofessional teaching sessions. The VPs can be quite easily aligned with other curricular courses e.g., based on key symptoms.

Target group: Medical students in year 3

Learning units: Person-centered approach to clinical reasoning
Level: Novice

Description: The learning unit was integrated into the Laboratory Training of Clinical Skills. This course consists of six face-to-face meetings for groups of around 10 students. Within the meetings the students discuss various topics related to communication skills and person-centered approach. The fifth of the six meetings of the course was replaced by the DID-ACT learning unit. Students could then refer back to this learning unit during final session of the course.
Mode: Part of a regular curricular activity

Technical Integration: Access to the DID-ACT Moodle via EduGain, synchronous sessions were held face-to-face. If possible, a uniform technology to provide access to the online resources in the class (using university tablets in our case) and support of the technical staff on-site was helpful to lower the technical barrier.

Tips & Tricks: Changing of the standard format of classes is interesting and motivating for the students. Virtual patients are a tool to present authentic clinical scenarios which are appreciated by the students. A blend of role-play and virtual patients allows students to take advantages of the two methods to reach the learning objectives.

Target group: Medical students in year 2 and nursing students in year 3 across Europe

Learning units: Collaboration of Health Professions in Clinical Reasoning
Level: Intermediate

Description:
This learning unit was implemented with , aside from clinical reasoning, tow additional objectives: 1) providing internationalization experience for students without traveling. 2) Deepening the knowledge of one's own professional roles and responsibilities and knowledge of another profession .
Mode: Extra-curricular interprofessional learning session with international participants

Technical Integration: Self-registration on DID-ACT Moodle, synchronous sessions via zoom.

Tips & Tricks: Facilitators running this course and supporting the discussions should be careful about supporting and encouraging all the participating professions' perspectives.

Target group: Medical students in year 1 and 2 (preclinical) across Europe

Learning units: Introduction into Clinical Reasoning & Health profession roles in clinical reasoning
Level: Novice

Description: The course was offered as a blended learning module to students from different medical schools in Europe as an international elective. Synchronous phases were held online via Zoom. The international aspect was very motivating for students and they learnt a lot from each other by exchanging their perspectives and how they are taught clinical reasoning vaspects. This teaching mode could also be implemented as an activity to welcome or prepare new Erasmus students. Facilitairs were trained by attending the train-the-trainer learning unit on "Differences and similarities in clinical reasoning among health professions".
Mode: Elective course

Technical Integration: Self-registration on DID-ACT Moodle, synchronous sessions via zoom and use of Padlet for interactivities.

Tips & Tricks: The organization with the registration of students was a bit complex, so, we suggest just setting dates and let students book. Allow enough (more time) for discussion and introduction rounds as participants do not know each other and are eager to hear and learn from peers at other schools and countries. Ideally, this session could also be held interprofessionally, however, it makes the finding of suitable dates even more complex.

How to teach synchronously in a virtual setting

  • You need a reliable camera, microphone, and virtual platform and be familiar with its features, such as whiteboard, chat, polling, breakout rooms, etc.
  • At the beginning establish communication rules, e.g. whether participants should raise their (virtual) hand, use the chat, and/or just speak. Also, we recommend asking participants to turn on their camera
  • For small group work break out rooms work very well, just be clear about the tasks the groups should work on prior to dividing them into the groups.
  • For collaboration the use of integrated virtual whiteboards or other platforms such as Padlet are very useful. Just make sure prior to the session that you have everything setup and the links at hand, e.g. to post them in the chat.
  • Allow a bit more time for starting the session and the group works as there might be participants who are not familiar with the platform or technical problems might occur.

How to motivate unprepared participants

  • Make clear that the asynchronous assignments are a core part of the course and that its content will not be repeated. Even if it is difficult, stick to that when starting the synchronous teaching session.
  • If you expect unprepared participants, you can start the session with a student-centered group exercise mixing prepared and unprepared students to increase peer-pressure and make them realize that being unprepared does not feel good.  
  • Use the introductory or closing quizzes / tests so that participants can self- assess whether they have the required knowledge and you as a facilitator can see the level of knowledge and preparation of your participants.

Further recommended reading:

How to involve participants with different levels of experience

  • To account for such different levels, we recommend making use of the asynchronous preparatory phases which also include introductory quizzes in which participants can self-assess their prior knowledge and you as a facilitator can assess the differences within your group. Participants with less prior experience can also be guided to additional preparatory resources.
  • Encourage participants to work in pairs or small groups when preparing so that they can help and learn from each other. You could even facilitate this by dividing them into groups with different levels of experience.
  • Similarly, during the synchronous phases, we recommend forming groups with participants different levels of experience and emphasize the peer support aspects of such group activities.
  • We also recommend starting with rather smaller groups and allow more time than stated in the course outlines, if you expect a heterogenous level of experience. This way you can better manage this challenge.
  • Encourage your participants to ask questions, emphasizing that nobody knows everything and that it is important for learning to ask questions.  
  • Especially in the train-the-trainer course you might have to deal with over-confident participants, who especially in an interprofessional setting can dominate the group. This is a complex cultural challenge, but you could try to establish (and follow) communication rules at the beginning of a session.  

How to address potential overlaps or redundancies

  • Identify what is already included and what is missing in your curriculum related to clinical reasoning outcomes and compare it to the DID-ACT blueprint. Prioritize learning outcomes that are not yet covered but regarded as important.
  • Identify activities, resources, or teaching sessions with similar learning outcomes that might be in need for change anyway because of low evaluation results, teachers or students struggle with it. These could be suitable for adding or replacing parts with DID-ACT activities.
  • Ask teachers and students about overlaps and gaps they see in their teaching / learning of clinical reasoning and where they struggle. This could also be done by a reflection round after related teaching activities in the curriculum
  • Although ideally a longitudinal integration is aimed at, we recommend to starting small with a pilot implementation to gain experience and develop a show case.

How to teach in an interprofessional setting

  • Allow for enough time prior to the teaching for the organization and motivation / encouragement of stakeholders and participants
  • Allow for enough time and guidance during the course so that the participants from the different professions can get to know each other and their professions and discuss their different perspectives. This might mean that you need to calculate some extra time in addition to the suggested duration of the learning unit.
  • There may be a different understanding of clinical reasoning in the different health professions, so we recommend making participants aware of this. You could for example use and adapt activities from the learning units on the health profession roles to facilitate this.
  • Courses in an interprofessional setting should not come too early in the curriculum (not before professions have formed their own professional identity - however, this also depends on the aim of the course). 
  • Make sure you have enough participants from different professions. If possible, the facilitator could divide the participants in smaller groups with an equal distribution of professions. 
  • Similarly, you need an equal distribution of facilitators / facilitators from different professions.
  • Develop customized learning materials considering the different professions. If needed you can adapt the material and activities provided in the DID-ACT curriculum.

Further recommended reading:

van Diggele, C., Roberts, C., Burgess, A. et al. Interprofessional education: tips for design and implementation. BMC Med Educ 20, 455 (2020). (Link)

Skip to content