The use of artificial intelligence and game-based learning in removable partial denture design: A comparative study

1 INTRODUCTION

Artificial intelligence (AI) is a collective term defined as the science and engineering of making intelligent machines and computer programs.¹ AI has become a common feature of today's world. From AI-powered online search engines to self-driving cars, we have come to rely on AI technology to complete our daily tasks.² Game-based learning is a concept used to describe teaching methods that adopt game mechanics to enhance student learning.³ Gamification and game-based learning have been shown to increase user engagement,^4-7 and critical thinking skills.⁸ This is achieved through the ability of gamification techniques to create a low-stakes learning environment that encourages experimentation.^{7, 9} AI is an essential component of video games, as it provides the algorithms that dictate the rules of the game.¹⁰ Since its inception, many great advancements in AI have been achieved through ongoing video game development.¹¹

The healthcare sector in general has also seen many advancements in AI implementation¹² such as early detection of disease as well as medical image processing.¹³ In dentistry, AI has been used to assist in restoration design,^14-18 interpretation of radiographs,¹⁹ implant type detection,²⁰ root fracture detection,²¹ and periodontal status diagnosis.^{22, 23} Treatment planning and decision-making are one of the most critical steps in dental treatment. Findings suggest that there are substantial variations between dentists in both diagnosis and treatment decisions.²⁴ Moreover, findings show deficiencies in concordance between clinical practice and evidence-based dentistry.²⁵ Those issues are especially magnified in dental education where calibration of dental faculty is an ongoing challenge.²⁶ Such challenges arise mainly due to differences in faculty training and experience that oftentimes results in a degree of subjectivity or bias. Conversely, the strengths of AI systems include their ability to reliably produce consistent results over time,²⁷ as well as maintain objectivity and immunity to bias.^{28, 29} Therefore, AI presents itself as a possible solution to overcome inherent human deficiencies. Removable partial denture (RPD) design is an area in dental treatment planning that has been shown to exhibit great variability between practitioners.^{30, 31} There is growing evidence that AI-assisted education has the potential to improve medical student learning.^32-34 There is also evidence that suggests that gamification techniques may improve health professional education.³⁵ Additionally there is evidence to support that innovation and change can have a positive impact on education.³⁶

Designing RPDs is a highly complex task, but it is based on a set of rules. A novel logic-based RPD design software was developed called AiDental.³⁷ The purpose of the study was to compare student performance in RPD design during a pre-clinical RPD course with and without using the AiDental software. Additionally, student perceptions associated with the use of this software were assessed. The current study tested the hypothesis that students who use AiDental software as an adjunct to traditional course material will achieve higher test grades when compared to students who have only received traditional course material. The study also tested the null hypothesis that there are no associations between students’ perceptions of AiDental software and their age or previous experiences with gamification techniques.

2 MATERIALS AND METHODS

The AiDental RPD Designer is an Algorithm based software with basic AI functionality as a decision-making tool for RPD design. AiDental's algorithms were coded with the logic of RPD design, allowing the software to process data input from the user and design RPD frameworks. Currently, the software supports the RPD design philosophies of McCracken's Removable Prosthodontics,³⁸ Stewarts’ Removable Partial Prosthodontics,³⁹ and Stratton's An Atlas of Removable Partial Denture Design.⁴⁰ The AiDental software allows input of a given case scenario, including missing teeth and abutment tooth conditions like tooth undercut location and depth, vestibule characteristics, patient esthetic requirements, and occlusal contact location. The software analyzes the number and distribution of missing teeth and provides an illustration and a list of compatible RPD designs including components such as clasps, major connectors, minor connectors, and rests (Figure 1). An additional module was developed which adds a layer of automated feedback to the software which is called the “AiDental game” (Figure 2). The AiDental game provides the user with a randomized RPD case. The user then selects the different RPD components, including abutment teeth, major connectors, rests, and clasps based on the provided abutment conditions. The AiDental game then compares the user's proposed answer to the ideal answer design and provides feedback as well as a grade that reflects how closely the user's design matches the ideal answer. The AiDental designer and game are web-based software that can be accessed from any internet-capable device with no need for downloads or installation, therefore minimizing issues with compatibility.

An Institutional Review Board (IRB) approval was obtained for this study (IRB 202201016/202101217). The current study was carried out during an RPD pre-clinical course at the University of Iowa College of Dentistry. The inclusion criteria of the potential participants are; a second-year (D2) student currently enrolled in the RPD pre-clinical class. A total of 80 D2 students were invited to participate in the study. Seven students declined the invitation, resulting in the study enrolling a total of seventy-three D2 students from the University of Iowa, College of Dentistry.

Before the study, faculty and instructors received training on using the software and were informed of the philosophy that will be used in the course. All participants were introduced to the University of Iowa, College of Dentistry's RPD design philosophy.

The current study was designed to include two phases. For phase one of the study, participants were randomized by gender and equally assigned into the AiDental group and the conventional group. To do that, the College of Dentistry Information Technology Department (IT) generated a unique Study ID for each participant. Participants received an E-mail from a University of Iowa E-mail address to their university E-mail address assigning them randomly to groups (i.e., AiDental group or conventional group), with equal gender distribution between the two groups. Directions related to the study were also included in the E-mail. The principal investigator (PI) was blinded from the study ID generation, and the group assignment and was not involved in student interaction for the duration of phase one of the study. Both groups experienced traditional RPD course content in the classroom referencing Stewarts's Removable Partial Prosthodontics³⁹ including traditional practice exercises and homework. The AiDental group was granted access to AiDental software as an adjunct to their conventional RPD coursework including the ability to use the AiDental game module which provided a virtually infinite supply of scenarios with model answers and feedback. Participants in the AiDental group were instructed to only use the software outside the classroom and not share information from the software system with others. By not sharing information with others, validity could be maintained within the study. Both groups had 2 weeks to prepare for a mock exam. Exercises were provided equally to both groups during the RPD class (8 h a week). Homework exercises were also given to both groups. However, the AiDental group had additional access to AI-generated exercises to practice at home. During these 2 weeks, no interaction was made with the PI to maintain blinding. During the mock practical, participants of both groups were asked to design one maxillary framework and one mandibular framework on paper and without the aid of notes or the AiDental software. Participants were informed that their scores for the mock practice test would have no impact on their course grades. The unique study ID was used on the practice sheets and no identifying information was used. After the mock practical exam, all answer sheets were collected by a study coordinator and graded by the Principal Investigator. Grading was done using a rubric where each arch is graded according to categories of the correctness of design choices, the accuracy of the drawing, correct mention of the purpose of the components used, and neatness. Each category was evaluated according to the rubric for needs for improvement, meeting expectations, or surpassing expectations. The maxillary and mandibular arches were scored separately on a scale of 0–4 consistent with the grading scheme used by the students in all other pre-clinical course work with a score of 0 representing no points awarded and 4 representing a perfect score for each arch. Afterward, an average was made of both arch design scores to result in one final grade. A score of 3.6–4 equates to a letter A grade, 3.2–3.59 equates to a letter B grade, 2.8–3.19 equates to a letter C grade, and any score below 2.8 equates to a letter F grade.

During the mock practical, participants of both groups were also asked to indicate their age and gender information and previous experience with gamification techniques in education (1 = low, 2 = moderate, and 3 = high).

A results spreadsheet was created containing each participant's university ID and their associated score. The results spreadsheet was given to the College of Dentistry IT Department, where a key was used to correlate the university ID with the study ID. The IT Department also performed randomization on the spreadsheet to ensure blinding. A University of Iowa statistician then analyzed the modified results spreadsheet.

In phase two, all participants in the class were given access to the AiDental Designer Software as an adjunct to their pre-clinical RPD education. The RPD course continued for an additional two months and on the final day of the course, all participants received a survey that was developed to emulate surveys and scoring systems used by the Accreditation Council for Graduate Medical Education.^{41, 42} Furthermore, to evaluate the validity of the survey questions, five faculty members teaching prosthodontics and operative dentistry were asked to review the survey and evaluate the relevance of the survey questions to the aim of the study. Faculty members were given the opportunity to rate questions as relevant or irrelevant. Questions with less than 80 Percent approval of relevance were excluded from the survey. Question 1 asked for the participant's age. Question 2 evaluated the participants' experience with gamification techniques in education. Question 3 comprised five questions on the perceptions on utilization of AiDental Designer Software which evaluated: 3.1) ease of use of the software, 3.2) whether the software improves decision-making and critical thinking relative to RPD design choices, 3.3) whether the software could be used as a reference for RPD design philosophies, and 3.3 and 3.4) whether the participants wanted to experience more AI and gamification techniques in education. Except for age, all responses were measured on a scale of 1–10 with 1 for strongly disagree to 10 for strongly agree. Additionally, participants had the option to leave comments in their own words on the advantages and disadvantages of using the software.

Descriptive statistics were conducted to provide an overview of participant characteristics in the study. Variables of interest related to perceptions of both the AiDental designer and game were assessed using Spearman's rank correlation test. The associations of use of the software with each exam item score and final grade or associations of the final grade, previous experience with gamification, and education benefit were evaluated using the chi-square test, Fisher's exact test, and non-parametric Wilcoxon rank-sum test as appropriate. The Shapiro-Wilks test was applied to verify the assumption of data normality when the nonparametric statistical procedure was carried out for the ordinal variables. Moreover, a thematic analysis was conducted on the responses to the optional comments section using the Braun and Clarke method.⁴³ All tests employed a significance level of 0.05, and statistical analysis was conducted using the statistical package SAS System version 9.4 (SAS Institute Inc., Cary, North Carolina, USA).

3 RESULTS

Seventy-three D2 dental students (46 females and 27 males), were randomly assigned into the AiDental group with software access and the conventional group (n = 37) without software access. The mean age of all participants was 24.3 ± 1.9 years, ranging from 22 to 32 years, and the mean ± SD/median score of previous experience with gamification techniques in education was 4.31 ± 2.50/4.00. Note that no significant difference in gender and age between the two groups, with (p = 0.524 by the chi-square test and p = 0.574 by the nonparametric Wilcoxon rank-sum test respectively (Table 1).

The data from phase one of the study revealed that subjects in the AiDental group were more likely than those in the conventional group to get a final grade of A or B. Participants in the AiDental group were also more likely to achieve a “surpass expectation” grade on the correctness of maxillary or mandibular designs and the purpose of maxillary components.

Considering variables collected at phase one from all groups, The analysis showed no associations of final grade with gender, age, and previous gamification experience. Moreover, of the thirty-six subjects who had software access, 80% had low previous experience with gamification, 63.9% enjoyed or enjoyed greatly using the software, and 88.9% felt the software was somewhat or very beneficial to their understanding.

Table 2 presents phase two descriptive statistics of survey data on the perceptions related to the use of AiDental software. According to a 10-point rating scale with 10 indicating the highest level of agreement, the mean∖median scores of responses to the survey questions from the highest to the lowest were: question 3.4 (mean = 8.85), question 3.3 (mean = 8.79), question 3.5 (mean = 8.61), question 3.1 (mean = 7.72), and question 3.2 (mean = 7.48).

The correlations of age or previous gamification experience with each perception rating score were evaluated using the Spearman correlation test presented in Table 3. The data showed that age was significantly negatively correlated with question 3.1, ease of use of the software (r = −0.33, p = 0.005); question 3.2, the software improves decision making and critical thinking relative to RPD design choices (r = −0.24, p = 0.043); question 3.4, the desire of the participant to experience more AI in education (r = −0.28, p = 0.017); and question 3.5, the desire to experience more gamification techniques in education (r = −0.27, p = 0.024). However, no correlation between age and the use of the software as a reference was noted (p = 0.198). Additionally, no significant correlation was found between previous gamification experience and each of the perceptions regarding the AiDental designer and game (p > 0.05 in all instances).

It was found that 33 participants optionally wrote comments related to the “advantages” question and 29 participants wrote comments related to the “disadvantages” question. Results of the thematic analysis show that the participants mentioned the advantages of using the software as follows: 1) provides practice (12 mentions); 2) ease of use (10 mentions); 3) aids in understanding RPD (six mentions); 4) fast feedback (six mentions); 5) use as a reference (six mentions); and 6) less human error (3 mentions). Results of the thematic analysis related to the disadvantages of the software are as follows: 1) insufficient feedback for incorrect answers (nine mentions); 2) graphics not appealing (five mentions); 3) inability of the user to draw in the game (four mentions), and 4) lack of a tutorial (four mentions). Results of the thematic analysis regarding the advantages and disadvantages are summarized in Tables 4 and 5, respectively

4 DISCUSSION

The current study evaluated the use of AI and game-based learning in RPD design. The AiDental software was built with algorithms that represent known design philosophies to be used as a reference guide during patient care, a feedback tool for students, and a calibration tool for faculty. AI use in education has been rising in popularity. In the latest Horizon report, the advantages of using AI in education include improving student performance and improving the student learning experience.⁴⁴ Examples of the use of AI in RPD design have been mentioned in the literature. Davenport et al.^{45, 46} and Beaumont et al.⁴⁷ created an early logic-based software that could delineate connections between missing edentulous spans, the project later evolved to incorporate traversing three-dimensional space on virtual models of partially edentate arches.⁴⁸

Chen et al.⁴⁹ created a case-based decision tree, which was able to output recommendations on major connector and clasp assembly selection for RPDs in text format based on the arrangement of missing teeth. However, abutment tooth characteristics like undercut depth and location were not considered and did not affect the outcome of the design. Further development of the software allowed for the display of major connectors and clasp assemblies on an electronic health record.⁵⁰

The findings of the current study indicate that 60% of participants in the AiDental group achieved a grade of A or B compared to only 31% of participants in the conventional group. This difference is statistically significant and can be attributed to the fact that participants in the AiDental group were able to receive significantly more practice through the software than the participants in the conventional group. This is possible because the software can produce a virtually infinite number of RPD scenarios with corresponding model answers. Moreover, the AiDental group with software access received instant and consistent feedback on their choices of abutments, clasps, rest seats, and major connectors. In contrast, participants in the conventional group had a limited number of practice scenarios provided during class, and feedback was only received during class hours. Standardization between groups in practice time was not possible since instructors cannot provide an infinite number of practice exercises and feedback to students. The ability to provide infinite amounts of exercise and feedback is among the advantages of automated feedback and gamification software like AiDental.

A 10-point Likert scale survey was given to participants at the end of class which evaluated participants’ perception of the software. Participants had a generally positive perception of the ease of use (question 3.1) and the ability of software to improve decision-making and critical thinking relative to RPD design choices (question 3.2). However, it was noted that this score had a negative correlation with participant age. This indicates that younger participants perceived the software more favorably than older participants. This is in line with findings from Czaja et al.^{51, 52} Sharit et al.,⁵³ Ellis et al.,⁵⁴ and Tacken et al.⁵⁵ that age was negatively associated with computer knowledge and computer interest and positively associated with computer anxiety. Conversely, results showed a high agreement with the statement that the software can be used as a reference for different RPD design philosophies (question 3.3) with no negative correlation with age. This suggests that while age might play a factor in the ease of use, most participants identify the software as a useful reference for RPD design philosophies regardless of age. Questions 3.4 and 3.5 asked if the participants agreed that Dental education should include more AI and “gamification” techniques respectively. The findings indicate that the mean scores were 8.85 for question 3.4 and 8.61 for question 3.5, which indicates that participants largely believe AI and gamification should be further implemented in dental education. However, because question 3.4 was negatively correlated with age, there may be a suggestion that older age groups might see AI as a possible threat to employment as seen in the findings by Walsh et al.⁵⁶ There was no correlation between question 3.5 and age, which suggests that participants of all age groups largely feel that gamification should be used more often in dental education. Evidence shows that game-based learning has been used in medical education⁵⁷ however, there is little documentation on game-based learning being applied in dental education.

Borit et al.⁵⁸ used a board game model where students interact by creating questions that are then answered by their peers. Results show that the use of game-based learning increased students’ attentiveness, motivation to learn, and enjoyment. Clinical implications of this AiDental software include using it as an adjunct to decision-making in clinical practice. The software can also be used to standardize and calibrate designs within each philosophy as well as identify distinctions between them.

It was found that more participants left comments on the question of the advantages than they did on the disadvantages question. The results of the thematic analysis show that the majority of participants identified advantages of the software to be its ability to provide practice exercises as well as ease of use. This corroborates the findings of the current study because participants in the AiDental group (who had unlimited practice exercises) had statistically significantly higher grades than those in the conventional group. Additionally, participants indicated that the software helped their understanding of RPD, provided fast feedback, and could be used as a reference. Additionally, they noted that it reduced the possibility of human error. The majority of the participants found that the disadvantage of the software was its lack of written explanation of incorrect answers. While the software can flag an incorrect choice along with displaying the correct answer, it does not explain why the answer is incorrect. This lack of explanation might not be a problem in an educational setting since students are usually expected to study material provided in class independently or with the help of the instructors. However, during patient treatment where the software may be used, an explanation of incorrect answers might be a more valuable function.

In regards to the interface, participants found that the graphics were not appealing on the software. The graphics were limited to what was necessary to achieve a usable and beneficial web-based learning environment. By limiting the fidelity of the graphics large downloads and slower response times could be avoided. Another issue that might arise is the color scheme used for RPD design components that might differ from what the participant or institution normally uses. Lastly, participants found the inability to draw within the software and a lack of training tutorials to be among the disadvantages, which are recommendations to help improve the usability of the software.

It is important to acknowledge the limitations of the current software and study. Most available RPD design philosophies provide general guidelines rather than specific design choices for every possible missing tooth combination.^38-40 While philosophies themselves might have a degree of bias based on the specific views of the authors, the algorithm of AiDental provides an objective reflection of the selected philosophy that is consistent and unbiased. The AiDental RPD designer aims to standardize designs associated with different philosophies. Another limitation of the software is the inability to include every possible treatment plan-altering consideration, such as operculum, short clinical crowns, patient preferences, pre-existing coronal attachments, and periodontal implications. Final design choices must be made by the dentist overseeing all the different factors contributing to the final design. AiDental should only be used as an adjunct reference to the dentist during RPD design.

Limitations of the study include the relatively small sample size which may reduce the statistical power to detect potentially significant associations. Another possible limitation was the inability to test the grading module within the RPD game. The AiDental game comes with a grading feature that can score the student's proposed answer against the model answer provided by the algorithm. Due to the restricted timeframe of the study, there was insufficient time to thoroughly evaluate this feature. Another limitation of this study was the fact that participants in the AiDental group received more practice, in total as compared to the conventional group. However, one of the main benefits of using AiDental as an educational tool is the ability of students to self-regulate the amount of practice they need beyond the classroom. The AiDental group was able to use the software beyond the classroom to gain extra practice. There is no conventional analog to this, so a similar situation for the conventional group was not possible. Another limitation of the study is the possibility that the introduction of the AiDental software as a new tool could have increased student interest and therefore prompted more usage and consequently more exercises for the AiDental group. This novelty factor was difficult to avoid because to test the effectiveness of the software on performance in RPD, there should not be any prior exposure of the control participants to the application. Other issues include time constraints associated with the curriculum, along with the close proximity of case and control, since they were both within the same class. Another limitation of the study was the focus of the hypothesis on the relationship of participant perception to participant age and previous experiences with gamification techniques. Future studies may address other possible associations of perceptions to the use of the software.

5 CONCLUSION

The current study compared student performance in RPD design in a pre-clinical RPD course with and without using AiDental software. It also evaluated student perceptions associated with using this software. The results showed improved student grades when using the software. Furthermore, the results of the phase two survey revealed positive student perceptions of both the designer and the game. Perceptions of participants point to the advantages of using the software such as aiding in the understanding of RPD, providing practice cases, providing fast feedback, reducing chances of human error, and providing reference capability. However multiple disadvantages have been noted as well, including the lack of detailed explanations for the incorrect answers, unappealing graphics, the user's inability to manually draw in the game, and the lack of tutorials for beginners. Those findings highlight the potential of the use of AI in prosthodontic and dental education. Time-consuming or exhausting tasks such as practice case creation and grading can be streamlined, and the instructor's energy diverted to addressing students’ deeper understanding of complex concepts. Future research on the current software should be directed towards evaluating the grading module of the game as well as evaluating the use of the software in a clinical setting. Future research for AI in prosthodontics in general should aim at exploring different avenues where optimizations can be made using the power of AI technology. Based on the results of the study, the authors accept the hypothesis that students who use the AiDental software as an adjunct to traditional course material will achieve higher grades when compared to the students who have only received traditional course material. However, the authors reject the null hypothesis that there are no associations between students’ perceptions of AiDental software and their age or previous experiences with gamification techniques. Further studies will be needed to confirm the findings of the current study.