2.1 Selection of Target Large Language Model (LLM) Chatbots

The selection of the LLM chatbots used in this study was based on the target groups that we defined as learners and educators with an interest in insects and their conservation but limited prior knowledge. To address this group, we chose the most popular LLM chatbots, which are easily accessible to the public and have models with tens of billions of parameters (Kasneci et al. 2023; Zhao et al. 2023; Helbling et al. 2024). We tested the following 10 large language model chatbots between July 15, 2024, and August 8, 2024, which at the time of the study were exclusively US-based: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o (all three from OpenAI, San Francisco, USA), Llama 3.1. (Meta, Cambridge, USA), Gemini (Google, Mountain View, USA), Copilot (conversation style set to precise) (Microsoft, Redmond, USA), wikichat: https://wikichat.genie.stanford.edu/ (style set to most factual and not balanced; therefore operating on ChatGPT-4) (Stanford University, Stanford, USA; Semnani et al. 2023), Claude 3.5 Sonnet (Anthropic, San Francisco, USA): https://claude.ai/, and Perplexity Standard and Perplexity Pro (both from Perplexity.ai, San Francisco, USA).

The hypotheses and corresponding prompts used are shown in Table 1. The responses provided by the 10 LLM chatbots to these hypotheses are given in the Supporting Information File S1.

2.2 Selection of Insect Groups for the Investigation

The main aim of this study was to compare the responses of major LLM chatbots in relation to bees and wasps, with the hypothesis that bees are generally liked while wasps are often disliked, similar to the way society's responses to these two groups were studied by Sumner et al. (2018). In addition, we also included other insect groups for comparison, some of which are generally liked and others which are generally disliked. To this end, we categorized lepidopterans (which are generally liked) into two groups: butterflies (liked) and moths (disliked). For dipterans (generally disliked), we divided them into two subgroups: Brachycera (disliked) and Nematocera (strongly disliked). Since our study targets learners and teachers with limited prior knowledge in a context where the scientific names of these groups are less familiar, we address Brachycera in the prompts as “flies” and Nematocera as “mosquitoes”.

To obtain a qualitative assessment of each insect group to test Hypothesis 1, we prompted each LLM chatbot to provide 10 words associated with bees, wasps, butterflies, moths, flies, and mosquitoes, respectively.

The output words were corrected into the singular form (e.g., “pollinators” to “pollinator”) and standardized for similar words with the same meaning (e.g., “beauty” to “beautiful”). We then compiled word clouds, in which the size of a word is proportional to its frequency of occurrence for each insect group. Word clouds were generated using the layout algorithm implemented on https://www.jasondavies.com/wordcloud/ to visualize the frequency of occurrence of each word. We selected the following parameters: spiral = rectangular; 4 orientations from −40° to +40°; scale = n.

To evaluate the emotional tone and sentiment toward the predefined list of words related to each insect group, we performed sentiment analyses using R Software. First, words from each list were classified as either positive or negative through an inner join operation using the established “Bing sentiment lexicon” (Hu and Liu 2004). This analysis was then cross-verified using the “NRC Emotion Lexicon” (Mohammad and Turney 2013), which categorizes each word into two sentiment categories (positive and negative) as well as eight basic and prototypical emotions defined by Plutchik (1991). The results of the NRC analysis, which include only those words from the predefined lists that were present in the lexicon and could thus be assigned one or more sentiment categories, are visualized in a bar plot created using “ggplot2” to display the distribution of sentiment types across the predefined words.

To test whether the LLMs have adopted the human bias that favors bees but disfavors wasps and whether there is a similar preference for butterflies over moths and flies over mosquitoes (Hypotheses 2 and 3), the selected LLM chatbots were prompted to rate these insect taxa on a scale of −5 (strongly negative) to +5 (strongly positive) based on their training data. The responses of the 10 LLM chatbots are visualized in boxplots. Due to the small sample size (n = 10) for each taxon, we refrained from further statistical analyses.

To test which insect groups LLMs suggest as vital for conservation (Hypothesis 4), we prompted the LLM chatbots to provide a list of the 10 insects they consider the highest priority for conservation. The LLMs listed these in order, and points were assigned based on their position. The first group received 10 points, the second group one point less, and so on, with the last group receiving one point.

If multiple insect groups were named in the same position (e.g., Hymenoptera: bees, wasps, ants), each group received the same score (e.g., if all were listed in position one, they each received 10 points). If a taxon was mentioned multiple times within the list by a single LLM chatbot, points were awarded for each position (i.e., positions seven and eight would yield a total of seven points, with four points for the 7th position and three points for the 8th (4 + 3)). The points were then summarized for each insect group, and a bar chart was created in Microsoft Excel for Mac (Version 16.78.3).

To test which insect species LLMs suggest as important for nature conservation (Hypothesis 5), we asked each LLM chatbot for the ten insect species they considered most important for conservation efforts. We obtained a total of 100 species, which were reduced to 88 valid species after removing 12 irrelevant species (including non-insect species such as spiders and birds, extinct species, and LLM-generated hallucinations of species that do not exist). Next, we assigned the zoogeographical realms of these 88 species based on the classification from Holt et al. (2013). We conducted a chi-squared goodness-of-fit test to assess whether there is a geographical bias in the occurrence of the insect species provided by the LLM chatbots.

2.3 R Software

Analyses were conducted in R, version 4.3.2 (R Core Team 2023), within the RStudio IDE, version 2023.06.1 + 524 (Posit team 2024). Sentiment analyses were performed using the following packages: “dplyr” (DOI: 10.32614/CRAN.package.dplyr); “ggplot2” (DOI: 10.32614/CRAN.package.ggplot2); “textdata” (DOI: 10.32614/CRAN.package.textdata); “tibble” (DOI: 10.32614/CRAN.package.tibble); “tidytext” (DOI: 10.32614/CRAN.package.tidytext). Boxplots were generated using the base R function “boxplot” The χ² test was performed using the base R functionality in the “stats” package, with visualizations created using the “graphics” package.

All figures were post-processed using Affinity Publisher 2 (Version 2.5.3).

4.1 Origins of Insect-Related Biases in LLM Chatbots

The word associations provided by the LLM chatbots for different insect groups highlight a clear bias that reflects common human perceptions, similar to those found by Sumner et al. (2018) in their questionnaire study. Our research confirms that bees are strongly associated with positive and ecologically significant terms, especially those emphasizing their role in ecosystems, particularly pollination services, which are greatly appreciated in society. Similarly, butterflies are predominantly associated with positive terms that reflect their cultural and aesthetic appeal, symbolic power for transformation and beauty, and ecological importance (Sumner et al. 2018; Iriani et al. 2021). The universal appreciation of butterflies may also be attributed to the fact that they do not evoke fear or disgust, unlike other insect groups (Gerdes et al. 2009; Schlegel and Rupf 2010).

Interestingly, the perception of moths was slightly positive despite the negative impact that the larval stages of numerous prominent pest species within this group have on agriculture and forestry (e.g., Sree and Varma 2015) and the adverse reactions caused by the bristles of some caterpillars (Ellis 2021). Outside agricultural settings, the public perception of moths is based mostly on their adult life stages, which are most noticeable at night when attracted to light sources. In these settings, moths are less likely to trigger strong negative emotions such as fear or disgust compared to wasps or mosquitoes, which are more likely to be associated with bites or disease transmission. Neither are moths associated with particularly repelling habitats such as feces or carcasses as compared to flies.

In contrast, the other insect groups—particularly wasps, flies, and mosquitoes—are associated with both neutral and negative terms, often emphasizing harmful aspects or the risk of diseases. Both bees and wasps are perceived as dangerous due to their capacity to sting, which can cause fatal anaphylactic reactions in hypersensitive individuals (Fitzgerald and Flood 2006; Gerdes et al. 2009). Yet, wasps are generally perceived as more negative than bees, partly due to their representation in the media, which typically portrays wasps as aggressive and focuses on the nuisance or the dangers posed by wasp stings rather than their ecological benefits (Sumner et al. 2018; Oi et al. 2024). Flies have long been depicted in literature and folklore as symbols of sin, moral failure, and pestilence. Consequently, they are perceived as pests or annoying rather than beneficial insects (Sumner et al. 2018; Santos et al. 2023). Flies and mosquitoes are also perceived by society to cause disease, death, and discomfort (Kellert 1993; Bhattacharya et al. 2016).

The consistent positive ratings for bees and butterflies and the negative ratings for wasps, flies, and mosquitoes by LLMs reflect biases present in human society that are then transmitted through the training data to these models (Bender et al. 2021; Bommasani et al. 2021. These biases favor more charismatic or aesthetically appealing species, which attract more public interest and empathy. Such biases, in turn, influence conservation priorities and public policy, leading to a skewed focus on species that are more popular or appealing (Schlegel and Rupf 2010; Ducarme et al. 2013; Soga and Gaston 2016; Courchamp et al. 2018).

Our study also reveals a clear trend toward taxonomic simplification. Many of the words LLMs associate with bees, such as “apiary,” “colony,” “hive,” “honey,” “queen,” “swarm,” “wax,” and “worker,” refer specifically to honeybees (Apis sp.) Their predominantly positive perception and their prioritization for conservation, suggested by all 10 LLM chatbots, can be attributed to a combination of cultural, economic, and ecological factors. For millennia, the mutualistic relationship between honeybees and humans has been documented in human culture, with historical references in early cave art, ancient Egyptian carvings and hieroglyphs, folklore, and cultural and religious symbols (Ransome 2004; Prendergast et al. 2021). Their economic value is well recognized today, with beekeeping products and their indispensable pollination services. Furthermore, the “save the bees” movement, triggered by reports of colony collapse disorder in the late 2000s, has heightened public awareness and contributed to the prioritization of honeybees in conservation efforts (van Engelsdorp et al. 2009; Suryanarayanan and Kleinman 2016). The plethora of scientific literature and popular science articles surrounding honeybees is another factor that presumably led to the LLMs' recommendation for honeybees as a top priority for conservation.

Additionally, we found a significant geographic bias toward Nearctic species in the LLMs conservation prioritization. The results of our study show that LLMs disproportionately recommend species occurring in the Holarctic region, especially in the Nearctic, with a significant underrepresentation of the Global South. This overemphasis on Nearctic species reflects the bias inherent in the LLMs' training data, which may disproportionately come from Western scientific literature and databases, as the dominance of Western perspectives often leads to an uneven global representation of biodiversity (Martin et al. 2012; Amano and Sutherland 2013).

4.2 Implications of Insect-Related Biases in LLM Chatbots

In light of the ongoing loss of biodiversity and the anticipated increase in the use of LLMs as sources of information, it is important for us to address the biases identified in this study. As LLMs become more integrated into educational platforms, these biases will likely entrench themselves in the perceptions of learners and teachers who rely on these AI systems as a source of information. Reinforcing these human biases through generative artificial intelligence can profoundly impact conservation efforts.

The LLMs' bias in conservation prioritization, coupled with their trend toward taxonomic simplification, may not only divert attention and resources away from less charismatic but equally important groups (Soga and Gaston 2016; Courchamp et al. 2018; Verma et al. 2023) but also diminish public awareness and support for the conservation of these taxa, which are often perceived as pests rather than essential components in ecosystems (Cardoso et al. 2011; Eisenhauer et al. 2019; Raguso 2020). This imbalance is particularly troubling when neglected groups already face steeper declines, as evidenced by recent studies showing that insect population declines differ substantially across taxonomic groups (Sánchez-Bayo and Wyckhuys 2019). An illustrative case of this imbalance is the marked underrepresentation of Diptera in conservation programs, despite flies and mosquitoes being extremely species-rich and biologically diverse. These groups also occupy key ecological roles as pollinators, biocontrol agents, indicator species, and as food sources for numerous insectivorous animals (Adler and Courtney 2019; Dunn et al. 2020; Orford et al. 2015; Raguso 2020).

Similarly, the LLMs' tendency to associate wasps with aculeate wasps, especially the social Vespidae, overlooks the broader ecological roles of the group. While aculeate wasps are undoubtedly important for ecosystems worldwide—acting as predators, pollinators, decomposers, biological indicators, and seed dispersers (Brock et al. 2021)—focusing only on Aculeata ignores approximately 70% of wasps, including Symphyta and various hyperdiverse parasitoid wasp lineages (Aguiar et al. 2013). Despite the irrefutable significance of parasitoid wasps as biological control agents in agriculture and ecosystem maintenance (Narendran 2001; Wang et al. 2019), these insects are still largely overlooked by the public and underrepresented in scientific research (Sumner et al. 2018). Moreover, parasitoid wasps are highly susceptible to ecosystem changes due to their high trophic level and specialization (Shaw and Hochberg 2001; Shaw 2006). They are also more susceptible to pesticides and less variable in response to pesticide exposure than predatory arthropods (Theiling and Croft 1988).

In the worst-case scenario, following LLMs' suggested conservation priorities could inadvertently lead to the implementation of measures that ultimately harm biodiversity. For example, prioritizing the conservation of honeybees (Apis sp.), a genus of approximately 11 species (Michener 2007), would neglect the vast diversity of wild bees, which includes over 20,000 described species (Michener 2007). Many wild bee species are solitary and are therefore only partially affected by the drivers that cause colony collapse disorder in eusocial honeybees. More concerning still is the mounting evidence that beekeeping practices can negatively affect pollen availability and contribute to the spread of parasites and pathogens, further straining wild bee populations, especially in areas where honeybees are kept at high densities or in nonnative ranges (Mallinger et al. 2017; Geldmann and González-Varo 2018; Valido et al. 2019; Iwasaki and Hogendoorn 2022; MacInnis et al. 2023).

Lastly, the geographical bias identified in our study could seriously hamper global conservation efforts. The underrepresentation of species from zoogeographical regions in the Global South could lead to the neglect of critical species that are essential to biodiversity and the ecological balance of these areas. Such neglect exacerbates global inequality, particularly in many biodiversity hotspots that are characterized by exceptional concentrations of endemic species and are undergoing significant habitat loss. These biodiversity hotspots often overlap with areas of human poverty (Fisher and Christopher 2007), making them further neglected in scientific research and conservation efforts (Meyer et al. 2015).

4.3 Overcoming Biases for Education and Science Communication

Addressing the biases inherent in LLMs is crucial for ensuring their reliability as tools for both conservation and public education. Our findings underscore the challenge of sentiment biases, taxonomic simplification, and geographic bias. These biases mirror and potentially exacerbate existing human prejudices, thereby skewing public perception and influencing conservation priorities (Schlegel and Rupf 2010; Ducarme et al. 2013; Soga and Gaston 2016; Courchamp et al. 2018). To mitigate these biases, it is essential to improve the quality and diversity of the training data used for LLMs. Strategies such as hyperparameter tuning (Ghosal et al. 2024), incorporating a broader and more balanced data set, and actively supplementing otherwise inaccessible sources (specifically “dark texts” as described by Page 2016) in training datasets can help create more fact-based models. Furthermore, ongoing collaboration between taxonomists, ecologists, data scientists, and conservationists can refine these models to better align with the realities of real-world conservation conditions. Utilizing enhanced models responsibly can support more balanced education efforts and help shift policy toward a more integrative and holistic approach to biodiversity conservation.

LLMs hold the potential to revolutionize science education by offering interactive and adaptive learning experiences (Li et al. 2024). Teachers can leverage these tools to deepen their students' understanding of complex ecological interactions and convey the importance of diverse insect groups in ecosystem health and resilience. Furthermore, LLMs can be used to tailor educational content to diverse learning styles and educational backgrounds, thus broadening accessibility and inclusivity in science education (Alqahtani et al. 2023). By improving the quality and diversity of training data and incorporating the latest ecological research findings, LLMs can serve as dynamic educational resources that evolve with new scientific discoveries. Such advancements not only enhance the educational experience but also foster critical thinking and informed decision-making among students, encouraging a more informed and proactive endorsement of biodiversity conservation.