2.1 BERTopic

BERTopic is a neural topic model with a class-based (c) Term Frequency–Inverse Document Frequency (TF-IDF) algorithm that mainly uses BERT, Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), Uniform Manifold Approximation and Projection (UMAP), and c-TF-IDF [8]. It can understand the input text using a neural network and extract several topics discussed in the text, each containing several keywords. The technical flow chart is shown in Figure 1.

2.2 Document embeddings

BERTopic assumes that documents that contain the same topic are semantically similar. The documents are embedded through Sentence-BERT (SBERT) [9]. This is a framework that can convert sentences and paragraphs to dense vector representations using the BERT pre-trained model.

In this study, for TCM prescriptions, the Chinese-supported paraphrase-multilingual-MiniLM-L12-v2 pre-trained model is used to perform the embedding step for each prescription, which results in prescription embedding.

2.3 Dimension reduction

The embeddings have hundreds of dimensions, which makes differences in distance measurement unclear. To improve the clustering effect, the dimensionality of embeddings needs to be reduced. BERTopic uses UMAP for dimension reduction to reduce the dimensionality of document embeddings.

UMAP is an advanced dimension reduction algorithm based on the theoretical framework of Riemannian geometry and algebraic topology [10]. It assumes that the data satisfy the following three conditions: (1) data should be uniformly distributed on an existing Riemannian manifold; (2) the underlying manifold of interest is locally connected; and (3) the topology of the manifold is constant.

The algorithm performs excellently and has been shown to retain more global and local features of high-dimensional data in the lower dimension, as in the research sentence vector with hundreds of dimensions are able to reduce to no more than 10 vector groups or even less; hence, it works well with real-world data and has been widely applied in various fields [11]. Through manifold modeling, the equivalent fuzzy topology can be used to obtain the projection of prescription embeddings in lower dimensions.

2.4 Document clustering

The dimensionality reduced embeddings are clustered by HDBSCAN. HDBSCAN is a spatial clustering algorithm based on hierarchical density with noise, which can solve clustering problems with various densities through parameter control [12]. It mainly includes five steps: spatial transform, building the minimum spanning tree, constructing the hierarchical structure of cluster groups, compressing the cluster tree, and extracting clusters.

Compared with traditional clustering algorithms, the HDBSCAN algorithm effectively improves the clustering effect of multiple topics.

2.5 Topic representation

Term Frequency–Inverse Document Frequency is a statistical method that can be used to assess the importance of a word in a document and extract its keywords:

()

where the size of the TF-IDF product reflects the importance of term t in document d. Term frequency (TF) refers to the frequency with which term t appears in document d:

()

where T represents the number of occurrences of term t in document d and D represents the total number of words in document d. The Inverse Document Frequency (IDF) reflects the frequency with which term t appears in the corpus:

()

where N represents the total number of documents in the corpus and c represents the number of documents in the corpus that contain term t.

For TCM prescriptions, the value of TF is the frequency with which a single TCM appears in a series of prescriptions and the value of IDF reflects the frequency with which the TCM appears in the corpus. The product of TF and IDF indicates the importance of the drug for a series of prescriptions.

Extracting too many documents directly through TF-IDF results in too many unclassified keywords, which is the main disadvantage. BERTopic has improved the TF-IDF algorithm and proposed a class-based TF-IDF algorithm, with only document d making some changes on concatenating all documents in a cluster (namely a class) as a new single document d:

()

where c represents a class, A represents the average number of words in class c, TF_t,c represents the frequency of term t in class c and TF_t represents the frequency of term t in all classes.

According to the clustering results, c-TF-IDF splices documents in each class and calculates TF-IDF, thereby effectively mining keywords in each topic class.

3.1 Dataset

The prescription data used in this study was sourced from the Department of Ophthalmology at Zhejiang Hospital of TCM, and covered the period from January 2016 to December 2021. After the data were filtered, a total of 1276 valid prescriptions for the treatment of corneal disorders were selected for analysis in this study.

Microsoft Excel was used as the storage format for the data, which recorded the drugs and their dosages in each prescription. Among the 1276 prescriptions used for treating corneal disorders, a total of 127 Chinese herbal medicines were included. Through frequency analysis, it was found that Buddleja officinalis and Bidens pilosa appeared the most frequently in the prescriptions (Figure 2).

3.2 Experimental setup

The process of extracting topic keywords from the documents using BERTopic is similar to the process of extracting core prescriptions. Traditional Chinese Medicine can be regarded as words, whereas TCM prescriptions are sentences. A series of prescriptions for treating specific diseases can be considered as a paragraph in the document with a clear theme. Therefore, the collected TCM prescriptions were organized into paragraphs so that BERTopic could obtain their topics and information, which were used to analyze the medication rules in prescriptions.

The collected clinical TCM prescriptions were formatted as an Excel table. The openpyxl library in Python 3.9 easily reads the table into arrays. Then, the drugs in each prescription were concatenated, separated by spaces, and finally constructed into strings to form a document that BERTopic could understand. All the prescription strings were stored in an array, thereby forming paragraphs that could be analyzed.

The paraphrase-multilingual-MiniLM-L12-v2 model was used, for which each prescription is embedded in a 384-dimensional sentence vector through a Sentence Transformer. To improve the clustering effect, the 384-dimensional sentence vector group was reduced to a five-dimensional vector group through UMAP. Then, the vector group was clustered through HDBSCAN to obtain a set of several compatible prescriptions for the disease. Finally, c-TF-IDF extracted topic words from prescriptions in each classification to obtain potential core drug combinations for the disease.

4.1 Cluster analysis

After the prescriptions were embedded using BERT, dimensionality was reduced through UMAP to improve the clustering effect. Figure 3 shows the clustering results after dimensionality reduction. All prescriptions were classified into four categories and noise was ignored. The prescriptions were used to treat corneal diseases and each category was considered as a different direction for treatment.

4.2 Core drug combinations

Through the analysis of BERTopic, four topics were extracted from the prescriptions, which identified four potential directions for medication for the treatment of the disease. The results are presented in Figure 4.

4.3 Analysis of drug association

The normalization results for the Topic 0 frequency data are shown in Table 1, with Eriocaulon buergerianum, Raw Rehmannia glutinosa, Prunella vulgaris, and Notopterygium incisum as the main drug combinations.

The normalization results for the Topic 1 frequency data are shown in Table 2, with Lycii Fructus, Bidens pilosa, and Buddleja officinalis as the main drug combinations.

The normalization results for the Topic 2 frequency data are shown in Table 3.

The normalization results for the Topic 3 frequency data are shown in Table 4, with Kochiae Fructus and Cortex Dictamni as the main drug combinations.

4.4 Drug frequency analysis

Among the 1276 TCM prescriptions for the treatment of corneal diseases, 127 drugs appeared, with a cumulative frequency of 8521 times. There were 22 high-frequency drugs with a frequency greater than or equal to 90, which accounted for 70.70% of the total drug frequency. Table 5 shows the frequency of prescriptions for corneal disorders.

5.1 Cluster analysis

Currently, in cluster analysis, experts are often used to manually specify the vector weights of each TCM, which is an error prone and time-consuming process. The K-means algorithm is used for clustering [13]. This method is simple and easy to implement; however, the clustering effect depends on the choice of the K value and initial point [14] and is easily disturbed by noisy data.

5.2 Term Frequency–Inverse Document Frequency

TF-IDF is also a TCM prescription data mining method [15]. Its goal is to calculate the TF-IDF value of each TCM. Table 6 shows the results of TF-IDF ≥5. The results showed that drugs commonly used in the treatment of corneal diseases included Buddleja officinalis, Bidens pilosa, Angelica sinensis, Eriocaulon buergerianum, Lycii Fructus, Raw Rehmannia glutinosa, Processed Pheretima asiatica, Epimedium, and Prunella vulgaris.

The results of TF-IDF only represent the types of core drugs in the batch of prescriptions, which is a macro analysis of prescriptions without local information. It is difficult to specify the combination of several drugs and hence the compatibility relationship cannot be explained well.