Ecology and Evolution
Volume 14, Issue 2 e10952
RESEARCH ARTICLE
Open Access

Decomposing diversity into measures of evenness, similarity, and richness

Bingzhang Chen, 

Corresponding Author

Bingzhang Chen

Department of Mathematics and Statistics, University of Strathclyde, Glasgow, UK

Correspondence

Bingzhang Chen, Department of Mathematics and Statistics, University of Strathclyde, 26 Richmond Street, Glasgow G1 1XH, UK.

Email: [email protected]

Contribution: Conceptualization (supporting), Data curation (lead), Funding acquisition (lead), ​Investigation (supporting), Methodology (supporting), Software (equal), Visualization (equal), Writing - review & editing (supporting)

Search for more papers by this author
Michael Grinfeld, 

Michael Grinfeld

Department of Mathematics and Statistics, University of Strathclyde, Glasgow, UK

Contribution: Conceptualization (lead), Formal analysis (lead), Methodology (lead), Writing - original draft (lead), Writing - review & editing (lead)

Search for more papers by this author
First published: 13 February 2024
https://doi.org/10.1002/ece3.10952
Citations: 2

Abstract

It has long been recognized that diversity has many measurable aspects, such as richness, evenness, and similarity among species. However, given a diversity index, it is unclear whether it necessarily can be decomposed into components that reflect these different aspects. Here, we present a scheme to decompose the Leinster and Cobbold diversity index, which subsumes and generalizes many other indices, into the components of richness, evenness and taxonomic similarity. Our approach addresses the problem that in general a vector of equal relative abundances does not maximize diversity. Furthermore, our approach uses all available information to give unbiased estimates of both evenness and similarity.

1 INTRODUCTION

It has long been recognized that diversity has multiple aspects such as “richness” or “variety” reflecting the number of species present, dominance or rarity relations among the constituent species (“evenness” or “balance”), and “similarity” (or “disparity”) among the species (measures of distance between species based on taxonomic, phylogenetic, or functional traits relationships) (Daly et al., 2018; Leinster & Cobbold, 2012; Purvis & Hector, 2000; Stirling, 2007). Stirling (2007) even claims that “there seems no other obvious candidate for a fourth important general property of diversity beyond these three.”

In ecology, claimed biogeographical patterns of diversity often depend on the particular aspect of diversity being investigated. For example, Stuart-Smith et al. (2013) show that biodiversity hot spots can shift from the tropics to higher latitudes if in addition to considering abundances, one takes account of functional traits similarity of species.

Several promising diversity indexes have been defined in the literature (Chao et al., 2014; Leinster & Cobbold, 2012; Rao, 1982; Stirling, 2007). However, a priori it is not clear whether from any given index one can extract information about richness, evenness, and species similarity. There is no argument why such a decomposition should exist and whether it has to be unique. In cases of non-uniqueness, one has to define what is the optimal way to perform such a decomposition. The situation here is similar to the decomposition of γ diversity into α and β diversity indices either additively or multiplicatively (Anderson et al., 2011; Jost, 2007).

Being able to decompose a diversity index into biologically significant components would help us understand which aspect of diversity contributes to the changes in the relationship between diversity and environmental variables (e.g., latitude or productivity). Such a decomposition is also helpful in understanding how different aspects of biodiversity influence the functioning of ecosystems (Hillebrand et al., 2018; Maureaud et al., 2019).

In an important study, van Dam (2019) proposed a straightforward decomposition of the Leinster–Cobbold (2012) diversity index that contains information about the richness, evenness, and similarity aspects of diversity. However, as will be detailed below, van Dam's approach fails to consider the fact that a homogenous community (i.e., in which all constituent species have equal relative abundances) may not maximize diversity if similarity among species is taken into account. Leinster and Meckes (2016) showed that in general, there exists a unique abundance vector p* that maximizes diversity that can differ from the equal abundance vector which we denote by p h $$ {\boldsymbol{p}}_{\mathrm{h}} $$ . In other words, the maximal evenness of some communities is not achieved when all species have equal relative abundances. The criticism of van Dam's (2019) assumption that the component of evenness is maximal for an equal abundance vector can also be applied to the work of Daly et al. (2018) who stated that “the diversity measure is maximal for a fixed number of species S when all species' abundances are equal.”

In this study, we develop a decomposition approach that addresses the above problem. We focus on the Leinster–Cobbold (LC) index (Leinster & Cobbold, 2012) because it subsumes many other diversity indices that are widely used in ecology, such as the Hill numbers (Hill, 1973) and Rao's index (Rao, 1982; Ricotta & Moretti, 2011; Stirling, 2007), and has many other merits to be discussed later. For recent applications of this index, please see Mugabushaka et al. (2016) and Veresoglou et al. (2014). We take advantage of intrinsic properties of the LC index (e.g., the existence of the vector p*) to achieve an unbiased (in a sense to be explained below) decomposition of the index.

The structure of this article is as follows. In Section 2, we discuss the notions of richness, evenness, and similarity using the material in Chao et al. (2014), Chiu et al. (2014), Daly et al. (2018), and Gregorius and Gillet (2022). Then, in Section 3, we comment biodiversity indices in general and present the required information about the LC index following Leinster and Cobbold (2012) and Leinster and Meckes (2016). In Section 4, we present first van Dam's, and then our decomposition and its consequences. Finally, in Section 5, we discuss the merits and limitations of our approach and suggest future work.

We also note that even if the quotation from Stirling (2007) reflects a deep property of biodiversity indices, it is not clear that a decomposition of a biodiversity index cannot contain additional terms. In fact, one interpretation of the decomposition we suggest contains an additional term, related to Hill's numbers and to the maximal diversity of a biocommunity (defined below). That is, the additional term does not carry any new information but is needed “to balance the books.” The quotation from Stirling also does not prescribe how richness, evenness, and similarity should be defined.

2 DIVERSITY COMPONENTS

2.1 Notation

For clarity, we first establish notation. Everywhere below we assume that the number of species (i.e., richness) in a community is fixed at n > 1 $$ n>1 $$ .

We use p = p 1 … p n $$ \boldsymbol{p}=\left({p}_1,\dots, {p}_n\right) $$ to denote the unit vector of relative abundances, that is,
p i > 0 for all i = 1 , … n and ∑ i = 1 n p i = 1 . $$ {p}_i>0\kern0.6em \mathrm{for}\kern0.5em \mathrm{all}\kern0.6em i=1,\dots n\kern0.5em \mathrm{and}\kern0.5em \sum \limits_{i=1}^n{p}_i=1. $$
We use the vector
p h = 1 n … 1 n , $$ {\boldsymbol{p}}_{\mathrm{h}}=\left(\frac{1}{n},\dots, \frac{1}{n}\right), $$ ()
where the subscript h stands for “homogeneous” to represent the equal relative abundance of species in a community. Below 1 will stand for an n-vector with all components equal to 1.

Next, we need to discuss n × n $$ n\times n $$ matrices to record the information of species similarity. We denote the n × n $$ n\times n $$ identity matrix by I n $$ {I}_n $$ which indicates that all species in the community are totally dissimilar with each other. We also use the notation J n $$ {J}_n $$ for the n × n $$ n\times n $$ matrix of ones which represents an extreme case that all species are the same in the community (i.e., effectively we have only one species).

In this article, for simplicity, we will work with ultrametric matrices; this choice is motivated by the fact that similarity matrices (see Section 3.2) constructed using taxonomic trees are necessarily ultrametric and using ultrametric matrices simplifies the theory of Leinster and Mecke (2016). For more information on ultrametric matrices, please see Dellacheria et al. (2014), Leinster (2013), and Leinster and Meckes (2016).

2.2 Evenness

Classically, the “most even” population of n species is one for which the vector of relative abundances is the homogeneous vector p h $$ {\boldsymbol{p}}_{\mathrm{h}} $$ , that is, one where every species is equally represented (Daly et al., 2018). For thorough discussions of evenness see, for example, Chao et al. (2014), Chao and Ricotta (2019), Gregorius and Gillet (2022), and references therein. As the case of richness discussed below, the term “evenness” itself seems to be precluding discussion. As rightly pointed in Gregorius and Gillet (2022), a definition of maximal evenness in terms of p h $$ {\boldsymbol{p}}_{\mathrm{h}} $$ leaves open the discussion of what would constitute “maximum unevenness.” Instead of “evenness,” van Dam (2019) uses “balance,” which seems to us a better term; this is the concept which, after defining it properly (see (12)) we will use below.

2.3 Richness

Richness is usually defined as the number of species (Daly et al., 2018). Such a definition is open to the same criticism as the notion of maximal evenness defined by p h $$ {\boldsymbol{p}}_{\mathrm{h}} $$ discussed above. It is “species-centric” without considering species similarity, and takes into account only the last level of taxonomic classification. We prefer to call the number of species n “species richness.”

2.4 Similarity

In this section, we discuss the construction of taxonomic similarity matrices Z for a community with n species.

The usual way of constructing similarity matrices Z (which are automatically ultrametric) is to assign distances between different levels of a taxonomic tree. Then the taxonomic distance d i , j $$ d\left(i,j\right) $$ between two species is the sum of distances from the nodes corresponding to these species to the first common node. Once the distance between nodes is defined, we could put Z ij = e − d i , j $$ {Z}_{ij}={e}^{-d\left(i,j\right)} $$ or, if the maximal distance in the tree has been normalized to 1, we could put Z ij = 1 − d i , j $$ {Z}_{ij}=1-d\left(i,j\right) $$ . As an example, consider the tree in Figure 1 and set the species–genus and the genus–family distance to be 0.3. If we use the additive recipe for defining Z, that is, put Z ij = 1 − d i , j $$ {Z}_{ij}=1-d\left(i,j\right) $$ , we obtain the similarity matrix
Z 1 = 1 0.7 0.4 0.7 1 0.4 0.4 0.4 1 . $$ {Z}_1=\left(\begin{array}{ccc}1& 0.7& 0.4\\ {}0.7& 1& 0.4\\ {}0.4& 0.4& 1\end{array}\right). $$ ()
Details are in the caption following the image
FIGURE 1
Open in figure viewerPowerPoint
An example taxonomic tree with the root at the family level.

3 THE LEINSTER–COBBOLD DIVERSITY INDEX

The Leinster and Cobbold (LC) diversity index was introduced in Leinster and Cobbold (2012). The LC index is a far-reaching generalization of Hill numbers, for discussions of which, see Chiu et al. (2014). For details on its properties, see Leinster and Cobbold (2012) and Leinster and Meckes (2016); here, we just collect the bare minimum in the framework of taxonomic (ultrametric) similarity matrices; ultrametricity is the blanket assumption on similarity matrices we make from now on.

Note that compared to the diversity indices proposed by Chao et al. (2014), the LC index has the flexibility to take into account taxonomic, phylogenetic, and functional diversity simultaneously, although the resulting similarity matrices will in general not be ultrametric. We leave this as a more general case for future study.

Before we start discussing the LC index, let us review the requirements that have to be satisfied by any diversity index.

3.1 Requirements for an diversity index

Leinster and Cobbold (2012, pp. 482–483) formulate conditions that any biodiversity index should satisfy; they number nine such conditions. The LC index satisfies all of them, that is, in brief, the LC index provides effective numbers, is modular, has the property of replication, is symmetric, is not changed if a species identical to one of those already present is added or by adding a new species of zero abundance, decreases if similarity between species increases, is larger if similarities between species are ignored, and always takes a value between 1 and the number of species n. Hence, the LC index is a legitimate starting point for the decomposition work we are interested in. We also note that while Leinster and Cobbold (2012) write that any index with sensitivity parameter q should be a monotone decreasing function of q, they do not add this criterion to the requirements on an index, as we do below.

3.2 Definition of the LC index

As in the definition of Hill numbers, below q ∈ 0 , ∞ $$ q\in \left[0,,,\infty \right) $$ is the sensitivity parameter, measuring the (un)importance given to rare species. Then for a community of n species with relative abundance vector p and similarity matrix Z, we have

Definition 1.The LC diversity of order q is

F Z , p , q ≔ ∑ i = 1 n p i Z p T i q − 1 1 / 1 − q . $$ F\left(Z,\boldsymbol{p},q\right):= {\left(\sum \limits_{i=1}^n{p}_i{\left(Z{\boldsymbol{p}}^T\right)}_i^{q-1}\right)}^{1/\left(1-q\right)}. $$ ()

Leinster and Cobbold (2012) show that the LC index indeed satisfies all the requirements imposed on a diversity index in Section 3.1.

Note that Leinster and Cobbold (2012) use a different notation, similar to the Hill number notation in the literature; they denote the right-hand side of (3) by   q D Z p $$ ^q{D}^Z\left(\boldsymbol{p}\right) $$ . We prefer the notation used here as it clearly shows functional dependencies and allows easy generalization, which we discuss briefly in Section 5. We collected the required properties of the LC index below and in Section 3.3.

The LC index has the following properties:
  1. F p , Z , q $$ F\left(\boldsymbol{p},Z,q\right) $$ is a monotone decreasing function of q;
  2. F p , Z , q < F p I n q $$ F\left(\boldsymbol{p},Z,q\right)<F\left(\boldsymbol{p},{I}_n,q\right) $$ for all q if Z ≠ I n $$ Z\ne {I}_n $$ ;
  3. F p h I n q = n $$ F\left({\boldsymbol{p}}_h,{I}_n,q\right)=n $$ for all q;
  4. F p J n q = 1 $$ F\left(\boldsymbol{p},{J}_n,q\right)=1 $$ for all q.
For proofs of (a) and (b), see Leinster and Cobbold (2012); the other statements are immediate.

Following Leinster and Meckes (2016), we now discuss the concept of a maximally balanced abundance vector for a community of n species with an ultrametric similarity matrix Z which is the key starting point of our decomposition approach.

3.3 p*: A crucial property of the LC index

Leinster and Meckes (2016) prove that for each similarity matrix Z, there exists a unique abundance vector p* that maximizes the diversity index F Z , p , q $$ F\left(Z,\boldsymbol{p},q\right) $$ for every value of q ∈ 0 , ∞ $$ q\in \left[0,,,\infty \right) $$ . We call p* the maximally balanced abundance vector (which replaces ph if taxonomic similarity is taken into account) and call F(Z, p*, q) (which is independent of q) the maximal diversity of a biocommunity with similarity matrix Z.

If Z $$ Z $$ is ultrametric, computing p* is a simple matter of solving a system of linear equations and normalizing:
p i * = w i ∑ j = 1 n w j $$ {p}_i^{\ast }=\frac{w_i}{\sum_{j=1}^n{w}_j} $$ ()
where w $$ \boldsymbol{w} $$ solves the system of equations Z w = 1 $$ Z\boldsymbol{w}=\mathbf{1} $$ , with 1 $$ \mathbf{1} $$ is a column vector of ones.
For the similarity matrix Z1 in (2), we have
p * ≈ 0.286,0.286,0.429 . $$ {\boldsymbol{p}}^{\ast}\approx \left(\mathrm{0.286,0.286,0.429}\right). $$
which differs from p h = 0.333,0.333,0.333 $$ {\boldsymbol{p}}_h=\left(\mathrm{0.333,0.333,0.333}\right) $$ . We can check that when q = 0 $$ q=0 $$ , F Z , p * , q = 1.522 $$ F\left(Z,{\boldsymbol{p}}^{\ast },q\right)=1.522 $$ which is greater than F Z p h q = 1.508 $$ F\left(Z,{\boldsymbol{p}}_h,q\right)=1.508 $$ . For q = 1 , 2 $$ q=1,2 $$ , F Z , p * , q $$ F\left(Z,{\boldsymbol{p}}^{\ast },q\right) $$ remains unchanged, but F Z p h 1 = 1.504 $$ F\left(Z,{\boldsymbol{p}}_h,1\right)=1.504 $$ and F Z p h 2 = 1.500 $$ F\left(Z,{\boldsymbol{p}}_h,2\right)=1.500 $$ .

Note that an alternative way of computing p* is provided in Leinster and Meckes (2016, lemma 6).

There are similarity matrices Z ≠ I n $$ Z\ne {I}_n $$ for which p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_h $$ . However, it is beyond the scope of this study to characterize exhaustively the set of such matrices. For now, we simply define a taxonomic tree to be taxonomically equilibrated if p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_h $$ . Of course if at each level of the tree all the nodes have the same degree, the taxonomic tree is equilibrated. However, the converse statement is not true. An example is provided by the tree in Figure 2.

Details are in the caption following the image
FIGURE 2
Open in figure viewerPowerPoint
A taxonomic tree that does not have the same degree at all nodes but allows p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_h $$ .

It is not hard to show that assigning of species–genus, genus–family, and family–order distances to be 0.25, and using the additive recipe to compute Z ij $$ {Z}_{ij} $$ , results in a similarity matrix for which p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_h $$ . Thus, there is a trichotomy of taxonomic trees: those for which p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_h $$ holds for every assignment of distances; those where such assignments can be chosen, as in Figure 2, and such that no assignment of distances results in a homogeneous maximally balanced abundance vector; an example of such a tree is shown in Figure 3.

Details are in the caption following the image
FIGURE 3
Open in figure viewerPowerPoint
A taxonomic tree for which p * ≠ p h $$ {\boldsymbol{p}}^{\ast}\ne {\boldsymbol{p}}_h $$ is guaranteed.

4 AN UNBIASED DECOMPOSITION SCHEME

We are now ready to propose a decomposition scheme for the LC index. It is best to start with the decomposition scheme suggested by van Dam (2019) and see why it has to be modified. van Dam writes
F p , Z , q = F p I n q F p h I n q ⋅ F p , Z , q F p I n q ⋅ F p h I n q . $$ F\left(\boldsymbol{p},Z,q\right)=\frac{F\left(\boldsymbol{p},{I}_n,q\right)}{F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right)}\cdot \frac{F\left(\boldsymbol{p},Z,q\right)}{F\left(\boldsymbol{p},{I}_n,q\right)}\cdot F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right). $$ ()
The second fraction in the right-hand side of (5),
D vD p , Z , q ≔ F p , Z , q F p I n q , $$ {D}_{vD}\left(\boldsymbol{p},Z,q\right):= \frac{F\left(\boldsymbol{p},Z,q\right)}{F\left(\boldsymbol{p},{I}_n,q\right)}, $$ ()
is clearly a measure of dissimilarity, while the first fraction should therefore be measure of balance,
B vD p , q ≔ F p I n q F p h I n q ; $$ {B}_{vD}\left(\boldsymbol{p},q\right):= \frac{F\left(\boldsymbol{p},{I}_n,q\right)}{F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right)}; $$ ()
F p h I n q = n $$ F\left({\boldsymbol{p}}_h,{I}_n,q\right)=n $$ , so the last term on the right-hand side is the standard measure of richness. Note that, as it should be, the two fractions always lie in the interval 1 / n , 1 $$ \left(1/n,1\right] $$ (property (b) of the LC index ensures that the value 1 / n $$ 1/n $$ cannot be reached).

The difficulty with (5) is in accepting (7) as a measure of balance. First of all, it does not use the information contained in the taxonomic similarity matrix Z $$ Z $$ . Secondly, it introduces p h $$ {\boldsymbol{p}}_{\mathrm{h}} $$ which a priori has no meaning in theories that take similarity into account. B vD p , q $$ {B}_{vD}\left(\boldsymbol{p},q\right) $$ will take the maximal value 1 when p = p h $$ \boldsymbol{p}={\boldsymbol{p}}_{\mathrm{h}} $$ for all values of q $$ q $$ , while in the framework of the LC index, the measure of balance should be maximal when p = p * $$ \boldsymbol{p}={\boldsymbol{p}}^{\ast } $$ . This argument makes the van Dam decomposition unusable in the framework of the LC index for which it is intended.

The dissimilarity measure F p , Z , q F p I n q $$ \frac{F\left(\boldsymbol{p},Z,q\right)}{F\left(\boldsymbol{p},{I}_n,q\right)} $$ does use information from both p $$ \boldsymbol{p} $$ and Z $$ Z $$ . Hence, the decomposition (5) can be said to be asymmetrically biased.

Note that an alternative decomposition
F p , Z , q = F p , Z , q F p h , Z , q ⋅ F p h , Z , q F p h I n q ⋅ F p h I n q , $$ F\left(\boldsymbol{p},Z,q\right)=\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left({\boldsymbol{p}}_{\mathrm{h}},Z,q\right)}\cdot \frac{F\left({\boldsymbol{p}}_{\mathrm{h}},Z,q\right)}{F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right)}\cdot F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right), $$ ()
in addition to having the drawbacks of (5) of using the often irrelevant abundance vector p h $$ {\boldsymbol{p}}_{\mathrm{h}} $$ and having an asymmetrically biased dissimilarity measure, also has the problem that its measure of balance, the first term in the right-hand side, can take values larger than 1 if Z $$ Z $$ is not a similarity matrix of an taxonomically equilibrated tree and p = p * $$ \boldsymbol{p}={\boldsymbol{p}}^{\ast } $$ .
To ensure normalization and to be consistent with Leinster and Meckes (2016), consider the following decomposition:
F p , Z , q = F p , Z , q F p * , Z , q ⋅ F p * , Z , q F p * I n q ⋅ F p * I n q . $$ F\left(\boldsymbol{p},Z,q\right)=\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}\cdot \frac{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },{I}_n,q\right)}\cdot F\left({\boldsymbol{p}}^{\ast },{I}_n,q\right). $$ ()
It is asymmetrically biased as the second factor, a dissimilarity measure, does not use the information from the actual abundance vector p. We will discuss the interpretation of F p * , I n , q $$ F\left({\boldsymbol{p}}^{\ast },,,{I}_n,,,q\right) $$ later.
To obtain an unbiased decomposition, we multiply (5) and (9) and take a square root. The result is
F p , Z , q = F p , Z , q F p * , Z , q F p I n q F p h I n q ⋅ F p , Z , q F p I n q F p * , Z , q F p * I n q ⋅ nF p * I n q , $$ F\left(\boldsymbol{p},Z,q\right)=\sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}\frac{F\left(\boldsymbol{p},{I}_n,q\right)}{F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right)}}\cdot \sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left(\boldsymbol{p},{I}_n,q\right)}\frac{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },{I}_n,q\right)}}\cdot \sqrt{nF\left({\boldsymbol{p}}^{\ast },{I}_n,q\right)}, $$ ()
since F p h I n q = n $$ F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right)=n $$ .
Note that the last term in the right-hand side of (10) can be rewritten as
nF p * I n q = F p * I n q n n ≔ E Z , q n , $$ \sqrt{nF\left({\boldsymbol{p}}^{\ast },{I}_n,q\right)}=\sqrt{\frac{F\left({\boldsymbol{p}}^{\ast },{I}_n,q\right)}{n}}n:= E\left(Z,q\right)n, $$ ()
which is appropriate notation as E Z , q $$ E\left(Z,q\right) $$ is determined by Z $$ Z $$ . The term E Z , q $$ E\left(Z,q\right) $$ expresses the lack of equilibration in the taxonomic tree and always lies in the interval 0 , 1 $$ \left(0,1\right] $$ . In the case of similarity matrices of taxonomically equilibrated trees, for which we have p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_{\mathrm{h}} $$ and hence F p * I n q = n $$ F\left({\boldsymbol{p}}^{\ast },{I}_n,q\right)=n $$ , we have that E Z , q = 1 $$ E\left(Z,q\right)=1 $$ . If Z $$ Z $$ does not correspond to a taxonomically equilibrated taxonomic tree, F p * I n q $$ F\left({\boldsymbol{p}}^{\ast },{I}_n,q\right) $$ is dependent on q $$ q $$ ; E Z , q $$ E\left(Z,q\right) $$ is monotone decreasing in q $$ q $$ ; the limit of E Z , q $$ E\left(Z,q\right) $$ as q → ∞ $$ q\to \infty $$ exists and may reflect properties of Z $$ Z $$ ; we leave analysis of E Z , q $$ E\left(Z,q\right) $$ to future work.
The difficulty with the decomposition (10) as written is in interpreting
B 0 p , Z , q = F p , Z , q F p * , Z , q F p I n q F p h I n q . $$ {B}_0\left(\boldsymbol{p},Z,q\right)=\sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}\frac{F\left(\boldsymbol{p},{I}_n,q\right)}{F\left({\boldsymbol{p}}_{\mathrm{h}},{I}_n,q\right)}}. $$ ()
as a measure of balance as it is not necessarily maximized by p = p * $$ \boldsymbol{p}={\boldsymbol{p}}^{\ast } $$ for all values of q $$ q $$ different from zero.
However, note that (10) can also be written as
F p , Z , q = F p , Z , q F p * , Z , q ⋅ F p , Z , q F p I n q ⋅ F p * , Z , q F p I n q , $$ F\left(\boldsymbol{p},Z,q\right)=\sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}}\cdot \sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left(\boldsymbol{p},{I}_n,q\right)}}\cdot \sqrt{F\left({\boldsymbol{p}}^{\ast },Z,q\right)F\left(\boldsymbol{p},{I}_n,q\right)}, $$ ()
In both (13), we have an unbiased measure of balance,
B p , Z , q ≔ F p , Z , q F p * , Z , q , $$ B\left(\boldsymbol{p},Z,q\right):= \sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left({\boldsymbol{p}}^{\ast },Z,q\right)}}, $$ ()
which correctly takes its maximal value 1 $$ 1 $$ at p = p * $$ \boldsymbol{p}={\boldsymbol{p}}^{\ast } $$ , and an unbiased measure of dissimilarity
D p , Z , q ≔ F p , Z , q F p I n q , $$ D\left(\boldsymbol{p},Z,q\right):= \sqrt{\frac{F\left(\boldsymbol{p},Z,q\right)}{F\left(\boldsymbol{p},{I}_n,q\right)}}, $$ ()
which for any given p is maximized by choosing I n $$ {I}_n $$ .
We suggest to define
R p , Z , q ≔ F p * , Z , q F p I n q , $$ R\left(\boldsymbol{p},Z,q\right):= \sqrt{F\left({\boldsymbol{p}}^{\ast },Z,q\right)F\left(\boldsymbol{p},{I}_n,q\right)}, $$ ()
the harmonic mean of Hill's number of order q $$ q $$ , F p I n q $$ F\left(\boldsymbol{p},{I}_n,q\right) $$ , and F p * , Z , q $$ F\left({\boldsymbol{p}}^{\ast },Z,q\right) $$ , the maximal diversity for all q $$ q $$ for a given similarity matrix Z $$ Z $$ (see Section 3.3), to be our notion of effective richness of order q $$ q $$ . Note that it arises naturally in the process of unique unbiased decomposition of the LC index. This is not an inconsistent definition: if Z = I n $$ Z={I}_n $$ and p = p h = p * $$ \boldsymbol{p}={\boldsymbol{p}}_{\mathrm{h}}={\boldsymbol{p}}^{\ast } $$ , that is, for a maximally balanced population of totally unrelated species, we have that R p h I n q = n $$ R\left({\boldsymbol{p}}_h,{I}_n,q\right)=n $$ , the species richness, for all q ≥ 0 $$ q\ge 0 $$ . So our notion of richness truly generalizes species richness and takes into account similarity among species, the abundance vector p $$ \boldsymbol{p} $$ and the sensitivity parameter q $$ q $$ .
Hence, with this definition of richness, (13) becomes
F p , Z , q = B p , Z , q D p , Z , q R p , Z , q . $$ F\left(\boldsymbol{p},Z,q\right)=B\left(\boldsymbol{p},Z,q\right)D\left(\boldsymbol{p},Z,q\right)R\left(\boldsymbol{p},Z,q\right). $$ ()

An advantage of our non-“species-centric” definition of richness is that it allows to compare two communities with different number of species.

5 DISCUSSION

We have proposed a decomposition of the LC index into components that include well-defined measures of balance (evenness) and (dis)similarity. Compared to a previous version of decomposition (van Dam, 2019), our approach delivers measures of balance and of dissimilarity of the community that employ all the available information and do not use the homogeneous abundance vector p h $$ {\boldsymbol{p}}_h $$ that does not have any biological significance in realistic communities.

The price for having three components in the decomposition was having to redefine richness as in (16). We could alternatively write (13) as
F p , Z , q = B p , Z , q D p , Z , q n ⋅ R p , Z , q n , $$ F\left(\boldsymbol{p},Z,q\right)=B\left(\boldsymbol{p},Z,q\right)D\left(\boldsymbol{p},Z,q\right)n\cdot \frac{R\left(\boldsymbol{p},Z,q\right)}{n}, $$ ()
in which case we must retain in our decomposition an in general q-dependent fourth term.

However, if the choice is between (17) and (18), a decomposition into four terms, we argue that (17) is preferable: the last term in the right-hand side of (18) can only be interpreted as the ratio between a measure of “effective richness” and the species richness and hence provides the same information as (17).

One could ask whether one could not start with the components (14), (15), n $$ n $$ , and define an index I p , Z , q $$ I\left(\boldsymbol{p},Z,q\right) $$ by
I p , Z , q = B p , Z , q D p , Z , q n . $$ I\left(\boldsymbol{p},Z,q\right)=B\left(\boldsymbol{p},Z,q\right)D\left(\boldsymbol{p},Z,q\right)n. $$

However, this “index” is not decreasing in q, so does not define a bona fide diversity index.

5.1 Incorporating more information streams

Although Stirling (2007) argues that there are only three aspects (i.e., richness, balance, and disparity) that should be considered in diversity and though LC index also uses only three “information streams”: the number of species n $$ n $$ , the relative abundance vector p $$ \boldsymbol{p} $$ , and the similarity matrix Z $$ Z $$ , we could in theory consider a diversity index F c 1 … c m q $$ F\left({c}_1,\kern0.5em \dots, {c}_m;q\right) $$ with m > 3 $$ m>3 $$ , where c 1 , … , c m $$ {c}_1,\dots, \kern0.5em {c}_m $$ are various information streams. For example, one may wish to consider both the phylogenetic and functional diversity in the diversity index and we can have more than one Z $$ Z $$ . We could then follow the decomposition process of Section 4: find m ! $$ m! $$ biased decompositions, multiply them together, take the m ! $$ m! $$ -th root, and simplify. However, this is already unwieldy in the case of m = 3 $$ m=3 $$ .

One approach to circumvent this problem is to take advantage of the flexibility of the LC index by incorporating different information streams into one unified similarity matrix. As explained in Section 2.4, one can define a similarity matrix by setting Z ij = e − d i , j $$ {Z}_{ij}={e}^{-d\left(i,j\right)} $$ , where d i , j $$ d\left(i,j\right) $$ is some suitably defined distance between species i $$ i $$ and j $$ j $$ . Hence, incorporating more information streams can be thought about as changing the distance function d ⋅ ⋅ $$ d\left(\cdot, \cdot \right) $$ . In the process of incorporating such information, such as functional similarity, the ultrametricity of the similarity matrix may be lost; it is possible that the resulting function d ⋅ ⋅ $$ d\left(\cdot, \cdot \right) $$ will no longer be a metric, becoming more generally a divergence measure. The point is that p $$ \boldsymbol{p} $$ and (a suitably redefined) Z $$ Z $$ dependence of a diversity index is sufficient to incorporate all relevant information.

5.2 Limitations and future work

Our approach relies on the computation of p* which is not always possible if the similarity matrix Z $$ Z $$ is not ultrametric. Leinster and Meckes (2016) also note that if the number of species becomes high enough, the computation of p* in the non-ultrametric case can be quite demanding. Extension of our work to the non-ultrametric case relevant to phylogenetic trees and physiological traits similarity, and exploration of the ensuing computational issues is therefore a priority.

Sensitivity of similarity matrices, and hence the LC index and our decomposition, to the abundance of a species that is significantly distinct from other species in a community, is an area that needs to be explored.

Another good (linear algebra) research question is an exhaustive characterization of the trichotomy in taxonomic trees with respect to taxonomic tree equilibration, that is, understanding for which taxonomic trees any choice of similarity matrix leads to p * = p h $$ {\boldsymbol{p}}^{\ast }={\boldsymbol{p}}_h $$ , as opposed to ones only possible for particular choices of a similarity matrix, or impossible for any similarity matrix.

AUTHOR CONTRIBUTIONS

Bingzhang Chen: Conceptualization (supporting); data curation (lead); funding acquisition (lead); investigation (supporting); methodology (supporting); software (equal); visualization (equal); writing – review and editing (supporting). Michael Grinfeld: Conceptualization (lead); formal analysis (lead); methodology (lead); writing – original draft (lead); writing – review and editing (lead).

ACKNOWLEDGEMENTS

The authors sincerely thank the editors, and two anonymous reviewers for thorough and insightful comments that substantially improved the article.

    FUNDING INFORMATION

    B. Chen was supported by a Leverhulme Trust Research Project Grant (RPG-2020-389).

    CONFLICT OF INTEREST STATEMENT

    The authors declare no conflict of interest.

    DATA AVAILABILITY STATEMENT

    This article does not involve any dataset.

      The full text of this article hosted at iucr.org is unavailable due to technical difficulties.