We consider the following fractional-order system with Hartree-type nonlinearity: where N ≥ 2, $$, and × denote the convolution of functions. Throughout this paper, for any $$, we denote by $$ the Sobolev space with the norm as follows: where $$ denotes the Fourier transformation of function f, that is, In addition, we will also use the notation $$ to denote the fractional Sobole space, whose detailed definition is seen in Di Nezza et al. [1].

A pair of weak solutions (u, v) to Equation (1) is defined in the distributional sense. That is, we say that $$ is a pair of weak solutions to Equation (1) if and only if for any $$, where By taking limits, we remark that Equation (2) is also true for any $$. It is well-known that the fractional-order equations and systems have many practical applications [2–5]. Especially, the fractional-order systems of type in Equation (1) naturally arise in the Hartree–Fock theory of the nonlinear Schrödinger equations and systems ([6] and the references therein). Indeed, if α = β and u = v in Equation (1), then the system is Equation (1) reduces to the following nonlocal equation: This type of nonlocal equation goes back to the description of the quantum theory of a polaron at rest by Pekar in 1954 [7] and the modeling of an electron trapped in its own hole in 1976 in the work of Choquard, as a certain approximation to Hartree–Fock theory of one-component plasma [8]. And the solutions can be regarded as a crucial threshold or criterion for global well-posedness and scattering in the focusing case. Thus, the study on the regularity and classification of solutions to Equation (1) are more meaningful and important. In the past years, there are many literatures on the classification of positive solutions to fractional-order equations and systems or integral equations and systems ([9–14] and the references therein). Especially, Liu [15] obtained the classification results of positive solutions to Equation (3) with α = 2. Dai et al. [10, 16] classified all the $$ weak solutions to Equation (3) via the method of moving planes. It is well-known that the method of moving planes is a powerful tool in the research of classification of positive solutions, and the method was initially introduced by Alexandrov [17] and refined by Serrin [18]. Later, it was further developed by Gidas et al. [19], Caffarelli et al. [20], Chen and Li [21], Li and Zhu [22] (moving spheres), Chen et al. [9], Chen et al. [23], and many others. In addition, we also mention that Ma et al. [24] established a lifting regularity lemma and hence derived the Lipschitz continuity of solutions for an integral system of Wolff type. The lifting regularity lemma has been widely applied to study the integrability and regularity of solutions for some nonlinear problems ([16, 25] and the references therein).

In this paper, we will follow the general lines of the regularity lifting lemma and the method of moving planes and give the regularity and classification of weak solution to Equation (1). Here, we emphasis that the problem in Equation (1) is quite different from the previous literature, since the problem in Equation (1) has two nonlocal terms at the same time; that is, the convolution and the operator $$ (s = α, β) are all nonlocal. Secondly, α and β may be also not equal in Equation (1), which have caused us some obstacles to obtain the classification of positive solutions. Fortunately, by more careful and refined analysis, we can overcome those obstacles and obtain the classification results. More specifically, we first establish the equivalence relationship between Equation (1) and the following integral system: where R_s,N = Γ((N − s)/2)/(π^N/22^sΓ(s/2)). We then use the regularity lifting lemma and the method of bootstrap to obtain the integrability and smoothness of the weak solutions. We finally prove that the solutions of Equation (4) are radially symmetric about some point $$ via the method of moving planes in an integral form and derive the explicit forms for u and v. Further, by the equivalence of Equations (1) and (4), we conclude that the weak solutions of Equation (1) have the same classification results. More specifically, we have the following theorems.

In this section, we will give the detailed proof of Theorems 1–3. For the sake of talking convenience, let us first give the regularity lifting lemma [24, Lemma 2.2], which will play an important role in showing the integrability and smoothness of the solutions. That is, let V be a topological vector space and assume that there exist two extended norms defined on V: Let

In addition, we also need the Hardy–Littlewood–Sobolev inequality [12]. That is, let p, q > 1 and 0 < s < N with $$, $$. Then, one has where C depends on s, p, q, and N.

Proof of Theorem 1. Let Firstly, we assume that $$ is a pair of weak solutions to Equation (1). Define Then $$ and $$. By the definition in Equation (2) of weak solutions to Equation (1), we have the following: Obviously, by the definition of $$ and integrating by parts, one has the following: In addition, by exchanging the order of integration, we have the following: It follows from Equations (8)–(10) that By the arbitrariness of ϕ and Equation (11), we have the following: Similar to the proof of Equation (11), if we set Then we also have which also implies that Hence, we prove the (u, v) is also a pair of solutions to Equation (4).

In what follows, we prove the solutions to Equation (4) must be also the solution to Equation (1). Indeed, let $$ be a pair of solutions to Equation (4). Then, by taking the Fourier transform to the two equations of (4), we get the following: For any $$, it follows that and This means that $$ is a pair of weak solutions of Equation (1).

Proof of Theorem 2. For the sake of brevity, we first use the notation C to denote a general positive constant that may depend on s, p, q, and N. Also, it may change from line to line. Now, let P_α and P_β are as in Equation (7). Then, by Equation (6), it is easy to verify that Similarly, we also have Hence, we obtain $$ and $$.

Now let T_α and T_β be defined as follows: and Then, for any $$ and $$, by using Equation (6) and Hölder’s inequlity, one has the following: and Obviously, from Equations (15) and (16), we know that T_α maps $$ into itself and T_β maps $$ into itself. To apply Lemma 1, let and let A > 0: where χ_Ω denotes the characteristic function of set Ω. For any A > 0, define where It follows from Equations (15) and (16) that Since $$ and $$, we can choose A large enough such that This means that Therefore, for any $$ and $$, we know that T_A is a contracting operator from $$ to itself for sufficiently large A. Let and Then, if (u, v) is a pair of solutions to Equation (4), it is easy to verify that (u, v) satisfies the following operator equation: Meanwhile, by the definition of F and G, for given A > 0, it is easy to verify that Indeed, it is obvious that F(x) ≥ 0 for all $$. Besides, for given A > 0, one has the following: Similarly, we also have the following: Therefore, for any $$, by Equations (17) and (18), one has the following: and On the other hand, for any $$ and $$, by Equation (6) and Hölder’s inequality, it is also easy to verify that if $$, then In particular, when $$, one also has the following: It follows from Equations (19)–(21) that whenever r and s are in the following range: Hence, for any $$, it follows from Lemma 1 that $$ whenever r and s are in the range in Equation (23). Let us repeat the above processes n-times until N ≤ min{(2n + 1)α, (2n + 1)β}. Then, we finally get $$ whenever r and s are in the following range: Further, we also can obtain the following: Now, for given R > 0, denote B_R = B_R(x), we rewrite the following: Note that $$ and $$. Then, it is from that Hölder’s inequality that and where and Similar to Equations (24) and (25), we also have the following: It follows from Equations (24)–(26) that $$ and $$. Further, due to $$, by Hölder’s inequality, we also have the following: That is to say, $$. Similarly, it is also not difficult to see that $$. Hence, we know that $$ for any $$. Now, for simplicity, let and Then, by Equation (1), we deduce that $$ for (p, q) ∈ A_α,β, which implies that $$ for any (r, s) ∈ B_α,β, and hence, we have $$. Further, if $$, then $$ for any (p, q) ∈ B_α,β. Therefore, by Equation (6) and Hölder’s inequality, one has the following: This means that $$ for any (r, s) ∈ B_α,β, and hence, we get $$. This completes the proof of Theorem 1.2.

Proof of Theorem 3. Let $$ and λ be an arbitrary real number, Define where P_α and P_β are as in the proof of Theorem 1. Let

Step 1. There exists $$ such that for all $$, Indeed, for any x ∈ Σ_λ, by the fact that |x^λ − y| = |x − y^λ| and |x^λ − y^λ| = |x − y|, we have the following: Here and hereafter, $$ is defined as follows: Similar to the proof of Equation (29), for any x ∈ Σ_λ, we also have the following: Now, in order to obtain Equation (28), let Then, for any $$, it follows from Equation (29) that where we have used the estimate |x − y| ≤ |x^λ − y| for any x, y ∈ Σ_λ and Similarly, for any $$, it follows from Equation (30) that where Let it follows from Equations (31) and (6) and Hölder’s inequality that where Now, let Note $$. Hence, similar to the proof of Equation (28), from Equation (32), one has the following: By the integrability of P_α and P_β, we can choose N₀ > 0 such that if λ < −N₀, Then, it follows from Equations (33) and (34) that and Note that $$. Then, from Equations (36) and (37), we deduce that By the integrability of u and v, we can also choose $$ such that if $$, It follows from Equation (39) that $$ and hence $$ must be measure zero and empty for all $$. Similarly, $$ must also be measure zero and empty for all $$. This completes the proof of Step 1.

In fact, Equation (28) provides a starting point to move the plane T_λ, and hence, we can start from this neighborhood of x₁ = −∞ and move the plane to the right as long as Equation (28) holds to the limiting position and argue that the solution pair (u, v) must be symmetric about the limiting plane. More precisely, let By using the similar argument in Step 1 and moving the plane T_λ from near x₁ = +∞ to the left, we must have λ₀ < ∞. Now, we will show that Otherwise, we may assume on $$: Obviously, by Equations (29) and (30), we can know that $$ on $$ if $$ on $$ and vice versa. Furthermore, by means of Equations (29) and (30), we can derive $$ and $$ in the interior of $$. Duo to this fact, we may show that the plane can be moved further to the right. More precisely, there exists ε > 0 such that for any λ ∈ [λ₀, λ₀ + ε), This will be a contradiction with the definition of λ₀. To prove the contradiction, it can be clearly seen from Equations (33) and (34) in Step 1 that our primary task is to prove that we can choose ε > 0 sufficiently small such that for all λ ∈ [λ₀, λ₀ + ε), and where C is the same as in Equations (33) and (34). In fact, by the integrability conditions on P_α, P_β, u, and v, we can choose R > 0 large enough, such that and Now, we fix the above R; in order to derive Equations (41) and (42), by the integrability conditions on P_α, P_β, u, and v, we only need to show that In fact, for any δ > 0 and λ > λ₀, let and Then, it is easy to verify that For fixed η > 0, by Equation (46), we can choose δ > 0 small enough such that μ(F_δ) ≤ η. For this fixed δ, we claim that Indeed, for any $$, we have the following: Hence, we obtain the following: It follows from Chebyshev inequality that as λ → λ₀, where $$. This finished the proof of Equation (48). Hence, by Equations (46)–(48), we have the following: By the arbitrariness of η > 0, Equation (50) implies that Similar to the proof of Equation (51), we can also verify that Obviously, by Equations (51) and (52), we finished the proof of Equation (45). Therefore, from Equations (43)–(45), one has Equations (41) and (42). This means that there exists ε > 0 such that for all λ ∈ [λ₀, λ₀ + ε), This is a contradiction with the definition of λ₀. Hence, Equation (40) must be hold true. Furthermore, by the definition of P_α and P_β, it is easy to verify that $$ and $$ for all $$. In addition, because of the fact that Equation (4) is invariant under rotation and the x₁ direction can be chosen arbitrarily, this means that the positive solution $$ must be radially symmetric and monotone, decreasing about some point $$. That is, we finished the proof of Step 2.

We use the analytic method in Chen et al. [9] and divide several small steps to complete the proof of this step. Firstly, by the conclusion of Lemma 4.1 in Dai et al. [16], we know that Equation (5) is a pair of solutions to Equation (4). Meanwhile, we claim that the system in Equation (4) is invariant under the scaling In fact, if (u, v) is a pair of solutions to Equation (4), then Similarly, we also have the following: From Equations (54) and (55), we know that the system in Equation (4) is invariant under the scaling of Equation (53).

Secondly, let (u, v) be a pair of solutions to Equation (4); we claim that there exist two positive numbers u_∞ and v_∞ such that and Now let us verify Equations (56) and (57) by contradiction. Without loss of generality, assume that Equation (56) does not hold, and let Note that $$ and $$ are usually called the Kelvin transform u and v, respectively. Then $$ also satisfies the integral system in Equation(4) in $$. Indeed, let Then, by a direct calculation, one has the following: Similarly, we also have the following: Hence, by Equation (59), one has the following: By Equation (60), similar to the proof of Equation (60), one has the following: From Equations (61) and (62), we know that $$ also satisfies the integral system in Equation (4) in $$. Let x¹ ≠ x² be any two points in $$ and $$.

Now we consider the Kelvin transform of u, v centered at x⁰. That is, let us define the following: and According to our hypothesis (Equation (56) does not hold), U(x) must have a singularity at x₀. Meanwhile, from $$, one has the following: In addition, by the above proof, we know that (U, V) is also a pair of solutions to Equation (4). Therefore, by the same arguments as in Step 1 and Step 2, we can deduce that U and V are radially symmetric and monotone, decreasing about point x⁰, since x₀ is a singular point of U. This means that u(x¹) = u(x²) for any two points x¹ and x² in $$, and hence, u must be a constant. Further, by the system of integral in Equations (4) or fraction order differential system in Equation (1), we know that u must be zero, which contradicts with u > 0 in $$. Therefore, we proved that u and v must satisfy the asymptotic property in Equations (56) and (57), respectively.

Thirdly, let $$. From Step 1 and Step 2, we can assume that (u, v) is a pair of solutions to Equation (4) centered at zero and s^N−αu(ξ) = u_∞ and s^N−βv(ξ) = v_∞. Then, we claim the following: In fact, we only need to consider ξ = 0. Otherwise, Equation (63) follows by considering (u(⋅+ξ), v(⋅+ξ)) instead of (u, v). Let x⁰ ≠ 0 and $$. Let Then we have $$ and $$. Moreover, $$ is a pair of solutions to Equation (4). By the proof of Step 1 and Step 2, it is easy to verify that $$ must be radially symmetric about $$. Hence, similar argument as in the proof of Lemma 3.1 in Chen et al. [9], we get Equation (63):