Six simple piecewise deterministic oil production models from economics are discussed by using solution tools that are available in the theory of piecewise deterministic optimal control problems.

1. Introduction

Six simple piecewise deterministic models for optimal oil-owner behavior are presented. Their central property is sudden jumps in states. The aim of this paper is to show in admittedly exceedingly simple models how available tools for piecewise deterministic models, namely, the HJB equation and the maximum principle, can be used to solve these models analytically. We are looking for solutions given by explicit formulas. That can only be obtained if the models are simple enough. The models may be too simple to be of much interest in themselves, but they can provide some intuition about features optimal solutions may have in more complicated models.

Piecewise deterministic models have been used a number of times in economic problems in the literature; some few scattered references are given that contain such applications [1–4]. I have not been able to find references directly concerned with piecewise deterministic oil production problems. For different probability structures, and for discrete time, a host of related problems has been discussed in the literature; references to such literature have been left out, with one exception. Problems of control of jump diffusions, see [5], encompass piecewise deterministic problems, and some problems appearing in [5] are related to the ones discussed below. A classic reference to piecewise deterministic control problems is [2].

In all models below, an unbounded number of jumps in the state can occur at times 0 < τ₁ < τ₂ < τ₃ < ⋯, and, when τ_j is given, τ_j+1 is exponentially distributed in [τ_j, ∞) (all τ_i − τ_i−1 independent). Sometimes, the size of the jumps is influenced by stochastic variables V_j. Let ω = (τ₁, V₁, τ₂, V₂, …). At time t, we imagine that the control values chosen can be allowed to depend on what has happened, that is, on τ_i, V_i for τ_i < t, but not on future events, that is, τ_i, V_i for which τ_i > t. Such controls (written u(t, ω)) are called nonanticipative. Corresponding state solutions denoted by x(t, ω) are then also nonanticipative. (A general set-up, with further explanations, is given in Appendix.) Frequently below, x(t, ω) will be the wealth of the oil owner. In infinite horizon economic models, the weakest terminal condition that is natural to apply is an expectational no-Ponzi-game condition; namely, liminf _T→∞E[e^−kTx(T, ω)] ≥ 0, where k is the discount factor. Note that some stronger conditions will be used in some of the models presented in the sequel.

2. Model 1

Consider the optimization problem

()

where u is the control, subject to the state dynamics,

()

where k and x₀ are fixed positive constants,

()

α_i fixed. (Formally, only sup _iα_i < ∞ is needed.) Let τ₀ = 0. There is a fixed intensity λ of jumps; that is, for i = 1,2, …, given

, τ_i ≥ τ_i−1 (all τ_i − τ_i−1 independent), and we assume δ > k.

The interpretation of the model is that u is consumption rate, α_i is the size (in dollars) of an oil field found at time τ_i, x is wealth, and kx is interest. Oil fields are sold immediately after discovery.

Let us solve problem (1)–(3) by using the extremal method (see the appendix). Now, with H = e^−δtu^γ/γ + p(kx − u), solving the first-order condition for maximum of H, we get that u = e^δt/(γ−1)p^1/(γ−1) maximizes the Hamiltonian.

The adjoint equation (see (A.8) in Appendix) is

()

we guess that we do not need the fact that p(t, j) depends on arbitrary initial points (s, y), as in appendix. Equation (4) is satisfied by

(we try the possibility that p(t, j) is even independent of j).

Now, for , ρ : = (k − δ)/(γ − 1) > 0, we get u(t) = e^δt/(γ−1)(p(t)) ^1/(γ−1) = be^−ρt. Because H is concave in (x, u), ax is linear, and α_i is independent of x, sufficient conditions based on concavity (see Theorem 3 in Appendix) then give us that the control u = be^−ρt is an optimal control for problem (1)–(3). The optimal control is independent of λ and the α_i’s.

Write β : = b/(k + ρ). For u = be^−ρt, we have that

. The solution of this equation equals, for t ∈ (τ_i, τ_i+1),

()

To see this, find

and simply check that the differential equation is satisfied. Moreover, evidently x^*(·) satisfies x^*(τ_i+) = x^*(τ_i−) + α_i. (To ensure that x^*(t)≥0 a.s., we could have postulated at the outset that a is so large that x₀ ≥ β.)

Write x^*(t) = e^ktΦ_i(t, ω), for t ∈ (τ_i, τ_i+1), where .

We are now going to replace the bequest function ax(T) by the terminal constraint Ex(T, ω) ≥ 0. For this purpose, we are now going to vary a and hence also and β. For t = T, consider the expectation of Φ_i(t, ω). We will not give an explicit formula for this expectation, but we mention that it can be calculated in two steps, first given that i jumps have happened in [0, T] and then the expectation with i being stochastic (the rule of double expectations is then used). The expectation of the sum containing α_j as well as x₀ is positive, and the term in front of β is negative. There thus exists a unique positive value β_T of β and, hence, of a (denoted by a_T) such that EΦ_i(T, ω) = 0; that is, Ex^*(T, ω) = 0.

If we drop the term ax(T) in the criterion but add the terminal condition Ex(T, ω) ≥ 0, the free end optimality obtained in the original problem for a = a_T evidently means optimality of in the end constrained problem (the found value a_T appears in u, though now not in the criterion). (Alternatively, we could use the sufficient conditions for end constrained problems in Theorem 3 in Appendix to obtain optimality in the present end constrained problem. This would require us to check condition (A.13), which is easily done.)

Let us discuss a little more the value of β for large values of T. Now,

()

And, continuing, we get

Then, .

Now, with probability 1 an infinite number of jumps occur in [0, ∞), and

()

Denote by β_∞ the value of β for which EΦ_∞(∞, ω) = 0, and define b_∞ : = β_∞(ρ + k). For T being large, EΦ_i(T, ω) ≈ EΦ_i(∞, ω) = EΦ_∞(∞, ω), so β_T is close to β_∞ and b_T : = β_T(ρ + k) is close to b_∞. So for T being large, the optimal control is approximately b_∞e^−ρt. Note that ρ belongs to (δ − k, ∞) and both ρ and b_∞ are increasing in γ ∈ (0,1), so a larger γ means we consume more in the beginning. In the problem where ax(T) is replaced by the terminal constraint Ex(T, ω) ≥ 0, the optimal control depends on α_i, λ. It is immediately seen that β_∞ is increasing in λ and in each α_i, so also b_∞ has these properties, indicating, for T being large, that even β_T and b_T have these properties. This conclusion actually follows for any T, because

is evidently increasing in λ and in each α_i.

We may assume that the jumps are stochastic, that is, that x(τ_i+) − x(τ_i−) = V_i, where take values in a common bounded set and are independent, and are independent of the τ_i’s. If we then assume that ∑E(V_i) < ∞, the solution in this problem is the same as the one above for α_i = E(V_i).

Note that we must assume that we have a deal with the bank in which our wealth is placed that it accepts the above behavior. That is, before time 0, we have got an acceptance for the possibility of operating with this type of admissible solutions, which means that only in expectation we leave a wealth in the bank ≥0. In actual runs, sometimes we leave in the bank a positive wealth (that the bank gets), and for other runs a negative wealth (debt) that the bank has to cover.

Consider now the case where T = ∞, a = 0 in (1). Then, for

()

when β = β_∞. In this case, u = b_∞e^−ρt is optimal in the problem where liminf _T→∞E[p(T)x(T, ω)] ≥ 0 is added as an end constraint. (See the appendix, (A.14).) The latter condition is equivalent to the so-called no-Ponzi-game condition.

3. Model 2

(This model Is related to exercise 4.1 in Øksendal and Sulem [5].)

Consider the following problem:

()

where u is the control, subject to

()

where α is a given positive number, δ > γk, τ_i, i = 1,2, …, is exponentially distributed in (τ_i−1, ∞) with intensity λ_i and λ_i is decreasing towards zero (τ₀ = 0, all τ_i − τ_i−1 independent).

In contrast to problem (1)–(3), now the jumps are linearly dependent on x(τ_i−). To defend such a feature, one might argue that the richer we are, the more we are able to generate large jumps (the jump may actually represent a collection of oil finds). On the other side, we will assume that such jumps occur with smaller and smaller intensities.

It is easy to see that the current value function

is of the form

when we replace x₀ by an arbitrary start value y. For any pair (x(t, ω), u(t, ω)) satisfying (10), we can write u(t, ω) = v(t, ω)x(t, ω), v(t, ω) ≥ 0. Then, for t ∈ (τ_i, τ_i+1),

()

so u(t) = y times some function w(t, ω), t ∈ (τ_i, τ_i+1). Then, the value of the criterion for this v(·), denoted by

, is proportional to y^γ and so is also sup _v(,·,)

. A similar argument works for

, the optimal value of the criterion when j ≥ 1 jumps have already occurred at time t = 0. So

, j = 0,1, …. Writing J and A_j instead of

and

, the current value function J satisfies the following HJB equation:

()

The first order condition for maximum is γu^γ−1 − γA_jx^γ−1 = 0. So u = B_jx, for B_j = (A_j) ^1/(γ−1). Then,

()

or dividing by x^γ and rearranging,

()

Dividing by A_j gives

()

Let A_(λ) satisfy

()

Evidently, relationship (16) would be obtained from (15) if all λ_j = λ(⇒A_j = A_j+1); in that case, the optimal value function would be A_(λ)y^γ.

Now, assume first, for some k^*, that . Then, is the optimal value in the problem with no jumps, so . Given , (14) determines , . It must be the case that A_j is decreasing. Given the same start point x, the optimal value when j + 1 jumps have occurred already at time 0 must be smaller than the optimal value when j jumps have occurred at time 0; in the former problem, prospective jumps have smaller probabilities for happening. (Let us show by backwards induction that A_j, j = 0,1, 2, …, k, is nonincreasing. Evidently, A_k = A_k+1. Assume by induction that A_j ≥ A_j+1. If A_j−1 < A_j, then, by (15), + λ_j(1 + α) ^γA_j/A_j−1>−δ + γk − λ_j+1 + λ_j+1(1 + α) ^γA_j/A_j−1 > −δ + γr − λ_j+1+λ_j+1(1 + α) ^γA_j+1/A_j = . Because γ − 1 < 0 and z → (z) ^1/(γ−1) is decreasing, a contradiction is obtained.) As we will let k^* vary, denote the sequence defined by (14) for by .

Now, , so . In fact, as is easily seen, this holds for all j: . (Compare the optimal values in the case where jump j occurs at (before) t = 0 in the two problems where λ_m = 0, m ≥ k^* + 1, and where λ_m = 0, m ≥ k^* + 2.) Assume now that λ_j > 0 for all j and let . (It is shown below that, for some , , for all k^*, j.) Then, lim _j→∞A_j = A₍₀₎, and the A_j’s satisfy (14).

Let x_*(t, ω) be the solution for u ≡ 0. By (11), , where . Choose the smallest j^* such that . If j^* jumps occur at once at 0 (λ_j = ∞, j ≤ j^*), while further jumps occur with intensity , then EΓ would equal . Hence, . For any admissible solution x(t, ω), x(t, ω) ≤ x_*(t, ω), for all (t, ω); hence, for t = T, 0 ≤ E[e^−δTA_j(x(T, ω)) ^γ] ≤ when T → ∞. So (A.7) in the appendix is satisfied. Hence, sufficient conditions hold (see Theorem 2 in Appendix) and , j = 0,1, …, are optimal. That is, if and x^*(t, ω) = x₀(1 + for t ∈ (τ_i, τ_i+1) (see (11)), then u^*(t, ω) = v^*(t, ω)x^*(t, ω) is optimal (x^*(t, ω) evidently satisfies x^*(t, ω) > 0, for all t). Hence, . (In the appendix, for the sufficiency of the HJB equation to hold, it is required that u^*(t, ω) is bounded if T < ∞ (for T = ∞, we need boundedness for t in all bounded intervals). This is not the case here. But we could have replaced u by vx, v being the control. Assume that we require v(t, ω) to be bounded in the above manner. Now, v^*(t, ω) is so bounded, and it is then optimal in the set of such bounded v(·, ·)’s.)

We can show that the A_j’s are increasing in α and in each λ_i, i > j. The simplest argument is that this must be so, when we now know that A_jy^γ is the optimal value function after j jumps at (before) t = 0.

If all λ_j = λ < (δ − kγ)/α, then, using (16), when t ∈ (τ_i, τ_i+1), where 0 < ρ^* = ρ + [λ − λ(1 + α) ^γ]/(1 − γ) < ρ = (δ − k)/(1 − γ). (The inequality αλ < δ − kγ implies δ − kγ + λ − λ(1 + α) ^γ > δ − kγ + λ − λ(1 + α) > 0, so A_(λ) > 0. In particular, . The above calculations show that in the problem where , for j > j^*, the optimal value functions are , where for j ≥ j^*, given by backward induction using (14) for j < j^*, again . Evidently, . Now, for j ≥ j^* and, from (16), it follows that when j → ∞.) Now, relate the present case to what happened in Model 1. As and ρ^* < ρ, in each interval (τ_i, τ_i+1), (d/dt)u^*(t) > d/dt(be^−ρt) (be^−ρt is the optimal control in Model 1); moreover, when t passes each τ_i u^*(t) changes by a factor (1 + α).

4. Model 3

In this model, the physical volume of oil production is constant = 1, but the oil price z jumps up and down at Poisson distributed time points τ_i, i = 1,2, …. Let τ₀ = 0. Consider the problem

()

where u is the control, subject to the differential equations below and

()

where V_i is taking a finite number of values

with given probabilities p_j, all V_i are identically distributed, and τ_i is exponentially distributed in (τ_i−1, ∞) with intensity λ (all the random variables τ_i − τ_i−1, V_i independent). Let v = EV. The two states y and z are governed by (18) (no jumps in y) and

()

y(0) = y₀, z(0) = z₀, where y₀ and z₀ are given constants; in fact, it will be convenient to assume that z₀ = v. The end constraint E[y(T)] ≥ 0 is required to hold. Here, y is wealth and ay is interest earned.

Let us use the extremal method (see the appendix) for solving this problem. In what follows, it is guessed that adjoint functions do not depend on arbitrarily given initial points (which in the appendix are denoted by (s, y)). The adjoint function corresponding to y is hence denoted by p(t, i). We make the guess that p(t, i) simply equals Ke^−at (independent of i), K being an unknown. It does satisfy the adjoint equation

()

(Ep_i(t), the expectation with respect to V_i, simply equals p(t, i), since p(t, i) does not depend on V_i.) The adjoint function p^z(t, i) corresponding to the state variable z satisfies

, with p^z(T, i) = 0 (z(T) is free), we do not need the formula for p^z(t, i).

Next, maximum of the Hamiltonian H = e^−δtu^γ/γ + p(ay + z − u) is obtained for u satisfying the first order condition u^γ−1e^−δt = p, so u = (e^δtp) ^1/(γ−1) = K^βe^−κt, where κ = (a − δ)/(γ − 1) > 0, β = 1/(γ − 1).

Now,

, so

()

where C = y₀ − K^β/(a + κ).

Note that, for any s, Ez(s) = v. We want y(t) to satisfy the condition Ey(T) = 0; that is,

()

which determines K. From now on, denote the solutions y(t) and z(t) by y^*(t, ω) and z^*(t, ω), with u^*(t) = K^βe^−κt. Sufficient conditions (see Theorem 3 in the appendix) are easily seen to be satisfied. We need to check the end condition E[p(T, ω)(y(T, ω) − y^*(T, ω))] ≥ 0 (p (·, ω) = p(·)) for all admissible solutions y(t, ω) (see (A.13) in the appendix). The terminal condition is Ey(T, ω) ≥ 0, for any admissible y(t, ω), and by construction y^*(t, ω) satisfies Ep(T)y^*(T, ω) = 0, so the just-mentioned end condition is satisfied. We hence have optimality of u^*(t), y^*(t, ω), and z^*(t, ω) among admissible triples (u(t, ω), y(t, ω), z(t, ω)) for which Ey(T, ω) ≥ 0 (z(t, ω) = z^*(t, ω)).

If T = ∞, the end condition required for using the sufficient condition in Theorem 2 in the appendix is liminf _T→∞E[p(T, ω)(y(T, ω) − y^*(T, ω))] ≥ 0. The condition lim _TE[p(T)y^*(T, ω)] = 0 determines K; it now reduces to the condition y₀ + v/a = K^β/(a + κ). Moreover, we now have optimality among all triples u(t, ω), y(t, ω), z(t, ω) for which liminf _T→∞E[p(T)y(T, ω)] ≥ 0. (This inequality means that the next-to-last inequality is satisfied.) Note that K^β and so the control, quite expectably, are increasing in v = EV.

5. Model 4

In this example, y is the rate of production (production per unit of time) of oil. The oil price z jumps up or down between two values at Poisson distributed time points. The income the owner does not consume, that is, zy − u, can be used to increase the rate of production, with the change

being proportional to zy − u. (If zy − u is negative, it means that he takes money out of his business and runs it down.) For simplicity, the proportionality factor is put equal to 1, and, simply, z jumps between the values 1 and 2. So consider

()

δ > 2γ, u ≥ 0 the control, subject to

()

where τ_i is exponentially distributed in (τ_i−1, ∞) with intensity λ (all V_i, τ_i − τ_i−1 independent V₀ = 1, z₀ = 0). There are no jumps in y and y(0) = y₀ > 0, y₀ being fixed.

Let J(y, j), j = 1,2, be the current optimal value function sought, where j = z ∈ {1,2}. The current value HJB equation is as follows

()

Let us try J(y, 1) = A₁y^γ, J(y, 2) = A₂y^γ. Then, the maximizing u equals u = B_jy, B_j = (A_j) ^1/(γ−1). In the two cases j = 1 and j = 2, we get the two equations

()

The equations can be solved for positive A₁ and A₂ (shown in a moment). Automatically, the solution y^*(t, ω) corresponding to u(y, z) = u(y, j) = B_jy satisfies y^*(t, ω) ≥ 0 a.s., for all t.

To show the existence of a solution of (26) and (27), note that if we put A₂ = A₁ in (26), then by dividing by A₁, we get , and κ₁(A₁) = 0 yields . Similarly, if we put A₁ = A₂ in (27), we get , and κ₂(A₂) = 0 yields . Now, . Denote the right-hand sides in (26) and (27) by ϕ(A₁, A₂) and ψ(A₂, A₁), respectively. Both are increasing in the second place and decreasing in the first place. When , , and (the only difference in ψ(A, A) and ϕ(A, A) is the third term in the formulas). Thus, ψ(A₂, A₁) = 0 can be uniquely solved with respect to A₂ for A₁ in , denote the solution A₂(A₁), it is evidently increasing in A₁ and , . Moreover, = ψ(A₁, A₁)⇒A₂(A₁) > A₁. Consider the function β(A₁): = ϕ(A₁, A₂(A₁)) > ϕ(A₁, A₁). Now, and , so exists for which 0 = β(A₁) = ϕ(A₁, A₂(A₁)). So this and , A₂ > A₁, satisfy both (27) and (26).

For any admissible y(t, ω), , where y_*(t) is the solution of (i.e., y_*(T) = y₀e^2t), so, for all admissible y(t, ω), 0 ≤ lim _T→∞E[e^−δTA_j(y(T, ω)) ^γ] ≤ . Hence, (A.7) in the appendix holds; the sufficient conditions in Theorem 2 are satisfied, and if u = y(A_j) ^1/(γ−1) when V_i = j ∈ {1,2}, then u is optimal. (Hence, if + and y^*(t, ω) is the solution of , then u^*(t, ω) = v(t, ω)y^*(t, ω) is optimal.)

Define A^* = [(δ − 3γ/2)/(1 − γ)] ^γ−1 (i.e., A^* satisfies 0 = −δA + A^γ/(γ−1) + 3γA/2 − γAA^1/(γ−1)). When λ = 0 then , and one may show that when λ > 0 decreases, A₂ increases towards while A₁ decreases towards . When λ increases to infinity, A₁ increases to A^*, and A₂ decreases to A^*. In fact, and . To prove convergence to A^*, note first that A₁, A₂ are bounded uniformly in λ. If A₁ and A₂ did not converge to a common number, say A_*, when λ → ∞, the two last terms in (27) and (26) would blow up to infinity for (certain) large values of λ, while the remaining terms would be bounded, a contradiction. Summing the two equations in (27) and (26) gives . Hence, A_* satisfies ; that is, A_* = A^*.

Two very simple models discussed below contain the feature that the owner can influence the chance of discovery but that it is costly to do so. In the first one, the intensity of discoveries λ is influenced by how much money is put into search at any moment in time; in the second one, it is costly buildup of expertise that matters for the intensity of discoveries.

6. Model 5

Consider the problem

()

and λ = μu^1−κ/(1 − κ), where μ > 0, κ are given constants, κ ∈ (0,1), u ≥ 0 is the control, and λ is the intensity of jumps. All fields found are of the same size α. We imagine that fields are sold immediately when they are found or that they are produced over a fixed period of time, with a fixed production profile. In both cases, we let income from a field discounted back to the time of discovery be equal to g₀ : = βα, for a given β > 0. The oil owner wants to maximize the sum of discounted incomes earned over [0, ∞).

Let us try the proposal that the current value optimal value function J = a, where a, an unknown. The current value HJB equation is

()

which implies that the maximizing u satisfies μg₀u^−κ = 1, so u = (μg₀) ^1/κ, −u + μg₀u^1−κ/(1 − κ) =

, and

. Trivially, (A.7) in the appendix holds, so the sufficient conditions in Theorem 2 are satisfied and u = (μg₀) ^1/κ is optimal. Because a > 0, in expectation (but not in all runs), over [0, ∞), a positive sum of discounted incomes ( = a) has been earned.

Often, it is the case that the best fields are exploited first hence, x(τ_i+) − x(τ_i−) = α_i, where α_i is decreasing and even α_i ↓ 0. Then, when j jumps are imagined to have happened already at time 0, we guess J(j) = a_j. Define

. Then, the HJB equation is

()

and

yields maximum. Postulate for the moment that, for some j,

. Then,

= ν_j(a_j+1). Thus, if we try a_j = a_j+1, we get δa_j < ν_j(a_j). If we try

, we get δa_j > ν_j(a_j) = 0. So a correct value of

exists such that δa_j = ν(a_j). This continues backwards and gives a₀ > a₁ > ⋯>a_j+1. If, for some j^*, α_j = 0, for j ≥ j^* + 1, then

, so in this case, (30) gives

. If all α_j > 0, letting j^* → ∞, it is easily seen that there exists an infinite decreasing sequence a_j ↓ 0 when j → ∞ satisfying (30) (compare a similar argument in Model 2).

Still, trivially condition (A.7) below holds, sufficient conditions in Theorem 2 in the appendix are satisfied and on [τ_j, τ_j+1), j = 0,1, 2, …, are optimal. (Uniqueness of the a_j’s follows from optimality.)

7. Model 6

Consider the problem

()

no jumps in y, λ = μy, μ, κ are fixed positive numbers, κ < 1, λ is the intensity of jumps in x.

All fields found are of the same size α. We again imagine that fields are sold immediately when they are found or that they are produced over a fixed period of time, with a fixed production profile. Let income from a field discounted back to the time of discovery be equal to g₀ : = βα, for a given β > 0. The state y(t) is the amount of expertise available for finding new fields, built up over time according to , where u is money per unit of time spent on building expertise.

Let us try the proposal that the current value function J equals a + ky, a and k being unknowns. The current value HJB equation is

()

Then, kδ = μg₀, so k = μg₀/δ. Moreover, the maximizing u equals u = k^1/κ, and −u + ku^1−κ/(1 − κ) =

. The optimal solution y^*(t) equals tb, where b = k^(1−κ)/κ/(1 − κ). Evidently, (A.7) below is satisfied, because J = e^−δT(a + ky(T)) ≥ 0 for any admissible x(·), y(·) and e^−δT(a + ky^*(T)) → 0 when T → ∞. Hence, u = k^1/κ = (μg₀/δ) ^1/κ is optimal.

8. Comparisons

In Models 1, 2, 5, and 6, oil finds are made at stochastic points in time; in Models 3 and 4, it is the price of oil that changes at stochastic points in time. In Model 1, we operate with the constraint Ex(∞, ω) ≥ 0 (for T = ∞, a = 0), where x is the oil-owners’ wealth. Here, for some runs, x(∞, ω) can be negative (β > x₀) and, for other runs, positive. In Model 2, we required x(t, ω) > 0 for all t, all ω. (The results in that model would have been the same if we had required only x(∞, ω) ≥ 0 for all ω.) In Model 2, the optimal control comes out as stochastic and not deterministic as in Model 1. Moreover, as a comment in Model 2 says, as a function of time, the optimal control decreases more rapidly in Model 1 as compared to Model 2. The latter feature stems from the fact that, in Model 2, we enhance future income prospects by not decreasing x too fast, because the jump term (the right-hand side of the jump equation) depends positively on x, which is not the case in Model 1.

In Models 3 and 4, the oil price exhibits sudden stochastic jumps. In Model 3, the rate of oil production is constant, but income earned (as well as interest) is placed in a bank after subtraction of consumption. In Model 4, income earned, after subtraction of consumption, is reinvested in the oil firm to increase production. In Model 4, the optimal control is stochastic; it depends on whether the current price is high or low. In Model 3, the control is deterministic, and it depends only on the expectation v of the stochastic price. Consider the case where, in Model 3, the expected price v is zero and a = 3/2. Then, u = K^βe^−κt, where κ = (3/2 − δ)/(γ − 1) and K^β = y₀(3/2 + κ). We saw in Model 4 that when the intensity of jumps λ is very high, A₁≃A₂≃A^*, where A^* = [δ − 3γ/2)/(1 − γ)] ^γ−1; we hardly pay attention to what the current price is, because it changes so frequently. Now u = y(A_j) ^1/(γ−1) when the current price is j( = z), or u≃y(A^*) ^1/(γ−1) when λ is large. When λ is large (so z switches very frequently between 1 and 2), the stochastic path of the equation most often is very close to the deterministic path of = (3/2)y − [(δ − 3γ/2)/(1 − γ)]y = −κy. The latter equation has the solution y = y₀e^−κt, and the corresponding u equals = (3/2)y₀e^−κt + κy₀e^−κt = κy₀e^−κt = y₀(3/2 + κ)e^−κt, the same control as that obtained in Model 3 in the case a = 3/2, v = 0.

In the extremely simple Models 5 and 6, the frequency of oil finds is not fixed but influenced by a control. In Model 5, the current frequency (or intensity) is determined by how much money is put into search at that moment in time. In the simplest case considered in Model 5, a find today does not influence the possibility of making equally sized discoveries tomorrow. Then it, is not unreasonable that the optimal control (which equals (μg₀) ^1/κ) is independent of the discount rate δ but dependent on the fixed value of the finds g₀ = βα. (Here, actually α could be the expected size of a find, in both Model 5 and Model 6; we could have had the sizes of the finds being independently stochastic, with α being the expected value of the sizes.) In Model 6, it is the ability to discover oil that is built up over time, so with greater impatience (higher δ), we should expect less willingness of devoting money to increase this ability, and this shows up in the formula u = (μg₀/δ) ^1/κ.

Acknowledgment

The author is very grateful for useful comments received from a referee that made it possible to improve the exposition and remove some typographical errors.

Appendix

Consider the problem

()

x⁰, U, T > 0 fixed, u is the control, with criterion

()

Here, τ_j and V_j, j = 1,2, …, are random variables, with all τ_j − τ_j−1 and V_j being independent (τ₀ = 0). Given τ_j−1, the intensity of the jump τ_j in [τ_j−1, ∞) is λ_j(t, x, u). (So for given nonanticipative function x(t) = x(t, ω), u(t) = u(t, ω), for τ ≥ τ_j−1, the probability distribution of τ = τ_j is

Assume that the five given functions f₀, f, g₀, g, and h₀ are continuous and Lipschitz continuous with respect to x with a common rank κⁿ for u ∈ U∩B(0, n), independently of (t, u, V), and also that these functions satisfy an inequality of the form |ϕ(t, x, u, V)| ≤ α_n + κ_n|x| for all (t, x, u), u ∈ U ∩ B(0, n), all V. Also, λ_j(·, ·, ·) is continuous and for all n^′. Define u(t, ω) to be admissible if u(t, ω) ∈ U, for all (t, ω), sup _t,ω|u(t, ω)| < ∞, and if u(t, ω) is nonanticipative and is separately piecewise continuous in each real variable on which it depends. Then, corresponding to u(t, ω), for each ω, there exists a nonanticipative function x(t, ω) (also called admissible), which is piecewise continuous in t, satisfying (A.2) and (A.1), for t ∈ (τ_j, τ_j+1), j = 0,1, … (x(t, ω) becomes piecewise continuous in each real variable on which it depends). If there are terminal conditions in the problem, u(t, ω) and x(t, ω) are called admissible if also the terminal conditions are satisfied. For pairs u(t, ω) and x(t, ω) to be called admissible, we can allow additional restrictions; namely, for given sets Q_j(t) ⊂ ℝⁿ, t ∈ (0, T), j = 0,1, …, it can be required that x(t, ω) has to belong to Q_j(t) a.s. for t ∈ (τ_j, τ_j+1).

HJB Equations. The following condition is sufficient for optimality. (Below, j = 0,1, 2, … indicates the number of jumps that have already happened).

Theorem 1. Assume that there exist functions J(s, y, j) and controls u_j(t, x), j = 0,1, 2, …, with J(T, y, j) = h₀(y), C¹ in (s, y) ∈ [0, T] × Rⁿ, j = 0,1, 2, …, such that the functions J(s, y, j), for all (s, y), satisfy the HJB equation

()

and assume that each u_j(s, y) yields maximum in this equation. Moreover, assume that a solution x^*(t, ω) of

, with jumps given by (A.2), exists such that

. Assume also that, for some constants a_J and b_J, |J(s, y, j)| ≤ a_J + b_J|y| for all (s, y, j). Then,

is optimal.

See pages 147 and 155 (and for a proof, see page 168) in [6]. (Formally, we only need to assume that the nonanticipative function u^*(t, ω) is measurable and bounded.)

In case of restrictions of the form x(t, ω) ∈ Q_j(t), we must assume that x^*(t, ω) satisfies these restrictions, in this case J(s, y, j) needs to only be C¹ in a neighborhood of each point in , with J(s, y, j) C⁰ in cl (cl = closure). Then, (A.4) needs to hold only at each point in .

If there are terminal conditions in the problem, replace J(T, x, j) = h₀(x) by

()

for all admissible x(·, ·).

If h₀ ≡ 0, [0, T] is replaced by [0, ∞) and if f₀ = e^−βtf⁰(x, u), g₀ = e^−βtg⁰(x, V), β > 0, and g, λ_j, and f are independent of t, and given a sequence u_j(x), then a sufficient condition for optimality of u^*(t, ω) (catching up optimality in case infinite values of the criterion appear) is as follows.

Theorem 2. For j = 0,1, 2, …, assume that some functions u_j(y) and (C¹ in y) exist, such that satisfy the current value HJB equation

()

for j = 0,1, 2, …, with u_j(y) yielding maximum. The functions g⁰, f⁰, f, g, and

are postulated to be continuous and Lipschitz continuous in x with rank κⁿ for u ∈ U∩B(0, n) and all V and to satisfy a growth conditions of the form |ϕ(x, u, V)| ≤ α_n + κ_n|x| for all V, x, u ∈ U∩B(0, n). Finally,

for each n^′. (It suffices to assume that, for any admissible pair (x(t, ω), u(t, ω), for t in bounded intervals, sup _t,ω λ_j(x(t, ω), u(t, ω)) < ∞.) Assume that a solution x^*(t, ω) of

, with jumps given by (A.2), exists. Assume also that, for some constants a_J and b_J,

for all (y, j). Define u^*(t, ω): =

and assume that sup _t,ω|u^*(t, ω)| < ∞ for t in bounded intervals. (If restrictions x(t, ω) ∈ Q_j, t ∈ (τ_j, τ_j+1) are present, one must have x^*(t, ω) ∈ Q_j; in this case, it suffices that

is C¹ in some open set containing Q_j, with

in cl Q_j, and (A.6) needs to hold only in Q_j.) Then, u^*(t, ω) is optimal, provided

()

for all admissible x(·, ·).

(This is the infinite horizon current value form of (A.4); compare (3.71) and (3.69), page 150 in [6].)

The Extremal Method. The so-called extremal method (see page 117 and pages 126–130 in [6]) now to be described yields solutions satisfying a maximum principle (necessary condition). Now, it is assumed that sup _jλ_j < ∞ and that λ_j does not depend on (t, x, u). To the assumptions above on f₀, f, g₀, g, and h₀, we now add the assumption that these five functions have derivatives with respect to x that are continuous in (t, x, u, V). Let I be the identity matrix and let H = f₀ + pf. For arbitrary (s, y), we seek controls u(t; s, y, j) and solutions x(t; s, y, j), j = 0,1, …, u(t; s, y, j) yielding maximum of H(t, x(t; s, y, j), u, p(t; s, y, j)), where each p(t) = p(t; s, y, j) is a solution on [s, T] of

()

with end condition

and where each x(t) = x(t; s, y, j) is a solution of

with initial condition x(s; s, y, j) = y (j = the numberofjumpsthathashappened).

Frequently, to find x(t; s, y, j), u(t; s, y, j), and p(t; s, y, j), as in deterministic problems, one first seeks a control

that maximizes H(t, x, u, p), and then one solves the equations

and

()

with side conditions

()

Evidently, to solve the equations

and (A.9) to obtain x(t) = x(t; s, y, j) and p(t) = p(t; s, y, j), knowledge of p(t; s, y, j + 1) is needed. If and only if λ_N+1 = λ_N+2 = ⋯ = 0 for some N, a backward recursion is possible, finding first x(·, ·, N) and p(·, ·, N), then x(·, ·, N − 1), p(·, ·, N − 1), x(·, ·, N − 2) and p(·, ·, N − 2), and so on. When x(t; s, y, j) and p(t; s, y, j) are known,

The entities x(t; s, y, j) and u(t; s, y, j) are used to piece together a nonanticipative pair x^*(t, ω), u^*(t, ω) defined for t∈[0, T]. On [0, τ₁], define u^*(t, ω) = u(t): = u(t; 0, x₀, 0) and x^*(t, ω) = x(t): = x(t; 0, x₀, 0). On (τ₁, τ₂],

()

Generally, when x^*(t, ω) = x(t, τ₁, V₁, …, τ_j−1, V_j−1) has been defined on (τ_j−1, τ_j], then on (τ_j, τ_j+1]

()

and

(where, for j = 0, the u- and x-functions in the sums are simply u(t) and x(t)). Let

. (These entities in fact satisfy a maximum principle not stated here: see page 126 and page 131 in [6].) (If there are restrictions of the form x(t − , ω) ∈ Q_j(t) in the problem, one only needs to construct solutions for (s, y) ∈ Q_j(s), for j > 0, for j = 0, only for (s, y) = (0, x₀). Eventually, one has to test if x^*(t − , ω) ∈ Q_j for t ∈ (τ_j, τ_j+1.)

So far, we have rendered conditions pertaining to a free terminal state problem. If there are hard terminal conditions (hard = holding a.s.) or soft terminal conditions (holding in expectation), then certain transversality conditions have to be satisfied by the p(t, ω)-functions; we may have to allow H-functions p₀f₀ + pf, where p₀ ∈ {0,1}, and for hard terminal conditions, certain controllability conditions have to be satisfied. Such conditions are omitted here, since our main interest is sufficient conditions; what we then need is a sort of transversality condition at the terminal time as given by (A.13), working for all types of terminal restrictions.

Assume now that entities x(t; s, y, j), u(t; s, y, j), p(t; s, y, j), x^*(t, ω), and u^*(t, ω) have been constructed by means of the extremal method described above, and define . Then, the following sufficient condition holds (here p₀ = 1).

Theorem 3. If all the entities h₀(x), g₀(t, x, V), , and p(t; s, y, j)g(t, x, V) are concave functions of x, then the control u^*(t, ω) is optimal in the free end case. (In case of restrictions of the form x(t − , ω) ∈ Q_j(t), again x^*(t, ω) must of course satisfy them, Q_j(t) must be convex, and, for each j, the just-mentioned concavity needs to hold only on Q_j(t). If U is convex, then concavity of (x, u) → H(t, x, u, p(t; s, y, j)) in Q_j(t) × U suffices for concavity of in Q_j(t).) In case one has terminal conditions on the solutions, then u^*(t, ω) is optimal, provided also that the following condition holds:

()

that is, for all x(·, ω) satisfying the terminal conditions.

In the end constrained case, p(T, ω) is perhaps different from h_0x(x^*(T, ω)). See proof of Theorem 2, page 30 in [7]. Note that (A.13) is implied by standard transversality conditions for soft end constraints that are omitted here; see (3.62), page 146 in [6] for the case h₀ ≡ 0.

When T = ∞, these concavity conditions again ensure optimality (perhaps catching up optimality) provided (see [7], page 32)

()

Boundedness, Lipschitz—and growth—conditions on u^*, g⁰, f⁰, f, and g are required, as in Theorem 2.

References

All articles

Optimal Oil-Owner Behavior in Piecewise Deterministic Models

Abstract

1. Introduction

2. Model 1

3. Model 2

4. Model 3

5. Model 4

6. Model 5

7. Model 6

8. Comparisons

Acknowledgment

Appendix

References

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Optimal Oil-Owner Behavior in Piecewise Deterministic Models

Abstract

1. Introduction

2. Model 1

3. Model 2

4. Model 3

5. Model 4

6. Model 5

7. Model 6

8. Comparisons

Acknowledgment

Appendix

References

References

Related

Information