Search

Conservation and management decision making in natural resources is challenging due to numerous uncertainties and unknowns, especially relating to understanding system dynamics. Adaptive resource management (ARM) is a formal process to making logical and transparent recurrent decisions when there are uncertainties about system dynamics. Despite wide recognition and calls for implementing adaptive natural resource management, applications remain limited. More common is a reactive approach to decision making, which ignores future system dynamics. This contrasts with ARM, which anticipates future dynamics of ecological process and management actions using a model-based framework. Practitioners may be reluctant to adopt ARM because of the dearth of comparative evaluations between ARM and more common approaches to making decisions. We compared the probability of meeting management objectives when managing a population under both types of decision frameworks, specifically in relation to typical uncertainties and unknowns. We use a population of Sandhill Cranes as our case study. We evaluate each decision process under varying levels of monitoring and ecological uncertainty, where the true underlying population dynamics followed a stochastic age-structured population model with environmentally driven vital rate density dependence. We found that the ARM framework outperformed the currently employed reactive decision framework to manage Sandhill Cranes in meeting the population objective across an array of scenarios. This was even the case when the candidate set of population models contained only naïve representations of the true population process. Under the reactive decision framework, we found little improvement in meeting the population objective even if monitoring uncertainty was eliminated. In contrast, if the population was monitored without error within the ARM framework, the population objective was always maintained, regardless of the population models considered. Contrary to expectation, we found that age-specific optimal harvest decisions are not always necessary to meet a population objective when population dynamics are age structured. Population managers can decrease risks and gain transparency and flexibility in management by adopting an ARM framework. If population monitoring data has high sampling variation and/or limited empirical knowledge is available for constructing mechanistic population models, ARM model sets should consider a range of mechanistic, descriptive, and predictive model types., (© 2018 by the Ecological Society of America. This article has been contributed to by US Government employees and their work is in the public domain in the USA.)

We deal with the problem of managing a project or a complex operational process by controlling the execution pace of the activities it comprises. We consider a setting in which these activities are clearly defined, are subject to precedence constraints, and progress randomly. We formulate a discrete-time, infinite-horizon Markov decision process in which the manager reviews progress in each period and decides which activities to expedite to balance expediting costs with delay costs. We derive structural properties for this dynamic project expediting problem. These enable us then to devise exact solution methods that we show to reduce computational burden significantly. We illustrate how our method generalizes and can be used to tackle a wide range of so-called stochastic shortest-path problems that are characterized by an intuitive property and can capture other applications, including medical decision-making and disease-modeling problems. Moreover, we also deal with the state identification issue for our problem, which is a challenging task in and of itself, owing to precedence constraints. We complement our analytical results with numerical experiments, demonstrating that both our solution and state identification methods significantly outperform extant methods for a supply chain example and for various randomly generated instances. This paper was accepted by Chung Piaw Teo, optimization. Funding: R. Mogre acknowledges support from the U.S.-UK Fulbright Commission and the Lloyd's Tercentenary Research Foundation through the Fulbright-Lloyd's Scholar Award, which allowed him to spend an extended period of time at Massachusetts Institute of Technology. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.4876. [ABSTRACT FROM AUTHOR]

Container slot allocation represents a critical operational decision-making challenge within the liner shipping industry, which necessitates making decisions on the transportation of loaded containers and the repositioning of empty ones under stochastic demand. We study novel dynamic allocation policy that leverage sequentially-revealed demand information to determine the slot allocation decision for both loaded and empty containers at each stage. In this paper, we develop a stochastic dynamic programming (DP) model to optimize the slot allocation decision for maximizing the expected total revenue over the planning horizon. To solve this model, we design an efficient allocation policy that makes slot allocations at each stage based on current empty container stocks, realized demand, and the mean demand at future stages. Comprehensive numerical experiments on both synthetic and realistic data demonstrate substantial revenue improvement of our approach over the commonly-used benchmark policies in practice and literature. • We develop a stochastic dynamic program for container slot allocation with empty container repositioning under stochastic demand. • We design a dynamic allocation policy that sequentially determines the slot allocation decision based on the realized information at each stage. • A series of numerical experiments are carried out to show that our approach can outperform the benchmark policies used in practice and literature. [ABSTRACT FROM AUTHOR]

Considerable resources have been invested in ecological restoration projects across the globe to restore ecosystem integrity. Restoration strategies are often diverse and have been met with mixed success. In this article, we describe the Chinook salmon (Oncorhynchus tshawytscha) decision‐support models developed by the Central Valley Project Improvement Act Science Integration Team as part of a larger structured decision‐making effort aimed at maximizing natural adult production of Chinook salmon in California's Central Valley, the United States. We then describe the decision‐analytic tools the stakeholder group used to solve the models and explore model results, including stochastic dynamic programming, forward simulation, proportional scoring, relative loss, expected value of perfect information, response profile analyses, and indifference curves. Using these tools, the stakeholder group was able to develop and evaluate restoration strategies for multiple Chinook salmon runs simultaneously, a first for the restoration program. We found that actions targeted at one run were detrimental to others, which was unexpected. Furthermore, information uncovered during this process was used to direct efforts towards targeted research/monitoring to reduce critical uncertainties in salmon demographic rates and make better restoration decisions moving forward. The decision sciences have established a wide range of analytical tools and approaches to simplify complex problems into key components, and we believe the concepts described in this article are of great interest and can be applied by many restoration practitioners that undoubtedly face similar difficulties when implementing restoration strategies for complex systems. [ABSTRACT FROM AUTHOR]

In this paper, the medical equipment replacement strategy is optimised using a multistage stochastic dynamic programming (SDP) approach. The outcome is an optimal path which shows whether to keep an existing piece of medical equipment (defender) or replace it with a more economical alternative (challenger). We assume that each decision can result in a number of different possible outcomes, each with a known probability. Contrary to deterministic dynamic programming, the state at the next stage is not completely determined by the state and policy decision at the current stage. Instead, the next stage depends on the operation and maintenance cost which is modelled as a stochastic variable. A Keep-Replace sequence of the highest returns (lowest costs) is the result of solving the problem using forward decision making. The benefit of the SDP solution versus that of keeping medical equipment until the end of its expected life is investigated for three scenarios: (1) no revenue for the defender and the challenger, (2) equal revenues for both, and (3) higher revenue for the challenger. The percentage of benefits relative to the current acquisition cost for the three scenarios are 616.9%, 728.2%, and 789.29%, respectively. Each percentage represents the relative difference between the equipment life cycle cost of the optimal sequence and that of the conventional sequence. [ABSTRACT FROM AUTHOR]

In this paper, a condition‐based maintenance model for a multi‐unit production system is proposed and analyzed using Markov renewal theory. The units of the system are subject to gradual deterioration, and the gradual deterioration process of each unit is described by a three‐state continuous time homogeneous Markov chain with two working states and a failure state. The production rate of the system is influenced by the deterioration process and the demand is constant. The states of the units are observable through regular inspections and the decision to perform maintenance depends on the number of units in each state. The objective is to obtain the steady‐state characteristics and the formula for the long‐run average cost for the controlled system. The optimal policy is obtained using a dynamic programming algorithm. The result is validated using a semi‐Markov decision process formulation and the policy iteration algorithm. Moreover, an analytical expression is obtained for the calculation of the mean time to initiate maintenance using the first passage time theory. [ABSTRACT FROM AUTHOR]

Problem definition: We analyze a catalyst-activated batch-production process with uncertainty in production times, learning about catalyst-productivity characteristics and decay of catalyst performance across batches. The goal is to determine the quality level of batches and to decide when to replenish a catalyst so as to minimize average costs, consisting of inventory-holding, backlogging, and catalyst-switching costs. Academic/practical relevance: This is an important problem in a variety of process-industry sectors, such as food processing, pharmaceuticals, and specialty chemicals, but has not been adequately studied in the academic literature. This paper also contributes to the stochastic economic lot-sizing literature. Methodology: We formulate this problem as a semi-Markov decision process (SMDP) and develop a two-level heuristic to solve this problem. The heuristic consists of a lower-level problem that plans the duration of batches within the current campaign to maximize the efficiency of the catalyst while ensuring that the target attribute level for each batch is set to meet a quality specification represented by an average attribute level across all the batches in a campaign. The higher-level problem determines when to replace the costly catalyst as its productivity decays. To evaluate our heuristic, we present a lower bound on the optimal value of the SMDP. This bound accounts for all costs, as well as the randomness and discreteness in the process. We then extend our methods to multiple-product settings, which results in an advanced stochastic economic lot-sizing problem. Results: We test our proposed solution methodology with data from a leading food-processing company and show that our methods outperform current practice with an average improvement of around 22% in costs. In addition, compared with the stochastic lower bounds, our results show that the two-level heuristic attains near-optimal performance for the intractable multidimensional SMDP. Managerial implications: Our results generate three important managerial insights. First, our simulation-based lower bound provides a close approximation to the optimal cost of the SMDP, and it is nearly attainable by using a relatively simple two-level heuristic. Second, the reoptimization policy used in the lower-level problem adequately captures the value of information and Bayesian learning. Third, in the higher-level problem of choosing when to replace a catalyst, the intractable multidimensional state of the system is efficiently summarized by a single statistic: the probability of inventory falling below a specific threshold. [ABSTRACT FROM AUTHOR]

The knowledge gradient (KG) policy was originally proposed for online ranking and selection problems but has recently been adapted for use in online decision-making in general and multi-armed bandit problems (MABs) in particular. We study its use in a class of exponential family MABs and identify weaknesses, including a propensity to take actions which are dominated with respect to both exploitation and exploration. We propose variants of KG which avoid such errors. These new policies include an index heuristic, which deploys a KG approach to develop an approximation to the Gittins index. A numerical study shows this policy to perform well over a range of MABs including those for which index policies are not optimal. While KG does not take dominated actions when bandits are Gaussian, it fails to be index consistent and appears not to enjoy a performance advantage over competitor policies when arms are correlated to compensate for its greater computational demands. [ABSTRACT FROM AUTHOR]

This paper studies a model of on-the-job search and savings under reference-dependent preferences that implies loss aversion in a worker's consumption behaviors. The model analyzes how loss aversion affects the worker's consumption decisions in job search. The results demonstrate that the presence of loss aversion will lead to a set of high steady-state consumption levels and the range of steady-state consumption levels is wider if the worker is more loss averse. Nevertheless, we show that there is a unique steady-state consumption level, which is a lower bound of the set, in the absence of loss aversion. In addition, we also find that great loss aversion may reduce consumption level, while small loss aversion not only causes consumption to remain at a high level, but also induces that the worker's future consumption level goes down when the employment status changes. [ABSTRACT FROM AUTHOR]

In the classic revenue management (RM) problem of selling a fixed quantity of perishable inventories to price-sensitive non-strategic consumers over a finite horizon, the optimal pricing decision at any time depends on two important factors: consumer valuation and bid price. The former is determined exogenously by the demand side, while the latter is determined jointly by the inventory level on the supply side and the consumer valuations in the time remaining within the selling horizon. Because of the importance of bid prices in theory and practice of RM, this study aims to enhance the understanding of the intertemporal behavior of bid prices in dynamic RM environments. We provide a probabilistic characterization of the optimal policies from the perspective of bid-price processes. We show that an optimal bid-price process has an upward trend over time before the inventory level falls to one and then has a downward trend. This intertemporal up-then-down pattern of bid-price processes is related to two fundamental static properties of the optimal bid prices: (i) At any given time, a lower inventory level yields a higher optimal bid price, which is referred to as the resource scarcity effect; (ii) Given any inventory level, the optimal bid price decreases with time; that is referred to as the resource perishability effect. The demonstrated upward trend implies that the optimal bid-price process is mainly driven by the resource scarcity effect, while the downward trend implies that the bid-price process is mainly driven by the resource perishability effect. We also demonstrate how optimal bid price and consumer valuation, as two competing forces, interact over time to drive the optimal-price process. The results are also extended to the network RM problems. [ABSTRACT FROM AUTHOR]

This paper presents a new Two Stage Bayesian Stochastic Dynamic Programming (TS-BSDP) model for real time operation of cascaded hydropower systems to handle varying uncertainty of inflow forecasts from Quantitative Precipitation Forecasts. In this model, the inflow forecasts are considered as having increasing uncertainty with extending lead time, thus the forecast horizon is divided into two periods: the inflows in the first period are assumed to be accurate, and the inflows in the second period assumed to be of high uncertainty. Two operation strategies are developed to derive hydropower operation policies for the first and the entire forecast horizon using TS-BSDP. In this paper, the newly developed model is tested on China's Hun River cascade hydropower system and is compared with three popular stochastic dynamic programming models. Comparative results show that the TS-BSDP model exhibits significantly improved system performance in terms of power generation and system reliability due to its explicit and effective utilization of varying degrees of inflow forecast uncertainty. The results also show that the decision strategies should be determined considering the magnitude of uncertainty in inflow forecasts. Further, this study confirms the previous finding that the benefit in hydropower generation gained from the use of a longer horizon of inflow forecasts is diminished due to higher uncertainty and further reveals that the benefit reduction can be substantially mitigated through explicit consideration of varying magnitudes of forecast uncertainties in the decision-making process. [ABSTRACT FROM AUTHOR]

In this research, a novel optimal single machine replacement policy in finite stages based on the rate of producing defective items is proposed. The primary objective of this paper is to determine the optimal decision using a Markov decision process to maximize the total profit associated with a machine maintenance policy. It is assumed that a customer order is due at the end of a finite horizon and the machine deteriorates over time when operating. Repair takes time but brings the machine to a better state. Production and repair costs are considered in the model and revenue is earned for each good item produced by the end of the horizon, there is also a cost for the machine condition at the end of the horizon. In each period, we must decide whether to produce, repair, or do nothing, with the objective of maximizing expected profit during the horizon. [ABSTRACT FROM AUTHOR]

Global change in climate and consequent large impacts on regional hydrologic systems have, in recent years, motivated significant research efforts in water resources modeling under climate change. In an integrated future hydrologic scenario, it is likely that water availability and demands will change significantly due to modifications in hydro-climatic variables such as rainfall, reservoir inflows, temperature, net radiation, wind speed and humidity. An integrated regional water resources management model should capture the likely impacts of climate change on water demands and water availability along with uncertainties associated with climate change impacts and with management goals and objectives under non-stationary conditions. Uncertainties in an integrated regional water resources management model, accumulating from various stages of decision making include climate model and scenario uncertainty in the hydro-climatic impact assessment, uncertainty due to conflicting interests of the water users and uncertainty due to inherent variability of the reservoir inflows. This paper presents an integrated regional water resources management modeling approach considering uncertainties at various stages of decision making by an integration of a hydro-climatic variable projection model, a water demand quantification model, a water quantity management model and a water quality control model. Modeling tools of canonical correlation analysis, stochastic dynamic programming and fuzzy optimization are used in an integrated framework, in the approach presented here. The proposed modeling approach is demonstrated with the case study of the Bhadra Reservoir system in Karnataka, India. [ABSTRACT FROM AUTHOR]

We consider the stochastic obstacle scene problem wherein an agent needs to traverse a spatial arrangement of possible obstacles, and the status of the obstacles may be disambiguated en route at a cost. The goal is to find an algorithm that decides what and where to disambiguate en route so that the expected length of the traversal is minimized. We present a polynomial-time method for a graph-theoretical version of the problem when the associated graph is restricted to parallel avenues with fixed policies within the avenues. We show how previously proposed algorithms for the continuous space version can be adapted to a discrete setting. We propose a generalized framework encompassing these algorithms that uses penalty functions to guide the navigation in real time. Within this framework, we introduce a new algorithm that provides near-optimal results within very short execution times. Our algorithms are illustrated via computational experiments involving synthetic data as well as an actual naval minefield data set. [ABSTRACT FROM AUTHOR]

In this study, a fuzzy-Markov-based interval stochastic dynamic (FM-ISDP) analysis method is developed for reservoir operation management (ROM). In ROM, uncertainties could be represented as discrete intervals, fuzzy sets, and fuzzy-stochastic forms. FM-ISDP is framed on the integration of interval analysis, fuzzy vertex analysis, stochastic dynamic optimization approaches and the fuzzy-Markov chain. The developed model is applied to a case of ROM within a multi-user, multi-uncertainty and multistage context. The results indicate that reasonable solutions can be generated, helping the authority identify desired water release policies and water allocation patterns. Besides, through comparing with the conventional interval stochastic dynamic method, the results also interpret parameters' vagueness is an imperative factor which has a significant effect on the total system benefit. [ABSTRACT FROM AUTHOR]

Application of adaptive management to complex natural resource systems requires careful evaluation to ensure that the process leads to improved decision-making. As part of that evaluation, adaptive policies can be compared with alternative nonadaptive management scenarios. Also, the value of reducing structural (ecological) uncertainty to achieving management objectives can be quantified., A multispecies adaptive management framework was recently adopted by the Atlantic States Marine Fisheries Commission for sustainable harvest of Delaware Bay horseshoe crabs Limulus polyphemus, while maintaining adequate stopover habitat for migrating red knots Calidris canutus rufa, the focal shorebird species. The predictive model set encompassed the structural uncertainty in the relationships between horseshoe crab spawning, red knot weight gain and red knot vital rates. Stochastic dynamic programming was used to generate a state-dependent strategy for harvest decisions given that uncertainty. In this paper, we employed a management strategy evaluation approach to evaluate the performance of this adaptive management framework. Active adaptive management was used by including model weights as state variables in the optimization and reducing structural uncertainty by model weight updating., We found that the value of information for reducing structural uncertainty is expected to be low, because the uncertainty does not appear to impede effective management. Harvest policy responded to abundance levels of both species regardless of uncertainty in the specific relationship that generated those abundances. Thus, the expected horseshoe crab harvest and red knot abundance were similar when the population generating model was uncertain or known, and harvest policy was robust to structural uncertainty as specified., Synthesis and applications. The combination of management strategy evaluation with state-dependent strategies from stochastic dynamic programming was an informative approach to evaluate adaptive management performance and value of learning. Although natural resource decisions are characterized by uncertainty, not all uncertainty will cause decisions to be altered substantially, as we found in this case. It is important to incorporate uncertainty into the decision framing and evaluate the effect of reducing that uncertainty on achieving the desired outcomes. [ABSTRACT FROM AUTHOR]

Under increasing environmental and financial constraints, ecologists are faced with making decisions about dynamic and uncertain biological systems. To do so, stochastic dynamic programming ( SDP) is the most relevant tool for determining an optimal sequence of decisions over time., Despite an increasing number of applications in ecology, SDP still suffers from a lack of widespread understanding. The required mathematical and programming knowledge as well as the absence of introductory material provide plausible explanations for this., Here, we fill this gap by explaining the main concepts of SDP and providing useful guidelines to implement this technique, including R code., We illustrate each step of SDP required to derive an optimal strategy using a wildlife management problem of the French wolf population., Stochastic dynamic programming is a powerful technique to make decisions in presence of uncertainty about biological stochastic systems changing through time. We hope this review will provide an entry point into the technical literature about SDP and will improve its application in ecology. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new model for availability maximization under partial observations for maintenance applications. Compared with the widely studied cost minimization models, few structural results are known about the form of the optimal control policy for availability maximization models. We consider a failing system with unobservable operational states. Only the failure state is observable. System deterioration is driven by an unobservable, continuous-time homogeneous Markov process. Multivariate condition monitoring data which is stochastically related to the unobservable state of the system is collected at equidistant sampling epochs and is used to update the posterior state distribution for decision making. Preventive maintenance can be carried out at any sampling epoch, and corrective maintenance is carried out upon system failure. The objective is to determine the form of the optimal control policy that maximizes the long-run expected average availability per unit time. We formulate the problem as an optimal stopping problem with partial information. Under standard assumptions, we prove that a control limit policy is optimal. A computational algorithm is developed, illustrated by numerical results. [ABSTRACT FROM AUTHOR]

A decision maker records measurements of a finite-state Markov chain corrupted by noise. The goal is to decide when the Markov chain hits a specific target state. The decision maker can choose from a finite set of sampling intervals to pick the next time to look at the Markov chain. The aim is to optimize an objective comprising of false alarm, delay cost, and cumulative measurement sampling cost. Taking more frequent measurements yields accurate estimates but incurs a higher measurement cost. Making an erroneous decision too soon incurs a false alarm penalty. Waiting too long to declare the target state incurs a delay penalty. What is the optimal sequential strategy for the decision maker? This paper shows that under reasonable conditions, the optimal strategy has the following intuitive structure: when the Bayesian estimate (posterior distribution) of the Markov chain is away from the target state, look less frequently; while if the posterior is close to the target state, look more frequently. Bounds are derived for the optimal strategy. Also the achievable optimal cost of the sequential detector as a function of transition dynamics and observation distribution is analyzed. The sensitivity of the optimal achievable cost to parameter and strategy variations is bounded in terms of the Kullback divergence. Also structural results are obtained for joint optimal sampling and measurement control (active sensing). [ABSTRACT FROM AUTHOR]

Most sequential decision-making problems in conservation can be viewed conceptually and modelled as a Markov decision process. The goal in this context is to construct a policy that associates each state of the system with a particular action. This policy should offer optimal performance in the sense of maximizing or minimizing a specified conservation objective, Dynamic programming algorithms rely on explicit enumeration to derive the optimal policy. This is problematic from a computational perspective as the size of the state space grows exponentially with the number of state variables., We present a state aggregation method where the idea is to capture the most important aspects of the original Markov decision process, find an optimal policy over this reduced space and use this as an approximate solution to the original problem., Applying the aggregation method to a species reintroduction problem, we demonstrate how we were able to reduce the number of states by 75% and reduce the size of the transition matrices by almost 94% (324 vs. 5184), and the abstract action matched the optimal action more than 86% of the time., We conclude that the aggregation method is not a panacea for the curse of dimensionality, but it does advance our ability to construct approximately optimal policies in systems with large state spaces. [ABSTRACT FROM AUTHOR]

Analyses of global climate policy as a sequential decision under uncertainty have been severely restricted by dimensionality and computational burdens. Therefore, they have limited the number of decision stages, discrete actions, or number and type of uncertainties considered. In particular, two common simplifications are the use of two-stage models to approximate a multi-stage problem and exogenous formulations for inherently endogenous or decision-dependent uncertainties (in which the shock at time t+1 depends on the decision made at time t). In this paper, we present a stochastic dynamic programming formulation of the Dynamic Integrated Model of Climate and the Economy (DICE), and the application of approximate dynamic programming techniques to numerically solve for the optimal policy under uncertain and decision-dependent technological change in a multi-stage setting. We compare numerical results using two alternative value function approximation approaches, one parametric and one non-parametric. We show that increasing the variance of a symmetric mean-preserving uncertainty in abatement costs leads to higher optimal first-stage emission controls, but the effect is negligible when the uncertainty is exogenous. In contrast, the impact of decision-dependent cost uncertainty, a crude approximation of technology R&D, on optimal control is much larger, leading to higher control rates (lower emissions). Further, we demonstrate that the magnitude of this effect grows with the number of decision stages represented, suggesting that for decision-dependent phenomena, the conventional two-stage approximation will lead to an underestimate of the effect of uncertainty. [ABSTRACT FROM AUTHOR]

Increasing use of spatial management tools in fisheries requires an understanding of fleet response, and in particular to where displaced fishing effort is likely to move. We develop a state-dependent decision-making model to address the spatial allocation of effort in an Australian tuna longline fishery. We assume that fishers have an economic objective in deciding where to fish, but that decisions in any period are also influenced by the remaining quota held at the time of the decision. Key features of the model include endogenous price dynamics, a moving stock and a competitive pool of different vessel types operating from different port locations. We utilize this model to illustrate fleet responses to marine reserves and limits on fishing effort. The results illustrate that the model framework provides advantages over statistically based models in that decisions made in response to the imposition of a reserve are not consistent with a proportional reallocation of effort. Rather, the stochastic dynamic model yielded an overall profit level of ∼4% higher relative to scenarios with no reserve. Incorporating the opportunity cost of a quota into the model resulted in an optimal utilization of effort, in which effort was concentrated in time periods and locations yielding maximized profit. Under a low level of effort relative to the season length, the model indicated an overall profit level 43% greater than the highest obtained when the same level of effort was applied solely within any given quarter of the season. [ABSTRACT FROM AUTHOR]

Making plans about how much to consume and how much to invest in risky assets over an uncertain lifetime is a fundamental economic challenge. The leading models of this planning problem use either additive or habit-forming preferences. For the most part, these models assume an individual is either correlation neutral or correlation seeking in consumption, respectively. In this paper, we introduce two habit-forming, correlation-averse preference models. With these preferences, we find closed-form solutions to the classic consumption and portfolio planning problem. Our solutions recommend that a correlation-averse decision maker follow a habit in their consumption plans. While such habits traditionally have been associated with correlation-seeking preferences, our model leads to consumption habits from correlation-averse preferences. [ABSTRACT FROM AUTHOR]

In this study, a fuzzy-Markov-chain-based stochastic dynamic programming (FM-SDP) method is developed for tackling uncertainties expressed as fuzzy sets and distributions with fuzzy probability (DFPs) in reservoir operation. The concept of DFPs used in Markov chain is presented as an extended form for expressing uncertainties including both stochastic and fuzzy characteristics. A fuzzy dominance index analysis approach is proposed for solving multiple fuzzy sets and DPFs in the proposed FM-SDP model. Solutions under a set of α-cut levels and fuzzy dominance indices can be generated by solving a series of deterministic submodels. The developed method is applied to a case study of a reservoir operation system. Solutions from FM-SDP provide a range of desired water-release policies under various system conditions for reservoir operation decision makers, reflecting dynamic and dual uncertain features of water availability simultaneously. The results indicate that the FM-SDP method could be applicable to practical problems for decision makers to obtain insight regarding the tradeoffs between economic and system reliability criteria. Willingness to obtain a lower benefit may guarantee meeting system-constraint demands; conversely, a desire to acquire a higher benefit could run into a higher risk of violating system constraints. [ABSTRACT FROM AUTHOR]

In agriculture Markov decision processes (MDPs) with finite state and action space are often used to model sequential decision making over time. For instance, states in the process represent possible levels of traits of the animal and transition probabilities are based on biological models estimated from data collected from the animal or herd. State space models (SSMs) are a general tool for modeling repeated measurements over time where the model parameters can evolve dynamically. In this paper we consider methods for embedding an SSM into an MDP with finite state and action space. Different ways of discretizing an SSM are discussed and methods for reducing the state space of the MDP are presented. An example from dairy production is given. [ABSTRACT FROM AUTHOR]

Abstract: The quality of environmental decisions should be gauged according to managers’ objectives. Management objectives generally seek to maximize quantifiable measures of system benefit, for instance population growth rate. Reaching these goals often requires a certain degree of learning about the system. Learning can occur by using management action in combination with a monitoring system. Furthermore, actions can be chosen strategically to obtain specific kinds of information. Formal decision making tools can choose actions to favor such learning in two ways: implicitly via the optimization algorithm that is used when there is a management objective (for instance, when using adaptive management), or explicitly by quantifying knowledge and using it as the fundamental project objective, an approach new to conservation. This paper outlines three conservation project objectives – a pure management objective, a pure learning objective, and an objective that is a weighted mixture of these two. We use eight optimization algorithms to choose actions that meet project objectives and illustrate them in a simulated conservation project. The algorithms provide a taxonomy of decision making tools in conservation management when there is uncertainty surrounding competing models of system function. The algorithms build upon each other such that their differences are highlighted and practitioners may see where their decision making tools can be improved. [Copyright &y& Elsevier]

We consider an ecosystem with two distinct equations of motion that are separated by a threshold value of the state variable. We find that increasing uncertainty (both uncertainty embedded in the natural system and uncertainty of the decisionmaker about the location of the threshold) can lead to nonmonotonic changes in precaution: a reduction in uncertainty can first increase and then decrease optimal precautionary activity. This nonmonotonicity can help to explain why regulators often give conflicting arguments about optimal abatement policies in the face of uncertainty. For example, some regulators argue for an immediate reduction in pollutant loading until uncertainty about the underlying process is reduced while others call for no costly reductions in pollutant loading until the same uncertainty is reduced. These statements can be consistent even if both sides agree on both economic objectives and the system dynamics, but have different priors on the uncertainty involved. [Copyright &y& Elsevier]

Abstract: Real-time hydropower reservoir operation is a continuous decision-making process of determining the water level of a reservoir or the volume of water released from it. The hydropower operation is usually based on operating policies and rules defined and decided upon in strategic planning. This paper presents a fuzzy rule-based model for the operation of hydropower reservoirs. The proposed fuzzy rule-based model presents a set of suitable operating rules for release from the reservoir based on ideal or target storage levels. The model operates on an ‘if-then’ principle, in which the ‘if’ is a vector of fuzzy premises and the ‘then’ is a vector of fuzzy consequences. In this paper, reservoir storage, inflow, and period are used as premises and the release as the consequence. The steps involved in the development of the model include, construction of membership functions for the inflow, storage and the release, formulation of fuzzy rules, implication, aggregation and defuzzification. The required knowledge bases for the formulation of the fuzzy rules is obtained form a stochastic dynamic programming (SDP) model with a steady state policy. The proposed model is applied to the hydropower operation of “Dez” reservoir in Iran and the results are presented and compared with those of the SDP model. The results indicate the ability of the method to solve hydropower reservoir operation problems. [ABSTRACT FROM AUTHOR]

Markov decision processes (MDP) with finite state and action space have often been used to model sequential decision making over time in dairy herds. However, the length of each stage has been at least 1 mo, resulting in models that do not support decisions on a daily basis. The present paper describes the first step of developing an MDP model that can be integrated into a modern herd management system. A hierarchical MDP was formulated for the dairy cow replacement problem with stage lengths of 1 d. It can be used to assist the farmer in replacement decisions on a daily basis and is based on daily milk yield measurements that are available in modern milking systems. Bayesian updating was used to predict the performance of each cow in the herd and economic decisions were based on the prediction. Moreover, parameters in the model were estimated using data records of the specific herd under consideration. This includes herd-specific lactation curves. [ABSTRACT FROM AUTHOR]

The article presents a study which examines the structured decision making process as conceptual framework to identify threshold for conservation and management. Numerical examples of structured decision processes that included all three kinds of thresholds are provided. Ecological thresholds, utility thresholds and decision thresholds are the three categories of thresholds being distinguished. The focus on the approach is on the objectives of management with an aim to provide decisions that are optimal with respect to those objectives.

Negotiations generally tend to focus efforts on attaining optimality in single-problem contexts that are ad hoc, disparate and temporary in nature. Once negotiators reach agreement, the process usually attains closure and the long-term impact of the outcome is rarely considered. In organizational settings, decisions involving quid pro quos are, however, made on a continuous basis. Since organizational environments are constantly in flux, negotiated solutions that appeared successful on a given problem at first might no longer work out to be effective in the long run. We postulate that organizations evolve from one state to another and negotiations play an important part in these transitions. From this perspective, decision-making in organizations or between them can be modeled using sequential Markov chains that converge on homeostasis. This leads to a prescriptive approach for transitional negotiations that allow for assessment of the long-term impact of decisions and suggest acceptance of possible short-term losses in favor of the better payoffs that are to come. We provide a hydraulic dam example to illustrate the transitional aspect of decision-making over time. Based on earlier successful GDSS, we also suggest a software architecture that would allow the proposed theoretical model to be implemented as an organization negotiation support system with practical benefits. [ABSTRACT FROM AUTHOR]

Stochastic dynamic programming (DP) is a strong mathematical tool allowing modeling and solving many multiperiod decision processes. Multiple objective and dynamics characterize many sequential decision problems. In the paper we consider returns in partially ordered criteria set as a way of generalization of single criterion DP models to multiobjective case. In the present paper, on the basis of theoretical findings, described in our previous papers we consider exemplary stochastic DP profit maximization processes. Because of the lack of space we omit the general, formal description of such a process and concentrate on explanation, how the theory of DP models in partially ordered criteria space works. Both in level-volume and velocity-volume process we will consider formulated problems step by step, first as single criterion problems and next as bi- -criteria ones. Conclusions are presented in the last section. [ABSTRACT FROM AUTHOR]

In this paper, we present a Decision-Making Framework (DMF) for reducing ozone pollution in the metropolitan Atlanta region. High ground-level concentrations of ozone continue to be a serious problem in several US cities, and Atlanta is one of the most serious of these cases. In contrast to the "trial and error" approach utilized by state government decision-makers, our DMF searches for dynamic and focused control strategies that require a lower total reduction of emissions than current control strategies. Our DMF utilizes a rigorous stochastic dynamic programming formulation and includes an Atmospheric Chemistry Module to represent how ozone concentrations change over time. This paper focuses on the procedures within the Atmospheric Chemistry Module. Using the US EPA's Urban Airshed Model for Atlanta, we use mining and metamodeling tools to develop a computationally efficient representation of the relevant ozone air chemistry. The proposed approach is able to effectively model changes in ozone concentrations over a 24-hour period. [ABSTRACT FROM AUTHOR]

Translocation is a useful management option for conservation of threatened animal species. It can be used to increase the range of a species, augment the numbers in a critical population, or establish new populations and hence spread the risk of extinction through local catastrophes. As it is an important and expensive conservation tool, translocation management decisions must be carefully considered, with the objective of the translocation project in mind. By analysing the translocation problem within a decision-theory framework, we find optimal management decisions that are rational and transparent. We illustrate our approach using a case study of the bridled nailtail wallaby (Onychogalea fraenata). Our particular translocation question is: if we have a set number of wallabies to translocate in each time period and two translocation sites, how many wallabies should we put at each site given the state of each population to maximise the benefit to the species? We model the translocated populations with first-order Markov chain stochastic population models, and use stochastic dynamic programming to determine the optimal management decisions. We look at two sites with different growth rates – one increasing and one decreasing – and compare the optimal strategies for two different objective functions. The first is a long-term persistence objective function, which maximises the persistence of translocated populations a large number of time steps after the end of the translocation program. The second maximises total population size at the end of the translocation program. Although these objective functions are similar, they generate surprisingly different optimal translocation strategies. When maximising the long-term persistence of the translocated populations, translocation decisions are not important as long as an increasing population is established. This indicates that site quality – rather than the number and timing of translocations – primarily determines the long-term persistence of populations. When maximising total population size, the optimal strategy is to add to the increasing population unless it is above a size where it is likely to reach its carrying capacity over the planning timeframe. As translocation decisions are important in fulfilling the objective, this objective function is more useful in creating practical advice for translocation managers. The discrepancy between the optimal strategies given by the two objectives demonstrates the importance of careful consideration when specifying the goals of a project. This observation applies not only to translocation programs, but any project where clear decision-making is needed. [Copyright &y& Elsevier]

This paper concerns two families of Markov decision problem that fall within the family of (bi-directional) restless bandits, an intractable class of decision processes introduced by Whittle. The spinning plates problem concerns the optimal management of a portfolio of reward-generating assets whose yields grow with investment but otherwise tend to decline. In the model of asset exploitation called the squad system, the yield from an asset tends to decline when it is used but will recover when the asset is at rest. In all cases, simply stated conditions are given that guarantee indexability of the problem, together with conditions necessary and sufficient for its strict indexability. The index heuristics for asset activation that emerge from the analysis are assessed numerically and found to perform very strongly. [ABSTRACT FROM AUTHOR]

The algorithmic framework developed for improving heuristic solutions of the new version of deterministic TSP [Comput. Chem. Eng. (2004)] is extended to the stochastic case. To verify the algorithmic framework for the stochastic case, a new variant of the stochastic TSP with an optional task, in which key parameters (cost matrix) of the problem change according to an underlying Markov model, is introduced in this work as a prototypical stochastic optimization problem. The optional task is the performing of an “investigation” which improves the information for decision making at some cost. The stochastic dynamic programming is performed in the subset of the states that is obtained by simulating a set of heuristics. The proposed algorithmic framework finds the approximated optimal cost-to-go only for the states in the subset. This reduces the computational time dramatically compared to the stochastic DP in the entire states space, without significant loss in the solution quality. The results show that the algorithmic framework also works efficiently for stochastic problems by improving the heuristic policies for making decisions for different realizations of the Markov process. [Copyright &y& Elsevier]

A large number of problems in production planning and scheduling, location, transportation, finance, and engineering design require that decisions be made in the presence of uncertainty. Uncertainty, for instance, governs the prices of fuels, the availability of electricity, and the demand for chemicals. A key difficulty in optimization under uncertainty is in dealing with an uncertainty space that is huge and frequently leads to very large-scale optimization models. Decision-making under uncertainty is often further complicated by the presence of integer decision variables to model logical and other discrete decisions in a multi-period or multi-stage setting.This paper reviews theory and methodology that have been developed to cope with the complexity of optimization problems under uncertainty. We discuss and contrast the classical recourse-based stochastic programming, robust stochastic programming, probabilistic (chance-constraint) programming, fuzzy programming, and stochastic dynamic programming. The advantages and shortcomings of these models are reviewed and illustrated through examples. Applications and the state-of-the-art in computations are also reviewed.Finally, we discuss several main areas for future development in this field. These include development of polynomial-time approximation schemes for multi-stage stochastic programs and the application of global optimization algorithms to two-stage and chance-constraint formulations. [Copyright &y& Elsevier]

The accelerated pace of technological change has led to rapid obsolescence of productive capacity in electronics and other industries. Managers must consider the impact of future technologies while making acquisition and replacement decisions in such environments. We consider a problem where a sequence of technological breakthroughs are anticipated but their magnitude and timing are uncertain. A firm, operating in such an environment, must decide how much capacity of the current technology to acquire to meet future demand growth. It must also determine whether to upgrade any of the older vintages. We formulate this problem and present some structural results. Using these results, we then develop a highly efficient regeneration point-based dynamic programming algorithm. The effectiveness of the proposed algorithm is illustrated through a computational study. The sensitivity of the first period decision to various parameters is also explored. [ABSTRACT FROM AUTHOR]