Search

The field of computer applications in education, characterized by continuous change in technology and nomenclature, is documented by diverse information sources. This paper names methods of organizing information on computer-based instructional systems and lists sources under literature surveys and reviews, conferences and symposia (1965-68, U.S.A.), publishers, and professional organizations. Interactive uses of computers for instruction include drill strategy, author-controlled tutorial, and "dialogue" tutorial; simulation and gaming; information handling, computation and display (scholarly aids); computer aids for instructional management; and computer-based tools for the author and researcher. The variety of lessons, systems, and languages for computer-based instruction depends on instructional objectives and on records of student performance. (TI)

This familiar card game has interested mathematicians, economists, and psychologists as a model of decision-making in the real world. It is now serving as a vehicle for investigations in computer science. (Author/MA)

The main problem in computer gaming research is the initial decision of choosing the type of gaming method to be used. Free-form games lead to exciting open-ended confrontations that generate much information. However, they do not easily lend themselves to analysis because they generate far too much information and their results are seldom replicable. On the other hand, rigid-form games, characterized by the prespecification of objects and rules, tend to be easily controlled and analyzed, but do not enhance the individual creativity necessary for insightful play. In order to reconcile these extremes, this paper discusses the merits of the one-person quasi-rigid rule game. This recent innovation defines the individual decision as the relevant unit of observation; because of this approach the data analysis is made easier without destroying the initiative of the players. The authors feel that this new gaming method will be a precursor of stronger, more scientific interest and activity in computer gaming for educational purposes. (MC)

Question and answer format concludes a series of articles on computing. (Editor)

Several factors influence the effectiveness of human-AI collaborations, including inherent human biases. Our research explores the role of a deceptive agent in enhancing the success of these systems by compensating for these biases. We investigate under what conditions an AI can compensate for human biases and where its use might be ethically justified. Contrary to traditional views that cast strategic deception in a negative light, our findings suggest it can, under specific conditions, improve cooperative outcomes and thereby enhance human decision-making, contributing to broader societal benefits. Our study employs game theory and reinforcement learning to observe how deceptive behaviors naturally emerge within the ongoing learning dynamics of AI agents. We support our theoretical claims with simulation results derived from Markov Decision Processes (MDP) and a signaling game example, providing a practical glimpse into how these agents learn and interact. Building on these insights, we propose an ethical framework to evaluate the permissibility of employing deceptive algorithms and reflect on the nuance the developer must adopt while deploying these algorithms. By advocating a careful approach to strategic deception, we aim to advance human-AI teamwork and decision-making, steering these collaborations toward outcomes that are both ethically sound and socially beneficial.

When multiple clients are collaboratively learning and training a shared model, incentives problems can arise. The clients may have different learning objectives and application domains, or they may be competitors whose participation in the learning system could reduce their competitive advantage. While collaborative learning is a powerful framework that leverages vast networks of compute and data to generate a better model for all, participants may defect from collaboration if their incentives are misaligned with the guarantees of the system. In this dissertation, I examine three areas where accounting for incentives is critical in designing an effective collaborative learning system. I. When clients in the system have heterogeneous data distributions and divergent learning tasks, full collaboration within the system can result in a global model which performs poorly for individual clients. Personalization of the global model to clusters of clients with similar learning objectives is a solution to this problem. We propose a personalization method which has optimal convergence guarantees and is provably robust to malicious attackers. II. Clients who are competitors may not want to participate in collaborative learning system if their contributions will benefit their competitors and disadvantage themselves. We design a collaborative learning scheme which guarantees that clients lose no utility by participating. Additionally, we show that even as clients focus on increasing their own revenues, their model qualities converge to the Nash bargaining solution, thus optimizing for joint surplus.III. Finally, privacy concerns are a major deterrent for joining collaborative learning systems. In the final chapter, we look at privacy dynamics in systems of learning agents more broadly. Specifically, we study a repeated-interaction game between potentially antagonistic learning agents -- a buyer and a price-discriminating seller -- and show that privacy-protecting behavior endogenously arises at equilibrium.

A textbook property of optimization algorithms is their ability to solve the problems under generic regularity conditions. Two examples are simplex method and gradient descent (GD) method. However, the performance of these fundamental and general-purpose optimization algorithms is often unsatisfactory; they often run slowly and perhaps return the suboptimal solutions in generic settings. In my view, this is the price of their generality; indeed, the generic algorithms are an achievement, but for many problems, the gains from leveraging spe- cial structure can be huge. A basic question then arises: how can we harness problem-specific structure within our algorithms to obtain fast, practical algorithms with strong performance guarantees? As more structured data-driven decision-making models emerge, this question has become increasingly pressing and relevant to practitioners.For example, the GD is known to get stuck at a suboptimal saddle points in nonconvex optimization. Nonetheless, a line of recent works have shown that random initialization or perturbation changes the dynamics of GD and makes it provably converge to a global optimal solution. In addition, both Markov decision process (MDP) and discrete optimal transport (OT) problems can be solved using large-scale linear programs. Rather than using generic LP algorithms, the policy iteration and the Sinkhorn iteration exploit special structures in MDP and OT and thus perform better in practice. Adapting algorithms to problem-specific structure is generally referred to as structure-driven algorithm design.Although this line of research – which has been studied extensively for over 70 years – has enjoyed widespread success, the machine-learning success stories have introduced new formulations ripe for deep theoretical analysis and remarkable practical impact. My research pushes this frontier by identifying special structure of reliable machine learning (minimax optimization) and multi-agent machine learning (high-order optimization and beyond) and design optimal algorithms for computing the appropriately defined optimal solutions; and other structured problems, such as efficient entropic regularized optimal transport, gradient- free nonsmooth nonconvex optimization, and adaptive and doubly optimal learning in games.

Nowadays with the growth of large-scale societal infrastructure systems, there has been significant research attention on improving efficiency, guaranteeing safety, reducing operational costs, and decreasing the carbon footprint of these systems. In particular, this thesis is focused on Human-Cyber-Physical Systems (H-CPS) (e.g. smart grid, electric transportation networks, autonomous driving). An H-CPS is any physical system in which a mechanism is controlled by both computer-based algorithms and human inputs. With the increasing complexity of human-machine interfaces, the traditional engineering and operating strategies are not adequate to manage. In fact, a mix of tools from stochastic control, distributed optimization, machine learning, and game theory is required. For example, in modern electric transportation systems, without appropriate demand management and coordination schemes, Electric Vehicle (EV) charging patterns could create problems for power transmission and distribution networks, and hence reduce the environmental benefits of transportation electrification.Furthermore, when managing demand to reduce costs in a power system, it is necessary to ensure that the operating constraints of the power grid are not violated as a result of our actions. Additionally, because of the availability of real-time data from these infrastructure systems, training a large-scale model over a vast amount of data requires sophisticated techniques that accelerate the training of learning models. Therefore, it becomes important to develop algorithms that are computationally efficient and consider the critical safety requirements of these systems.The aforementioned problems are characterized by many challenges including: How can we encourage customers to act in a way that is more likely to benefit society even when it may conflict with their own interests? How do we make sure that the infrastructure systems' safety criteria are not disregarded while we are learning the proper procedures to optimize customer behavior? How do we make sure that our proposed algorithms are computationally efficient? This thesis is focused on developing optimization and machine learning frameworks that promote efficiency and flexibility in large-scale societal infrastructure systems with the active involvement of humans. In the first part of the thesis, we focus on designing optimal price and routing mechanisms for a public charging stations network in electric transportation systems to coordinate customers (i.e., EV drives) towards a socially optimal behavior given their heterogeneity. In the second part of the thesis, we provide two theoretical learning guarantees for online decision-making problems in safety-constrained unknown linear systems. Moving on to the third part, we develop two methods to speed up the learning process of the online learning algorithms in new tasks based on their limited past experience with unknown linear systems. We also support our theoretical results in all three parts by significant improvement in numerical experiments.

Strategic interactions in multi-agent systems can be conveniently modeled, manipulated and characterized within the analytical framework provided by game theory and mechanism design, and used to extract engineering insights regarding systems of interest. Accordingly, in this dissertation, we adopt such a framework and pursue research directions aimed at understanding how information and externalities impact the strategic outcomes that can emerge in systems with multiple, non-cooperative decision makers. We consider two types of mechanisms: decision-based mechanisms and preference-based mechanisms which respectively manipulate either the players' action sets or the players' utilities -- the two major building blocks of a game. We conduct our analysis of decision-based mechanisms on Colonel Blotto games which are popular models for competitive resource allocation in adversarial environments. Within this framework, we first consider settings where the information to a competitor is obfuscated, and quantify the value of information relating to competitive objectives and the opponent's strength. We then consider the role of pre-emption under this framework, and show that -- perhaps surprisingly -- revealing information to competitors can also offer strategic benefits in competitive interactions. Our results on preference-based mechanisms focus on the design of taxes in congestion games to optimize the system performance. In our study, we consider three performance measures corresponding with the worst-case equilibrium efficiency (Price of Anarchy), the best-case equilibrium efficiency (Price of Stability), and the transient system performance (Price of Urgency). Within this context, our first set of results focus on optimizing the Price of Anarchy; we derive tractable methodologies for computing the optimal taxes within this setting. We then investigate the consequences of optimizing for the worst case on the other performance measures: we show that the taxes that optimize the Price of Anarchy necessarily have Price of Stability equal to the Price of Anarchy, and that optimal Price of Anarchy guarantees can correspond with arbitrarily poor Price of Urgency. We supplement this last set of results by proposing techniques for characterizing the respective trade-off curves. We conclude with a discussion on future directions for both decision- and preference-based mechanisms.

Over the past three decades, the reach of recommender systems has grown exponentially. Today, recommender systems are deployed on all major internet platforms, influencing our opinions, decisions, careers, and relationships. However, despite their far-reaching impact, these algorithms and their consequences are still poorly understood. In this thesis, we argue that this is due to the challenging dynamics of the recommendation problem. We outline four problems that distinguish the dynamics of recommendation from other dynamical systems, making them particularly hard to reason about: (1) direct measurement and experimentation are often infeasible, (2) feedback effects make it difficult to reason about cause and effect, (3) the scale of internet platforms requires increased algorithmic complexity, and (4) incentives created by recommender systems cause users to behave strategically. We build the foundations necessary to understand and remedy these four problems, paving the way for a complete understanding of the dynamics of recommender systems and their consequences.

During the 2010s decade, the increase of connectivity in the world led to the development of navigation applications that help vehicles to travel within the transportation network using real-time traffic information. Information-aware routing changed traffic patterns by spreading congestion in the network. Before information-aware routing, route choice was dictated by direction signs, themselves prescribed by city traffic plans. As a consequence, with these new routing behaviors, some traffic plans have been outdated.From a game theoretical point of view, by providing vehicles with the fastest path to reach their destination, information-aware routing suggestions direct the state of traffic toward a Nash equilibrium. The gap between the state of a game and a Nash equilibrium can be measured with the average deviation incentive. Using the restricting path choice model, we show that the average deviation incentive monotonically decreases when information-aware routing behaviors increase.On the ground, if information-aware routing behaviors might (or might not) increase the overall transportation efficiency, some local roads received more traffic than the one they are designed to sustain. In Los Angeles, CA, we measure a 3-fold increase in the flow of one I-210 off-ramp between 2014 and 2017. To our knowledge, this is likely a consequence of the rise of information-aware routing behaviors due to an increase in navigational-app usage. Travel time data shows the travel time equalization between the main Eastbound I-210 route between Pasadena, CA, and Azusa, CA, and some alternate routes using local roads. The travel time equalization expresses that the state of traffic is a Nash equilibrium, which demonstrates the presence of information-aware routing behaviors. However, the severity of information-aware routing cannot be quantitatively assessed globally due to the lack of available floating-car data (trajectory data). Qualitatively, many cities and neighborhoods reported negative externalities of cut-through traffic due to information-aware routing: Los Angeles, CA recorded several crashes on the steep Baxter Street, Leonia, NJ reported a fatality on the Ford Lee Road, and Fremont, CA faced the challenges of cut-through traffic on the Mission Boulevard. All around the world, cut-through traffic might induce higher travel times, delays, unreliable travel times for residents, noise, gas emissions, traffic accidents, decrease accessibility in affected neighborhoods, wrong directions, and infrastructure damage, among others. To mitigate these issues, many cities changed the design of their road network. Others used cap-and-trade techniques (e.g., access restriction for non-residents on the Ford Lee Road in Leonia, NJ), or Pigovian taxes (e.g., road pricing on Lombard Street in San Francisco, CA). The city of Fremont erected new stop signs, built speed bumps, and updated its traffic signal timing plans to decrease the Mission boulevard's attractivity.To pick the best mitigation techniques to fight against cut-through traffic, we suggest using a digital twin of the city traffic. Traffic microsimulators can replicate, in the digital world, the movement of each vehicle within the road network. Because the challenges of running a traffic simulation are not apparent until one creates their own, we provide a blueprint to develop, calibrate and validate a traffic microsimulation. A traffic simulation of the Fremont, CA neighborhood affected by cut-through traffic due to information-aware routing is made open source, for anyone that would like to understand how information-aware routing might lead to cut-through traffic.On the way, we realized that simulating the behaviors of each vehicle in the network is computationally expensive. Therefore, we proposed to cluster vehicles into a mean-field by developing a mean-field routing game. While macroscopic routing models already exist to estimate how, knowing the traffic demand, traffic evolves in the network, we envision that the mean-field routing game is the best tool to perform large-scale dynamic routing control. We also envision that large-scale dynamic routing control is enabled by navigational applications, with already existing features like eco-routing that have been launched in 2021 by Google Maps.

This dissertation describes the work I have done on the subject of computing equilibrium solutions of connected quadratic programs (QPs), particularly for the connected programs arising in the context of autonomous system design. Reasoning about the interaction between multiple mathematical programs enables the analysis and solution to a wealth of problems inaccessible by standard optimization formulations. One of the most commonly arising examples of connected optimization problems is the bilevel program, comprised of an outer-level and inner-level program. The outer-level problem is subject to a constraint that a subset of decision variables solve the inner-level problem. Another class of connected optimization problems are those of Nash equilibrium problems, in which optimization problems are solved simultaneously. Beyond these and some other simple organizations of problem, general interactions among optimization problems are not well studied, despite the potential for modeling many interesting real-world problems. Presumably this is because with generalization in problem organization comes increased complexity, both from an analysis and computation perspective. To address this, it is claimed in this work that it is possible to compute equilibrium solutions to a broad range of connected optimization problems, assuming those problems take the form of convex quadratic programs. This claim is based off a result that equilibrium solutions to a collection of QPs with piecewise linear constraints can be represented as a piecewise linear mapping. Recursive application of this result leads to the results for generally nested equilibrium problems. Theoretical results and computational approaches for this class of problem are developed. The efficacy of the presented methodologies is demonstrated on various problems faced by autonomous vehicles.

Large-scale autonomous systems are systems comprised of many components, each acting according to its own preferences, local information and capabilities. Such systems are ubiquitous in our world and include automated warehouses, UAV swarms, traffic systems, sensor networks, the internet of things, auctions, ridesharing systems, and social networks. We focus on autonomous systems which are engineered: each component is human-designed.While such systems are attractive because they can process a high amount of data and are generally robust against single points of failure, there are often many challenges in their design. System designers must take into account that each component has its own capabilities, model of the environment, data set, and local objective. Furthermore, the system designer often cannot make decisions for each component at each time step, rather, decision-making rules are assigned to allow components to react autonomously. Small adjustments to such rules can often have cascading effects throughout the system. Finally, the interconnected nature of the system opens doors to new kinds of system-wide attacks and vulnerabilities.In this work, we focus on the challenge of information sharing constraints: each agent does not have access to all of the system information at every given time step. These constraints often arise naturally (i.e., no router can access all available data on the internet before making a routing decision), but they can also arise from privacy, trust or political issues. Thus it is imperative for the system designer to understand the relevant information constraints on the system and their effects on the emergent behavior. In this work we endeavor to answer the two following questions:1. How do a set of information sharing constraints impact the resulting emergent system-wide behavior?2. How can a system designer strategically set decision-making rules for the components to offset any negative effects caused by information sharing constraints?We answer the first question by assuming that the emergent behavior is a system equilibrium, and then comparing the value of the worst-case equilibrium to the value of the optimal decision set, where value is based on the system designer objective. Different types of information sharing constraints among the components are evaluated on this basis. We answer the second question in certain settings by showing that small deviations from standard decision-making rules can improve the system's performance guarantees.These questions are addressed in two settings: first in a cooperative setting, where the system designer can design the decision-making rules for each agent. The system designer objective function is assumed to be submodular, and this property is leveraged to show closeness of equilibrium to optimal. The second is a noncooperative setting, where the system design must operate in the presence of an attacker. Here, the constraints on information sharing are related to how much knowledge about the attacker the other players have.

This thesis contains research conducted on various topics in quantum Hall physics and deep learning theory. The first chapter studies a particular aspect of quantum Hall systems, namely their behavior around the $\nu = 1/2$ Landau level (LL) state. This work is motivated by the need to understand better this particular state in light of the two proposed distinct theoretical descriptions existing for the same. Specifically, we analyze quantum oscillations around the $\nu = 1/2$ LL state using one of the propositions to support the latter. The second and third chapters study two distinct domains in deep learning, multi and single-objective models. In particular, the second considers a specific type of multi-objective model, zero-sum games, to demonstrate existing issues in training such setups and develop an efficient optimization scheme. The final chapter involves studying a particular aspect of the generalization behavior of deep neural networks (DNNs). Specifically, it attempts to provide a theoretical framework to explain the recently observed phenomenon of "epoch-wise double descent" in such DNNs.

Since 2011, the private ride-hailing companies Didi (2019), Uber (2019) and Lyft (2021) have expanded into more and more cities. These ride-hailing services (RHS) bring convenience to our life; however, at the same time they, also raise security concerns for users. For example, several recent news items show that a considerable number of registered drivers whose licenses have been revoked are still taking RHS orders on the respective platforms; this phenomenon directly leads to insecurity on part of its users and the bad reputation of the ride-hailing service provider (SP). The traditional solution to solve this problem is to periodically check the validity of the drivers' licenses; however, it is a considerably time-consuming and costly process since the SPs have to manually interact with the governing authorities. Therefore, in this paper, we have presented an auditable self-sovereign identity system (named AudiSSI), which provides an efficient approach for the SPs to manage their registered drivers' qualifications in a decentralized and automatic manner. Further, using smart contract technology, we propose a safety guarantee insurance in the form of an auditing contract to enable the RHS rider to check their driver's qualifications before the trip starts and get incentives once they detect a disqualified driver. We designed an incentive mechanism and have provided a game theoretical analysis. Finally, we implemented a prototype of AudiSSI and deployed it on Hyperledger Indy and Fabric to show that self-sovereign identity system for RHS driver with qualification auditing is efficient and technically feasible.

We consider here how a generalized signaling game may self-assemble as the saliences of the agents evolve by reinforcement on those sources of information that in fact lead to successful action. On the present account, generalized signaling games self-assemble even as the agents coevolve meaningful representations and successful dispositions for using those representations. We will see how reinforcement on successful information sources also provides a mechanism whereby simpler games might compose to form more complex games. Along the way, we consider how an old game might be appropriated to a new context by reinforcement on successful associations between old and new saliences.

Edge computing has been recently introduced as a way to bring computational capabilities closer to end users of modern network-based services, in order to support existent and future delay-sensitive applications by effectively addressing the high propagation delay issue that affects cloud computing. However, the problem of efficiently and fairly manage the system resources presents particular challenges due to the limited capacity of both edge nodes and wireless access networks, as well as the heterogeneity of resources and services' requirements. To this end, we propose a techno-economic market where service providers act as buyers, securing both radio and computing resources for the execution of their associated end users' jobs, while being constrained by a budget limit. We design an allocation mechanism that employs convex programming in order to find the unique market equilibrium point that maximizes fairness, while making sure that all buyers receive their preferred resource bundle. Additionally, we derive theoretical properties that confirm how the market equilibrium approach strikes a balance between fairness and efficiency. We also propose alternative allocation mechanisms and give a comparison with the market-based mechanism. Finally, we conduct simulations in order to numerically analyze and compare the performance of the mechanisms and confirm the theoretical properties of the market model., Comment: Corrected typos and figure orientation. in IEEE Transactions on Mobile Computing

We design a novel encoding model based on Lagrange coded computing (LCC) for private, secure, and resilient distributed mobile edge computing (MEC) systems, where multiple base stations (BSs) act as masters offloading their computations to edge nodes acting as workers. A two-fold objective of the scheme is: i) efficient allocation of computing tasks to the workers; ii) providing the workers with appropriate incentives to complete their tasks. As such, each master must decide on its offloading requests to the workers including the allocated tasks and service fees to be paid. This problem is complex due to the following reasons: i) masters can be privately-owned or managed by different operators, i.e., there is no communication and no coordination among them; ii) workers are heterogeneous non-dedicated nodes with limited and nondeterministic transmission and computing resources. As a result, the masters must compete for constrained resources of workers in a stochastic partially-observable environment. To address this problem, we define the interactions between masters and workers as a direct stochastic first-price-sealed-bid (FPSB) auction. To analyze the auction, we represent it as a stochastic Bayesian game and develop a Bayesian learning framework to perfect the auction solution.

Vehicular Content Centric Network (VCCN) has been proposed to address the issues of user mobility and sporadic connectivity in VANET. The in-network caching of VCCN can indeed improve the performance of content distribution. However, data caching unfortunately suffers from Cache Pollution Attack (CPA) that sends out fabricated requests to pollute the data cache. To prevent the degradation of network performance caused by such an attack, we propose a detection and defense scheme of CPA by adopting game theory. We model the poisoning attack scenario into a non-cooperative game model. First, we prove the non-existence of pure strategy Nash Equilibrium (NE) in the attacker and defender game and propose a normal form game model with complete information and an extensive Bayesian form game model with incomplete information. Under the guidance of NE, we design a punishment strategy to prevent the coordinated attack and make the attackers behave normally. Moreover, we design a cooperative detection scheme to give the road side unit the final decision on CPA based on the direct and indirect suspicious lists which are generated by each node according to the observed traffic pattern. Simulation evaluations demonstrate that our scheme has a better performance compared to other state-of-the-art schemes.

Eco-safe driving is a driving style equally oriented towards both fuel-saving and safety. Existing studies have proposed feedback and alerting systems to sensitize a driver towards eco-safe driving. These approaches, however, have been shown to have lesser driver acceptance. The game theory based approach can sustain driver’s interest and engagement towards safe and economical driving, thus outperforming training and feedback mechanisms. In this work, a driving game based on a Bayesian Nash Equilibrium (BNE) gaming strategy has been designed to recommend optimal vehicle manoeuvres in real time. The BNE strategy enhances the driving experience of all drivers from comfort, mileage, and safety perspectives. The drivers have demonstrated improvement of 5%–17% in driving scores post application of BNE gaming mode. Results thus prove effective reinstatement of eco-safe driving behaviour due to proposed strategy.

A 360-degree streaming system can provide immersive, interactive, and autonomous experiences surrounding the user by viewpoint changing to see different angles of the 360-degree video. Due to the limited and highly dynamic cellular network conditions, high-resolution 360-degree video playback over mobile devices often suffer from playback freezing, and inevitable bandwidth waste appears in delivering those out-of-viewpoint parts. In this paper, a hybrid control scheme is presented for segment-level continuous bitrate selection and tile-level bitrate allocation for 360-degree streaming over mobile devices to increase users' quality of experience. First, a deep reinforcement learning (RL) method is proposed to predict the segment bitrate and avoid playback freezing events. Second, a viewpoint-prediction-map-based cooperative bargaining game theory is proposed for bitrate allocation optimization to choose a suitable bitrate for each tile to reduce unreasonable bandwidth waste. The proposed scheme is compared with state-of-the-art approaches over a wide variety of mobile network conditions with multiple viewpoint traces and 360-degree video contents. Experimental results indicate that the proposed method outperforms the state-of-the-art approaches in terms of different experimental objectives over mobile devices.

In this article, we apply a high-order control to a dynamical system with uncertainty. There are two characteristics. First, the uncertain part, which is time-varying but bounded, is described in a fuzzy aspect. Specifically, the uncertainty lies within a fuzzy set and the bound is regarded as a fuzzy number. Second, the systems are uniformly bounded and uniformly ultimately bounded with a deterministic controller based on the Lyapunov theory. To obtain better system performance and lower control input, we apply the noncooperative game theory to optimize the parameters by establishing a Nash game. Then, the D-operation is proposed for the uncertainty related to fuzzy numbers. Finally, we perform the numerical simulations of the steering-by-wire system for verification.

There is an increasing need for charging station recommendation to minimize the overall charging time for electric vehicles and balance load for the charging stations. To grant this need, we model the recommendation problem as a Markov Decision Process (MDP) problem. However, the traditional MDP model has the issue of `curse of dimensionality'. To address this issue, we propose an extension of MDP: multiple-phase MDP, in which the state transition of MDP is decomposing into several phases, so as to reduce the state space and state transition complexities. This is done by introducing two states other than the normal state defined in MDP: post decision state and intermediate decision state. Then, we propose an online learning based algorithm to solve the formulated multiple-phase MDP model. Thanks to the reduced complexities of the state space and state transition, the proposed online algorithm can converge fast. By comparing to other recommendation mechanisms, such as game theory based recommendation and Q-learning based recommendation, our simulation evaluation demonstrates that our proposition can bring good performance.

The deployment of connected, automated vehicles (CAVs) provides the opportunity to enhance the safety and efficiency of transportation systems. However, despite the rapid development of this technology, human-driven vehicles are predicted to predominate the vehicle fleet, compelling CAVs to be able to operate in a mixed traffic environment. The key to achieving a reliable and safe human-CAV collaboration in such environments is to characterize the interactions between the actors and incorporate the underlying decision-making mechanism of human drivers into CAVs' motion planning algorithms. Towards this goal and extending a previously developed game theoretical model, the present study proposes a decision-making dynamic to achieve more realistic models of human behavior when making conflicting maneuvers at intersections. A novel field test is conducted to extract the required modeling data directly from CAVs' perception system, facilitating the incorporation of the model into CAV navigation algorithms. Model validation and sensitivity analysis provided invaluable insights into the nature of human decisions and indicated that the proposed structure is robust to environmental uncertainties and can well capture the real-world behavior of human drivers in unprotected left-turn maneuvers. The derived knowledge can be directly used in CAV motion planning algorithms to provide the vehicle with more accurate predictions of human actions when operating in mixed traffic environments.

Edge computing, as an emerging and prospective computing paradigm, allows a service provider to serve its users by allocating them to nearby edge servers delivering services with low latency. From the service provider's perspective, a cost-effective service user allocation aims to allocate maximum service users to minimum edge servers. Such an allocation leverages multi-tenancy to reduce the resources hired by the service provider for serving the service users. However, the allocation of excessive service users to an edge server may result in severe interference and consequently impact their data rates. There is a trade-off between multi-tenancy and interference in the pursuit of a cost-effective service user allocation. In this paper, we formally model this service user allocation (SUA) problem. To solve the SUA problem effectively and efficiently, we propose a game-theoretic approach, namely MI-SUAGame, to formulate the SUA problem as a potential game. We analyze the game and prove its admission to a Nash equilibrium. Then, a novel decentralized algorithm is designed for finding a Nash equilibrium in the game as the solution to the SUA problem. The performance of MI-SUAGame is theoretically analyzed and experimentally evaluated against the state-of-the-art approach.