Preview only show first 10 pages with watermark. For full document please download

W Cooperative Resource Management In Cloud-enabled Vehicular Networks Member, Ieee

   EMBED


Share

Transcript

7938 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 Cooperative Resource Management in Cloud-Enabled Vehicular Networks Rong Yu, Member, IEEE, Xumin Huang, Jiawen Kang, Jiefei Ding, Sabita Maharjan, Member, IEEE, Stein Gjessing, Member, IEEE, and Yan Zhang, Senior Member, IEEE Abstract—Cloud-enabled vehicular networks are a new paradigm to improve the quality of vehicular services, which have drawn considerable attention in industry and academia. In this paper, we consider the resource management and sharing problem for bandwidth and computing resources to support mobile applications in cloud-enabled vehicular networks. In such an environment, cloud service providers (SPs) can cooperate to form coalitions to share their idle resources with each other. We propose a coalition game model based on two-sided matching theory for cooperation among cloud SPs to share their idle resources. As a result, the resources can be better utilized, and the QoS for users can be improved. Numerical results indicate that our scheme can improve resource utilization and increase by 75% the QoS of the applications compared with that without cooperation. Moreover, the higher service cost of cooperation brings negative effect on coalition formation. The higher cooperation willingness of cloud SPs and the lower service cost support more service applications. Index Terms—Cloud-enabled vehicular network, coalition game, resource utilization, two-sided matching. Manuscript received December 31, 2014; revised July 1, 2015 and August 20, 2015; accepted September 9, 2015. Date of publication September 24, 2015; date of current version November 6, 2015. This work was supported in part by programs of the National Natural Science Foundation of China under Grant 61422201, Grant 61370159, Grant U1201253, and Grant U1301255, in part by the Guangdong Province Natural Science Foundation under Grant S2011030002886, in part by the Special-Support Project of Guangdong Province under Grant 2014TQ01X100, in part by the High Education Excellent Young Teacher Program of Guangdong Province under Grant YQ2013057, in part by the Science and Technology Program of Guangzhou under Grant 2014J2200097 (Zhujiang New Star Program), in part by the 240079/F20 Project funded by the Research Council of Norway, and in part by the European Commission Seventh Framework Programme for Research and Technological Development (FP7) through the Cognitive Radio Oriented Wireless Networks (CROWN) Project under Grant PIRSESGA-2013-627490. R. Yu, X. Huang, J. Kang, and J. Ding are with the School of Automation, Guangdong University of Technology, Guangzhou 510006, China (e-mail: [email protected]; [email protected]; jiawen.kang@ ieee.org; [email protected]). S. Maharjan is with the Simula Research Laboratory, 1364 Fornebu, Norway (e-mail: [email protected]). S. Gjessing and Y. Zhang are with the Simula Research Laboratory, 1364 Fornebu, Norway, and also with the Department of Informatics, The Faculty of Mathematics and Natural Sciences, University of Oslo, 0373 Oslo, Norway (e-mail: [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIE.2015.2481792 I. I NTRODUCTION W ITH the rapid development of wireless communication technologies, connected vehicles may use vehicle-toinfrastructure (V2I) and vehicle-to-vehicle (V2V) communications to form vehicular ad hoc networks (VANETs). Due to the high potential for a large variety of applications, VANETs have received considerable attention in academia and industry. At the same time, we have witnessed fast-growing applications catering to drivers [1], [2]. Massive mobile applications are typically supported by vehicular onboard units (OBUs). However, the limited computing capability of OBUs brings new challenges to VANETs. VANETs may integrate with cloud computing and are evolving toward cloud-enabled vehicular networks [3]. In cloud-enabled vehicular networks, there are many mobile applications in connected vehicles, e.g., online video streaming [4], location-based services, social services [5], and accident alerts [6], [7]. Generally, these applications are supported by both bandwidth and computing resources. A cloud service provider (SP) reserves a certain amount of long-term bandwidth resources from network providers and reserves some long-term computing resources (e.g., CPU, memory, and storage [8]) from data centers, which are owned by the cloud providers. All these resources are used to support different mobile applications. The number of mobile applications running on an SP is depending upon the SP’s available bandwidth and computing resources, thus possibly degrading the service quality and the user experience when the resources are limited. Various mobile applications have different resource requirements. In some cases, when running certain applications, an SP may fall short of one kind of resource while it has sufficient other resources. Therefore, resource sharing should be encouraged in cloud-enabled vehicular networks. Similar cooperative ideas can be found in vehicular delay-tolerant networks [9]–[11]. Vehicles share their storage, bandwidth, and energy resource with each other to secure, store, carry, and forward network data for the mutual enhancement of the overall network performance. In this paper, we focus on bandwidth and computing resource cooperation in a cloud environment. As a result, it is important for SPs to design efficient resource management and sharing schemes to satisfy the service quality requirements of the applications. Compared with traditional mobile cloud computing (MCC) networks, cooperative resource management problems are more challenging in cloud-enabled vehicular networks. On one hand, unlike traditional devices or terminals, vehicles have a unique feature in their potential high mobility. A vehicle may pass 0278-0046 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS by several roadside units (RSUs) during the service time of a running application. In this case, the vehicle has to: 1) perform radio handoff and access to different RSUs and 2) have its virtual cloud resource synchronously migrate to the RSUs for the continuity of the application. This “cloud resource mobility” problem is an essential new issue in vehicular cloud computing [12]. On the other hand, the high mobility of vehicles also induces wireless channel fading [13]. To guarantee the QoS, it is necessary to consider the impact of vehicle high mobility on wireless channel quality in cloud-enabled vehicular networks. There are several previous studies that address related issues [12], [14]–[20]. In [14] and [15], vehicles may cooperate to run applications through V2V communication without additional infrastructure. However, it is not easy to ensure secure communications among the connected vehicles [21]–[23]. The high mobility of vehicles and the short transmission range may result in service outage. Some researchers utilize machine-to-machine communication to establish the connectivity between vehicles and to provide services [16]. As the resource is distributed in this scenario, it may not be enough to support massive applications. Some research focuses on how to efficiently allocate computing resources in cloud-enabled vehicular networks [12], [17]. In [17], a Bayesian coalition game is proposed to avoid the poor utilization of the computing resource and to reduce energy consumption. The problem of the intelligent usage of virtual machines (VMs) can be studied for efficiently utilizing the computing resource and balancing the load during sharing the resource. Similar work can be found in [12], which allocates the computing resource to achieve the efficient handling of frequent changes in data center topologies. Compared with cloud data centers, a vehicular cloud data center is a more dynamic environment because of the mobility of the connected vehicles. Both bandwidth and computing resources must be scheduled to enhance the performance of cloud-enabled vehicular networks. A new vehicular cloud architecture using clustering techniques was designed in [18] and [19], where a new Q-learningbased helper selection was employed to improve resource management. Clustering techniques are used to group vehicles and promote resource cooperation. However, the service delay caused by the complex resource allocation algorithm cannot be neglected in practice and may become the bottleneck of performance for high-speed vehicles. In [20], some SPs cooperate to create a resource pool to share their bandwidth and computing resources. All the SPs can use the resource from the resource pool when required, thereby increasing resource utilization. The revenue of an SP is based on the contribution to a user’s demand. However, it may not be the optimal approach to maximize the revenues of individual SPs using grand coalition. In this paper, we introduce a coalition game-based model for resource sharing among the different SPs in cloud-enabled vehicular networks. In this paper, resource sharing is conducted in two phases. In the first phase, each SP evaluates its revenue and decides to either work alone or join a coalition in the cloud market. In the second phase, an SP will either rent the resource from others or lease the resource to others. The two groups of SPs effectively match their demands by using a two-sided matching approach. As a result, the cooperation between the SPs not only improves the revenue of the SPs but also increases 7939 the number of users that can access the services. This will increase the QoS for both the users and the SPs. The major contributions of this paper are summarized as follows. • We combine prices with resource demands in cloudenabled vehicular networks and adopt a coalition-game model, which is close to a realistic business model in the cloud market. • We apply two-sided matching theory in forming coalitions. Our scheme not only improves the fairness of transactions but also optimizes the resource utilization of the SPs, and a win-win situation for the SPs is achieved in every coalition. • The revenue of an SP includes not only the profit from the users who use mobile applications but also the benefit from leasing its idle resource to other SPs. Therefore, our scheme also enhances the revenues from idle resources. The rest of this paper is organized as follows. The system model and assumptions are described in Section II. We present our problem formulation in Section III and analyze the classification of SPs in Section IV. In Section V, we elaborate our proposed scheme. Extensive simulations are conducted and discussed in Section VI. Section VII concludes this paper. II. S YSTEM M ODEL A. Cloud-Enabled Vehicular Networks Cloud-enabled vehicular networks are an emerging paradigm in MCC, which combines the advantages of VANETs with cloud computing. There are two kinds of cloud computing for vehicular networks, i.e., infrastructure-based vehicular clouds and autonomous vehicular clouds. By communicating with the RSUs, a vehicle can access services in the infrastructurebased vehicular cloud. Vehicles can form an autonomous cloudenabled vehicular network to support emergencies and other ad hoc events [3]. Fig. 1 shows that a mobile application in a vehicle can be divided into two parts, i.e., a local computing part running on a mobile device and a remote computing part running on a server. These two parts communicate with each other through RSUs. Therefore, the part running on a remote server requires both bandwidth and computing resources. Cloud-enabled vehicular networks provide services at three levels. The first level is the fundamental level called Infrastructure as a Service. The infrastructures, such as storage, computing, sensing, communicating devices, and software, are created as VMs in the cloud. The second level is called Platform as a Service, whose components and services are provided and configured as a service. The third level is Software as a Service, whose applications are provided in a “pay-as-you-go” fashion. B. Resource Cooperation in Cloud-Enabled Vehicular Networks Fig. 2 shows that cloud-enabled vehicular networks have three layers. In the first layer, a vehicle communicates with other vehicles or its nearest RSU, which provides access to a local cloud in order to send requests. In the second layer, the cloud SP will allocate resources to vehicles. In the last layer, the 7940 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 Fig. 1. Model of a cloud-enabled vehicular network. Fig. 2. Cloud-enabled vehicular network environment. central cloud is the data center that runs mobile applications on the application server. The data center manages and allocates all the resources to the SPs. The SPs provide resources to vehicles through RSUs. With the resources from the cloudenabled vehicular networks, the mobile devices in the vehicles can run applications. 1) Vehicular Communication: The vehicles are equipped with OBUs, GPSs, and pseudonymous recorders, which make mobile Internet available on the move. V2V communication allows the vehicles to communicate with nearby vehicles and to form a cloud-enabled vehicular network. The cloud-enabled vehicular networks provide low-price computational services to vehicles. Through V2I communication, the vehicles can get resources from the SPs in the cloud. 2) Cloud SPs: An SP is a virtual network operator that reserves a certain amount of bandwidth resource from a network provider and computing resource from a data center (cloud provider) on a long-term basis. After checking the availability of the resource with the SP, the application server accepts requests from users. The application server initializes the remote computing modules in the data center when there are enough resources with the SP. Various applications require different resources from the SPs. Therefore, an SP can provide more than one resource. In Fig. 2, the SPs in the cloud possess two kinds of resources, i.e., bandwidth resources and computing resources. The long-term resources are the fixed assets of each SP, which are cheaper than the resources obtained on an on-demand basis. However, the total number of available resources determines the maximum number of supported applications. If the applications surpass the capabilities of the SPs, the SPs will either drop the new service requirement or rent resources from others. In this paper, a coalition game is proposed to resolve the problem of resource sharing and cooperation in order to enhance the SPs’ revenue and improve resource utilization. Fig. 2 shows that an SP with more bandwidth and computing resources can share its resources with other SPs who have a shortage of these resources. In coalition 1, SP2 may expand its capacity to support applications by renting more extra longterm resources with a high cost from network providers or the cloud provider. On the other hand, another SP (e.g., SP1 ) can lease short-term access to its own long-term reserved resource to SP2 at a lower cost. This way, SP2 can increase the number of applications that can be accessed or run to obtain a higher QoS. Moreover, SP1 improves its revenue by utilizing its idle resources. SP3 rents short-term access to bandwidth resources in SP2 . In coalition 2, SP4 can lease computing and bandwidth resources to different SPs (e.g., SP5 and SP6 ). There are several kinds of coalitions, but the deal is only made based on the condition of mutual benefit among SPs. 3) Data Centers: A data center provides hardware facilities and infrastructure for clouds. SPs rent long-term reserved resources from data centers to be a third party for resource management. Finally, a number of servers are linked with highspeed networks to provide services for vehicles. YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS Note that round-trip requirements of mobile cloud applications are a unique characteristic compared with normal applications. Since it is crucial to reduce the necessary roundtrip latency in mobile cloud applications, we are motivated to leverage live VM migration technology in this paper. The basic idea of using live VMs in vehicular cloud computing is to keep vehicles close to the physical resource such as in the platforms, e.g., PlanetLab [24]. Previous studies on these platforms indicate that VM migration technology is significantly beneficial in reducing the round-trip latency for mobile cloud applications. Therefore, the SPs adopt the live VM migration technology to support mobile applications. III. P ROBLEM F ORMULATION A. Wireless Network and Data Center There are N cloud SPs in a cloud-enabled vehicular network. The SPs can negotiate with each other and share their reserved bandwidth and computing resources. A cloud SP i at time t is indexed by SPti , where i = 1, 2, . . . , N . Resource management in cloud-enabled vehicular networks includes the bandwidth resources and the computing resources, which are t t and CSP , respectively. There are k vehidenoted as BSP i i cles denoted as V = {v1 , v2 , v3 , . . . , vk }. The vehicles obtain services by sending application requests to the cloud. The mobile applications are indicated by A = {a1 , a2 , a3 , . . . , ak }. In addition, a set of bandwidth resources and a set of computing resources are represented by B = {b1 , b2 , b3 , . . . , bk } and C = {c1 , c2 , c3 , . . . , ck }, respectively. In this paper, we consider the following features in our model. First, the arrival and departure times of the mobile applications of vehicles follow a Poisson distribution [25]. Second, for a vehicle j (denoted as vj ), its mobile application ai will consume bandwidth resource bi that is a random value in the range [bmin,i , bmax,i ]. In a vehicular network, vehicles generally travel at speeds up to 60 mi/h. The drastic variation of wireless channels may result in significant fading effects. Vehicle speed becomes an essential factor in the channel quality model. Cimini [13] presented that the fading rate is directly proportional to the vehicle speed when the speed is higher than a specific threshold value. The threshold value of the speed is dependent on the communication capability of vehicular devices. To guarantee the needed QoS, the running mobile application may need more bandwidth and computing resources when the vehicle is driven at a high speed. A vehicle vj consumes bandwidth resource bi,n when its speed is not higher than threshold s∗j . If vj drives at speed sj , the wireless channel fades at the rate of rj , which is given by  ksj , sj > s∗j rj = 0, sj ≤ s∗j where k is a predefined constant. During time period Δt, the maximum value of the available bandwidth for mobile application ai is  bmax,i = bi,n + rj (sj )dΔt. 7941 Based on these features, application ai can obtain the amount of bandwidth qib from SPti , which can be defined as  Bi , Bi ≤ bmax,i b (1) qi = bmax,i 1, Bi > bmax,i where Bi is the bandwidth resource provided by SPti . At the same time, mobile application ai needs computing resource ci whose value follows an arbitrary distribution in the range [cmin,i , cmax,i ] with probability density function ρc (ci ) [26]. Similarly, as SPti can provide computing resource ci , the amount of computing resource service qic can be defined as ⎧ ⎪ ci ≤ cmin,i ⎨0,  ci c qi = (2) cmin,i ρc (ci )dci , cmin,i < ci ≤ cmax,i ⎪ ⎩ 1, ci > cmax,i . Therefore, the QoS requirement of user i is the minimum value between qib and qic , i.e., QoSi = min(qib , qic ). Next, we evaluate the ability of each SP to run an application. B. Quantification of Resource Each SP has a certain amount of long-term reserved resources. Bandwidth resources and computing resources can be shared through short-term cooperation. Let ηi and γi represent the differences between the amount of the required resources and the amount of the actual resources as follows: ηi = |Bi − bmax,i | γi = |Ci − cmax,i | . If the available bandwidth resource Bi is more than the upper bound of bandwidth resource bmax,i , ηi indicates the amount of resources that SPti can offer to rent out. On the contrary, SPti wants to rent some bandwidth resources ηi from other SPs for short-term usage. In the equation, γi is obtained by the difference between the available computing resource Ci and the upper bound of computing resources cmax,i . C. Utility Function The utility function of cloud SP SPti consists of two parts, i.e., the satisfaction function and the cost function. Satisfaction function Sit in the case of renting extra short-term resources from other SPs (denoted as Rfrom ) is different from the case of leasing own resources to others (denoted as Rto ). However, in both cases, the satisfaction functions are convex and start from zero. For simplicity, we take bandwidth resource cooperation as an example to analyze our model. The satisfaction function in Rfrom is defined as t t S(wb,i , xtb,i ) = wb,i log(1 + xtb,i ). For presentation, we simply write the aforementioned equation as t t Sb,i = wb,i log(1 + xtb,i ) (3) 7942 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 where 0 ≤ xtb,i ≤ 1, and t = wb,i α 1+ e−β(ϑ−Bi ) . (4) ∗ Here, xb,i represents the amount of leased bandwidth resources t of cloud SP i. The willingness of SPti is denoted as wb,i , which is an S function curve. The predefined constants α and β are determined by the user preferences. ϑ is a constant indicating the maximum required resource. Apparently, the larger the available resource Bi , the lower is the willingness of the renting resource from other SPs. This means that more and more idle resources should be leased out. For Rto , xb,i represents the bandwidth resources that can be leased out. Thus, it is a negative value. Alternatively, we adopt the satisfaction function in this case as follows: t (log(2 + xtb,i ) − 1), stb,i = wb,i the first-order optimality condition (∂utb,i (SPti ))/(∂xtb,i ) = 0, we obtain where −1 ≤ xtb,i ≤ 0. (5) t t > 0 and Pb,to < 0 denote the unit price in Rfrom Let Pb,fr and Rto , respectively. The sign is an indication of the cost or the earning. The unified utility functions in the two cases of cloud SP i in time slot t are given by t t t , xtb,i , Pb,to , Pb,fr U wb,i ⎧ ⎨wt log 1 + xt − P t xt , 0 ≤ xtb,i ≤ 1 b,i b,i b,fr b,i = ⎩wt log(2 + xt ) − 1 + P t xt , −1 ≤ xt < 0. b,i b,i b,to b,i b,i xtb,i = (6) In time slot t, each SP will choose a proper xtb,i to maximize utility utb,i (SPti ) according to the given price. IV. D ISCUSSION ON THE R OLE OF SP S The SPs in this paper can be classified into two categories according to their potential behaviors, i.e., Rfrom and Rto . In this paper, the prices in both sides are set by the network provider. The network provider only has one price for renting. However, different coalitions have different cooperative prices. We first analyze the characteristic of the utility in Rfrom . Differentiating utb,i (SPti ) with respect to xtb,i , we obtain t ∂utb,i SPti wb,i t − Pb,fr = ∂xtb,i t 1 + xb,i ln 2 t 2 t t ∂ ub,i SPi wb,i = − < 0. 2 2 ∂xtb,i 1 + xtb,i ln 2 Clearly, the utility function of Rfrom is concave, which indicates that the maximal value of this function exists. Therefore, using −1 (7) ∗ where xtb,i is called the best response, which maximizes the ∗ t . Then, we substitute xtb,i utility on the condition of price Pb,fr into (6) and have  t t wb,i wb,i t ∗ t t t − + Pb,fr . (8) ub,i (SPi ) = wb,i log t ln 2 Pb,fr ln 2 Therefore, the utility function of cloud SPs SPti can be cont . Then, verted into an optimal utility function in terms of Pb,fr ∗ t t if we take the first and second derivatives of ub,i (SPi ) with respect to xtb,i , we obtain ∗ t wb,i ∂utb,i SPti +1 =− t t ∂xb,i Pb,fr ln 2 ∗ t ∂ 2 utb,i SPti wb,i > 0. = 2 2 ∂xtb,i P tb,fr ln 2 Clearly, the optimal utility function is convex. Using ∗ t (∂utb,i (SPti ))/(∂Pb,fr ) = 0, we can get the ceiling price P tb,fr,i when ∗ P tb,fr,i = In order to simplify the aforementioned equation, we use the following form to express the unified utility function: utb,i (SPti ) ⎧ ⎨wt log 1 + xt − P t xt , 0 ≤ xtb,i ≤ 1 b,i b,i b,fr b,i = ⎩wt log(2 + xt ) − 1 + P t xt , −1 ≤ xt < 0. b,i b,i b,to b,i b,i t wb,i t Pb,fr ln 2 t wb,i . ln 2 (9) The ceiling price is interpreted as a threshold for SPti . Under different given prices set by the network provider, SPti may have different best utilities as follows. ∗ t t < P b,fr,i , the best utility is obtained with the best • If Pb,fr ∗ t ∗ response xb,i > 0, and accordingly, utb,i (SPti ) > 0. In this case, the SP would like to rent resources. ∗ t t = P b,fr,i , the best utility is obtained with the best • If Pb,fr ∗ t ∗ response xb,i = 0, and accordingly, utb,i (SPti ) = 0. In this case, the SP does not expect to rent resources. ∗ ∗ t > P tb,fr,i , the best response is xtb,i < 0. This • If Pb,fr ∗ violates the limit of xtb,i ≥ 0. In this case, the SP refuses to rent resources. Following a similar analysis to the utility function of Rto , i.e., (6), we have t wb,i ∂utb,i SPti t = − Pb,s ∂xtb,i t 2+x ln 2 ∂ 2 utb,i SPti ∂xtb,i 2 b,i t wb,i < 0. = − 2 2 + xtb,i ln 2 The utility function in Rto is also concave. Thus, we can obtain the best response of (∂utb,i (SPti ))/(∂xtb,i ) = 0 as  t wb,i t ∗ xb,i = − 2 + t . (10) Pb,to ln 2 YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS 7943 Now, substituting (10) into (6), we get the optimal utility t as function for Rto in terms of Pb,s  t t wb,i wb,i t t ∗ t t t − wb,i − 2Pb,to − . ub,i SPi = wb,i log − t Pb,to ln 2 ln 2 (11) ∗ The first-order and second-order derivatives of utb,i (SPti ) with t respect to price Pb,to are written as ∗ t ∂utb,i SPti wb,i −2 = − t t ∂Pb,to Pb,to ln 2 ∗ t ∂ 2 utb,i SPti wb,i > 0. = 2 t 2 ∂P b,to P tb,to ln 2 Fig. 3. Roles of the SPs in each case. TABLE I K EY N OTATIONS AND D ESCRIPTIONS IN F IG . 3 ∗ From the first-order optimality condition (∂utb,i (SPti ))/ ∗ t (∂Pb,to ) = 0, we obtain the minimal price P tb,to,i that the SP is willing to lease out idle resources for revenue as ∗ P tb,to,i = − t wb,i . 2 ln 2 (12) We also have three cases to estimate the preference of the SP t by comparing with price Pb,to in Rto . ∗ t | > |P tb,to,i |, the best utility is obtained • When |Pb,to ∗ with the best response xtb,i > 0, and accordingly, ∗ utb,i (SPti ) > 0. In this case, the SP prefers to lease its resources to others. ∗ t | = |P tb,to,i |, the best utility is obtained • When |Pb,to ∗ with the best response xtb,i = 0, and accordingly, ∗ utb,i (SPti ) = 0. In this case, the SP does not expect to lease its resources to others. ∗ t | < |P tb,to,i |, the best utility can be obtained • When |Pb,to ∗ with the best response xtb,i < 0. This violates the posit tivity requirement of xb,i . In this case, the SP refuses to lease its resources to others. By considering both the cases of renting resources from other SPs and leasing to others, the rules used to determine the actions of an SP are shown in Fig. 3. The key notations and corresponding descriptions of Fig. 3 are listed in Table I. ∗ ∗ t t • In the case that Pb,fr < P tb,fr,i and Pb,to < |P tb,to,i |, the SP prefers to rent resources from other SPs. ∗ ∗ t t ≥ P tb,fr,i and Pb,to < |P tb,to,i |, the • In the case that Pb,fr SP prefers to be idle. ∗ ∗ t t ≥ P tb,fr,i and Pb,to ≥ |P tb,to,i |, • In the case that Pb,fr the SP prefers to lease resources to other SPs. ∗ ∗ t t • In the case that Pb,fr < P tb,fr,i and Pb,to ≥ |P tb,to,i |, the action of the SP is determined by the utility, which is further analyzed as follows. To further analyze the actions of the SP in the zone of ∗ ∗ t t < P tb,fr,i and Pb,to ≥ |P tb,to,i |, which is defined as the Pb,fr discussion zone, we compare (8) with (11) and thus obtain t ∗∗ the critical price P b,i . For the same cloud SP, the prices t t in Rfrom and Rto are the same. Thus, Pb,fr = −Pb,to . We compare the different best utility functions for one cloud SP i, and have t wb,i log t wb,i t ln 2 Pb,fr  t wb,i t + Pb,fr ln 2  t t w wb,i b,i t t t + 2Pb,fr − wb,i = wb,i log − . (13) t Pb,fr ln 2 ln 2 − Therefore, P tb,i ∗∗ t = wb,i . ∗ (14) ∗∗ ∗ t . The clasObviously, we get that |P tb,to,i | < P tb,i < P b,fr,i sification of the SP is based on the price given by the network provider Pbt and follows the rules shown by the utility in the discussion zone in Fig. 3 [27]. ∗∗ t < P tb , the SP prefers to rent extra resources • If Pb,i from others. It is represented by the shaded area in the discussion zone in Fig. 3. ∗∗ t > P tb , the SP prefers to lease its resources to • If Pb,i others. It is represented by the area that is not shaded in the discussion zone in Fig. 3. ∗∗ t = P tb , the SP will be self-sufficient. It is repre• If Pb,i sented by the red point marked in the discussion zone in Fig. 3. The analysis for the cases of computing resources is similar and thus straightforward. 7944 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 V. S OLUTION AND D ISCUSSION A. Pareto Optimality In a coalition game, players hope to join different coalitions to improve their utility. There are several possible options: 1) an individual player would like to join a certain coalition if the utility is improved in the coalition; 2) a player of coalition A would like to leave A but join coalition B if B provides a better utility; and 3) a player would like to leave a coalition and work alone if leaving results in an increased utility. We employ a simple but effective mechanism, i.e., the merge-andsplit mechanism, to derive the stable formation of the coalition game. In the merge-and-split mechanism, Pareto optimality is used as a criterion of the options of the players. Definition 1: There are two sets of coalitions G1 = {G11 , 1 G2 , . . . , G1l } and G2 = {G21 , G22 , . . . , G2m }, which are two different partitions of the same set G ⊂ S. For a player SPi , let uk (SPi ) denote the utility of SPi in coalition Gk (k = 1, 2). Coalition G1 is preferred over G2 by the Pareto order, which is denoted by G1  G2 , if and only if u1 (SPi ) ≥ u2 (SPi ) ∀ SPi ∈ S  (15) with at least an inequality for a player SPk . Following the criterion of the Pareto order, the players will be reorganized so that coalitions are reformed for improved utilities. This procedure usually takes many rounds. In each round, all the coalitions should be involved so that their utilities are ensured to increase, or at least not to decrease. It means that the reorganization of coalitions is naturally a global operation. In order to facilitate the procedure, we decouple the global operation by a series of distributed operations using the following two fundamental rules. • Merge. For any set of coalitions {G1 , . . . , Gl }, if  { lj=1 Gj } {G1 , . . . , Gl }, then merge {G1 , . . . , Gl } to  { lj=1 Gj }, which is denoted by {G1 , . . . , Gl } →  { lj=1 Gj }. l Gj , if {G1 , . . . , Gl } • Split. For any coalitions Uj=1 l l { j=1 Gj }, then split { j=1 Gj } into {G1 , . . . , Gk },  which is denoted by { lj=1 Gj } → {G1 , . . . , Gk }. By using these rules of the merge-and-split mechanism, the SPs are allowed to negotiate and constitute the local coalitions. Therefore, the global Pareto optimal collection of coalitions can be consolidated gradually. B. Strategy of the Coalition Formation Game 1) Stage 1—Initialization: To increase the available resources for mobile applications, the SPs can share their resources by cooperation in the coalition game. Each SP in the MCC environment will be partitioned by G = {G1 , G2 , . . . , Gl }. Before the coalition is created, we have G = SP = {SP1 , . . . , SPN }, which means that all SPs work alone and have no cooperation at the beginning. The base station will send the price information to each SP. Based on the current price, SPs can be categorized into three types, i.e., renting extra resources as Rfrom , leasing its idle resources as Rto , and being idle. The idle SPs would not join the game since they are self-sufficient. As the game continues, there exist two forms, i.e., individual and cooperative coalitions (coalition is between different SPs). These two forms are called “member.” The twosided matching is performed to improve the efficiency of the coalition formation process. Therefore, there are two sides in the coalition game. The members that would like to rent resources (Rfrom ) are inviters. The members that would like to lease out their resources (Rto ) are invitees. In the game, the inviters will propose to the invitees to constitute coalitions. If an invitee accepts the proposal from an inviter, a new coalition is formed. The inviters will propose to the invitees according to their own preference lists. On the other hand, the invitees accept the proper proposals from the best inviters according to their own preference lists. Clearly, the matching between the inviters and the invitees, which is called inviter–invitee matching, is a twosided matching, which takes the fairness of the transaction into consideration. 2) Stage 2—Coalition Formation: The entire procedure of coalition formation has a number of rounds. Each round consists of three main steps as follows. • Update preference list. The preference list indicates the preference of the owner to cooperate with the other members. All the members of the two sides should regenerate their preference list in each round of the coalition formation. For a given inviter/invitee, it will specify the invitees/inviters one by one to hypothetically constitute coalitions and compute the best utilities. After that, the members in the preference list are sorted in descending order of the best utilities. • Inviter–invitee matching. The inviters will propose to the invitees. When receiving proposals, an invitee will select a candidate with the highest rank in its preference list. The other inviters are labeled as unselected. The selected inviter is merged with the invitee to form a new coalition. The unselected inviters will be compared with the tentative members of the invitee coalition. If an unselected inviter has the lowest rank, it is rejected and has to propose to other invitees. • Merge-and-split. When all inviters have proposed, the invitees will carry out the merge-and-split operation to ensure Pareto optimality in the list of invitees. The merge-andsplit operation consists of the following two operations. 1) Merge (G  = Merge{G}). If a higher utility is achievable after merging and the utilities of all SPs are improved in the Pareto order, coalitions or individual SPs in collection G will merge. 2) Split (G  = Split{G}). If a higher utility is achievable after splitting and the utilities of all SPs are improved in the Pareto order, one or more coalitions in collection G will split. The aforementioned three steps will be repeated until there is no more change of the members in both the inviters’ and the invitees’ sides. Finally, through the two-sided matching, the SPs in the coalition can reach a win-win situation. YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS 3) Stage 3—Cooperative Game: After the coalitions are stably constituted, all members in G(Gi ∈ G) will claim the amount of required resources negotiated. For an individual SP, it will claim the demand by (7) and (10). For a coalition, the inside SPs will work out their demands through a cooperative game. Since the deal is between the members, we only consider fair trade, which means that the amount of renting resources is equal on both sides. C. Coalition Result In the coalition game, both the members in Rfrom and in Rto try to maximize their utility values. Therefore, they prefer the conditions that make them achieve the maximum utility. There are two conditions, i.e., working alone or working in a coalition. 1) Condition 1—Working Alone: After the coalition game, some SPs may work alone since these SPs did not find other SPs to improve their utility when the numbers of members in the two sides are not equivalent. In addition, some SPs cannot improve their utility by working in any coalition, e.g., the selfsufficient SPs. 2) Condition 2—Resource Trading in a Coalition: For this condition, there exists at least one SP, i.e., SPtA , in Rfrom and one SP, i.e., SPtB , in Rto . We suppose that SPtA and SPtB are both their final choices in a coalition; thus, Gj = {SPtA , SPtB }. In this paper, we discuss the bandwidth resources, but the process of computing resources is similar. According to the rule of the equivalent quantity of trading resources, we use ηA and ηB to balance the ratio differences between SPtA and SPtB . Let t t Pb,fr and Pb,to denote the negotiated prices for renting resources t of SPA and SPtB , respectively. Here, the network provider will t charge a service fee, i.e., σPb,to (0 < σ < 1), which could be a very small part of the renting price. If σ = 1, no SP will participate in the coalition formation since the loss outweighs the gain. Therefore, (8) and (11) take the following form: ⎧ wt ⎪ xtb,A = P t b,Aln 2 − 1 ⎪ ⎪ b,fr,n ⎪ wt ⎪ ⎨ t xb,B = − P t b,Bln 2 + 2 (16) b,to,n ⎪ t t ⎪ ⎪ η x + η x = 0 A b,A B b,B ⎪ ⎪ ⎩ t t Pb,fr = −(1 + σ)Pb,to . After solving these equations, we can obtain the amount of t trading resources xtb,A and kxtb,B , and the two prices, i.e., Pb,fr t and Pb,to . Moreover, the balance ratio is denoted as k. These are defined as follows: xtb,A = −kxtb,B = t Pb,fr = t (2k + 1)wb,A −1 t + k(1 + σ)wb,B D. Stability of The Coalition Formation In this section, we mainly discuss the stability and convergence of the proposed strategy of coalition formation. We use Pareto optimal Dc -stable partitions to demonstrate the stability of the coalition according to the work in [28]. Definition 1: A collection of coalitions S = {S1 , . . . , Sk } is said to be Dc -stable if it satisfies two conditions as follows. 1) i ∈ {1, . . . , k} and for each partition {P1 , . . . , Pl } of l coalition Gi : u(G i) ≥ j=1 u(SPj ). 2) S ⊆ {1, . . . , k}: j∈S u(Gi ) ≥ u(∪i∈T Gi ). Theorem 1: The final coalition formation under the proposed strategy can be Dc -stable [29]. Proof: We first consider condition 1. In the final coalition set G = {G11 , G12 , . . . , G1l }, we assume that SPi is included in coalition Gk , i.e., SPi ∈ Gk . However, if SPi can obtain a higher utility by working alone or joining other coalitions Gl , condition 1 will be violated. Therefore, according to the splitand-merge rules, SPi will leave the current coalition Gk . Thus, coalition Gk will not exist. The coalition formations in G are unstable and cannot be the final coalition set. Therefore, condition 1 must be satisfied for any stable coalition generated under the proposed strategy. For condition 2, we consider the situation in the same final coalition set G = {G11 , G12 , . . . , G1l }. If coalition Gk can obtain a higher utility when it combines with other SPs and become a larger coalition Gk (Gk ⊆ Gk ), then Gk will merge into Gk , i.e., u(Gk ) < u(Gk ). G cannot be the final coalition set for the same reason. Therefore, for stable formation of the final coalition set, condition 2 needs to be satisfied. In summary, both conditions 1 and 2 are involved in the final coalition set in order to ensure the stability of the final result. Theorem 2: In Theorem 1, the final coalition formation is Dc -stable. Therefore, if this partition exists and is stable on the final coalition set, the Pareto optimal solution will be the only stable solution. Theorem 3: In the matching process, the coalition could be only formed on the different sides. Proof: If there exists a coalition in the side of Rfrom , both t SPA and SPB are included in coalition Gk . Note that Pb,to < ∗ ∗ t t P b,to,B < P b,to,A according to the rules of classification. Since the coalition will only exist on the mutual benefit condition, SPA will be the buyer, whereas SPB will be the seller. Coopera∗ t < PA,B < tion price PA,B will satisfy the condition Pb,to,B ∗ t Pb,to,A . It means that, after participating in the coalition, SPB is still the seller, and SPA is the buyer. Therefore, the coalition only exists when it contains members from both sides at the same time. VI. N UMERICAL R ESULTS t wb,A In this section, we introduce our hardware system and laboratory testbed. We also use real public traces of vehicles to evaluate our proposed scheme. t t wb,A + k(1 + σ)wb,B t Pb,to =− 7945 (2k + 1) ln 2 t wb,A t + k(1 + σ)wb,B (2k + 1) ln 2(1 + σ) ηB k= . ηA A. Hardware Platform (17) We have implemented a hardware testbed including a vehicular OBU and the cloud-enabled RSU named “cloud box.” The 7946 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 Fig. 4. Hardware testbed for real field demonstration. (a) Hardware system. (b) Architecture of the hardware testbed. Fig. 5. Comparison of the throughput and the CPU usage in experimental results. (a) Comparison of experimental results in terms of the throughput. (b) Comparison of experimental results in terms of the CPU usage. picture of the hardware system of the cloud box is shown in Fig. 4(a), and the architecture of the cloud box is shown in Fig. 4(b). The hardware system mainly includes an industrial personal computer and a high-performance wireless communication module to communicate with vehicles and data centers. By a wired or wireless communication module, the cloud box takes the role of the RSU with powerful capabilities to support vehicular services when connecting to the data center in the local cloud. Therefore, the cloud box is the key component to connect the vehicles and the data center. In this case, we carry out an experiment using the cloud box. In this experiment, we use several mobile devices that act as vehicles, and there are two SPs. SP1 has a 2-Mb/s bandwidth resource and 20% CPU resources (computing resources) of a computer. SP2 has a 1-Mb/s bandwidth resource and 10% CPU resources of the same computer. SP1 is overloaded and needs more bandwidth and CPU resources, whereas SP2 has idle resources in the 60s before reversing the state of SP1 and SP2 . The results of the experiment are provided in Fig. 5. Fig. 5 shows that the resource-limited SP with cooperation can improve its capacity by using another SP’s idle resources. To further analyze the performance of our proposed scheme, we will use real public traces of vehicles for performance analysis in the following section. B. Parameter Setting We consider a scenario with 20 cloud SPs and 10 service areas. The reserved bandwidth of each cloud SP from the base station is a random value that follows a uniform distribution and ranges from 1 to 10 Mb/s. The servers of each cloud SP in the data center are random values and follow a uniform distribution in the range [1, 10]. A set of mobile applications has bandwidth and server requirements that follow a uniform distribution that ranged from 1 to 4; thus, ϑ = 4. We adopt real traces of vehicles in the household transportation survey of San Francisco, California [30]. Generally, the normal speed of the vehicles is from 25 to 43 mi/h. However, some vehicles drive at the speed of 60 mi/h, which is the maximal speed of the observed vehicles. In Fig. 6, we can clearly find two main rush hours in a whole day, i.e., the morning rush hours and the evening rush hours. The arrival time of applications from 11:00 to 20:00 follows a Poisson distribution as the black curve shown [25], [31]. In addition, the works in [25] YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS Fig. 6. Observation of real vehicle traces. and [31] indicate that there are no direct connections between vehicle traces and the number of requested applications. Here, λ is the arrival rate of new applications. The service time of the application follows an exponential distribution, and the average value is μ [32]. C. Performance Analysis 1) Impact of SP Cooperation: Fig. 7(a) shows the average number of running applications in every observation time. The solid lines with squares represent the average number of running applications without cooperation during the observation time. The dotted lines with circles are the average number of running applications with cooperation among the SPs. Clearly, the average number of running applications with cooperation is larger than that without cooperation. For instance, at t = 6, the average number with cooperation is 2.5 times that without cooperation. Considering the whole observation duration, the average value of all running applications without cooperation is about 13 (dotted line), whereas that without cooperation is about 8 (solid line), which is 42.8% less than that with cooperation. This figure illustrates that cooperation among the SPs can improve resource utilization. Fig. 7(b) shows the average value of the QoS of applications in terms of time. The columns in blue represent the QoS of the applications provided by the SPs without cooperation during the observation time. The columns in red are the incremental portion of the applications’ QoS with cooperation among the SPs. The black line is the average value of the QoS without cooperation, whereas the red line is the average with cooperation, which is 75% higher than that without cooperation. Fig. 7(c) shows the influence of high mobility on the performance. It is observed that the applications’ QoS is decreasing when the average speed of the vehicles is increasing. This is because the increasing average speed of the vehicles causes a higher fading rate, thus having a potential impact on the bandwidth availability. Fig. 7(c) shows that the QoS with cooperation decreases less than that without cooperation. Therefore, 7947 the resource cooperation helps reduce the influence of fading and improves the application QoS. Fig. 7(d) shows that the total number of running applications increases with a higher Poisson arrival rate. However, when the arrival rate increases, the line without cooperation reaches its upper bound, whereas the line with cooperation still rises with the increasing arrival rate. This is because the limited resources of a single SP cannot support a large number of applications. The SPs with resource cooperation improve their capabilities to support many more applications. In summary, Fig. 7 illustrates that our coalition game-based resource sharing approach can improve the resource utilization and the number of served applications for the SPs, and it increases the QoS of the applications. 2) Impact of Pricing Policy: The pricing policy (i.e., service fee σ), which is announced by the network provider and the cloud provider, has an important impact on the cooperation among the SPs since it will add extra cost to the SPs. For simplicity, we consider that the service fees of the bandwidth and computing resources are with the same values. As aforementioned, a smaller σ means a lower service fee. Therefore, more and more SPs will participate in coalition formation to improve their utility. In Fig. 8(a), the line with rhombuses is for the case of β = 0 indicating the situation when no one is willing to participate in coalition formation. Without cooperation between different SPs, the total number of mobile applications is only decided by the long-term reserved resources of each SP. However, under the coalition game-based scheme, the total number of applications decreases as the service fee increases, but the number is still much higher than that when the SPs do not cooperate. The three curves for different willingness values of β show that a lower service fee during cooperation is conducive to improve resource cooperation. To further analyze the impact of service fees on coalition formation, we compare the average coalition price with respect to service fees. We use β = [0.005, 0.003, 0.001] indicated by blue, black, and red lines in Fig. 8(b), respectively. The lines with circles represent the coalition price for renting extra resources. The lines with squares represent the coalition price for leasing one’s own idle resources. The difference of the coalition price between the two cases will be enlarged as the service fee increases. In addition, the coalition price in Rfrom increases, whereas the coalition price in Rto decreases. Thus, it is intuitive that the extent of coalition formation will decrease when SPs need to pay higher extra prices for short-term resources, and the SPs will earn less. As shown in Fig. 8(c), the average QoS for each SP is also affected by the service fees of the system. The average QoS in the case of no coalition, when β = 0, is shown by the line with rhombuses that is a constant value. In the case of a coalition game, the average QoS will decrease as the service fee increases or as the willingness to cooperate is reduced. In summary, on one hand, the service fee will negatively affect coalition formation. On the other hand, if more SPs are willing to take part in the coalition game, SPs can achieve a high QoS. 3) Impact of Service Arrival Rate: Arrival rate λ is an important parameter for system performance. Fig. 9 shows the impact of the arrival rate on the total number of mobile applications. We consider different values of σ in Fig. 9(a) and 7948 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 Fig. 7. Performance comparison with and without SP cooperation. (a) Total number of applications (λ = 1, μ = 1). (b) QoS of applications (λ = 1, μ = 1). (c) QoS of applications for different average vehicle speeds. (d) Total number of applications for different Poisson arrival rates. different values of β in Fig. 9(b). In Fig. 9(a), the service fee is constant, i.e., β = 0.001, but the willingness values are different, i.e., σ = [0.2, 0.4, 1]. In Fig. 9(b), the willingness value is constant, i.e., σ = 0.3, but the service fee is β = [0.004, 0.001, 0]. If β = 0 or σ = 1, the SPs refuse to participate in a coalition. Thus, both cases represent the system without coalitions. In Fig. 9(a), the total number of applications decreases for all the three curves as λ increases. This is because the higher λ is, the less applications arrive in the limited time. Therefore, the demand of the SPs’ resources will also decrease. If the resources of all SPs are enough to run their own applications, there is no need for extra short-term resources. Then, the coalitions between the SPs will not exist, and the three curves will intersect at one point. By comparing the curves in Fig. 9(a), for λ = 0.5, the difference between the red curve (σ = 0.2) and the blue curve (σ = 1) is about 19.8%. In Fig. 9(b), when λ = 0.5, the difference between the red curve (β = 0.0004) and the blue curve (β = 0) is about 37.3%. Moreover, the effect of the arrival rate is the same on the systems with different willingness values to pay or different service fees. This means that resource cooperation is efficient for resource utilization and increases the number of running applications. 4) Impact of Round-Trip Time: We consider that the available bandwidth for live VM migration between data centers is set as 100 Mb/s. Each VM is allocated 1-GB random access memory and a 10-GB disk [33]. Fig. 10 shows that the roundtrip time without live VM migration is dramatically rising when the communication distance between the sender and the receiver becomes larger, which is part of the conclusion in [34]. Due to the live VM migration technology, the vehicles can keep close to the physical resources. The round-trip time with live VM migration is about 30 ms, which is acceptable for most applications [33], [35]. VII. C ONCLUSION In this paper, we have introduced a model-based coalition game for resource management and sharing among different YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS 7949 Fig. 8. Performance in terms of pricing policy. (a) Total number of applications (λ = 1, μ = 1). (b) Average coalition price (λ = 1, μ = 1). (c) QoS of applications (λ = 1, μ = 1). Fig. 9. Performance in terms of service arrival rate. (a) Number of applications with different pricing σ values (μ = 1, β = 0.001). (b) Number of applications with different willingness β values (μ = 1, σ = 0.03). 7950 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 12, DECEMBER 2015 Fig. 10. Performance in terms of round-trip time. SPs in cloud-enabled vehicular networks. As the mobile applications can remotely run on powerful servers in the cloud, the cloud SPs constitute a virtual resource network. The virtual resource network can provide cloud SPs with reserved bandwidth and computing resources in order to support mobile applications. The coalition game approach promotes resource cooperation, from which both the SPs and the users can benefit. Furthermore, we leverage the two-sided matching theory to speed up the coalition formation process. Simulation results indicate that our scheme enhances the resource utilization of the SPs and improves the QoS of the vehicular users. R EFERENCES [1] L. Da Xu, W. He, and S. Li, “Internet of things in industries: A survey,” IEEE Trans. Ind. Electron., vol. 10, no. 4, pp. 2233–2243, Nov. 2014. [2] J. C. Ferreira, V. Monteiro, and J. L. Afonso, “Vehicle-to-anything application (v2anything app) for electric vehicles,” IEEE Trans. Ind. Informat., vol. 10, no. 3, pp. 1927–1937, Aug. 2014. [3] S. Olariu, I. Khalil, and M. Abuelela, “Taking VANET to the clouds,” Int. J. Pervasive Comput. Commun., vol. 7, no. 1, pp. 7–21, 2011. [4] N. Kumar, S. Zeadally, and J. Rodrigues, “QoS-aware hierarchical web caching scheme for online video streaming applications in internet-based vehicular ad hoc networks,” IEEE Trans. Ind. Electron., to be published. [5] M. Baguena Albaladejo, C. Calafate, J. Cano, and P. Manzoni, “An adaptive anycasting solution for crowd sensing in vehicular environments,” IEEE Trans. Ind. Electron., to be published. [6] M. Tinghuai et al., “Social network and tag sources based augmenting collaborative recommender system,” IEICE Trans. Inf. Syst., vol. 98, no. 4, pp. 902–910, 2015. [7] N. Kumar, J. Lee, N. Chilamkurti, and A. Vinel, “Energy-efficient multimedia data dissemination in vehicular clouds: Stochastic-reward-netsbased coalition game approach,” IEEE Syst. J., to be published. [8] Y. Ren, J. Shen, J. Wang, J. Han, and S. Lee, “Mutual verifiable provable data auditing in public cloud storage,” J. Internet Technol., vol. 16, no. 2, pp. 317–323, 2015. [9] J. A. Dias, J. J. Rodrigues, and L. Zhou, “Performance evaluation of cooperative strategies for vehicular delay-tolerant networks,” Trans. Emerging Telecommun. Technol., vol. 25, no. 8, pp. 815–822, 2014. [10] J. Dias, J. Rodrigues, C. Mavromoustakis, and F. Xia, “A cooperative watchdog system to detect misbehavior nodes in vehicular delay-tolerant networks,” IEEE Trans. Ind. Electron., to be published. [11] J. A. Dias, J. J. Rodrigues, and L. Zhou, “Cooperation advances on vehicular communications: A survey,” Veh. Commun., vol. 1, no. 1, pp. 22–32, Jan. 2014. [12] T. Refaat, B. Kantarci, and H. Mouftah, “Dynamic virtual machine migration in a vehicular cloud,” in Proc. IEEE Symp. ISCC, Jun. 2014, vol. Workshops, pp. 1–6. [13] L. Cimini, “Analysis and simulation of a digital mobile channel using orthogonal frequency division multiplexing,” IEEE Trans. Commun., vol. 33, no. 7, pp. 665–675, Jul. 1985. [14] H. Mousannif, I. Khalil, and H. Al Moatassime, “Cooperation as a service in VANETs,” J. Universal Comput. Sci., vol. 17, no. 8, pp. 1202–1218, 2011. [15] H. Mousannif, I. Khalil, and S. Olariu, “Cooperation as a service in VANET: Implementation and simulation results,” Mobile Inf. Syst., vol. 8, no. 2, pp. 153–172, 2012. [16] G. Wu, S. Talwar, K. Johnsson, N. Himayat, and K. D. Johnson, “M2m: From mobile to embedded Internet,” IEEE Commun. Mag., vol. 49, no. 4, pp. 36–43, Apr. 2011. [17] N. Kumar, S. Zeadally, N. Chilamkurti, and A. Vinel, “Performance analysis of Bayesian coalition game-based energy-aware virtual machine migration in vehicular mobile cloud,” IEEE Netw., vol. 29, no. 2, pp. 62–69, Mar./Apr. 2015. [18] H. Arkian, R. Atani, and A. Pourkhalili, “A cluster-based vehicular cloud architecture with learning-based resource management,” in Proc. IEEE 6th Int. Conf. CloudCom, Dec. 2014, pp. 162–167. [19] H. Arkian, R. Atani, and S. Kamali, “FcVcA: A fuzzy clustering-based vehicular cloud architecture,” in Proc. 7th Int. Workshop Nets4Cars-Fall, Oct. 2014, pp. 24–28. [20] R. Kaewpuang, D. Niyato, P. Wang, and E. Hossain, “A framework for cooperative resource management in mobile cloud computing,” IEEE J. Sel. Areas Commun., vol. 31, no. 12, pp. 2685–2700, 2013. [21] Z. Xia, X. Wang, X. Sun, and Q. Wang, “A secure and dynamic multikeyword ranked search scheme over encrypted cloud data,” IEEE Trans. Parallel Distrib. Syst., to be published. [22] P. Guo, J. Wang, X. H. Geng, C. S. Kim, and J.-U. Kim, “A variable threshold-value authentication architecture for wireless mesh networks,” J. Internet Technol., vol. 15, no. 6, pp. 929–936, Nov. 2014. [23] Z. Fu, X. Sun, Q. Liu, L. Zhou, and J. Shu, “Achieving efficient cloud search services: Multi-keyword ranked search over encrypted cloud data supporting parallel computing,” IEICE Trans. Commun., vol. 98, no. 1, pp. 190–200, 2015. [24] B. Chun et al., “PlanetLab: An overlay testbed for broad-coverage services,” SIGCOMM Comput. Commun. Rev., vol. 33, no. 3, pp. 3–12, Jul. 2003. [25] C.-Y. Chow, M. F. Mokbel, and X. Liu, “A peer-to-peer spatial cloaking algorithm for anonymous location-based service,” in Proc. 14th Annu. ACM Int. Symp. Adv. Geograph. Inf. Syst., 2006, pp. 171–178. [26] F. Qian et al., “Profiling resource usage for mobile applications: a crosslayer approach,” in Proc. 9th Int. Conf. Mobile Syst., Appl., Services, 2011, pp. 321–334. [27] R. Yu, J. Ding, W. Zhong, Y. Liu, and S. Xie, “PHEV charging and discharging cooperation in V2G networks: A coalition game approach,” IEEE Internet Things J., vol. 1, no. 6, pp. 578–589, Dec. 2014. [28] N. Kato, A. Takeuchi, C. Wei, and Z. M. Fadlullah, “GT-CFS: A game theoretic coalition formulation strategy for reducing power loss in micro grids,” IEEE Trans. Parallel Distrib. Syst., vol. 25, no. 9, pp. 2307–2317, Sep. 2014. [29] F. P. Kelly, A. K. Maulloo, and D. K. Tan, “Rate control for communication networks: Shadow prices, proportional fairness and stability,” J. Oper. Res. Soc., pp. 237–252, 1998. [30] [Online]. Available: https://catalog.data.gov/dataset/californiahousehold-transportation-survey [31] G. Ghiani, F. Guerriero, G. Laporte, and R. Musmanno, “Real-time vehicle routing: Solution concepts, algorithms and parallel computing strategies,” Eur. J. Oper. Res., vol. 151, no. 1, pp. 1–11, 2003. [32] D. Niyato, P. Wang, E. Hossain, W. Saad, and Z. Han, “Game theoretic modeling of cooperation among service providers in mobile cloud computing environments,” in Proc. IEEE WCNC, 2012, pp. 3128–3133. [33] T. Wood, K. Ramakrishnan, P. Shenoy, and J. Van der Merwe, “Cloudnet: dynamic pooling of cloud resources by live wan migration of virtual machines,” ACM SIGPLAN Notices, vol. 46, no. 7, pp. 121–132, 2011. [34] M. Torrent-Moreno, F. Schmidt-Eisenlohr, H. Fßler, and H. Hartenstein, Packet Forwarding in VANETs, the Complete Set of Results. Karlsruhe, Germany: Fak. Informatik, Univ. Karlsruhe, 2006. [35] S. Xie and Y. Wang, “Construction of tree network with limited delivery latency in homogeneous wireless sensor networks,” Wireless Pers. Commun., vol. 78, no. 1, pp. 231–246, 2014. YU et al.: COOPERATIVE RESOURCE MANAGEMENT IN CLOUD-ENABLED VEHICULAR NETWORKS Rong Yu (S’05–M’08) received the Ph.D. degree from Tsinghua University, Beijing, China, in 2007. He is currently a Full Professor with the School of Automation, Guangdong University of Technology, Guangzhou, China. He is the coholder of over ten patents and the author or coauthor of over 70 international journal and conference papers. His research interests mainly include wireless communications and networking, including cognitive radios, wireless sensor networks, and home networking. Dr. Yu is currently serving as the Deputy Secretary General of the Internet of Things (IoT) Industry Alliance of Guangdong, Guangzhou, and the Deputy Head of the IoT Engineering Center of Guangdong, Guangzhou. He is a member of the Home Networking Standard Committee in China, where he leads the standardization work of three standards. Xumin Huang is currently working toward the Master’s degree in networked control systems in the School of Automation, Guangdong University of Technology, Guangzhou, China. His research interests include network performance analysis, simulation, and enhancement in wireless communications and networking. Jiawen Kang received the M.S. degree from Guangdong University of Technology, Guangzhou, China, where he is currently working toward the Ph.D. degree in the School of Automation. His research interests include resource management, security, and privacy protection in wireless communications and networking. Jiefei Ding received the M.S. degree from Guangdong University of Technology, Guangzhou, China. In 2015, she spent five months studying at The Hong Kong University of Science and Technology, Kowloon, Hong Kong, as a Postgraduate Visiting Internship Student. She is currently with the School of Automation, Guangdong University of Technology. Her research interests include cloud computing resource management, software-defined networks, and demand response management in smart grids. 7951 Sabita Maharjan (M’09) received the M.E. degree from Tokyo Institute of Technology, Tokyo, Japan, in 2008 and the Ph.D. degree in networks and distributed systems from the University of Oslo, Oslo, Norway, in 2013. She is currently a Postdoctoral Fellow with the Simula Research Laboratory, Fornebu, Norway. Her research interests include wireless networks, network security, smart grid communications, cyberphysical systems, and machineto-machine communications. Stein Gjessing (M’91) received the Dr.Philos. degree from the University of Oslo, Oslo, Norway, in 1985. He is currently an Adjunct Researcher with the Simula Research Laboratory, Fornebu, Norway, and a Professor of computer science with the Department of Informatics, The Faculty of Mathematics and Natural Sciences, University of Oslo, where he acted as the Department Head from 1987 to 1991. From February 1996 to October 2001, he was the Chair of the national research program “Distributed IT-System” founded by the Research Council of Norway. His current research interests include routing, transport protocols, and wireless networks, including cognitive radios and smart grid applications. Yan Zhang (SM’10) received the Ph.D. degree from Nanyang Technological University, Singapore. He is currently the Head of the Department of Networks, Simula Research Laboratory, Fornebu, Norway, and a part-time Associate Professor with the Department of Informatics, The Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway. His current research interests include wireless networks and reliable and secure cyberphysical systems (e.g., healthcare, transport, and smart grids). Dr. Zhang is a Senior Member of the IEEE Communications Society and the IEEE Vehicular Technology Society. He is an Associate Editor or on the Editorial Board of a number of well-established scientific international journals, e.g., Wiley Wireless Communications and Mobile Computing. He also serves as the Guest Editor for the IEEE T RANSAC TIONS ON I NDUSTRIAL I NFORMATICS, IEEE Communications Magazine, Wireless Communications, and the IEEE T RANSACTIONS ON D EPEND ABLE AND S ECURE C OMPUTING . He serves as a Chair for a number of conferences, including the 2016 IEEE International Symposium on Personal, Indoor and Mobile Radio Communications; the 2016 IEEE Consumer Communications and Networking Conference; the 2016 International Wireless Internet Conference; the 2015 IEEE International Conference on Smart Grid Communications; and the 2015 IEEE International Conference on Cloud Computing Technology and Science. He serves as a Technical Program Committee Member for numerous international conferences, including the IEEE International Conference on Computer Communications, the IEEE International Conference on Communications, the IEEE Global Communications Conference, and the IEEE Wireless Communications and Networking Conference. He was a recipient of seven Best Paper Awards.