(see next paragraph). The c1.xlarge (c1.xl) resource class provides the best per-core sustained performance from the Amazon EC2 instances; however, it aggregates eight cores and therefore has an increased cost per hour.

We also measured the resource provisioning time in the three Clouds, that is, the time elapsed from when resources are requested until they are accessible (see Fig. 11.3). The dps.cloud has an average provisioning time of about 5 min because of the slower hard-drive available. Amazon EC2 is the fastest and needs about 74 s, while GG is surprisingly slow and needs 20 min on an average. The dps.cloud provisioning time could be improved through faster storage hardware, while future versions of Eucalyptus also promise an improvement in image management and caching.

A characteristic of all Clouds that we surveyed is that they charge the resource consumption based on hourly billing increments, and not based on 1s billing increments, as assumed by the simulations performed in two recent related works [4,5]. Table 11.4 shows that for our relatively short workflows below 2 h, there can be a significant difference between the hourly and the 1s billing increment policies. This ratio is decreasing with the growing problem size from 4.4 for the smallest workflow to 1.64 for the largest workflow.

Finally, we define a new metric called $ per unit of saved time ($/T) as the ratio between the time gained by using Cloud resources and total cost of these resources. The results show that the medium and big workflows are the most convenient to be scaled on additional Cloud resources and cost between $0.08 and $0.18 per saved minute, while the small workflows exhibit high costs of up to $1.72 per minute because of the hourly billing increments.

11.5  Related Work

Deelman et al. [4] analyzed the cost of Cloud storage for an image mosaic workflow and a possible on-demand calculation of the results. The work is based on an Amazon EC2 and S3 simulation rather than the real execution. The computation cost model is based on 1s billing increment and the storage cost model on a byte- per-second billing increment, in contrast to the real Cloud providers that are charged based on hourly, gigabyte-per-month billing increments.

Assuncao et al. [5] described an approach of extending a local cluster with Cloud resources using two schedulers, one for the cluster and one for the Cloud, applying different strategies. The possible benefit of not violating deadlines and achieving higher cluster throughput is analyzed. The system concentrates on clusters and does not extend its scope to Grids or multiple Cloud providers. Their results are generated using simulation and do not take the real speed of Cloud resources into account.

Gropp et al. [28] check the usability of Cloud computing for scientific applications using several benchmarks, and shows that Cloud computing can be useful to scientific computing in general.

Yigitbasi et al. [29], present a framework to analyze the performance of Clouds and the results encourage the usability of Clouds for loosely-coupled jobs such as in workflows.

11.6  Conclusions and Future Work

In this chapter, we extended a Grid workflow development and computing environment to use on-demand Cloud resources in Grid environments offering a limited amount of high-performance resources. We presented extensions to the resourcemanagement architecture to consider Cloud resources comprising three new components: Cloud management for automatic image management, image catalog for management of software deployments, and security for authenticating with multiple Cloud providers. We presented experimental results of using a real-world application in the Austrian Grid environment, extended with an academic Cloud. Our results demonstrate that workflows with large problem sizes can significantly benefit from being executed in a combined Grid and Cloud environment. Similarly, the cost of using Cloud resources is more convenient for large workflows due to the hourly billing increment policies applied.

Our environment currently supports providers offering Amazon EC2-compliant interfaces, which we plan to extend for other Cloud providers. We also plan to investigate more sophisticated multi-criteria scheduling strategies, such as the effect of the resource class granularity (i.e. number of underlying cores) on the execution time, resource allocation efficiency, and the overall cost. In addition, we also intend to use the Cloud simulation framework presented in [30] for validating various scheduling and optimization strategies at a larger scale.

References

1.Iosup A, Dumitrescu C, Epema D, Li H, Wolters L (2006) How are real grids used? The analysis of four grid traces and its implications. In international conference on grid computing, IEEE Computer Society, pp 262–269

2.Costa GD, Dikaiakos MD, Orlando S (2007) Analyzing the workload of the south-east federation of the egee grid infrastructure, CoreGRID Technical Report, Tech. Rep. TR-0063, 2007

3.Yu J, Buyya R (2005) A taxonomy of scientific workflow systems for grid computing. ACM SIGMOD Rec 34(3):44–49

4.Deelman E, Singh G, Livny M, Berriman JB, Good J (2008) The cost of doing science on the cloud: the montage example. In proceedings of the ACM/IEEE conference on high performance computing, SC 2008. IEEE/ACM, Austin, Texas, USA, p 50

5.Assuncao ACM, Buyya R (2009) Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. In: Kranzlmüller D, Bode A, Hegering H.-G, Casanova H, Gerndt M (eds) 11th IEEE international conference on high performance computing and communications (HPCC 2009), ACM

6.Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youseff L, Zagorodnov D (2008) Eucalyptus: a technical report on an elastic utility computing architecture linking your programs to useful systems. UCSB Computer Science Technical Report, Tech. Rep. 2008–10, 2008

7.Chisnal D (2007) The definitive guide to the Xen hypervisor. Prentice-Hall, New Jersey

8.Fahringer T, Prodan R, Duan R, Nerieri F, Podlipnig S, Qin J, Siddiqui M, Truong HL, Villazón A, Wieczorek M (2005, November 13–14) Askalon: a grid application development and computing environment. In proceedings of the 6th IEEE/ACM international conference on grid computing (GRID 2005). IEEE, Seattle, Washington DC, pp 122–131

9. Fahringer T, Qin J, Hainzer S (2005) Specification of grid workflow applications with agwl: an abstract grid workflow language. In CCGRID. IEEE Computer Society, pp 676–685

10. Czajkowski K, Fitzgerald S, Foster I, Kesselman C (2001) Grid information services for distributed resource sharing. In 10th international symposium on high performance distributed computing. IEEE Computer Society Press

11.Ostermann S, Plankensteiner K, Prodan R, Fahringer T, Iosup A (2008) “Workflow monitoring and analysis tool for ASKALON”, in Grid and Services Evolution. Barcelona, Spain, pp 73–86

12. Amazon (2009) Elastic compute cloud (EC2). http://aws.amazon.com/ec2/. Accessed January 2009

13. GoGrid (2009) Cloud hosting: Instant windows and linux cloud servers. http://www.gogrid. com/. Accessed January 2009

14. “Elastichosts: Cloud hosting and cloud computing that’s flexible and easy to use. http://www. elastichosts.com/

15. Mosso (2009) The cloud, cloud computing, cloud hosting, cloud services @ mosso. http:// www.mosso.com/. Accessed January 2009

16. Amazon Inc. (2009) Amazon ec2 api. http://developer.amazonwebservices.com/connect/ kbcategory.jspa?categoryID=87. Accessed April 2009

17. Keahey K, Freeman T, Lauret J, Olson D (2007) Virtual workspaces for scientific applications. In Scientific discovery through advanced computing. Boston, MA

18. Buyya R, Yeo CS, Venugopal S (2008, 25–27 Sept) Market-oriented cloud computing: Vision, hype, and reality for delivering it services as computing utilities. In 10th IEEE international conference on high performance computing and communications (HPCC 2008). IEEE, Dalian, China, pp 5–13

19. Agathon Group (2009) https://www.agathongroup.com/. Accessed January 2009

20. FlexiScale (2009) Utility computing on demand. http://flexiscale.com/. Accessed January 2009 21. 3tera (2009) Applogic - grid operating system for web applications. http://www.3tera.com/

AppLogic/. Accessed January 2009

22. Novotny J, Tuecke S, Welch V (2001) An online credential repository for the Grid: MyProxy. In proceedings of the tenth international symposium on high performance distributed computing (HPDC-10), IEEE Computer Society Press

23. von Laszewski G, Foster I, Gawor J (2000) CoG kits: a bridge between commodity distributed computing and high-performance grids. In Java Grande Conference. ACM Press, pp 97–106 24. Blaha P, Schwarz K, Luitz J (2001) WIEN2k, a full potential linearized augmented plane wave

package for calculating crystal properties. TU Wien ISBN 3-9501031-1-2, 2001

25. Volkert J (2006, July 6–9) Austrian grid: Overview on the project with focus on parallel applications. In 5th international symposium on parallel and distributed computing (ISPDC 2006). IEEE Computer Society, Timisoara, Romania, p 14

26. Dongarra JJ, Luszczek P, Petitet A (2003) The LINPACK benchmark: past, present and future. Concurr Comput Prac Exp 15(9):803–820

27. Gropp W, Lusk E, Ashton D, Buntinas D, Butler R, Chan A, Ross R, Thakur R, Toonen B (2005) Mpich2 user’s guide, version 1.0.3, Mathematics and computer science division, Argonne national laboratory. Technical Report. http://www-unix.mcs.anl.gov/mpi/mpich/

28. Rehr JJ, Gardner JP, Prange M, Svec L, Vila F (2009) Scientific computing in the cloud. Computing Research Repository vol. abs/0901.0029, 2009

29. Yigitbasi N, Iosup A, Ostermann S, Epema D (2009) C-meter: A framework for performance analysis of computing clouds. In International Workshop on Cloud Computing (Cloud 2009)

30. Calheiros RN, Ranjan R, Rose CAFD, Buyya R (2009) Cloudsim: a novel framework for modeling and simulation of cloud computing infrastructures and services. Computing Research Repository, vol. abs/0903.2525, 2009

Chapter 12

Peer-to-Peer Cloud Provisioning: Service

Discovery and Load-Balancing

Rajiv Ranjan, Liang Zhao, Xiaomin Wu, Anna Liu, Andres Quiroz, and Manish Parashar

Abstract  Clouds have evolved as the next-generation platform that facilitates creation of wide-area on-demand renting of computing or storage services for hosting application services that experience highly variable workloads and requires high availability and performance. Interconnecting Cloud computing system components (servers, virtual machines (VMs), application services) through peer- to-peer routing and information dissemination structure are essential to avoid the problems of provisioning efficiency bottleneck and single point of failure that are predominantly associated with traditional centralized or hierarchical approaches. These limitations can be overcome by connecting Cloud system components using a structured peer-to-peer network model (such as distributed hash tables (DHTs)). DHTs offer deterministic information/query routing and discovery with close to logarithmic bounds as regards network message complexity. By maintaining a small routing state of O (log n) per VM, a DHT structure can guarantee deterministic look-ups in a completely decentralized and distributed manner.

This chapter presents: (i) a layered peer-to-peer Cloud provisioning architecture; (ii) a summary of the current state-of-the-art in Cloud provisioning with particular emphasis on service discovery and load-balancing; (iii) a classification of the existing peer-to-peer network management model with focus on extending the DHTs for indexing and managing complex provisioning information; and (iv) the design and implementation of novel, extensible software fabric (Cloud peer) that combines public/private clouds, overlay networking, and structured peer-to-peer indexing techniques for supporting scalable and self-managing service discovery and loadbalancing in Cloud computing environments. Finally, an experimental evaluation is presented that demonstrates the feasibility of building next-generation Cloud provisioning systems based on peer-to-peer network management and information

R. Ranjan (*)

SA Project, CRC Smart Services, Service Oriented Computing Research Group,

School of Computer Science and Engineering, University of New South Wales, Australia e-mail: rajivr@cse.unsw.edu.au

Systems and Applications, Computer Communications and Networks,

DOI 10.1007/978-1-84996-241-4_12, © Springer-Verlag London Limited 2010