9 A Reference Architecture Based on Workflow for Building Scientific Private Clouds |
157 |
focus on: (1) allocating the same task to three or more volunteer computing nodes;
(2) preemptive scheduling based on multi-queue scheduling mechanisms.
YML backend: YML backend encapsulates different underlying middleware and provides a consistent executable environment to tasks from the layer “core of YML.” Concurrently, it permits to utilize one or several middleware through a specific interface for each middleware. The back-end layer consists of three parts named Monitor, worker coordinator, and data manager. In general, YML backend sends requests for executing a task on a computing node and if the task finishes, it also notifies the scheduler that the task is terminated successfully. Data manager is a component for managing all data exchanges between nodes and data server. This component provides two services: distributing appropriate data to workers and retrieving the final results. Worker coordinator maintains a list of active requests and a list of finished requests. The status can change dynamically according to the status of computing nodes. It will allocate those tasks from YML scheduler to appropriate computing nodes in computing resource pools. Monitor component is used to monitor the status of computing nodes. The monitoring mechanism is based on users’ daily behavior, which is adopted to predict the available time of computing resources and make prediction for data migration.
9.4 Primary Experiments on YML-PC
In this section, three kinds of primary experiments (emulations) are made to show that: (1) the computing resource pool can be scaled very easily; (2) great improvements on platform efficiency can be made through emulating the data persistence;
(3) great improvements on platform efficiency can be made through emulating appropriate task distribution between different virtual organizations.
Here, inter-iterative parallel-based block-based Gauss-Jordan algorithm [27, 28] is used. According to the algorithm, q2 is the number of block-counts of matrix. The number of total tasks the algorithm will generate is q3. All these experiments are based on YML+OmniRPC, YML+XtremWeb, YML+OmniRPC/XtremWeb, and Grid 5,000 platform [29]. In our experiments, the computational resources can be described as follows (Table 9.1):
Table 9.1 |
Parts of computing resources in Grid’5000 platform |
|
||
|
|
|
|
|
Site |
Cluster |
Nodes |
CPU/memory |
|
|
|
|
|
|
Nancy |
Grelon |
120 |
2 |
× Inter xeon, 1.6 GHz/2 GB |
Rennes |
Paravent |
99 |
2 |
× AMD opteron, 2 GHz/2 GB |
Lyon |
Sagittaire |
70 |
2 |
× AMD opteron, 2.4 GHz/2 GB |
Bordeaux |
Bordereau |
93 |
2 |
× AMD opteron, 2.6 GHz/2 GB |
|
|
|
|
|
158 |
L. Shang et al. |
9.4.1 YML-PC Can Be Scaled Up Very Easily
In this experiment, we set the block-size of submatrix to 1,500 * 1,500, and the middleware is YML+Xtremweb. The reason is that XtremWeb can be easily scaled up for its “pull model” based task allocation mechanism. “R-B” represents that the computing resource pool has ten computing nodes, while “R-A” implies that the computing resources have scaled up to 20 computing nodes. Scale up occurs during the process of program execution.
Figure 9.8 shows that when the block-count is less than 32, there is little influence on the elapsed time whether computing resource pool scales up or not. But when the block-count is more than 32, scalability of computing resource pool has an important influence on the elapsed time. The reason stems from the algorithm itself. When the block-count is small, tasks generated are few; ten computing resources can be enough for generated tasks. So, the influence on the elapsed time is small. With the increase in block-count of matrix, the generated tasks increase greatly. More computing resources are needed. So, the influence on elapsed time becomes more obvious. In a word, from Fig. 9.8, we can conclude that, whether the block count is small or large, scalability of a computing pool can improve the efficiency of the platform. At the same time, this experiment testifies that YML-PC has the ability to scale.
Fig. 9.8 Feature of scalability of YML-PC
9 A Reference Architecture Based on Workflow for Building Scientific Private Clouds |
159 |
Fig. 9.9 Data persistence in YML-PC
9.4.2 Data Persistence in YML-PC
The efficiency of YML-PC can be improved with the help of data persistence technology. In this experiment, we set block-size of submatrix as 1,500 * 1,500. The middleware is YML+OmniRPC. YML in Fig. 9.9 represents that the platform does not support data persistence, while YML+DP stands for the YML-PC supporting data persistence.
Figure 9.9 shows that data persistence is very important for scientific computing especially for scientific computing with substantial data. It can save a lot of time and thus improve the efficiency of the platform. With increase in block-counts of matrix, more tasks are generated and therefore a lot of data transfers between data server and workers are generated. If we take data persistence technology in cloud computing platform, less communication overhead is generated and the efficiency of cloud platform can be improved.
9.4.3 Schedule Mechanism in YML-PC
Appropriate selection of computing resources based on trust model in YML-PC is very important. In this experiment, we set block-size as 1,500 * 1,500, and the middleware is YML+Xtremweb. ‘No fault’ in Fig. 9.10 represents the no