UnEncrypted

J. M. Cutillas Lozano, D. Gimenez,´ L. G. Cutillas Lozano

scheme (Algorithm 2), but the methodology is common to di erent metaheuristics and parallel implementations. For example, some metaheuristics include an improvement part in the initialization, and the number of threads in the two levels of this improvement are added to the number of threads for the initialization of the reference set, so obtaining

T hreadsIni = {threads−scheme1, threads−level1−scheme2, threads−level2−scheme2}. The metaheuristic scheme is used at two levels: for the hyperheuristic (HMS) and for the application of the metaheuristics determined from the metaheuristic parameters

(MetaheurParam) in each element of the reference set in the metaheuristic (MS) with which the hyperheuristic is implemented. Thus, parallelism can be applied in the hyperheuristic and in the metaheuristics, with a total of four parallelism levels, but it will be preferable to parallelize at a high level, and normally parallelism is applied only in the hyperheuristic.

4Modelling and auto-tuning of the shared-memory parameterized scheme

To reduce the execution time it is necessary to select the values of the parallelism parameters (T hreadsIni, T hreadsCom, T hreadsImp and T hreadsInc) appropriately, which means a model of the execution time must be obtained for each function, and the number of threads of a loop or the number of threads in the first and the second parallelism levels must be established. So, the value of some parameters (20 in our experiments with GRASP, Scatter Search, Genetic algorithms and Tabu Search as basic algorithms) must be selected, and an auto-tuning methodology is systematically applied. A scheme of the auto-tuning process is shown in Figure 1. It is divided in three phases:

Figure 1: Phases of the auto-tuning process

•First phase: Design. The routine is developed together with its theoretical execution time. A model is obtained for each basic routine. Because two types of parallelism

Modelling parameterized shared-memory hyperheuristics for auto-tuning

have been identified in these routines, two basic models can be used, one for one-level routines and another for nested parallelism. For example, the generation of the initial population in function Initialize with an initial number of elements in the reference set IN EIni can be modelled:

where kg represents the cost of generating one individual; kp the cost of generating one thread; and p is the number of threads. And the improvement of a percentage P EIIni of the initial elements with an intensification (extension of improvement for each neighbour) IIEIni is modelled:

For each of the other basic functions in Algorithm 2, the corresponding metaheuristic parameters are determined, and the model of the execution time is obtained as a function of those parameters and the parallelism parameters (the number of threads to select to be used in each routine and subroutine).

•Second phase: Installation. When the shared-memory parameterized scheme is being installed in a particular system, the value of the parameters influenced by the system are estimated. The parameters kg , ki, kp, kp,1 and kp,2 used in step 1 are some of those parameters, as are the corresponding parameters for the other basic routines in Algorithm 2. We summarize the results of the installation of the scheme in an HP Integrity Superdome SX2000 with 128 cores of Intel Itanium-2 dual-core Montvale with shared-memory. The optimum number of threads varies with the number of elements, and we are interested in the selection at running time of a number of threads

close to the optimum. The model in equation 1 is used, and parameters kg and kp in the model are obtained by least-squares for a one-level routine, with a small number of elements (in order to have low installation time). In the experiments with a hyperheuristic with IN EIni = 5, the values obtained are kg = 5.77 · 10−1 and kp = 4.91 · 10−2, all in seconds.

For a two-level routine, like the routine to improve elements after the initial generation or after combining or mutation, the values of the parallelism parameters are obtained by least-squares with experiments with parameters for the hyperheuristic IN EIni = 10, P EIIni = 100 and IIEIni = 1. The results are ki = 1.21, kp,1 = 1.04 · 10−1

J. M. Cutillas Lozano, D. Gimenez,´ L. G. Cutillas Lozano

and kp,2 = 9.89 · 10−2 seconds. By substituting these values in the theoretical model of the execution time (equation 2), the behaviour of the routine in the system is well predicted, as can be seen in Figure 2, where the theoretical and experimental speedups in the improvement of the initial population are represented for the hyperheuristic parameter combination IN EIni = 50, P EIIni = 50 and IIEIni = 1.

•Third phase: Execution. At execution time the number of threads in each basic function is selected from the theoretical execution time (equations 1 and 2) with the values of the hyperheuristic parameters being those of the hyperheuristic we are experimenting with and the values of the system parameters those estimated in the installation phase. The number of threads which gives the theoretical mimimum execution time is obtained by minimizing the corresponding equation after substituting in it the values of the hyperheuristic and system parameters. For example, for the initial generation of the reference set:

and for the improvement of the generated elements:

To validate the auto-tuning methodology the optimum number of threads and the maximum speed-up achieved are calculated from the model for di erent hyperheuristic parameters using the system parameters obtained in the installation. Tables 1 and 2 compare the results for the initial generation of the reference set and for the improvement of elements for two parameter combinations. The number of threads selected with the auto-tuning methodology is not far from the experimental optimum and, as a consequence, the speed-up achieved with auto-tuning is not far from the maximum and the auto-tuning methodology is useful for the reduction of the execution time of hyperheuristics, which have a high cost caused by the application of a large number of metaheuristics.

We can compare the results obtained for the hyperheuristic using the auto-tuning metodology with those achieved when directly applying individual metaheuristics to a problem of optimization of electrical costs. Since the metaheuristic scheme is the same, similar results would be expected in both cases, although there may be di erences due to di erent implementations. For example, in the improvement function of the MS, the second level was used to start more threads to work on the improvement of the fitness function (more

Modelling parameterized shared-memory hyperheuristics for auto-tuning

Speed-up

experimental theoretical

14

12

14

10

Figure 2: Theoretical and experimental speed-ups when varying the number of threads of the first and second level of parallelism for one parameter combination in a two-level parallel routine when applying the HMS.

Table 1: Speed-up and number of threads for IN EIni = 20 and 100 in the one-level parallel routine when applying the HMS. Optimum experimental values (optimum), values obtained with auto-tuning (model) and experimental speed-up values obtained from threads given by model (opt.-mod.).

J. M. Cutillas Lozano, D. Gimenez,´ L. G. Cutillas Lozano

neighbours are analysed) but not to reduce the execution time, from which the number of threads of second level could be taken as constant. So, in this case the model is slightly di erent. In the function of the initial generation of elements there are no di erences in the implementation. The behaviour of the one-level routine when applying the MS was well predicted, as can be seen in Figure 3, where the theoretical and experimental speed-up are represented.

Tables 3 and 4 compare the results for the initial generation of the reference set and for the improvement of elements for two parameter combinations using the MS. As in the case of the HMS, the number of threads and the speed-up selected with the auto-tuning methodology was not far from the experimental optimum. It can be seen that the technique applied for MS is also valid for HMS.

Table 2: Speed-up and number of threads for other two parameter combinations in the two-level parallel routine when applying the HMS. Optimum experimental values (optimum), values obtained with auto-tuning (model) and experimental speed-up values obtained from threads given by model (opt.-mod.).

Table 3: Speed-up and number of threads for IN EIni = 100 and 500 in the one-level parallel routine when applying the MS. Optimum experimental values (optimum), values obtained with auto-tuning (model) and experimental speed-up values obtained from threads given by model (opt.-mod.).

Modelling parameterized shared-memory hyperheuristics for auto-tuning

Figure 3: Theoretical and experimental speed-up when varying the number of threads for three parameters in a one-level parallel routine when applying the MS.

Table 4: Speed-up and number of threads for other parameter combinations in the two-level parallel routine when applying the MS. Optimum experimental values (optimum), values obtained with autotuning (model) and experimental speed-up values obtained from threads given by model (opt.-mod.).

J. M. Cutillas Lozano, D. Gimenez,´ L. G. Cutillas Lozano

5Conclusions and future work

The auto-tuning methodology previously applied to parameterized shared-memory metaheuristic schemes can be applied to hyperheuristics based on metaheuristic schemes. The auto-tuning in this case o ers additional di culties to those in previous works, but the benefit is more apparent, due to the large execution time of the resulting hyperheuristic. The applicability of the methodology has been shown with a problem of minimization of electricity consumption in wells exploitation and in a large shared-memory system. The methodology provides satisfactory values for the number of threads to use in the application of the parallel hyperheuristic, which has been shown for two basic functions in the hyperheuristic scheme, but the auto-tuning works the same way for the other functions.

The auto-tuning methodology has not yet been integrated in the metaheuristic scheme, and this is the most immediate step. Another possibility to improve the application of the hyperheuristic is to determine search ranges for each metaheuristic parameter, so reducing the possible values of the elements in the metaheuristic with which the hyperheuristic is implemented.

Similar parameterized, parallel metaheuristic schemes together with the corresponding auto-tuning methodology should be developed for message-passing or GPU, which may be preferable for the application of hyperheuristics with large reference sets or with a high cost of the fitness function (application of the underlying methodologies).

Acknowledgements

Partially supported by Fundaci´on S´eneca, Consejer´ıa de Educaci´on de la Regi´on de Murcia, 08763/PI/08, and the High-Performance Computing Network on Parallel Heterogeneus Architectures (CAPAP-H). The authors gratefully acknowledge the computer resources and assistance provided by the Supercomputing Centre of the Scientific Park Foundation of Murcia.

References

[1]Alba, E.: Parallel Metaheuristics: A New Class of Algorithms. Wiley Interscience (2005).

[2]Almeida, F., Cuenca, J., Gim´enez, D., Llanes-Castro, A., Mart´ınez-Gallar, J.P.: A framework for the application of metaheuristics to tasks-to-processors assignation problems. The Journal of Supercomputing. Published on-line (2009).

Modelling parameterized shared-memory hyperheuristics for auto-tuning

[3]Almeida, F., Gim´enez, D., L´opez-Esp´ın, J.J.: A parameterised shared-memory scheme for parameterised metaheuristics. The Journal of Supercomputing 58(3): 292–301 (2011).

[4]Clinton Whaley, R., Petitet, A., Dongarra, J.: Automated empirical optimizations of software and the ATLAS project. Parallel Computing 27, 3–35 (2001).

[5]Cuenca, J., Gim´enez, D., Gonz´alez, J.: Architecture of an automatic tuned linear algebra library. Parallel Computing 30, 187–220 (2004).

[6]Cuenca, J., Gim´enez, D., Mart´ınez-Gallar, J.P.: Heuristics for work distribution of a homogeneous parallel dynamic programming scheme on heterogeneous systems. Parallel Computing 31, 717–735 (2005).

[7]Cutillas-Lozano, J.M., Cutillas-Lozano, L.G., Gim´enez, D.: Modeling shared-memory metaheuristic schemes for electricity consumption. In: DCAI, 33–40 (2012).

[8]Cutillas-Lozano, J.M., Cutillas-Lozano, L.G., Gim´enez, D.: Resoluci´on de un problema de optimizaci´on de consumo el´ectrico en explotaci´on de pozos por medio de metaheur´ısticas parametrizadas. In: MAEB, 391–398 (2012).

[9]Cutillas-Lozano, L.G.: Metaheur´ıstica aplicada a la optimizaci´on de los criterios de producci´on de aguas subterr´aneas. Sondea Project. Final-year dissertation, University of Alicante, 2008.

[10]L´opez-Esp´ın, J.J., Gim´enez, D.: Genetic algorithms for simultaneous equation models. In: DCAI, 215–224 (2008).

[11]Raidl, G.R.: A unified view on hybrid metaheuristics. Hybrid Metaheuristics Third International Workshop, LNCS 4030, 1–12 (2006).

Proceedings of the 12th International Conference on Computational and Mathematical Methods

in Science and Engineering, CMMSE 2012 July, 2-5, 2012.

Evaluating the impact of cell renumbering of unstructured meshes on the performance of finite volume GPU solvers

Marc de la Asunci´on1, Jos´e M. Mantas1 and Manuel J. Castro2

1 Depto. Lenguajes y Sistemas Inform´aticos, Universidad de Granada

2 Depto. An´alisis Matem´atico, Universidad de M´alaga

emails: marc@correo.ugr.es, jmmantas@ugr.es, castro@anamat.cie.uma.es

Abstract

In this work, we study the impact of renumbering the cells of unstructured triangular finite volume meshes on the performance of CUDA implementations of several finite volume schemes to simulate two-layer shallow water systems. We have used several numerical schemes with di erent demands of computational power whose CUDA implementations exploit the texture and L1 cache units of the GPU multiprocessors. Two di erent reordering schemes based on reducing the bandwidth of the adjacency matrix for the volume mesh have been used. Several numerical experiments performed on a Fermi-class GPU show that enforcing an ordering which enhances the data locality can have a significant impact on the runtime, and this impact is higher when the numerical scheme is computationally expensive.

1Introduction

Currently, Graphics Processing Units (GPUs) are being used extensively to accellerate considerably numerical simulations in science and engineering. These platforms make it possible to achieve speedups of an order of magnitude over a standard CPU in many applications and are growing in popularity [16]. In particular, GPUs have been used in many applications based on finite volume numerical schemes [1, 2, 3, 7]. Currently, most of the GPU implementations of numerical schemes are based on the CUDA framework [14] which includes an extension of the C/C++ language to facilitate the programming of NVIDIA GPUs for general purpose applications.

Evaluating the impact of cell renumbering of unstructured meshes. . .

Although the performance of finite volume computations for unstructured meshes could be substantially improved by using GPU platforms, the irregularity of the memory access patterns hampers this goal.

Obtaining high performance on a CUDA-enabled GPU implementation of unstructured mesh computations is not easy because mesh data cannot be easily laid out so as to enable coalescing [15]. However, the renumbering of the data cells of a mesh has proved to be an important way to improve the performance in parallel numerical computations which work on unstructured meshes [4] because a suitable data ordering optimizes the cache usage. In GPU, this approach could be possible if the ordering enables enough locality to use textures and to improve the L1 cache usage.

In modern CUDA-enabled GPUs, reads from texture memory are cached in a manner that preserves spatial locality, meaning that data reads from nearby points in space will possibly be cache hits. On the other hand, in Fermi class GPUs, the same on-chip memory can be dedicated mostly as L1 cache for each kernel call to reduce bandwidth demand [15]. A better access to texture and global memory can be achieved by renumbering the elements in an unstructured mesh such that the elements nearby in the mesh remain nearby in texture and global memory, enabling a better exploitation of the texture and L1 cache. Thus, one can obtain substantial performance improvements without changing the code.

In this paper, we study the impact of renumbering the cells of unstructured triangular finite volume meshes on the GPU performance for CUDA implementations of several finite volume two-layer shallow water solvers. These CUDA solvers have been implemented to take advantage of the texture and L1 cache units of a Fermi-class GPU, and exhibit di erent numerical intensity profiles. In order to apply a cell reordering which enhances the data locality, two reordering schemes based on reducing the bandwidth of the adjacency matrix for the mesh are used. Our goal is to evaluate the e ect of these reordering techniques on the runtime of the finite volume CUDA solvers.

The outline of the article is as follows: the next section describes the underlying mathematical model and presents three finite volume numerical schemes to solve it. In Section 3 the CUDA implementation of the schemes is briefly described. Next, two bandwidth reduction techniques which will be used as renumbering strategies are introduced in Section 4. Section 5 shows and analyzes the performance results obtained when the di erent CUDA solvers are applied to two test problems on a NVIDIA GTX 580 GPU using di erent ordering strategies. Finally, conclusions are drawn in Section 6.

2Mathematical model and numerical schemes

The two-layer shallow water system [5] is a system of partial di erential equations which governs the 2d flow of two superposed immiscible layers of shallow fluids in a subdomain Ω R2. This system has been used as the numerical model to simulate ocean and estuarine