Efficiency analysis of parallel implementation of SIMPLE algorithm on multi-processor computers

Authors

  • Sergey Viktorovich Lashkin Public Corporation to Atomic Power Engeneering “Rosatom” FSUE “RFNC-VNIIEF”
  • Andrey Sergeevich Kozelkov Public Corporation to Atomic Power Engeneering “Rosatom” FSUE “RFNC-VNIIEF”
  • Andrey Vladimirovich Yalozo Public Corporation to Atomic Power Engeneering “Rosatom” FSUE “RFNC-VNIIEF”
  • Vitaliy Yurievich Gerasimov Public Corporation to Atomic Power Engeneering “Rosatom” FSUE “RFNC-VNIIEF”
  • Dmitriy Konstantinovich Zelensky Public Corporation to Atomic Power Engeneering “Rosatom” FSUE “RFNC-VNIIEF”

DOI:

https://doi.org/10.7242/1999-6691/2016.9.3.25

Keywords:

computational fluid dynamics (CFD), SIMPLE algorithm, multi-grid solver, modeling

Abstract

This paper describes the details of parallel implementation of a SIMPLE algorithm for numerical solution of the Navier-Stokes system of equations on arbitrary unstructured grids. Implemented iteration schemes of serial and parallel options of the SIMPLE algorithm are shown. When describing parallel implementation, special attention is paid to the description of computational data exchange between the processors under the condition of the grid model decomposition using fictitious cells. We discuss specific features for the distributed matrices storage and implementation of the vector-matrix operations in the parallel mode. We show that the suggested way of matrix storage will allow reducing the number of inter-processor exchange. A series of numerical experiments illustrates multi-grid SLAE solver tuning as it affects the general efficiency of the algorithm (it includes the types of the cycles used - V, W and F, the number of smoothing operator iterations, and the number of cells for coarsening). Two ways (direct and indirect) of the efficiency evaluation for the numerical algorithm parallelization are shown. The paper provides the results of the internal and external flow problem solution with parallelization efficiency evaluation by two algorithms. It is shown that the suggested parallel implementation makes it possible to do efficient computation on the problems on a thousand of processors. Based on the results produced, general recommendations are given on the choice of optimal tuning of the multi-grid solver and optimal number of cells per processor.

Downloads

Download data is not yet available.

References

Ferziger J.H., Peric M. Computational methods for fluid dynamics. - Berlin: Springer Verlag, 2002. - 423 p.
2. Fletcer K. Vycislitel’nye metody v dinamike zidkostej. - M.: Mir, 1991. - T. 1. - 504 s.
3. Patankar S. Numerical heat transfer and fluid flow. - New York: Hemisphere Publishing Corporation, 1980. - 197 p.
4. Moukalled F., Darwish M.A. A unified formulation of the segregated class of algorithms for fluid flow at all speeds // Numer. Heat Tr. B-Fund. - 2000. - Vol. 37, no. 1. - P. 103-139. DOI
5. Shterev K.S., Stefanov S.K. Pressure based finite volume method for calculation of compressible viscous gas flows // J. Comput. Phys. - 2010. - Vol. 229, no. 2. - P. 461-480. DOI
6. Jasak H. Numerical solution algorithms for compressible flows: Lecture Notes. Technical report. - Croatia, Zagreb: Wikki Ltd., 2006. - 206 p.
7. Katz A., Sankaran V. High aspect ratio grid effects on the accuracy of Navier-Stokes solutions on unstructured meshes // Comput. Fluids. - 2012. - Vol. 65. - P. 66-79. DOI
8. Xie B., Li S., Ikebata A., Xiao F. A multi-moment finite volume method for incompressible Navier-Stokes equations on unstructured grids: Volume-average/point-value formulation // J. Comput. Phys. - 2014. - Vol. 277. - P. 138-162. DOI
9. Lindholm E., Nickolls J., Oberman S., Montrym J. NVIDIA Tesla: a unified graphics and computing architecture // IEEE Computer Society, Hot Chips19, March-April 2008. - P. 39-55.
10. Barth M., Byckling M., Ilieva N., Saarinen S., Schliephake M., Weinberg V. Best Practice Guide Intel Xeon Phi v1.1. - 2014. - 51 p.
11. McDonough J.M. Lectures in computational fluid dynamics of incompressible flow: Mathematics, algorithms and implementations. - Departments of Mechanical Engineering and Mathematics, University of Kentucky, 1991, 2003, 2007. - 155 c.
12. Tikir M.M., Carrington L., Strohmaier E., Snavely A. A generic algorithms approach to modeling the performance of memory-bound computations // Supercomputing, 2007. SC’07. Proc. of the ACM/IEEE Conference, November 10-16, 2007. - 12 p. DOI
13. Emans M., Liebmann M. Velocity-pressure coupling on GPU // SFB-Report No. 2013-003. - 2013. - 21 p.
14. Gaburov E., Cavecchi Yu. XeonPhi meets astrophysical fluid dynamics. www.prace-ri.eu (data obrasenia 24.06.2016).
15. Volkov K.N., Derugin U.N., Emel’anov V.N., Karpenko A.G., Kozelkov A.S., Teterina I.V., Alozo A.V. Resenie zadac gazovoj dinamiki i teploobmena na graficeskih processorah // VANT. Seria: Matematiceskoe modelirovanie fiziceskih processov. - 2014. - No 4. - S. 22-34.
16. Kruckov I.A., Kopkin S.V. Programmnyj kompleks modelirovania metodom molekularnoj dinamiki dla gibridnyh vycislitel’nyh sistem // VANT. Seria: Matematiceskoe modelirovanie fiziceskih processov. - 2012. - No 1. - S. 59-65.
17. Belakov V.A., Linev A.V., Gorskov A.V., Krylov I.B. Modelirovanie relaksacii massiva kremnievyh nanokristallov po metodu Monte-Karlo s ispol’zovaniem graficeskih uskoritelej // Vestnik NNGU. - 2012. - No 4-1. - S. 260-267.
18. Kozelkov A.S., Derugin U.N., Laskin S.V., Silaev D.P., Simonov P.G., Tatuskina E.S. Realizacia metoda rasceta vazkoj neszimaemoj zidkosti s ispol’zovaniem mnogosetocnogo metoda na osnove algoritma SIMPLE v pakete programm LOGOS // VANT. Seria: Matematiceskoe modelirovanie fiziceskih processov. - 2013. - No 4. - S. 44-56.
19. Antonov A.S. Parallel’noe programmirovanie s ispol’zovaniem tehnologii OpenMP: Ucebnoe posobie. - M.: Izd-vo MGU, 2009. - 77 s.
20. Kozelkov A.S., Kurulin V.V., Puckova O.L., Laskin S.V. Modelirovanie turbulentnyh tecenij s ispol’zovaniem algebraiceskoj modeli rejnol’dsovyh naprazenij s universal’nymi pristenocnymi funkciami // Vycisl. meh. splos. sred. - 2014. - T. 7, No 1. - C. 40-51. DOI
21. Kozelkov A.S., Kurulin V.V., Tatuskina E.S., Puckova O.L. Modelirovanie turbulentnyh tecenij vazkoj neszimaemoj zidkosti na nestrukturirovannyh setkah s ispol’zovaniem modeli otsoedinennyh vihrej // Matem. modelirovanie. - 2014. - T. 26, No 8. - S. 81-96.
22. Kozelkov A.S., Sagaliev R.M., Dmitriev S.M., Kurkin A.A., Volkov K.N., Derugin U.N., Emel’anov V.N., Pelinovskij E.N., Legcanov M.A. Matematiceskie modeli i algoritmy dla cislennogo modelirovania zadac gidrodinamiki i aerodinamiki: Uceb. posobie. - Niznij Novgorod: NGTU im. R.E. Alekseeva, 2014. - 164 s.
23. Kozelkov A.S, Derugin U.N., Cibereva U.A., Kornev A.V., Denisova O.V., Strelec D.U., Kurkin A.A., Kurulin V.V., Saripova I.L., Rubcova D.P., Legcanov M.A., Tatuskina E.C., Laskin S.V., Alozo A.V., Acevic S.V., Tarasova N.V., Ginniatullin R.R., Sizova M.A., Krutakova O.L. Minimal’nyj bazis zadac dla validacii metodov cislennogo modelirovania turbulentnyh tecenij vazkoj neszimaemoj zidkosti // Trudy NGTU im. R.E. Alekseeva. - 2014. - No 4 (106). - C. 21-69.
24. Menter F.R. Two-equation eddy-viscosity turbulence models for engineering applications // AIAA J. - 1994. - Vol. 32, no. 8. - P. 1598-1605. DOI
25. Chorin A.J. A numerical method for solving incompressible viscous flow problems // J. Comput. Phys. - 1967. - Vol. 2, no. 1. - P. 12-26. DOI
26. http://openfoam.org/ (data obrasenia 29.06.2016).
27. Akobovskij M.V. Obrabotka setocnyh dannyh na raspredelennyh vycislitel’nyh sistemah // VANT. Seria: Matematiceskoe modelirovanie fiziceskih processov. - 2004. - No 2. - C. 40-53.
28. Evstigneev V.A. Primenenie teorii grafov v programmirovanii / Pod red. A.P. Ersova. - M.: Nauka, 1985. - 352 c.
29. Hendrickson B., Leland R. The Chaco user’s guide: Version 2.0. - Technical Report, SAND95-2344, Sandia National Laboratories, Albuquerque, NM, July 1995. - 44 p.
30. Leland R., Hendrickson B. An empirical study of static load balancing algorithms // Proc. Scalable High-Perfomance Computing Conference, IEEE, 23-25 May 1994. - P. 682-685. DOI
31. Bahvalov N.S., Zidkov N.P., Kobel’kov G.G. Cislennye metody. - M.: BINOM. Laboratoria znanij, 2003. - 630 s.
32. Volkov K.N., Derugin U.N., Emel’anov V.N., Kozelkov A.S., Teterina I.V. Algebraiceskij mnogosetocnyj metod v zadacah vycislitel’noj fiziki // Vycislitel’nye metody i programmirovanie. - 2014. - T. 15, No 2. - S. 183-200.
33. Volkov K.N., Derugin U.N., Emel’anov V.N., Karpenko A.G., Kozelkov A.S., Teterina I.V. Metody uskorenia gazodinamiceskih rascetov na nestrukturirovannyh setkah. - M.: Fizmatlit, 2014. - 536 s.
34. Ghia U., Ghia K.N., Shin C.T. High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method // J. Comput. Phys. - 1982. - Vol. 48, no. 3. - P. 387-411. DOI
35. Lojcanskij L.G. Mehanika zidkosti i gaza. - M.: Drofa, 2003. - 840 s.
36. Vogel J.C., Eaton J.K. Combined heat transfer and fluid dynamic measurements downstream of a backward-facing step // J. Heat Transfer. - 1985. - Vol. 107, no. 4. - P. 922-929. DOI
37. Uotkins D.S. Osnovy matricnyh vycislenij. - M.: BINOM, 2009. - 664 c.
38. Woodward C.S., Serban R. SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers. - Livermore: UCRL-PRES-213978.
39. Novaev D.A., Bartenev U.G., Lipov D.I., Kolpakov S.I., Kiselev A.B., Serova T.N., Hudakova L.V. Programmnye sredstva STK dla issledovania effektivnosti vypolnenia parallel’nyh prilozenij // VANT. Seria: Matematiceskoe modelirovanie fiziceskih processov. - 2011. - No 4. - S. 72-81.
40. Snir M., Otto S., Huss-Lederman S., Walker D., Dongarra J. MPI: The complete reference. - MIT Press, 1996.
41. Ahmed S.R., Ramm G., Faltin G. Some salient features of the time-averaged ground vehicle wake. - SAE Technical Paper 840300, 1984. - 34 p. DOI
42. Kozelkov A.S., Krutakova O.L., Kurkin A.A., Kurulin V.V., Tatuskina E.S. Zonnyj RANS-LES podhod na osnove algebraiceskoj modeli rejnol’dsovyh naprazenij // MZG. - 2015. - No 5. - C. 24-33. DOI
43. Kozelkov A.S., Kurulin V.V. Cislennaa shema dla modelirovania turbulentnyh tecenij neszimaemoj zidkosti s ispol’zovaniem vihrerazresausih podhodov // ZVMMF. - 2015. - T. 55, No 7. - S. 1255-1266. DOI
44. Kozelkov A., Kurulin V., Emelyanov V., Tyatyushkina E., Volkov K. Comparison of convective flux discretization schemes in detached-eddy simulation of turbulent flows on unstructured meshes // Journal of Scientific Computing. - 2016. - Vol. 67, no. 1. - P. 176-191. DOI
45. Kozelkov A.S., Kurkin A.A., Kurulin V.V., Pelinovskij E.N., Tatuskina E.S. Modelirovanie vozmusenij v ozere Cebarkul’ pri padenii meteorita v 2013 godu // MZG. - 2015. - No 6. - S. 134-143.
46. Kozelkov A.S., Kurkin A.A., Pelinovskij E.N. Vlianie ugla vhoda tela v vodu na vysoty generiruemyh voln // MZG. - 2016. - No 2. - S. 166-176.

Downloads

Published

2016-09-30

Issue

Vol. 9 No. 3 (2016)

Section

Articles

License

Copyright (c) 2016 Computational Continuum Mechanics

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

How to Cite

Lashkin, S. V., Kozelkov, A. S., Yalozo, A. V., Gerasimov, V. Y., & Zelensky, D. K. (2016). Efficiency analysis of parallel implementation of SIMPLE algorithm on multi-processor computers. Computational Continuum Mechanics, 9(3), 298-315. https://doi.org/10.7242/1999-6691/2016.9.3.25