HPC Systems for Computational Fluid Dynamics

Airplane

Note: This information was gathered by CFD software and service provider Resolved Analytics and shared on our website for the consumption of our audience, many of which work primarily in the CFD space.

CFD software is a processing tool used by engineers to analyze and solve problems involving fluid flows. CFD is used to simulate aerodynamics, hydrodynamics, weather analysis, heat transfer, engine combustion, etc. The more complex the problem, the more computational power needed to solve these problems. Open source software is available (OpenFOAM), however, licensed software (ANSYS Fluent, Siemens Star CCM+) offers more user-friendly interface, reliable simulation results, and support.

Computational Fluid Dynamics (CFD) is the analysis of fluid flows using numerical solution methods. Using CFD, engineers analyze complex problems involving fluid-fluid, fluid-solid or fluid-gas interaction. Engineering fields where CFD analyses are frequently used are aerodynamics and hydrodynamics, where quantities such as lift and drag or field properties as pressures and velocities are obtained. Fluid dynamics is involved with physical laws in the form of partial differential equations. Sophisticated CFD solvers transform these laws into algebraical equations and efficiently solve these equations numerically.

Stages of CFD

Pre-processing

The first step in CFD modeling, which starts with describing the geometry of the object being worked with, usually imported from a CAD drawing. The process of creating the environment in which the object will be simulated is broken into smaller, more manageable segments, known as meshing. Meshing can be handled in different ways, but ultimately, creates the fluid conditions in which the object in question will be simulated in.

Solving

The second step is the solving of the problem, once the physics problem has been identified, the environment has been meshed, and boundary conditions set, the solver processes this information and proceeds to run all the calculations necessary to solve the problem. There are multiple solvers available, varying in efficiency and capability of solving certain physical phenomena. (ANSYS FLUENT, Star CCM+, CFD++, OpenFOAM)

Post-processing

Finally, the results are visualized and analyzed in the post processing phase. At this stage the results and conclusions can be drawn based on the results of the solver. Results are presented in many different forms: static or moving pictures, graphs or tables.

Ansys Fluent:

Pros: powerful, efficient and validated numerical methods, full suite of physics and multiphysics capabilities

Cons: requirement for standalone software for pre-processing (SpaceClaim, only supported on Windows OS) and superior post-processing (Ensight), cost

  • Workflow: new “Watertight Workflow”, one only needs to use SpaceClaim (in place of Design Modeler) and then bring the geometry directly into Fluent for native meshing.
  • Physics Modeling:
  • If there is some sort of physics phenomena that isn’t built in and available, Fluent supports the use of user codes called UDF’s (User Defined Functions), which are fully customizable scripts that allow you to tap or “hook” into the flow variables in order to model the physics/behavior at each computational cell.
  • new hybrid Volume-of-Fluid (VOF) to Discrete Phase Model (DPM) that is used for spray nozzle simulations.
  • CAD Cleanup and Meshing: “Watertight Workflow”, This tree structure guides you from top to bottom as you import geometry, add meshing parameters, label boundaries and zones/regions, and create surface and volume meshes.
  • Meshing:
  • process is now straightforward, and the workflow guides the user though the remaining effort. the “preview” of the mesh size that allows the user to see how small/large the computation cells will be before anything is meshed. This can save a lot of time that could otherwise be wasted by meshing a geometry with too coarse or too fine of a grid.
  • new Mosaic meshing technology, also called “poly-hexcore”, this mesh type is showing an improvement in total cell counts for similar (or boosted) accuracy when compared to polyhedral alone (which increases speed). Compared to the polyhedral mesh, the poly-hexcore mesh had ~10% fewer cells in total.
  • improvement in meshing time from 3 minutes (on 4 cores) for the poly-hexcore compared to ~13.5 minutes for the corresponding all-polyhedra mesh. This speedup (~4.5x) could save major time for generating larger, more complex meshes.
  • Simulation: Test cases available, allow the user to verify that the software performs according to documentation and to provide the user with confidence that it can adequately solve various physics problems within tolerable levels of accuracy.
  • Post Processing: Ensight able to show both solid FEA model results as well as fluid CFD cases and can be quite impressive for analyzing and animating fluid-body-interaction data.

Siemens Star CCM+

Pros: powerful, efficient and validated numerical methods, full suite of physics and multiphysics capabilities, streamlined workflow and ease-of-use, post-processing

Cons: still looking 

  • Interface: Users can access all pre-processing, simulation, and post-processing tasks within single interface.
  • Physics Modeling: comes with a database of common materials in categories of solid, liquid, gas, and electrochemical species and a wide range of turbulence modeling options. A finite element solver has been added recently that allows basic solid mechanics modeling.
  • Clean Up & Meshing: Makes the process of importing, repairing, defining and meshing your CAD parts about as painless as it can be. Overall mesh generation is handled well.
  • Post Processing:
  • STAR-CCM+ provides the most striking and intuitive flow visualization techniques among all leading CFD software packages and which are comparatively easy to use.
  • Offers VR, opportunity to move around inside a simulation result which has the potential to provide more useful insight
  • Offers screenplay, the animation recording is no longer constrained by a single visualization and, instead, visualizations can dynamically change throughout the recording.
  • Simulation: STAR-Test, internal test system, the user has the opportunity to verify that the software received is able to reproduce the same results on the platform you are using (verification) while also providing an understanding of the accuracy to be expected when modeling specific physics use cases (validation). Only multiphysics simulation tool that has achieved ASME Nuclear Quality Assurance – 1 compliance.
  • Summary: makes lives of engineers easier by providing a full-suite of multi-physics capabilities, a streamlined workflow within a modern java based interface, best-in-class meshing capabilities and insightful, meaningful and impressive post-processing without the prerequisite of obtaining a Ph. D. in programming.

OpenFOAM (Open Source)

Pros:

  • Freely licensed, cost effective as user demand increases. Widely distributed, so its reliable and accurate as a result of being scrutinized and improved by large group of diverse developers, motivated to ensure code performs well.
  • If user has the skill and desire, they can add to the functionality through additional coding.

Cons:

– steep learning curve

– limited user support

– increased cost of ownership relating to reduced usability

– lack of specialized capabilities

– requirement of additional software including pre and post processors.

5 General CFD Categories:

Open Source: OpenFOAM is the most widely used open source CFD software.

Pros:

– Freely licensed, cost effective as user demand increases. Widely distributed, so its reliable and accurate as a result of being scrutinized and improved by large group of diverse developers, motivated to ensure code performs well.

– If user has the skill and desire, they can add to the functionality through additional coding.

Cons:

– steep learning curve

– limited user support

– increased cost of ownership relating to reduced usability

– lack of specialized capabilities

– requirement of additional software including pre and post processors.

Open Source Wrappers: Open source software such as OpenFOAM, with friendly user GUI (Graphical User Interface) environments bundled in such as: Visual-CFD, HELYX and simFlow. SimScale (cloud, web browser-based simulation) offers convenience of single interface, but suffer from separation between user and the execution code.

  • Pros: some convenience of full service commercial platform at much lower price
  • Cons: same limitations of open source; i.e. limited user support, lack of specialized capabilities, while adding another layer of software with its own potential for bugs.

CAD (Computer Aided Drawing) Integrated: Most widely used are SolidWorks and AutoDesk Inventor, are CFD add-ons within 3-D solid modeling platforms. Marketed towards product designers seeking to solve steady-state, single phase, non-reacting flow problems, with focus on ease of use.

Specialty: specialized functionality.

  • Converge is a multipurpose code, high sophistication in regards to moving meshes, multiphase flows and turbulent combustion as needed for focus in: automotive, internal combustion. (AVL Fire, also for automotive)
  • 6sigma: for datacenter ventilation.
  • EXA for aerodynamics
  • CFX for turbo machinery
  • New: EXN/Aero focus on improving performance on very large scale simulations through combined use of CPU and GPU processing, until purchased by JUUL e-cig maker.

Comprehensive Packages: Gold standard for CFD is ANSYS Fluent and Siemen Star-CCM+. Fluent seems to capture more market share in electronic and industrial product markets while Star-CCM+ in the aerospace, automotive and energy industries. Software such as COMSOL’s CFD Module and Altair’s AcuSolve are components of broader Multiphysics simulation platforms, now growing in scope and marketshare. Primary drawback of these packages is the cost.

  • Capability to import complex 3D solid and surface geometries from diverse sources.
  • All in one workflow, including pre-processing, solving and post-processing.
  • Broad multi-physics simulation capabilities.
  • Efficient data architectures, numerical methods and utilization of diverse hardware and software configurations.
  • Vendor initiated verification and validation of physics and numerical methods.
  • Limited requirements for user-coding and/or command line operations.

CAD embedded CFD:

CAD embedded into CFD software is touted as being a game changer, however, mostly overhyped. CAD software can easily be imported into CFD programs so there is no need to have them in the same interface.

CFD Applications

SolidWorks Flow Simulation: add on to SolidWorks CAD software, considered the “most CAD embedded” CFD program in its class.

Fluid volumes extracted for CFD analysis and boundary conditions needed for CFD simulation are linked directly to the native 3D CAD geometry surfaces eliminating the necessity of redefining the model setups when experimenting with simple CAD geometry changes.

Marketed as solution to “designers” working in industrial and electronics, those without experience in fluid dynamics, numerical analysis and CFD.

3D CAD cleanup and prep work is similar to that which you would find if exporting to a neutral file format for export to standalone CFD software.

Complicated, production-ready CAD models will need to simipified by suppressing details unnecessary to fluid-flow analysis, and combining common solids to eliminate unnecessary surface to surface interfaces.

The “check model” tool and “CAD cleanup” tool are useful as the can catch and repair some simple issues, but often require significant user intervention to troubleshoot problematic geometries and then repair in the native CAD environment.

Meshing in this program attempts to simplify the process by limiting user input, but ultimately results in average to poor quality, and is unable to benefit from multicore architectures.

Simulating in SolidWorks Flow uses a Finite Volume solver which has been found to be inefficient, compared to Finite Element methods. In a comparison, SolidWorks Flow required 4 – 100 times longer to perform simulation compared to ANSYS fluent while returning less accurate results (for lift and drag).

Autodesk CFD: Though not technically CAD embedded as it requires another application to launch from within the Inventor or Fusion 360 workspace.

Marketed toward industries serving electronics and architecture, given the other architecture related software that Autodesk has.

Convenient in that when CFD is launched form CAD program, it can push 3D models directly into CFD and automatically assigns settings from CAD so you can introduce new CAD design variations with ease. Can sometimes run into difficulties.

When importing CAD models directly into CFD, a toolkit can be activated to analyze the 3D model for model health, identifying: slivers, gaps and interfaces, but only identifies. Use needs to go back to original CAD program to make adjustments to CAD models.

Meshing is messy but possible.

Autodesk CFD is somewhat of outlier as it uses finite element solver, which results in less accurate and less efficient than finite volume solver. Autodesk CFD finite element solver is significantly slower than finite volume solvers.

ANSYS Discovery Live:  mesh less solver, similar to the Lattice Boltzman solvers, where GPU acceleration of coarse-lattice simulations can be run very quickly. This is how Hollywood studios have produced amazing life like fluid effects on large scales for years.

ANSYS is usually known for highly accurate, time consuming simulations. This was first release that made CFD most accessible where designers can seem immediate, real time impacts of design changes.

If ANSYS continues to improve upon this platform, it can result in solving fine lattice results, meaning more accurate real time results in unsteady flow environments, as opposed to highly calculated pre-preped environment that are planned in advance.

OpenFOAM: Common open source CFD solvers (besides OpenFOAM): SU2, Palabos, Fire Dynamics Simulator, MFIX.

Gained considerable credibility by growing user base: universities and corporations, Mercedes Benz, BASF, BMW, Volkswagen, Intel.

Coded in C++ instead of Fortran, to take advantage of object oriented abilities.

Linux based, or if running through Microsoft Windows, need to run through virtual machine. Or you can run Linux Bash Shell within windows. Or can run through a “containerized” environment, via Docker technology.

Strengths:

  • Many capabilities and multiple solvers that can be applied to numerous types of flow problems
  • Product has been developed and refined over 20 years, by experts specializing in solving CFD problems.
  • Advantages of wide user base, tutorials, and example problems, as well as ability to customize the code base to user liking.
  • Increased acceptance in academia and industry.
  • FREE!!

Drawbacks:

  • Steep learning curve, need for advanced user experience in determining what important physics to solve and how to best match those physics with numerical algorithms.
  • Need for Linux based OS, or version for Windows that lacks native capabilities/utilities. Need knowledge of Linux commands for file manipulation.
  • Lack of high performance built in meshing utility.
  • Need to learn additional post-processing software package.
  • Extra time required to setup and analyze model results due to disconnected work flow as compared to workflow optimized commercial software solutions that feature All-in-one packages for pre-processing, solve, post-processing.

COMSOL Multiphysics: Extremely user friendly, well designed GUI, limited capabilities.

originally partnered with US based Mathworks as European distributor of MATLAB, leading to developing their own software FEMLAB that expanded the capabilities MATLAB in solving partial differential equations.

Eventually they developed their own finite element based meshing and solving routines (FEMLAB).

In 2005, FEMLAB became known as COMSOL Multiphysics, utilizing GUI with MATLAB like programming interface.

Mainly marketed towards research in academia, more recently, COMSOL is attempting to make steps towards making it more useful to engineers.

HARDWARE RECOMMENDATIONS:

Main components to consider, in order of priority:

  • RAM: Determines maximum model size (DOF, degrees of freedom) that can be solved.
  • CPU: number of cores and clock speed determines how quickly a model can be solved. (Good metric to compare between CPU options is: Clock Speed x Number of Cores / Cost
  • Storage: determines how much data can be held on the system, and how quickly it can be input/read.
  • Graphics:
  • GPU: speed up complex solutions
  • Interconnects: enables high speed clustering

Leave a Reply

Your email address will not be published. Required fields are marked *