7 Things Worth Looking At When Assessing AI in Engineering Simulation

AI can make engineering simulation much faster, but speed alone does not make a model suitable for engineering decisions. A model may reproduce selected CFD or FEA results within milliseconds and still only work for one geometry family, a narrow operating range or exactly the solver configuration used to create its training data.

The technically relevant question is therefore not only how fast and accurate the model is on selected test cases. It is what the model has actually learned, within which limits it can be used, how these limits are detected and how its results fit into the real engineering and validation process.

Here are seven areas I would look at when assessing AI in engineering simulation.

What AI in Engineering Simulation Actually Replaces

The term “AI simulation” is often used very broadly. A faster solver running on GPUs is not automatically AI, because adaptive meshing, improved numerical algorithms, parallelisation, better preconditioners or lower numerical precision can also reduce calculation time significantly without machine learning.

Reduced-order models are also not necessarily AI. Classical model-reduction approaches, including Proper Orthogonal Decomposition and projection-based methods, have existed for many years. Machine learning can be used as part of a reduced-order workflow, but model reduction and AI should not automatically be treated as the same thing.

The first step should therefore be to clarify which mapping the model is expected to learn and which part of the existing simulation process it replaces or supports.

A relatively simple surrogate may use geometry parameters, material properties and boundary conditions as inputs and predict a few quantities of interest, such as maximum temperature, pressure loss, displacement or a critical stress value. A field surrogate attempts to predict a distributed result, for example a complete temperature, stress or flow field. This is already a considerably more complex task, especially when the geometry or mesh changes.

Neural operators go another step further and attempt to learn mappings between functions, for example from spatially varying boundary conditions, source terms or material properties to the corresponding solution of a partial differential equation. This makes them interesting for families of related simulation problems, but they still learn from a defined distribution of examples. They do not learn physics without boundaries or independently from the problem definition used during training.

Predicting one scalar value for a fixed product platform is therefore a very different technical challenge from predicting transient fields across different CAD geometries, meshes, boundary conditions and physical regimes. These cases should not be compared only based on an impressive inference time.

Check whether it is really a many-query problem

The strongest use case for an AI surrogate usually exists when an expensive simulation model needs to be evaluated many times. This can be the case in design-space exploration, optimisation, sensitivity studies, uncertainty propagation, digital twins, control applications, repeated product configuration or near-real-time decision support.

In these cases, the high initial effort can potentially be justified because the trained model is reused many times. However, the relevant comparison is not only the time required for one solver run compared with one AI inference.

The full cost of the AI approach includes the generation of suitable training simulations, data preparation, model selection, training, tuning, validation, workflow integration, deployment, monitoring, maintenance and later revalidation. The cost and effort of the engineers creating the reference simulations should also not be ignored.

This complete lifecycle cost has to be compared with the cost of continuing to use the conventional simulation process. If an organisation only needs a limited number of simulations, building and maintaining a surrogate may cost more than simply running the existing solver.

The case becomes stronger when the model is used frequently, when the problem definition remains sufficiently stable and when the trained model can support many design decisions over a meaningful period. It becomes weaker when every new product generation, geometry change or operating condition requires a new dataset, retraining and another complete validation cycle.

A statement such as “1,000 times faster than CFD” can therefore be technically correct for the inference stage and still not represent the overall engineering or economic value. The relevant question is how much time and effort are saved across the complete workflow, not only how quickly one prediction appears after the model has already been built.

The validity domain must be explicit

An AI surrogate does not automatically learn a general physical law. It approximates relationships represented in its training data, reference models, inputs and model structure.

The validity domain may depend on geometry families, parameter ranges, materials, material models, boundary conditions, initial conditions, operating ranges, flow regimes, contact situations, nonlinear behaviour, load combinations, mesh properties, time steps, solver versions, turbulence models, constitutive models and convergence criteria.

A model trained mainly on attached flow does not automatically remain accurate when strong separation appears. A thermal surrogate trained for natural convection does not automatically generalise to forced convection. A structural model trained below the yield point does not automatically predict plastic deformation, damage or failure.

Even when every individual input remains inside its minimum and maximum value, the combination of inputs can still be outside the region covered by the training data. A multidimensional validity domain is not simply a list of acceptable minimum and maximum values.

This also has consequences for testing. A random train-test split can be too optimistic when very similar samples from the same simulation campaign are distributed between training and test data. This mainly checks whether the model can interpolate between neighbouring cases, but it does not necessarily show whether it can handle a new geometry, product variant, operating region or physical regime.

A stronger validation approach may hold out complete geometries, complete parameter regions, complete load cases, different mesh resolutions or an entire product variant. The exact split should reflect how the model will later be used.

It is also important to examine local and worst-case errors, not only the mean error across all test cases. A low average error can hide a relevant failure close to a temperature threshold, structural limit, fatigue boundary or flow transition. In engineering, the few cases close to the limit can be more important than the majority of cases far away from it.

Matching the solver is not yet matching reality

When an AI model is trained using simulation data, it primarily learns the behaviour of the reference simulation. This includes not only the intended physical relationships but also the assumptions, simplifications, discretisation choices, boundary conditions, numerical tolerances and possible modelling errors of the reference model.

If the CFD model uses an unsuitable turbulence model, the AI surrogate may reproduce the resulting error very efficiently. If contact conditions in an FEA model are unrealistic, the surrogate can learn the same unrealistic response. If the mesh does not resolve a local hotspot, good agreement between the AI model and this simulation does not solve the underlying problem.

There are therefore at least two separate validation questions. The first is whether the AI model reproduces the reference simulation within an acceptable error. The second is whether the reference simulation adequately represents the physical product and the engineering decision being made.

These two questions should not be mixed.

Verification concerns whether the equations, numerical methods and software implementation are working as intended. Validation concerns whether the model represents reality well enough for its intended use. Uncertainty concerns what is not known exactly and how this affects the result. These principles remain relevant when AI is added to the simulation process.

A surrogate can only become more representative of reality than its simulation reference when additional information is included, for example physical measurements, test data or validated physical constraints. Even then, it has to be demonstrated that the additional data actually improves the relevant prediction and does not only improve selected test cases.

Without this additional evidence, an AI surrogate should normally be considered an approximation of the reference model, not an independent correction of the physics.

Accuracy must be assessed in engineering terms

A single metric such as R², mean squared error or mean relative error is rarely enough to judge whether a model is suitable for engineering use.

The error assessment should be connected to the decision the model supports. For scalar outputs, relevant questions include the maximum absolute error, systematic bias, ranking between different designs, correct identification of feasible and infeasible solutions, behaviour close to constraints and stability under small changes in the inputs.

For field predictions, global error metrics can hide important local failures. A model may reproduce most of a temperature field well but miss a small hotspot. It may predict overall displacement accurately but underestimate a local stress concentration. It may reproduce the average flow field while shifting a separation point, recirculation zone or vortex structure that is important for the product behaviour.

Depending on the application, validation may therefore need to include local field errors, integral quantities, gradients, extrema, conservation balances, symmetry, invariance, monotonic behaviour and physically meaningful limiting cases.

Governing-equation residuals can provide additional information, but a low residual alone does not prove that the correct physical solution was obtained. The result can still be affected by unsuitable boundary conditions, scaling problems, poor optimisation or a solution that satisfies the equations only in the tested regions.

Acceptance criteria should ideally be defined before reviewing the final model performance. Otherwise, there is a risk that the accuracy achieved by the model becomes the accuracy accepted by the organisation.

The required accuracy also depends strongly on the use case. A model supporting early concept screening can have wider tolerances than a model releasing a component, controlling industrial equipment or replacing a safety-relevant simulation or test. There is no useful accuracy target without a defined engineering decision.

Be precise about PINNs, neural operators and hybrid models

Different AI approaches address different types of problems, and the terminology can sometimes create expectations that are technically not justified.

Physics-informed neural networks include governing equations, boundary conditions or initial conditions in the training process. They can be useful for forward problems, inverse problems, parameter identification and situations where physical equations need to be combined with limited measurement data.

However, “physics-informed” does not mean “physically guaranteed”. In many PINN approaches, the equations and boundary conditions are represented through weighted terms in the training objective. The result can be sensitive to scaling, loss weighting, optimisation, stiffness, sharp gradients and competing boundary and domain objectives.

A model can therefore have difficulties even when the underlying PDE is well known. The limitation may come from the training and conditioning of the optimisation problem rather than from the theoretical representation capacity of the network.

PINNs can be useful tools, but they are not a general replacement for mature finite-element, finite-volume or finite-difference solvers. Their suitability depends strongly on the specific problem, available data, dimensionality, boundary conditions and required accuracy.

Neural operators such as Fourier Neural Operators or DeepONets aim to learn mappings between functions. This makes them interesting for repeated solutions across parameterised PDE families, especially when many related queries are required.

But operator learning does not remove the need for a clear validity domain. The performance still depends on the distribution of training functions, geometry representation, discretisation, parameter ranges, numerical reference data, architecture and physical regimes included in the training examples.

Good transfer between resolutions on a research benchmark does not automatically mean reliable transfer between industrial meshes, complex CAD variants or different operating regimes.

For many industrial applications, a hybrid approach may therefore be more credible than attempting to replace the complete simulation process. AI can help select the next simulation through active learning, accelerate design-space exploration, predict a useful initial solution, correct a lower-fidelity model, combine low- and high-fidelity data, estimate selected closure terms, support mesh or parameter selection, classify simulation outcomes or automate repetitive preparation and reporting work.

This can create value earlier because the established solver and engineering validation process remain part of the loop. However, hybrid does not automatically mean safe. Learned corrections, reduced models and closure relations can still violate conservation, stability or physical invariance if these properties are not considered explicitly and tested properly.

The real product is the complete engineering capability

A successful pilot shows that a method can work on selected data. It does not yet show that an organisation can use the model repeatedly, maintain it and trust the resulting decisions over time.

A usable engineering capability needs traceable training and validation data, documented simulation configurations, versioning of the reference solver, dataset and AI model, clear acceptance criteria, defined validity limits and a process for handling inputs outside the supported domain.

It should also define where uncertainty information is meaningful, when a fallback to conventional simulation or testing is required, who owns the model and which changes trigger revalidation.

Relevant changes can include a new geometry family, different materials, changed boundary conditions, new physical regimes, updated meshes, a new solver version, modified measurement systems, additional output quantities or retraining with new data.

Out-of-distribution detection can support this process, but it should not be treated as perfect protection. A model can produce a confident prediction for an unsupported case, and an input can appear statistically similar to the training data while being physically different in an important way.

The workflow should therefore clearly define when an AI prediction is sufficient, when an engineer must review the result, when a conventional simulation is necessary and when physical testing remains mandatory.

This also creates organisational responsibilities. Someone has to own the training data, reference simulation, AI model, validation evidence, release process and later maintenance. These responsibilities may be distributed across different teams, but they need to be explicit.

Without this ownership and lifecycle process, the model may remain an interesting research prototype even when the initial technical results look impressive.

Conclusion

AI can materially improve engineering simulation. It can reduce evaluation time, enable broader design-space exploration, support optimisation and make simulation knowledge available earlier in product development.

But a fast prediction is not automatically a trustworthy engineering result.

The important questions remain engineering questions: Which problem is being solved? Which physical regimes are covered? What is the reference truth? Which errors matter for the decision? What happens outside the tested domain? How is the model maintained? When is another simulation required, and when is physical testing still necessary?

The strongest AI-simulation applications will probably not be the ones making the largest general claims. They will be the ones with a clearly bounded problem, credible reference data, explicit validation and a realistic place within the engineering workflow.

AI changes the economics and workflow of engineering simulation. It does not remove the need for physics, validation and engineering judgement.