10.3 Neural networks for heat transfer 345

Example 10.4: We are trying to obtain a model for the temperature at any given point in a long fin purely using prior data. Sketch the process required to achieve this using neural networks. Comment on whether a linear model could work here.

Solution:

Note: Since the real problem requires a lot of data, we have outlined the theoretical process below without actual numerical data.

We follow the usual learning process here.

Step 1: As was discussed in Section 10.3.1, the mathematical formulation of the long fin involves defining x = (θb , x, m) and y = θ . Note that this requires us to already understand the physical parameters of the problem sufficiently to know that θ = T − T∞ is a useful reduction of parameters in the problem.

Step 2: We could naively select a linear model here. In this case, we would have θ = w0 + w1θb + w2 x + w3m . It should be clear from the physics of the problem that this is a bad model because, for a long fin, θ cannot keep increasing or decreasing unboundedly with x, which would happen in the linear model we have provided above. Similar problems will occur with any polynomial model. This is why the use of neural networks is particularly useful here. It allows us to fit a model that we can confidently use to approximate any nonlinear relation.

However, this leads to a question. How many layers and neurons should we use? Each network “architecture” is effectively a different hypothesis function. For engineering problems, it is always best to build intuition about the size of the network required by first working with a single hidden layer. It is a good idea to play with the number of neurons in this layer to see the number of neurons beyond which you will not gain any performance improvement. In practice, for the fin problem, it turns out that anywhere between 3 and 6 neurons in the hidden layer work really well.

Fig. 10.12 shows the schematic for the architecture in this problem. Note that while the neurons in the hidden layer will have a nonlinearity (say, a sigmoid), the

FIGURE 10.12

Schematic of ANN for Example 10.4, with one hidden layer.

346CHAPTER 10  Machine learning in heat transfer

output neuron cannot have a sigmoid as nonlinearity. Ideally, it is better to leave this neuron as a purely linear neuron (see Problem 10.5).

Step 3: Data collection simply involves collecting available data of long fins at

varying values of θb , x, m and measuring the corresponding θ . However, for the purposes of testing how neural networks work, we could generate synthetic data compu-

tationally by simply calculating θ = θbe− mx. This would require us to specify ranges of θb , x, m for which we will create the synthetic data. Even experimental data would have some range. For instance, we could choose to create 1000 such “synthetic” data points. These should now be split into training and testing sets separately, and only the training set should be utilized for finding the optimum parameters.

Step 4: The architecture of the network (number of neurons and layers) we chose in Step 2 fixes the form of the hypothesis function. We can now find the optimum parameters by optimizing over the training dataset. For neural networks, it is impossible to perform direct optimization due to their nonlinearity. We, therefore, use some iterative algorithms. Typically, a nonlinear optimization called Levenberg-Marquardt (see Balaji [2019] for details) is a good choice for these problems.

At the end of Step 4, we now have the complete surrogate model

This surrogate model can now be used wherever the analytical expression was. For example, we could use it in the design, optimization of fins, or for inverse problems in fins. While this is a simple problem, you can use the same process for any complicated problem.

10.4  Practical considerations in engineering problems

If the process of fitting neural networks is as simple as outlined in the previous section, why is the use of neural networks not universal across engineering applications of heat transfer? There are several reasons why this is so. We list some of the important ones below. These also serve as warning signs when an engineer attempts to apply the theory above to their practice. The issues are

•A neural network is best thought of as an interpolating, regression model. Usually, its applicability is limited to the range of data it has seen. It cannot extrapolate to discover new physics. In short, it is mathematics, not magic!

•In practice, it takes time and effort to pose the problem correctly. Even the specification of inputs and outputs in a problem is often nontrivial in engineering situations. Think about heat exchanger design, for instance. The appropriate inputs and outputs for this design are nontrivial. What one chooses to measure and designate as input and output is as important as how one fits a network to it. This requires domain expertise and deep knowledge of the field at hand.

•Collecting and curating data is an extremely important part of the problem.

There might be a variety of past data of varying reliability, completeness, and

10.5 Applications in heat transfer 347

precision. Once again, think of collecting past data about heat exchanger design. Combining heterogeneous data statistically and offering it in a digestible form to the algorithm is a very time consuming and difficult part of the overall process. In fact, sometimes, at the end of data cleaning, there is not much data left to train sophisticated neural networks.

•Deciding on the appropriate architecture (neurons, layers, nonlinearity, etc.) of the network is not always easy. In practice, there is a lot of iteration required in coming up with an accurate, cheap, and trainable network. Determining the appropriate architecture, in many cases, is a time-consuming task.

•Finding the optimal parameters for a network is, in practice, an extremely hard task in industrial problems. Networks can be stubborn in not converging to optimal parameters. Optimizing neural networks is, therefore, a very rich and active open research problem. It is perhaps the hardest technical part of the neural network framework.

•Even if one gets past the above steps, the neural network is not interpretable.

That is, we do not know what physical significance (if any) the neurons in the hidden layer have. If the network suggests an outlandish or nonintuitive design, it is very difficult to trust it without having a “feel” for the problem. Similarly, in case of faulty predictions, it is difficult for the engineer to know where to fix the surrogate model. Due to these reasons, in critical applications engineers may prefer a simpler, interpretable model to a more accurate but opaque model, such as a neural network.

10.5  Applications in heat transfer

1.Consider a heat exchanger that is being used in a factory setting.. From this exchanger, we obtain continuous data and can train the effectiveness of the exchanger as a function of the pertinent parameters through a neural network. This network can then be used to predict performance at off-design conditions or pave the way for possible optimization. The key advantage is that the network can be adaptively trained, meaning that the network will be dynamically updated with new data as and when they are collected and weights and biases too will keep evolving.

2.In continuation of (1) it is known that the heat transfer rate Q and pressure drop ∆ p are in fundamental conflict in a heat exchanger. With past data it is possible to develop one or two networks (i.e., either one for Q and ∆p together or two for each of these separately) to characterize the quantities of interest against the controlling variables and then try to optimize combining the two criteria. The network can be built with past data from an existing heat exchanger and, as explained before, can be dynamically updated, which will intrinsically take care of issues like fouling.

3.At a much simpler level, almost all of the charts we introduced and used in this text (e.g., Heisler charts for transient heat transfer, fin efficiency charts, and heat

348 CHAPTER 10  Machine learning in heat transfer

exchanger effectiveness charts) can be generated with a trained neural network once we have access to the original data. The advantages of a network will be convenience to use and easy programmability to look for, say, alternate fin designs followed by optimization.

4.In many other areas of heat transfer, where we solve an inverse problem—that is, obtaining the cause (e.g., thermal conductivity or thermal diffusivity) from the effects (e.g., temperature distribution) without explicitly mentioning about them—we can develop a neural network with temperatures as inputs and causes as output, which then is a straight solution to the inverse problem. Even so, large training data sets are required and this approach is easier said than done for involved problems such as those in convective heat transfer.

5.Advanced applications like determination of emissivity of special coatings used in spacecraft can be solved using measurements, a surrogate model, and an algorithm to give samples of trial emissivity values, which minimizes the difference between measurements and simulations of, for example, the temperature time history of the specimen with the coating whose emissivity is to be determined.

6.Almost all of the correlations that have been developed in convective heat transfer can be reworked using neural networks. The advantage is that experimental or numerical results obtained by several investigators can be combined with a view to come out with an ANN (artificial neural network)-based robust correlation. In fact many of the highly popular correlations such as those of Churchill and coworkers are based on this strategy, albeit without using a network.

7.Finally, there is a fascinating new subfield that has come to be known as physicsinformed neural networks (PINNs), where we actually solve engineering problems, including ones in heat transfer by incorporating a network and its derivatives in the original differential equation with a view to obtain the best set of weights to solve the problem. While data will be required for doing this, once the network is trained it will return results in a fraction of the time taken by a conventional solver. However, it is still early days as yet for PINNs, and they have a long way to go.

10.6  Summary

Neural networks are a subset of machine learning techniques for supervised learning. In heat transfer, they can be used to serve as approximate models for both forward and inverse problems. Neural networks can be seen as a regression technique using a nonlinear hypothesis function and, given sufficient data, can serve as universal approximators. Despite their flexibility, there are some lacunae while applying neural networks to practical problems. Nonetheless, machine learning, in general, and neural networks, in particular, hold tremendous potential for heat transfer applications. Recent methods such as those by Raissi et al. (2019) have shown tremendous promise in having novel methods for inverse heat transfer using a combination of physics

10.6 Summary 349

and data. We think that this direction is particularly interesting and that, in the future, there will be tighter integration of data and physics in heat transfer engineering.

Problems

10.1Complete the proof given in Section 10.3.2. That is, show that for m univariate

data points using the linear model yˆ = w0 + w1x , the optimal parameters w0 and w1 for minimizing the mean squared error are given by:

    where all summations ∑ should be read as ∑im=1 .

10.2 Determine the optimum parameters for a quadratic fit to the data above. That is, assuming a model of the form yˆ = w0 + w1x + w2 x2 , find the expressions for the optimum parameters w0 , w1 , and w2 for minimizing the mean squared error.

10.3The coefficient of thermal expansion, α, of steel is given at discrete values of the temperature in Table 10.2. We wish to fit a model to the data in order to find thermal expansions at intermediate temperatures not available in the table. Find the expressions and parameters for

a.The best fit linear model

b.The best fit modelfor the given data. Which of these models fits the data better? (Hint: Use the results of Problems 10.1 and 10.2).

10.4Consider a variant of Example 10.1 where the temperatures at 5 different locations are the same as given in the example. We are also given that the flux at x = 0 in the slab is 2250W / m2 . However, the thermal conductivity of the material is unknown. From the data given, estimate the thermal conductivity. This is an example of an inverse problem.

Table 10.2  Variation of coefficient of thermal expansion of steel with temperature.

350 CHAPTER 10  Machine learning in heat transfer

10.5Consider a very simple neural network with the following features. There is neuron only in the input, hidden, and output layers. Assuming no bias units and

asigmoid activation function at both layers, answer the following.

a.Write down the analytical expression for the hypothesis function this network represents.

10.6Consider the fin problem in Example 10.4. Answer the following questions.

a.Why is it important that the output neuron not have a sigmoid nonlinearity? (Hint: Think about the range of the sigmoid function vs. the range of the data we want the output neuron to represent.)

b.Assuming a single hidden layer with 6 neurons, how many adjustable weights does the full network have? How many bias weights (simply called biases generally) does the network have?

c.Assuming the same architecture as the previous part, and that all weights

and biases have been initialized to 1, what is the output of the network for an input data having θb = 70 , m = 4 , and x = 0.1?

References

Bejan, A., 1993. Heat Transfer. John Wiley and sons, NewYork.

Balaji, C., 2019. Thermal System Design and Optimization. Ane Books Pvt, New Delhi. Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep Learning. MIT press, Cambridge, MA. McCulloch, W., Pitts, W., 1943. A logical calculus of ideas immanent in nervous activity. Bull.

Math. Biophys. 5 (4), 115–133.

Raissi, M., Perdikaris, P., Karniadakis, G.E., 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707.

Rabault, J., Kuchta, M., Jensen, A., Réglade, U., Cerardi, N., 2019. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. J. Fluid Mech. 865, 281–302.