Добавил:
ivanov666
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Intelligent data analysis in medicine. Study aid
.pdf
41
Linear regression
Python: scikit-learn;
Method: linear_model.LinearRegression.
Linear regression is a statistical method for determination of the relationship
between input data (x: predictors, independent variables) and output data (y: output or
dependent variable). As the name suggests, linear regression requires that y should be
calculated from a linear combination of input variables. The goal is to place a line of
best fit through all data points, allowing predictions to be made easily by minimizing
the residual (Fig. 9).
Fig. 9. Graphical interpretation of linear regression
When there is only one input variable x in the set, this method is known as simple
linear regression. Multiple linear regression evaluates two or more predictor variables.
Once the model has generalized the input parameters, they are used to predict
the values of y given the new, previously unused input data x. Popular methods used to
estimate regression coefficients are methods of the least squares and gradient descent.
Given a line of best fit through the sample data, standard least squares minimizes the
sum of squared distances from each data point to the regression line. The goal is to
minimize the sum of squared deviations. The coefficient values are iteratively
recalculated so that the model error decreases.
42
The sum of squared errors is calculated for each pair (input, output). The learning
rate is selected and acts as a scaling factor. The coefficients are selected to minimize
the error until it is impossible to obtain a further increase in the quality of the
implemented regression model.
The learning rate is a hyperparameter whose value is determined
experimentally. Different values are tried and the value that produces the best results
is used. The main criticism of linear regression centers on its simplicity, resulting in its
inability to capture complex relationships in the data. Noise in the data can also be
taken into account by the model, which will lead to incorrectly constructed
dependencies. Relationships between variables are not always linear, so a linear
regression model may not perform well on data with nonlinear relationships. However,
sometimes the original variables can be transformed to fit a linear regression model.
Linear regression also assumes that all variables in the vector have the same variance.
The above is a great demonstration of why different machine learning algorithms
should be used when training a model.
Advantages of linear regression:
– speed and ease of obtaining the model;
– interpretability of the model. The linear model is transparent and
understandable to the analyst. Based on the obtained regression coefficients, one can
judge how a particular factor affects the result and draw additional useful conclusions
on this basis;
– wide applicability. A large number of real processes in economics and business
can be described with sufficient accuracy by linear models;
– knowledge of this approach. For linear regression, typical problems (for
example, multicollinearity) and their solutions are known, tests for assessing the static
significance of the resulting models have been developed and implemented.
The main disadvantage of linear regression is that this method allows only
direct relationships to be described, although in practice it is almost often necessary to
create models with nonlinear relationships between data.
Logistic regression
Python: scikit-learn;
Method: linear_model.LogisiticRegression.
43
Linear regression applies to continuous variables, whereas logistic regression
predictions apply to discrete values. Logistic regression is often used for binary
classification problems. For example, logistic regression can be used to predict whether
a patient will experience an adverse event while taking a particular medication;
determining whether a patient is likely to be readmitted or has a specific medical
diagnosis. Unlike linear regression, the output of the model is expressed as a
probability ranging from 0 to 1. The predicted output is generated by taking a
logarithmic transformation of the input vector x and applying a logistic function. A
given threshold is used to convert this probability into binary classification.
As with linear regression, logistic regression assumes a linear relationship
between input variables and output data. Feature reduction may be required to
transform data into linear models. Logistic regression models are also prone to
overfitting, which can be addressed by removing highly correlated inputs. Finally, it is
possible that the coefficients do not converge, which can happen if the data is sparse
or highly correlated.
Support Vector Machine (SVM)
Python: scikit-learn;
Method: svm.SVC, svm.LinearSVC.
Support Vector Machine (SVM) is a non-probabilistic binary linear classifier
used for both classification and regression problems. SVM is often used in conjunction
with methods to analyze natural language text. This method is also used in image
recognition and handwritten digit recognition tasks.
The algorithm finds the hyperplane or line of best fit between two classes defined
by the support vectors.
Support vectors are the data points closest to the hyperplane that will change
the position of the hyperplane if removed (Fig. 10). The greater the instance value, or
the distance from the data points to the hyperplane, the greater the confidence that the
data is classified appropriately. The line of best fit is determined as a result of an
optimization procedure aimed at minimizing the error. SVM uses something known as
a kernel to map data to high-dimensional feature spaces.
44
Fig. 10. Graphical interpretation of the SVM method
The data is mapped using kernel transformations in more and more dimensions
until a hyperplane can be formed to classify it.
Advantages of the support vector machine:
– the convex quadratic programming problem is well studied and has a unique
solution;
– the support vector machine is equivalent to a two-layer neural network, where
the number of neurons in the hidden layer is determined automatically as the number
of support vectors;
– the principle of the optimal separating hyperplane leads to maximization of the
width of the separating strip, and therefore to a more confident classification.
Disadvantages of the classical implementation of the support vector machine:
– instability to noise: outliers in the source data directly affect the construction
of the separating hyperplane;
– general methods for constructing kernels and rectifying spaces that are most
suitable for a specific task are not described;
– no selection of features;
– it is necessary to select free model parameters using cross-validation.
45
Naive Bayes
Python: scikit-learn;
Method: GaussianNB, MultinomialNB, BernoulliNB.
Naive Bayes uses Bayes' theorem to calculate the probability that some event
will occur if another event has already occurred. The algorithm is considered naive
because it assumes that all variables are independent of each other, which is not typical
for real data samples. The Bayesian classifier is often used when the input data is of
high dimensionality. For example, given a threshold, this method can be used to
probabilistically classify whether a vector belongs to a particular class (Fig. 11).
Fig. 11. Graphical interpretation of the Bayesian classifier
These are the advantages of the Bayesian classifier:
– the algorithm easily and quickly predicts the class of the test data set. It also
handles multi-class prediction well;
– the performance of the Naive Bayes classifier is better than other simple
algorithms such as logistic regression. Moreover, less training data is required;
– the algorithm works well with categorical features (compared to numerical
ones). For numerical features, a normal distribution is assumed, which can be a serious
assumption for the accuracy of this algorithm.
Disadvantages of the Bayesian classifier:
– if a variable has a category (in the test dataset) that was not observed in the
training dataset, then the model will assign a probability of zero and will not be able to
make a prediction. This is often called zero frequency. To solve this problem, the
smoothing technique is usually used. One of the simplest smoothing methods is called
Laplace estimator;
46
– the values of predicted probabilities are not always sufficiently accurate;
– a limitation of this algorithm is the assumption of independence of features.
However, in real problems, completely independent features are extremely rare.
Nearest neighbors (kNN) method
Python: scikit-learn;
Method: neighbors.KNeighborsClassifier.
It is important not to confuse this algorithm with k-means clustering. The kNN
method classifies an unknown object by the majority of k-nearest neighbors and is used
in classification and regression problems. kNNs are a non-parametric method. This
algorithm stores the training data set as a representation of it and performs
classification of the new sample based on learning by analogy. Since there is no model
training as such, kNN is classified as a lazy learning method. Each sample object
represents a point in N-dimensional space. A neighbor is defined as closest if it has the
smallest distance in the feature space (Fig. 12). The distance between an invisible
object and its neighbor is typically calculated using Euclidean distance. It is possible
to use other methods for determining the distance between objects in the feature space.
The algorithm works as follows:
1. The distance between any two points is calculated.
2. Nearest neighbors are found based on comparison of these pairwise distances.
3. The majority votes for a class label based on the list of nearest neighbors.
Fig. 12. Graphical interpretation of the kNN method
47
The forecast is performed upon request. Regression problems use the mean or
median of the k-most similar cases. In classification problems, the class with the
highest frequency from the k-most similar instances is selected. As with most
algorithms, the deterministic method may vary. You can also use the Hamming
distance, Manhattan distance, etc. to determine the distance between sample objects.
The disadvantage of kNN is that it requires large computational resources to
classify an object, since it is necessary to calculate the distance for all neighbors in the
training dataset. This algorithm is not well suited for multidimensional data. Each input
variable can be viewed as a dimension of the n-dimensional input space. For example,
x1 will be one-dimensional, and x1, x2 will be two-dimensional, etc. Increasing the
dimension exponentially increases the volume of the input space. KNN is not well
suited for data with missing values because distances between vectors cannot be
calculated from missing data.
Advantages of the nearest neighbors method:
– the algorithm is simple and easy to implement;
– not sensitive to emissions;
– there is no need to build a model, adjust several parameters or make additional
assumptions;
– the algorithm is universal. It can be used for both types of problems:
classification and regression.
Disadvantages of the nearest neighbors method:
– the algorithm works significantly slower when the sample size, predictors or
independent variables increase;
– the argument above implies large computational costs at run time;
– it is always necessary to determine the optimal value of k.
Artificial neural networks
Python: scikit-learn;
Method: linear_model.Perceptron.
Neural networks are well suited for data mining tasks due to their ability to
model high-dimensional data and effectively find hidden patterns among them. Neural
networks can be used for prediction and classification problems. For example, the
process of evaluating the output and comparing it with the actual output is known as
the feedforward method.
48
Neural networks are a subset of machine learning algorithms that include
perceptrons, fully connected neural networks, convolutional neural networks, recurrent
neural networks, short- and long-term memory neural networks, deep neural networks,
and more. Most of them are trained using an algorithm called backpropagation.
Artificial neural networks are used in many tasks:
– classification of diseases: computers can learn what images of diseased organs,
such as kidneys, eyes, liver, look like in order to predict the likelihood of disease;
– speech recognition;
– translation and digitization of text;
– pattern recognition.
Neural networks can learn to perform a task by imitating the functioning of the
human brain — supervised, unsupervised and reinforcement learning.
Artificial neural networks are composed of perceptrons and usually contain one
or more hidden layers. There are several topologies, the simplest of which is the
feedforward network. Backpropagation is a technique for identifying an error or
output loss and propagating it back to the network. The weights at each node are
updated to minimize the corresponding error produced by each neuron.
Backpropagation seeks to minimize the error for the neurons and therefore the entire
model by updating the corresponding neuron weights.
The main training that needs to be done in neural network models is teaching the
neurons when they should fire. An epoch refers to one training iteration of forward and
backward propagation. During the training phase, nodes in the neural network adjust
their weights depending on the error of the last test result. Another important parameter
is the learning speed of the neural network. The number of perceptrons or neurons in
the model is equal to the number of variables in the original data. Some views may also
have an offset node. Typically, artificial neural networks have neurons organized in
layers. Each layer can perform different transformations on the input data it receives
from the previous layer.
Deep learning is a process that uses multi-layer neural network architectures.
Training a deep artificial neural network model requires more time and CPU power
compared to other types of models. Notably, the performance of deep neural network
models may not necessarily be better than standard supervised learning methods. Deep
learning is not a new technique, but its use is becoming more widespread due to
advances in hardware, especially in power and cost, that have made such computing
possible.
49
Recurrent neural networks store the layer's output and feed it back to the input
to improve the output layer's predictions. Thus, each node has memory when
performing calculations and uses sequentially arriving information.
Convolutional neural networks are deep feedforward neural networks.
Convolutional layers are often used in speech recognition, spatial recognition, natural
language, and computer vision tasks. Individual convolutional layers are typically
applied to different aspects of some specific problem. For example, to understand
whether there is a face in a photo, you can use separate convolutional layers to identify
different aspects of the face — eyes, nose, ears, mouth, etc.
Unsupervised Machine Learning
Unsupervised learning refers to the process of learning a model from unlabeled
data. This means that the input data (x) does not have a corresponding output data (y).
Semi-supervised learning occurs when only some output labels (y) are provided. The
more labeled data provided to the model for training, the more accurate the resulting
system will be. Unsupervised learning can require a lot of time, computational
resources, and expertise. Typically, significant improvements in model accuracy can
be achieved by improving data labeling. Unsupervised learning involves two main
classes of problems: clustering and association.
Clustering problem
Clustering refers to the process of discovering relationships in data. It is used for
a variety of health-related purposes, including:
– grouping the patients with similar data and symptoms for further joint
monitoring;
– detection of anomalies or deviations in requirements or transactions;
– activity tracking using motion sensors.
50
K-Means
Python: scikit-learn;
Method: cluster.KMeans.
K-Means clustering aims to find k-similar groups in the data. K refers to an
iterative algorithm that calculates the centroids of specific k-clusters, with all training
data belonging to a certain cluster. Data points are assigned to a certain cluster by the
distance between a given point and its center of gravity. A cluster centroid is a set of
attribute values that define the resulting clusters.
Questions for self-control
1. What is the main task of machine learning?
2. What tools and libraries are used to implement machine learning algorithms?
3. What problems can be solved using supervised learning algorithms?
4. What is the task of classification?
5. What is the problem of regression?
6. What is the idea of the decision tree method? Give a basic algorithm for
constructing decision trees.
7. Give the advantages and disadvantages of the gradient boosting method.
8. Give the advantages and disadvantages of linear regression.
9. Give an algorithm for the support vector machine (SVM).
10. Give the advantages and disadvantages of the support vector machine.
11. What mathematical principles is Naive Bayes based on?
12. Give an algorithm for the operation of the nearest neighbor method.
Соседние файлы в предмете [НЕСОРТИРОВАННОЕ]