Conventional methods of data processing
(short review)
Introductory Lecture “0”+7
Compared by Prof. Raoul R. Nigmatullin
1
Data treatment? – structural and information analysis.
How to take into account many random factors?
|
|
Type of data |
1. |
Numerical data |
|
2. |
Interval data |
Numerical data |
3. |
Rank data (ordered data) |
|
4. |
Nominal (textual) data |
|
The volume of the sampling (N>30). The more – the better. (N>>1)
Classification of the conventional data |
|
|
|
1. Cluster analysis |
|
|
|
Example (Price – quality) |
N |
Price (in kilo $) |
Quality |
|
2
Y – Price, X- quality (in balls). The proper
selection of variables! - Important
Number of variables, measure of similarity (clusterization of data)
Reduction of data:
Laws-Principles-Models-Methods
Scattering diagram
3
2. Factor analysis (many factors – to find their similarity) Idea – reduction of the factors and their classification
Initial factors (many) |
The reduced factors |
The further decreasing of |
|
(the grouped factors) |
the factors |
3. Neuron nets
input |
Number of neurons, number of layers? |
Data “feeding” |
|||
w1 |
1 |
|
|
Filter – step-similar function |
|
|
|
|
Output – the desired (known) function |
||
|
|
|
output |
||
|
|
|
|
|
|
w2 |
|
Filter |
|
W(i) > 0 – excitatory input, W(i) < 0 – |
|
2 |
S |
inhibitory input |
|
||
|
|
||||
|
|
|
|||
w3 |
|
|
|
Mathematical model of neuron net. |
|
3 |
|
|
|
|
|
|
|
|
|
|
|
4
List of problem that can be solved with the help of Neuron Nets
1.Classification of images
2.Clusterization
3.Approximations of output (functions) Artificial Intelligence
4.Forecasting of data
5.Optimization and [control = (management)].
Topology, education (optimization of the NN) play the essential role.
+ find and express of the significant factors , factors can be grouped to a specific syndrome
(factor analysis)
-But any NN expresses the abilities and possibilities of the specific expert group that created the partial NN.
-Any NN – is a black box. Why w(i) has a specific value – cannot be explained.
4. Trees of Solutions |
+ |
- |
Branching structure |
|
|||
|
+ |
- + |
- |
+ |
- + |
- + |
- |
5
Example of grant an advance
by a bank.
Where these trees are applied? - 1. Banking, 2. Industry, 3. Diagnostics of diseases
4.Consulting
5.Regression analysis
(LLSM) |
The simplest dependence
6
6. Correlation analysis
The principal component analysis
We have k-basic factors that influence on some random behavior of the function. In what cases is it possible to replace the influence of k-
components by small number of components m (m<k) making the influence of other components as insignificant?
Determination of the basic component
Criterion – maximal dispersion
7
Данные = Инфо + Шум
Инфо= прямая линия. Шум= данные – прямая линия (подгоночная ф-ция)
9
10
11