125 Кібербезпека / 4 Курс / 3.1_3.2_4.1_Захист інформації в інформаційно-комунікаційних системах / Лiтература / [Sumeet_Dua,_Xian_Du]_Data_Mining_and_Machine_Lear(BookZZ.org)

188    Data Mining and Machine Learning in Cybersecurity

Transversal endurance refers to the endurance of a proposed privacy preservation technique when it is applied in PPDM algorithms. A privacy preservation technique, such as a sanitization algorithm, is initially developed in pairs with a specific data-mining or machine-learning method. Given a number of data sets, the transversal endurance measures the ability of privacy protection provided by a variety of PPDM algorithms using the same privacy preservation methods.

Accompanied by the protection of sensitivity information, we need to consider the accuracy of mining results impacted by the loss of insensitive information. We call this metric functionality loss, because the modification of the original input data and new data-mining or machine-learning algorithms may lead to the downgrading data-mining or machine-learning results. Fundamentally, the functionality loss presents the difference between the mining results using the original data and the privacy preservation mining results using the modified data. Thus, this metric depends on both data set and PPDM algorithms. Various concepts have been proposed to present this metric in terms of data set, including accuracy, completeness, consistency, and so on.* Readers should refer to Bertina et al. (2005) for more details about these concepts. For example, the following classification error EC can present the total function loss of clustering algorithms:

K

EC = n1 ∑Clusterk ( X ) − Clusterk ( X M ), (8.2)

k =1

where |Clusterk(X)| and |Clusterk(XM)| denote the cardinalities of cluster k with the original data set X = {x1, …, xn} and modified data set XM = {xM1 , …, xMn }. With respect to association rules, function loss refers to the loss of nonsensitive rules. This loss results from the application of privacy preservation techniques that hides both sensitive and insensitive information.

Computation complexity evaluates the time requirements of privacy preservation algorithms. The proposed approaches include calculating the CPU time used by the algorithm measuring the operations needed for hiding the sensitive information and measuring computational cost by the degree of a polynomial. In SMC, we must include the communication cost in the final computation cost, which is caused by the amount of exchanged information between sites.

Scalability evaluates the efficiency of PPDM algorithms with the increasing amount of input data. The proliferation of information communicative techniques and data storage capabilities requires the development of more efficient PPDM algorithms to deal with the influx of data sets. Moreover, the increasing data dimensionality exacerbates the design of high-scalable PPDM methods.

*Completeness measures the loss of individual data information in the sanitized data; consistency measures the loss of correlation between data in the sanitized data.

Privacy-Preserving Data Mining    189

8.3Data-Mining and Machine-Learning Applications in PPDM

The objective of PPDM is to keep private data and private knowledge safe once the mining on the data has been completed. PPDM methods can be analyzed from the perspectives of data distribution, data modification, data mining algorithms, data or rule hiding, or privacy preservation (Verykios et al., 2004a). We categorize the principle PPDM methods in Table 1.1, according to data-mining algorithms and present their privacy preservation methods. In particular, the privacy preservation technique is the most important for the selective modification of the data, which are classified into three groups: heuristic-based techniques, cryptography-based techniques, and reconstruction-based techniques. For a more detailed analysis, readers should see Verykios et al. (2004a).

Researchers have investigated various data-mining algorithms in isolation of each other. Among them, the most important privacy preservation methods have been proposed for a number of data-mining algorithms, like support vector machines (SVM) classification (Yu et al., 2006), association rule mining algorithms (Evfimievski et al., 2004), K-means clustering (Vaidya and Clifton, 2003), decision tree inducers (Agrawal and Srikant, 2000), BN (Wright and Yang, 2004), KNN (Kantarcioglu and Clifton, 2004), ANN (Barni et al., 2006), and other statistical methods (Du et al., 2004). Readers should see Table 1.1 for our list of references on this topic.

8.3.1 Privacy Preservation Association Rules (Table 1.1, A.4)

Let us review the association rules as explained in Chapter 2. Let E = {I1,I2, …, Ik} be a set of items and X be a database consisting of n transactions T1, …, Tn, X = {T1, …, Tn}. Each transaction Tj, 1 ≤ j ≤ n is a subset of items with Tj ̂ E. Each transaction Tj has support s (Ti) in E, defined by the following conditional probability:

Given a threshold Th, if s (Tj) ≥ Th, we say transaction Tj is frequent; if we have another transaction Ti ̂ E and Tj ̂ Ti, we say s (Tj) ≥ s (Ti).

Itemset I causes a privacy breach of level ρ, if we randomize transaction Tj = R(Tj ), and we find that a frequent itemset I and the item Ir I have the conditional probability

190    Data Mining and Machine Learning in Cybersecurity

To solve this problem, Evifimievski et al. proposed randomization operators using “cut and paste” to insert “false” items to replace some true items (Evfimievski et al., 2002). Their method includes three steps as follows:

Step 1. Randomly select an integer number N between 0 and the cutoff Km. Step 2. Randomly select N items in transaction Tj and put them in the empty

transaction Tj .

Step 3. Select items in the remaining set E = {I1 , I2 , …, Ik } with probability ρm and continue to fill them in Tj .

In the above, transaction Tj has size m, and two parameters Km and ρm are optimized based on the desired level of privacy.

Using the notations as defined above, itemset I and X = {T1, …, Tn} were defined as |I| = d and |Ti| = m, i = 1, …, n. Then, using the definition of support given in Equation 8.3, the authors obtained partial support for itemset I as follows: ̂s =

(s0, …, sd) and sl = #{Tj X|#(I ∩ Tj) = l }/n.

They proposed a transition matrix M, to transform the original support of itemset I, ̂s , to the randomized support of its subsets, ̂s ′. Matrix M has d rows and m columns, and the following definition for each element at row l ′ and column l:

where E(̂s ′) denotes the expected value of the randomized partial support vector ̂s ′ and ̂s ′ follows multinomial distribution. Using Equation 8.6, the original partial support vector can obtain its unbiased estimator as

and the variance matrix for this estimator as

d

Cov(suest ) = n1 ∑sl P −1 D [l ](P −1 )T , (8.8)

l =1

where matrix D[l] has k + 1 rows and k + 1 columns, and each element D [l ]iD , jD =

PiD ,l δiD = jD − PiD ,l PjD ,l , iD = 1, ..., d , jD = 1, ..., m .

Privacy-Preserving Data Mining    191

Using the above randomization and randomized support analysis, the privacy preservation apriori association rules can be mined in the following steps:

Step 1. Initialize k = 1.

Step 2. Randomize the original data, and estimate partial support and variance σ2 of each candidate set using Equations 8.7 and 8.8.

Step 3. Keep those candidate sets that contain support values above the minimum support threshold smin.

Step 4. Form new candidate sets with all (k + 1)-sized itemsets that have k-sized sub-itemsets with all support values above smin − σ.

Step 5. Return to Step 2 with k = k + 1 and repeat steps until no candidate remains or the estimator deviates from the expected value significantly.

As shown in Figure 8.6, Evifimievski et al. proposed a framework for privacy preservation association rules mining from transactions by randomization methods. They evaluated the privacy breach level by checking Equation 8.1 and obtained the conditional probability by

where sl+ = P (#(Tj ∩ I )= l , Ir T j ), s0+ = 0 .

Collection of original data

Randomization of the original data

PP association rules mining

Evaluation of privacy breach

Figure 8.6 Workflow of privacy preservation association rules mining method. (Reprinted from Inform. Syst., 29, Privacy preserving mining of association rules, Evfimievski, A., Srikant, R., Agrawal, R., and Gehrke, J., 343–364, Copyright (2004), with permission from Elsevier.)

Privacy-Preserving Data Mining    195

data sets SA, held by Alice, and SB, held by Bob, and prevent privacy leaks between Alice and Bob. Each participant, Alice or Bob, only receives knowledge of the BN structure but no confidential information from the other partner.

Yang and Wright (2006) presented a protocol to construct a BN for vertically partitioned data. We use the notations and definitions presented in the Section 3.3.6.1 BN classifier to simplify the description of PPBN. Two research topics exist in the BN classifier: the recognition of BN structure and the training of BN model. Wright and Yang employed the K2 algorithm (Yang and Wright, 2006) in learning the BN structure and modified the scoring function in the K2 algorithm. Given a set of v variables or nodes, X = {x1, …, xi, …, xv}, the K2 algorithm attempts to maximize the score function f(xi,parent(xi)) in the sequence of parent candidates (nodes) up to a maximum of u (the number of upper bound) parents for a node. K2 consists of the following steps.

Step 1. Initialize parent set parent (xi) to be empty for each node, i = 1, …, v.

Step 2. Update f (xi, parent (xi)) with f (xi, parent(xi)) (cparent(xi) − parent(xi)), if f (xi, parent(xi)) < f (xi, parent(xi)) (cparent(xi) − parent(xi)); otherwise stop adding parents to node xi, where cparent(xi) refers to the pool of possible parents of node xi and parent(xi) refers to the pool including the selected parent nodes.

Step 3. Iterate Steps 1–2 until node xi has obtained u parents.

Step 4. Iterate Steps 1–3 until all nodes in the BN have been added as parents.

In the original K2 algorithm, the score function has the following definition for binary attributes:

where qi denotes the number of unique parents of variable xi, Pijk,k = 0 or 1 denotes the number of occurrences of variable xi taking value k and its parent(xi) taking the jth unique value. To preserve privacies in BN computation, Wright and Yang proposed a new score function as follows:

qi

g(xi , parent(xi )) = ∑12 (ln Pij 0 + ln Pij1 − ln(Pij 0 + Pij1 + 1))

j =1

+ (Pij 0 ln Pij 0 + Pij1 ln Pij1 − (Pij 0 + Pij1 + 1)ln(Pij 0 + Pij1 + 1)).

(8.14)

196    Data Mining and Machine Learning in Cybersecurity

Yang and Wright obtained this score function via the operation of a natural log on the score function in Equation 8.13 and simplified processing with respect to Stirling’s approximation. The log function and approximation have no effects on the ordering of the parent nodes in the K1 algorithm, while the operations preserve the original score information in the new score function. Following the example above, we will use Alice and Bob to illustrate the authors’ point. In the implementation of the g score function in the K2 algorithm, Alice and Bob jointly solved the maximum g scores associated with the parent nodes. In the proposed PPNB method, privacy-preserving protocols were employed on scalar products, computations of Pijk, score computations, and score comparisons. Alice and Bob shared the intermediate values of the above four computation results.

Wright and Yang designed a privacy-preserving scalar product protocol as follows. Given private vector VA from Alice and VB from Bob, the proposed scalar product protocol consisted of the following steps:

Step 1. Alice generated an encryption key and a decryption key using an encryption algorithm, and shared the encryption with Bob.

Step 2. Alice encrypted her private data elements in VA, E(VA1), …, E(VAn), and shared the encrypted data with Bob.

Step 3. Bob encrypted a random number R, obtained the encrypted R, E(R), and shared the encrypted data E(R) ∏yi with Alice, where if VAi = 1, yi =

E(VAi); otherwise yi = 1.

Step 4. Alice derived R + VA · VB from E(R) ∏yi .

In the above, Wright and Yang demonstrated that E(R) ∏yi = E(VA VB + R) in

Step 3, such that Alice could obtain R + VA · VB using the description key while Bob only maintained R. With respect to the computations of privacy-preserving parameters Pijk, both Alice and Bob generated a respective n-length vector compatible with i, j, k. Then, both partners obtained the shared parameters Pijk via the above private scalar product protocol. The output was the random shares of parameters Pijk.

In private score computation, Wright and Yang referred to Yang and Wright

(2006) for computing random shares of ln Pijk and Pijk ln Pijk. With respect to (Pij0 + Pij1 + 1) ln (Pij0 + Pij1 + 1), they let Alice and Bob separately compute

random shares. Then, Alice and Bob selected the maximum values among the m shared score values. Yang and Wright (2006) evaluated the proposed g score function on two data sets. The first (Asia) data set included eight features such as Asia, smoking, tuberculosis, lung cancer, bronchitis, either, x-ray, and dyspnoea. The second (synthetic) data set consisted of 10,000 data points and six features denoted 0–5. Wright and Yang compared the performance of the g and f score functions in both data sets and observed that the g score function approximated the f score function sufficiently for the K2 algorithm, e.g., all g scores fall into 99.8% of ln(f ). They noted several causes of information leaks: known structure