Uploaded byhktripathy
PPTX, PDF5,638 views

Lect6 Association rule & Apriori algorithm

The document discusses the Apriori algorithm for mining association rules from transactional data. The Apriori algorithm uses a level-wise search where frequent itemsets are used to explore longer itemsets. It determines frequent itemsets by identifying individual frequent items and extending them to larger sets as long as they meet a minimum support threshold. The algorithm takes advantage of the fact that subsets of frequent itemsets must also be frequent to prune the search space. It performs candidate generation and pruning to efficiently identify all frequent itemsets in the transactional data.

Embed presentation

Downloaded 300 times
Association Rule & Apriori Algorithm
Association rule mining • Proposed by Agrawal et al in 1993. • It is an important data mining model studied extensively by the database and data mining community. • Assume all data are categorical. • No good algorithm for numeric data. • Initially used for Market Basket Analysis to find how items purchased by customers are related. Bread  Milk [sup = 5%, conf = 100%] 2
What Is Association Mining? • Motivation: finding regularities in data • What products were often purchased together? — Beer and diapers • What are the subsequent purchases after buying a PC? • What kinds of DNA are sensitive to this new drug? • Can we automatically classify web documents?
• Given a set of transactions, find rules that will predict the occurrence of an item based on the occurrences of other items in the transaction. Market-Basket transactions TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke Example of Association Rules {Diaper}  {Beer}, {Milk, Bread}  {Eggs,Coke}, {Beer, Bread}  {Milk}, Implication means co-occurrence, not causality! Association rule mining
Basket Data Retail organizations, e.g., supermarkets, collect and store massive amounts sales data, called basket data. A record consist of transaction date items bought Or, basket data may consist of items bought by a customer over a period.  Items frequently purchased together: Bread PeanutButter
Example Association Rule 90% of transactions that purchase bread and butter also purchase milk “IF” part = antecedent “THEN” part = consequent “Item set” = the items (e.g., products) comprising the antecedent or consequent • Antecedent and consequent are disjoint (i.e., have no items in common) Antecedent: bread and butter Consequent: milk Confidence factor: 90%
Transaction data: supermarket data • Market basket transactions: t1: {bread, cheese, milk} t2: {apple, eggs, salt, yogurt} … … tn: {biscuit, eggs, milk} • Concepts: • An item: an item/article in a basket • I: the set of all items sold in the store • A transaction: items purchased in a basket; it may have TID (transaction ID) • A transactional dataset: A set of transactions 7
Transaction data: a set of documents • A text document data set. Each document is treated as a “bag” of keywords doc1: Student, Teach, School doc2: Student, School doc3: Teach, School, City, Game doc4: Baseball, Basketball doc5: Basketball, Player, Spectator doc6: Baseball, Coach, Game, Team doc7: Basketball, Team, City, Game 8
Definition: Frequent Itemset • Itemset • A collection of one or more items • Example: {Milk, Bread, Diaper} • k-itemset • An itemset that contains k items • Support count () • Frequency of occurrence of an itemset • E.g. ({Milk, Bread,Diaper}) = 2 • Support • Fraction of transactions that contain an itemset • E.g. s({Milk, Bread, Diaper}) = 2/5 • Frequent Itemset • An itemset whose support is greater than or equal to a minsup threshold TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke
The model: data • I = {i1, i2, …, im}: a set of items. • Transaction t : • t a set of items, and t  I. • Transaction Database T: a set of transactions T = {t1, t2, …, tn}. 10  I: itemset {cucumber, parsley, onion, tomato, salt, bread, olives, cheese, butter}  T: set of transactions 1 {{cucumber, parsley, onion, tomato, salt, bread}, 2 {tomato, cucumber, parsley}, 3 {tomato, cucumber, olives, onion, parsley}, 4 {tomato, cucumber, onion, bread}, 5 {tomato, salt, onion}, 6 {bread, cheese} 7 {tomato, cheese, cucumber} 8 {bread, butter}}
The model: Association rules • A transaction t contains X, a set of items (itemset) in I, if X  t. • An association rule is an implication of the form: X  Y, where X, Y  I, and X Y =  • An itemset is a set of items. • E.g., X = {milk, bread, cereal} is an itemset. • A k-itemset is an itemset with k items. • E.g., {milk, bread, cereal} is a 3-itemset 11
Rule strength measures • Support: The rule holds with support sup in T (the transaction data set) if sup% of transactions contain X  Y. • sup = probability that a transaction contains Pr(X  Y) (Percentage of transactions that contain X  Y) • Confidence: The rule holds in T with confidence conf if conf% of tranactions that contain X also contain Y. • conf = conditional probability that a transaction having X also contains Y Pr(Y | X) (Ratio of number of transactions that contain X  Y to the number that contain X) • An association rule is a pattern that states when X occurs, Y occurs with certain probability. 12
Support and Confidence • Support count: The support count of an itemset X, denoted by X.count, in a data set T is the number of transactions in T that contain X. Assume T has n transactions. • Then, n countYX support ).(   countX countYX confidence . ).(   13 Goal: Find all rules that satisfy the user-specified minimum support (minsup) and minimum confidence (minconf).
Example: Beer}Diaper,Milk{  4.0 5 2 |T| )BeerDiaper,,Milk(   s 67.0 3 2 )Diaper,Milk( )BeerDiaper,Milk,(    c  Association Rule An implication expression of the form X  Y, where X and Y are itemsets Example: {Milk, Diaper}  {Beer}  Rule Evaluation Metrics Support (s)  Fraction of transactions that contain both X and Y Confidence (c)  Measures how often items in Y appear in transactions that contain X TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke Definition: Association Rule
Is minimum support and minimum confidence can be automatically determined in mining association rules? • For the mininmum support, it all depends on the dataset. Usually, may start with a high value, and then decrease the values until to find a value that will generate enough paterns. • For the minimum confidence, it is a little bit easier because it represents the confidence that you want in the rules. So usually, use something like 60 % . But it also depends on the data. • In terms of performance, when minsup is higher you will find less pattern and the algorithm is faster. For minconf, when it is set higher, there will be less pattern but it may not be faster because many algorithms don't use minconf to prune the search space. So obviously, setting these parameters also depends on how many rules you want.
An example • Transaction data • Assume: minsup = 30% minconf = 80% • An example frequent itemset: {Chicken, Clothes, Milk} [sup = 3/7] • Association rules from the itemset: Clothes  Milk, Chicken [sup = 3/7, conf = 3/3] … … Clothes, Chicken  Milk, [sup = 3/7, conf = 3/3] t1: Bread, Chicken, Milk t2: Bread, Cheese t3: Cheese, Boots t4: Bread, Chicken, Cheese t5: Bread, Chicken, Clothes, Cheese, Milk t6: Chicken, Clothes, Milk t7: Chicken, Milk, Clothes 16
Basic Concept: Association Rules  Let min_support = 50%, min_conf = 50%:  A  C (50%, 66.7%)  C  A (50%, 100%) Customer buys diaper Customer buys both Customer buys beer Transaction-id Items bought 10 A, B, C 20 A, C 30 A, D 40 B, E, F Frequent pattern Support {A} 75% {B} 50% {C} 50% {A, C} 50%
Association Rule Mining Task • Given a set of transactions T, the goal of association rule mining is to find all rules having • support ≥ minsup threshold • confidence ≥ minconf threshold • Brute-force approach: • List all possible association rules • Compute the support and confidence for each rule • Prune rules that fail the minsup and minconf thresholds  Computationally prohibitive!
Frequent Itemset Generation • Brute-force approach: • Each itemset in the lattice is a candidate frequent itemset • Count the support of each candidate by scanning the database • Match each transaction against every candidate • Complexity ~ O(NMw) => Expensive since M = 2d !!! TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke N Transactions List of Candidates M w
Brute-force approach: Given d items, there are 2d possible candidate itemsets
Computational Complexity • Given d unique items: • Total number of itemsets = 2d • Total number of possible association rules: 123 1 1 1 1                             dd d k kd j j kd k d R If d=6, R = 602 rules
Mining Association Rules • Two-step approach: 1. Frequent Itemset Generation – Generate all itemsets whose support  minsup if an itemset is frequent, each of its subsets is frequent as well.  This property belongs to a special category of properties called antimonotonicity in the sense that if a set cannot pass a test, all of its supersets will fail the same test as well. 1. Rule Generation – Generate high confidence rules from each frequent itemset, where each rule is a binary partitioning of a frequent itemset • Frequent itemset generation is still computationally expensive
Frequent Itemset Generation • An itemset X is closed in a data set D if there exists no proper super- itemset Y* such that Y has the same support count as X in D. *(Y is a proper super-itemset of X if X is a proper sub-itemset of Y, that is, if X  Y. In other words, every item of X is contained in Y but there is at least one item of Y that is not in X.) • An itemset X is a closed frequent itemset in set D if X is both closed and frequent in D. • An itemset X is a maximal frequent itemset (or max-itemset) in a data set D if X is frequent, and there exists no super-itemset Y such that X  Y and Y is frequent in D.
Frequent Itemset Generation Strategies • Reduce the number of candidates (M) • Complete search: M=2d • Use pruning techniques to reduce M • Reduce the number of transactions (N) • Reduce size of N as the size of itemset increases • Used by DHP (Direct Hashing & Purning) and vertical-based mining algorithms • Reduce the number of comparisons (NM) • Use efficient data structures to store the candidates or transactions • No need to match every candidate against every transaction
Many mining algorithms • There are a large number of them!! • They use different strategies and data structures. • Their resulting sets of rules are all the same. • Given a transaction data set T, and a minimum support and a minimum confident, the set of association rules existing in T is uniquely determined. • Any algorithm should find the same set of rules although their computational efficiencies and memory requirements may be different. • We study only one: the Apriori Algorithm 25
• The algorithm uses a level-wise search, where k-itemsets are used to explore (k+1)-itemsets • In this algorithm, frequent subsets are extended one item at a time (this step is known as candidate generation process) • Then groups of candidates are tested against the data. • It identifies the frequent individual items in the database and extends them to larger and larger item sets as long as those itemsets appear sufficiently often in the database. • Apriori algorithm determines frequent itemsets that can be used to determine association rules which highlight general trends in the database. The Apriori algorithm
• The Apriori algorithm takes advantage of the fact that any subset of a frequent itemset is also a frequent itemset. • i.e., if {l1,l2} is a frequent itemset, then {l1} and {l2} should be frequent itemsets. • The algorithm can therefore, reduce the number of candidates being considered by only exploring the itemsets whose support count is greater than the minimum support count. • All infrequent itemsets can be pruned if it has an infrequent subset. The Apriori algorithm
• So we build a Candidate list of k-itemsets and then extract a Frequent list of k-itemsets using the support count • After that, we use the Frequent list of k-itemsets in determing the Candidate and Frequent list of k+1-itemsets. • We use Pruning to do that • We repeat until we have an empty Candidate or Frequent of k- itemsets • Then we return the list of k-1-itemsets. How do we do that?
KEY CONCEPTS • Frequent Itemsets: All the sets which contain the item with the minimum support (denoted by L𝑖 for ith itemset). • Apriori Property: Any subset of frequent itemset must be frequent. • Join Operation: To find Lk , a set of candidate k-itemsets is generated by joining Lk-1 with itself.
APRIORI ALGORITHM EXAMPLE
The Apriori Algorithm : Pseudo Code
Apriori’s Candidate Generation • For k=1, C1 = all 1-itemsets. • For k>1, generate Ck from Lk-1 as follows: – The join step Ck = k-2 way join of Lk-1 with itself If both {a1, …,ak-2, ak-1} & {a1, …, ak-2, ak} are in Lk-1, then add {a1, …,ak-2, ak-1, ak} to Ck (We keep items sorted). – The prune step Remove {a1, …,ak-2, ak-1, ak} if it contains a non-frequent (k-1) subset
Example – Finding frequent itemsets Dataset D TID Items T100 a1 a3 a4 T200 a2 a3 a5 T300 a1 a2 a3 a5 T400 a2 a5 1. scan D C1: a1:2, a2:3, a3:3, a4:1, a5:3  L1: a1:2, a2:3, a3:3, a5:3  C2: a1a2, a1a3, a1a5, a2a3, a2a5, a3a5 2. scan D  C2: a1a2:1, a1a3:2, a1a5:1, a2a3:2, a2a5:3, a3a5:2  L2: a1a3:2, a2a3:2, a2a5:3, a3a5:2  C3: a1a2a3, a2a3a5  Pruned C3: a1a2a3 3. scan D  L3: a2a3a5:2 minSup=0.5
Order of items can make difference in porcess Dataset D TID Items T100 1 3 4 T200 2 3 5 T300 1 2 3 5 T400 2 5 minSup=0.5 1. scan D  C1: 1:2, 2:3, 3:3, 4:1, 5:3  L1: 1:2, 2:3, 3:3, 5:3  C2: 12, 13, 15, 23, 25, 35 2. scan D  C2: 12:1, 13:2, 15:1, 23:2, 25:3, 35:2 Suppose the order of items is: 5,4,3,2,1  L2: 31:2, 32:2, 52:3, 53:2  C3: 321, 532  Pruned C3: 532 3. scan D  L3: 532:2
Generating Association Rules From frequent itemsets • Procedure 1: • Let we have the list of frequent itemsets • Generate all nonempty subsets for each frequent itemset I • For I = {1,3,5}, all nonempty subsets are {1,3},{1,5},{3,5},{1},{3},{5} • For I = {2,3,5}, all nonempty subsets are {2,3},{2,5},{3,5},{2},{3},{5}
• Procedure 2: • For every nonempty subset S of I, output the rule: S → (I - S) • If support_count(I)/support_count(s)>= min_conf where min_conf is minimum confidence threshold • Let us assume: • minimum confidence threshold is 60% Generating Association Rules From frequent itemsets
Association Rules with confidence • R1 : 1,3 -> 5 – Confidence = sc{1,3,5}/sc{1,3} = 2/3 = 66.66% (R1 is selected) • R2 : 1,5 -> 3 – Confidence = sc{1,5,3}/sc{1,5} = 2/2 = 100% (R2 is selected) • R3 : 3,5 -> 1 – Confidence = sc{3,5,1}/sc{3,5} = 2/3 = 66.66% (R3 is selected) • R4 : 1 -> 3,5 – Confidence = sc{1,3,5}/sc{1} = 2/3 = 66.66% (R4 is selected) • R5 : 3 -> 1,5 – Confidence = sc{3,1,5}/sc{3} = 2/4 = 50% (R5 is REJECTED) • R6 : 5 -> 1,3 – Confidence = sc{5,1,3}/sc{5} = 2/4 = 50% (R6 is REJECTED)
How to efficiently generate rules? • In general, confidence does not have an anti-monotone property c(ABC→D) can be larger or smaller than c(AB →D) • But confidence of rules generated from the same itemset has an anti- monotone property • e.g., L= {A,B,C,D} c(ABC→D) ≥ c(AB→CD) ≥ c(A→BCD) Confidence is anti-monotone w.r.t number of items on the RHS of the rule.
Rule generation for Apriori Algorithm
Rule generation for Apriori Algorithm Pruned the Rule
• Cdidate rule is generated by merging two rules that share the same prefix in the rule consequent • join (CD=>AB, BD=>AC) would produce the candidate rule, D=>ABC • Prune rule D=>ABC if its subset AD=>BC does not have high confidence Rule generation for Apriori Algorithm

More Related Content

PPTX
Association rule mining and Apriori algorithm
byhina firdaus
 
PDF
Lecture13 - Association Rules
byAlbert Orriols-Puig
 
PPTX
Association Rule Mining in Data Mining.pptx
bylahiruherath654
 
PPSX
Frequent itemset mining methods
byProf.Nilesh Magar
 
PPT
Chapter 8. Classification Basic Concepts.ppt
bySubrata Kumer Paul
 
PPT
Association rule mining
byAcad
 
PPT
3. mining frequent patterns
byAzad public school
 
PPT
Data Mining: Concepts and Techniques_ Chapter 6: Mining Frequent Patterns, ...
bySalah Amean
 
Association rule mining and Apriori algorithm
byhina firdaus
 
Lecture13 - Association Rules
byAlbert Orriols-Puig
 
Association Rule Mining in Data Mining.pptx
bylahiruherath654
 
Frequent itemset mining methods
byProf.Nilesh Magar
 
Chapter 8. Classification Basic Concepts.ppt
bySubrata Kumer Paul
 
Association rule mining
byAcad
 
3. mining frequent patterns
byAzad public school
 
Data Mining: Concepts and Techniques_ Chapter 6: Mining Frequent Patterns, ...
bySalah Amean
 

What's hot

PPTX
Apriori algorithm
byMainul Hassan
 
PDF
Data Mining: Association Rules Basics
byBenazir Income Support Program (BISP)
 
PPT
Disjoint sets
byCore Condor
 
PPTX
Association Rule Learning Part 1: Frequent Itemset Generation
byKnoldus Inc.
 
PPTX
Fp growth
byFarah M. Altufaili
 
PPTX
Data mining fp growth
byShihab Rahman
 
PPTX
Assosiate rule mining
byTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
 
PPTX
k medoid clustering.pptx
byRoshan86572
 
PPTX
Apriori Algorithm
byInternational School of Engineering
 
PPTX
Decision tree induction \ Decision Tree Algorithm with Example| Data science
byMaryamRehman6
 
PPTX
Data mining: Classification and prediction
byDataminingTools Inc
 
PPTX
Naive bayes
byAshraf Uddin
 
PPT
Fp growth algorithm
byPradip Kumar
 
PPTX
Association rule mining
byUtkarsh Sharma
 
PPTX
Classification techniques in data mining
byKamal Acharya
 
PPT
Decision tree and random forest
byLippo Group Digital
 
PDF
Data Science - Part V - Decision Trees & Random Forests
byDerek Kane
 
PPTX
Data mining techniques unit III
bymalathieswaran29
 
PPTX
3 Data Mining Tasks
byMahmoud Alfarra
 
PPTX
Mining Association Rules in Large Database
byEr. Nawaraj Bhandari
 
Apriori algorithm
byMainul Hassan
 
Data Mining: Association Rules Basics
byBenazir Income Support Program (BISP)
 
Disjoint sets
byCore Condor
 
Association Rule Learning Part 1: Frequent Itemset Generation
byKnoldus Inc.
 
Fp growth
byFarah M. Altufaili
 
Data mining fp growth
byShihab Rahman
 
Assosiate rule mining
byTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
 
k medoid clustering.pptx
byRoshan86572
 
Apriori Algorithm
byInternational School of Engineering
 
Decision tree induction \ Decision Tree Algorithm with Example| Data science
byMaryamRehman6
 
Data mining: Classification and prediction
byDataminingTools Inc
 
Naive bayes
byAshraf Uddin
 
Fp growth algorithm
byPradip Kumar
 
Association rule mining
byUtkarsh Sharma
 
Classification techniques in data mining
byKamal Acharya
 
Decision tree and random forest
byLippo Group Digital
 
Data Science - Part V - Decision Trees & Random Forests
byDerek Kane
 
Data mining techniques unit III
bymalathieswaran29
 
3 Data Mining Tasks
byMahmoud Alfarra
 
Mining Association Rules in Large Database
byEr. Nawaraj Bhandari
 

Similar to Lect6 Association rule & Apriori algorithm

PPTX
[email protected] .
bySaiM947604
 
PDF
AssociationRule.pdf
byWailaBaba
 
PPTX
Association Analysis in Data Mining
byKamal Acharya
 
PPT
dm14-association-rules (BahanAR-3).ppt
byMalaKetaren
 
PPTX
MODULE 5 _ Mining frequent patterns and associations.pptx
bynikshaikh786
 
PDF
BIM Data Mining Unit4 by Tekendra Nath Yogi
byTekendra Nath Yogi
 
PPTX
Rules of data mining
bySulman Ahmed
 
PPTX
Apriori and FP Growth in Data Warehousing DM .pptx
byrathishresto
 
PPT
Pert 06 association rules
byaiiniR
 
PDF
Mining Frequent Patterns And Association Rules
byRashmi Bhat
 
PPT
Association-rules in data analytiics.ppt
bykavita20193
 
PPTX
3 association rule mining
byVishal Dutt
 
PPTX
ASSOCIATION Rule plus MArket basket Analysis.pptx
bySherishJaved
 
PPTX
Association 04.03.14
byrahulmath80
 
DOCX
Data Warehouse and Data Mining Unit III.docx
byUjjwala Sachin Patil
 
PDF
Association Rule Mining(ARM) notes for class
byracoon97970301
 
PPTX
1.pptx .
bySaiM947604
 
PPTX
Dma unit 2
bythamizh arasi
 
PDF
associationunit3 - Copyassociationunit3 - Copyassociationunit3 - Copyassociat...
bySSMRITI1
 
PPT
Data mining arm-2009-v0
byPrithwis Mukerjee
 
[email protected] .
bySaiM947604
 
AssociationRule.pdf
byWailaBaba
 
Association Analysis in Data Mining
byKamal Acharya
 
dm14-association-rules (BahanAR-3).ppt
byMalaKetaren
 
MODULE 5 _ Mining frequent patterns and associations.pptx
bynikshaikh786
 
BIM Data Mining Unit4 by Tekendra Nath Yogi
byTekendra Nath Yogi
 
Rules of data mining
bySulman Ahmed
 
Apriori and FP Growth in Data Warehousing DM .pptx
byrathishresto
 
Pert 06 association rules
byaiiniR
 
Mining Frequent Patterns And Association Rules
byRashmi Bhat
 
Association-rules in data analytiics.ppt
bykavita20193
 
3 association rule mining
byVishal Dutt
 
ASSOCIATION Rule plus MArket basket Analysis.pptx
bySherishJaved
 
Association 04.03.14
byrahulmath80
 
Data Warehouse and Data Mining Unit III.docx
byUjjwala Sachin Patil
 
Association Rule Mining(ARM) notes for class
byracoon97970301
 
1.pptx .
bySaiM947604
 
Dma unit 2
bythamizh arasi
 
associationunit3 - Copyassociationunit3 - Copyassociationunit3 - Copyassociat...
bySSMRITI1
 
Data mining arm-2009-v0
byPrithwis Mukerjee
 

More from hktripathy

PPTX
Lect 3 background mathematics
byhktripathy
 
PDF
Lect 2 getting to know your data
byhktripathy
 
PDF
Lect 1 introduction
byhktripathy
 
PDF
Lecture7.1 data sampling
byhktripathy
 
PDF
Lecture5 virtualization
byhktripathy
 
PDF
Lecture6 introduction to data streams
byhktripathy
 
PDF
Lecture4 big data technology foundations
byhktripathy
 
PDF
Lecture3 business intelligence
byhktripathy
 
PDF
Lecture2 big data life cycle
byhktripathy
 
PDF
Lecture1 introduction to big data
byhktripathy
 
PPTX
Lect9 Decision tree
byhktripathy
 
PPTX
Lect8 Classification & prediction
byhktripathy
 
PPTX
Lect7 Association analysis to correlation analysis
byhktripathy
 
PPTX
Lect5 principal component analysis
byhktripathy
 
PPTX
Lect4 principal component analysis-I
byhktripathy
 
PPTX
Lect 3 background mathematics for Data Mining
byhktripathy
 
PPTX
Lect 2 getting to know your data
byhktripathy
 
PPTX
Lect 1 introduction
byhktripathy
 
Lect 3 background mathematics
byhktripathy
 
Lect 2 getting to know your data
byhktripathy
 
Lect 1 introduction
byhktripathy
 
Lecture7.1 data sampling
byhktripathy
 
Lecture5 virtualization
byhktripathy
 
Lecture6 introduction to data streams
byhktripathy
 
Lecture4 big data technology foundations
byhktripathy
 
Lecture3 business intelligence
byhktripathy
 
Lecture2 big data life cycle
byhktripathy
 
Lecture1 introduction to big data
byhktripathy
 
Lect9 Decision tree
byhktripathy
 
Lect8 Classification & prediction
byhktripathy
 
Lect7 Association analysis to correlation analysis
byhktripathy
 
Lect5 principal component analysis
byhktripathy
 
Lect4 principal component analysis-I
byhktripathy
 
Lect 3 background mathematics for Data Mining
byhktripathy
 
Lect 2 getting to know your data
byhktripathy
 
Lect 1 introduction
byhktripathy
 

Recently uploaded

PDF
Blood Group Incompatibility: Rh Factor and Erythroblastosis Fetalis
bySudhandraKarthi
 
PPTX
Coefficient of Multiple Correlation - R₁.₂₃, R₂.₁₃ & R₃.₁₂
bySundar B N
 
PPTX
PARENTAL ROUTES OF DRUGS ADMINISTRATION .pptx
byAneetaSharma15
 
PDF
Types of eggs and Egg membranes (Developmental Zoology)
bySudhandraKarthi
 
PPTX
Brahma_Muhurta_Presentation Ayurvedic Perspective
bySudha Sumathy
 
PPTX
META-ANALYSIS INTERPRETATION, PUBLICATION BIAS AND GRADE ASSESSMENT.pptx
bySystematic Reviews Network (SRN)
 
PPTX
Partial Correlation of Coefficient - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂
bySundar B N
 
PDF
Cultivating Greatness Pune's Best Preschools and Schools.pdf
byWellington College
 
PDF
Environmental Studies Module 4 BBA,BCCA Sem 1 NEP.pdf
byJayanti Pande
 
PPTX
ATTENTION -PART 2.pptx Shilpa Hotakar for I semester BSc students
bySHILPA HOTAKAR
 
PDF
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
PPTX
Isolation, Identification and Analysis of phytoconstituents.
byD. Y. Patil College of Pharmacy, Kolhapur.
 
PPTX
cis.jackson.alaysia.ResearchPresentation 1.Microsoft Teams vs Zoom vs Webex v...
byMerritt College
 
PPTX
ORAL & SUBLINGUAL ROUTE OF DRUGS ADMINISTRATION.pptx
byAneetaSharma15
 
PPTX
GROWTH & DEVELOPMET MILESTONES (TODDLER).pptx
byAneetaSharma15
 
PPTX
Access partner report in Odoo 18.2 _ Odoo 19
byCeline George
 
PDF
The voice of the rain poem class 11th.pdf
byReeta Baral
 
PPTX
Introduction-to-Anatomy-and-Physiology.pptx
byAshish Umale
 
PPTX
Multiple Correlation - Introduction, Meaning, Examples, Options, Formulas and...
bySundar B N
 
PPTX
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 
Blood Group Incompatibility: Rh Factor and Erythroblastosis Fetalis
bySudhandraKarthi
 
Coefficient of Multiple Correlation - R₁.₂₃, R₂.₁₃ & R₃.₁₂
bySundar B N
 
PARENTAL ROUTES OF DRUGS ADMINISTRATION .pptx
byAneetaSharma15
 
Types of eggs and Egg membranes (Developmental Zoology)
bySudhandraKarthi
 
Brahma_Muhurta_Presentation Ayurvedic Perspective
bySudha Sumathy
 
META-ANALYSIS INTERPRETATION, PUBLICATION BIAS AND GRADE ASSESSMENT.pptx
bySystematic Reviews Network (SRN)
 
Partial Correlation of Coefficient - Values of r₁₂.₃, r₂₃.₁ & r₁₃.₂
bySundar B N
 
Cultivating Greatness Pune's Best Preschools and Schools.pdf
byWellington College
 
Environmental Studies Module 4 BBA,BCCA Sem 1 NEP.pdf
byJayanti Pande
 
ATTENTION -PART 2.pptx Shilpa Hotakar for I semester BSc students
bySHILPA HOTAKAR
 
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
Isolation, Identification and Analysis of phytoconstituents.
byD. Y. Patil College of Pharmacy, Kolhapur.
 
cis.jackson.alaysia.ResearchPresentation 1.Microsoft Teams vs Zoom vs Webex v...
byMerritt College
 
ORAL & SUBLINGUAL ROUTE OF DRUGS ADMINISTRATION.pptx
byAneetaSharma15
 
GROWTH & DEVELOPMET MILESTONES (TODDLER).pptx
byAneetaSharma15
 
Access partner report in Odoo 18.2 _ Odoo 19
byCeline George
 
The voice of the rain poem class 11th.pdf
byReeta Baral
 
Introduction-to-Anatomy-and-Physiology.pptx
byAshish Umale
 
Multiple Correlation - Introduction, Meaning, Examples, Options, Formulas and...
bySundar B N
 
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 

Lect6 Association rule & Apriori algorithm

  • 1.
    Association Rule &Apriori Algorithm
  • 2.
    Association rule mining •Proposed by Agrawal et al in 1993. • It is an important data mining model studied extensively by the database and data mining community. • Assume all data are categorical. • No good algorithm for numeric data. • Initially used for Market Basket Analysis to find how items purchased by customers are related. Bread  Milk [sup = 5%, conf = 100%] 2
  • 3.
    What Is AssociationMining? • Motivation: finding regularities in data • What products were often purchased together? — Beer and diapers • What are the subsequent purchases after buying a PC? • What kinds of DNA are sensitive to this new drug? • Can we automatically classify web documents?
  • 4.
    • Given aset of transactions, find rules that will predict the occurrence of an item based on the occurrences of other items in the transaction. Market-Basket transactions TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke Example of Association Rules {Diaper}  {Beer}, {Milk, Bread}  {Eggs,Coke}, {Beer, Bread}  {Milk}, Implication means co-occurrence, not causality! Association rule mining
  • 5.
    Basket Data Retail organizations,e.g., supermarkets, collect and store massive amounts sales data, called basket data. A record consist of transaction date items bought Or, basket data may consist of items bought by a customer over a period.  Items frequently purchased together: Bread PeanutButter
  • 6.
    Example Association Rule 90%of transactions that purchase bread and butter also purchase milk “IF” part = antecedent “THEN” part = consequent “Item set” = the items (e.g., products) comprising the antecedent or consequent • Antecedent and consequent are disjoint (i.e., have no items in common) Antecedent: bread and butter Consequent: milk Confidence factor: 90%
  • 7.
    Transaction data: supermarketdata • Market basket transactions: t1: {bread, cheese, milk} t2: {apple, eggs, salt, yogurt} … … tn: {biscuit, eggs, milk} • Concepts: • An item: an item/article in a basket • I: the set of all items sold in the store • A transaction: items purchased in a basket; it may have TID (transaction ID) • A transactional dataset: A set of transactions 7
  • 8.
    Transaction data: aset of documents • A text document data set. Each document is treated as a “bag” of keywords doc1: Student, Teach, School doc2: Student, School doc3: Teach, School, City, Game doc4: Baseball, Basketball doc5: Basketball, Player, Spectator doc6: Baseball, Coach, Game, Team doc7: Basketball, Team, City, Game 8
  • 9.
    Definition: Frequent Itemset •Itemset • A collection of one or more items • Example: {Milk, Bread, Diaper} • k-itemset • An itemset that contains k items • Support count () • Frequency of occurrence of an itemset • E.g. ({Milk, Bread,Diaper}) = 2 • Support • Fraction of transactions that contain an itemset • E.g. s({Milk, Bread, Diaper}) = 2/5 • Frequent Itemset • An itemset whose support is greater than or equal to a minsup threshold TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke
  • 10.
    The model: data •I = {i1, i2, …, im}: a set of items. • Transaction t : • t a set of items, and t  I. • Transaction Database T: a set of transactions T = {t1, t2, …, tn}. 10  I: itemset {cucumber, parsley, onion, tomato, salt, bread, olives, cheese, butter}  T: set of transactions 1 {{cucumber, parsley, onion, tomato, salt, bread}, 2 {tomato, cucumber, parsley}, 3 {tomato, cucumber, olives, onion, parsley}, 4 {tomato, cucumber, onion, bread}, 5 {tomato, salt, onion}, 6 {bread, cheese} 7 {tomato, cheese, cucumber} 8 {bread, butter}}
  • 11.
    The model: Associationrules • A transaction t contains X, a set of items (itemset) in I, if X  t. • An association rule is an implication of the form: X  Y, where X, Y  I, and X Y =  • An itemset is a set of items. • E.g., X = {milk, bread, cereal} is an itemset. • A k-itemset is an itemset with k items. • E.g., {milk, bread, cereal} is a 3-itemset 11
  • 12.
    Rule strength measures •Support: The rule holds with support sup in T (the transaction data set) if sup% of transactions contain X  Y. • sup = probability that a transaction contains Pr(X  Y) (Percentage of transactions that contain X  Y) • Confidence: The rule holds in T with confidence conf if conf% of tranactions that contain X also contain Y. • conf = conditional probability that a transaction having X also contains Y Pr(Y | X) (Ratio of number of transactions that contain X  Y to the number that contain X) • An association rule is a pattern that states when X occurs, Y occurs with certain probability. 12
  • 13.
    Support and Confidence •Support count: The support count of an itemset X, denoted by X.count, in a data set T is the number of transactions in T that contain X. Assume T has n transactions. • Then, n countYX support ).(   countX countYX confidence . ).(   13 Goal: Find all rules that satisfy the user-specified minimum support (minsup) and minimum confidence (minconf).
  • 14.
    Example: Beer}Diaper,Milk{  4.0 5 2 |T| )BeerDiaper,,Milk(   s 67.0 3 2 )Diaper,Milk( )BeerDiaper,Milk,(    c  AssociationRule An implication expression of the form X  Y, where X and Y are itemsets Example: {Milk, Diaper}  {Beer}  Rule Evaluation Metrics Support (s)  Fraction of transactions that contain both X and Y Confidence (c)  Measures how often items in Y appear in transactions that contain X TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke Definition: Association Rule
  • 15.
    Is minimum supportand minimum confidence can be automatically determined in mining association rules? • For the mininmum support, it all depends on the dataset. Usually, may start with a high value, and then decrease the values until to find a value that will generate enough paterns. • For the minimum confidence, it is a little bit easier because it represents the confidence that you want in the rules. So usually, use something like 60 % . But it also depends on the data. • In terms of performance, when minsup is higher you will find less pattern and the algorithm is faster. For minconf, when it is set higher, there will be less pattern but it may not be faster because many algorithms don't use minconf to prune the search space. So obviously, setting these parameters also depends on how many rules you want.
  • 16.
    An example • Transactiondata • Assume: minsup = 30% minconf = 80% • An example frequent itemset: {Chicken, Clothes, Milk} [sup = 3/7] • Association rules from the itemset: Clothes  Milk, Chicken [sup = 3/7, conf = 3/3] … … Clothes, Chicken  Milk, [sup = 3/7, conf = 3/3] t1: Bread, Chicken, Milk t2: Bread, Cheese t3: Cheese, Boots t4: Bread, Chicken, Cheese t5: Bread, Chicken, Clothes, Cheese, Milk t6: Chicken, Clothes, Milk t7: Chicken, Milk, Clothes 16
  • 17.
    Basic Concept: AssociationRules  Let min_support = 50%, min_conf = 50%:  A  C (50%, 66.7%)  C  A (50%, 100%) Customer buys diaper Customer buys both Customer buys beer Transaction-id Items bought 10 A, B, C 20 A, C 30 A, D 40 B, E, F Frequent pattern Support {A} 75% {B} 50% {C} 50% {A, C} 50%
  • 18.
    Association Rule MiningTask • Given a set of transactions T, the goal of association rule mining is to find all rules having • support ≥ minsup threshold • confidence ≥ minconf threshold • Brute-force approach: • List all possible association rules • Compute the support and confidence for each rule • Prune rules that fail the minsup and minconf thresholds  Computationally prohibitive!
  • 19.
    Frequent Itemset Generation •Brute-force approach: • Each itemset in the lattice is a candidate frequent itemset • Count the support of each candidate by scanning the database • Match each transaction against every candidate • Complexity ~ O(NMw) => Expensive since M = 2d !!! TID Items 1 Bread, Milk 2 Bread, Diaper, Beer, Eggs 3 Milk, Diaper, Beer, Coke 4 Bread, Milk, Diaper, Beer 5 Bread, Milk, Diaper, Coke N Transactions List of Candidates M w
  • 20.
    Brute-force approach: Given ditems, there are 2d possible candidate itemsets
  • 21.
    Computational Complexity • Givend unique items: • Total number of itemsets = 2d • Total number of possible association rules: 123 1 1 1 1                             dd d k kd j j kd k d R If d=6, R = 602 rules
  • 22.
    Mining Association Rules •Two-step approach: 1. Frequent Itemset Generation – Generate all itemsets whose support  minsup if an itemset is frequent, each of its subsets is frequent as well.  This property belongs to a special category of properties called antimonotonicity in the sense that if a set cannot pass a test, all of its supersets will fail the same test as well. 1. Rule Generation – Generate high confidence rules from each frequent itemset, where each rule is a binary partitioning of a frequent itemset • Frequent itemset generation is still computationally expensive
  • 23.
    Frequent Itemset Generation •An itemset X is closed in a data set D if there exists no proper super- itemset Y* such that Y has the same support count as X in D. *(Y is a proper super-itemset of X if X is a proper sub-itemset of Y, that is, if X  Y. In other words, every item of X is contained in Y but there is at least one item of Y that is not in X.) • An itemset X is a closed frequent itemset in set D if X is both closed and frequent in D. • An itemset X is a maximal frequent itemset (or max-itemset) in a data set D if X is frequent, and there exists no super-itemset Y such that X  Y and Y is frequent in D.
  • 24.
    Frequent Itemset GenerationStrategies • Reduce the number of candidates (M) • Complete search: M=2d • Use pruning techniques to reduce M • Reduce the number of transactions (N) • Reduce size of N as the size of itemset increases • Used by DHP (Direct Hashing & Purning) and vertical-based mining algorithms • Reduce the number of comparisons (NM) • Use efficient data structures to store the candidates or transactions • No need to match every candidate against every transaction
  • 25.
    Many mining algorithms •There are a large number of them!! • They use different strategies and data structures. • Their resulting sets of rules are all the same. • Given a transaction data set T, and a minimum support and a minimum confident, the set of association rules existing in T is uniquely determined. • Any algorithm should find the same set of rules although their computational efficiencies and memory requirements may be different. • We study only one: the Apriori Algorithm 25
  • 26.
    • The algorithmuses a level-wise search, where k-itemsets are used to explore (k+1)-itemsets • In this algorithm, frequent subsets are extended one item at a time (this step is known as candidate generation process) • Then groups of candidates are tested against the data. • It identifies the frequent individual items in the database and extends them to larger and larger item sets as long as those itemsets appear sufficiently often in the database. • Apriori algorithm determines frequent itemsets that can be used to determine association rules which highlight general trends in the database. The Apriori algorithm
  • 27.
    • The Apriorialgorithm takes advantage of the fact that any subset of a frequent itemset is also a frequent itemset. • i.e., if {l1,l2} is a frequent itemset, then {l1} and {l2} should be frequent itemsets. • The algorithm can therefore, reduce the number of candidates being considered by only exploring the itemsets whose support count is greater than the minimum support count. • All infrequent itemsets can be pruned if it has an infrequent subset. The Apriori algorithm
  • 28.
    • So webuild a Candidate list of k-itemsets and then extract a Frequent list of k-itemsets using the support count • After that, we use the Frequent list of k-itemsets in determing the Candidate and Frequent list of k+1-itemsets. • We use Pruning to do that • We repeat until we have an empty Candidate or Frequent of k- itemsets • Then we return the list of k-1-itemsets. How do we do that?
  • 29.
    KEY CONCEPTS • FrequentItemsets: All the sets which contain the item with the minimum support (denoted by L𝑖 for ith itemset). • Apriori Property: Any subset of frequent itemset must be frequent. • Join Operation: To find Lk , a set of candidate k-itemsets is generated by joining Lk-1 with itself.
  • 30.
  • 31.
    The Apriori Algorithm: Pseudo Code
  • 32.
    Apriori’s Candidate Generation •For k=1, C1 = all 1-itemsets. • For k>1, generate Ck from Lk-1 as follows: – The join step Ck = k-2 way join of Lk-1 with itself If both {a1, …,ak-2, ak-1} & {a1, …, ak-2, ak} are in Lk-1, then add {a1, …,ak-2, ak-1, ak} to Ck (We keep items sorted). – The prune step Remove {a1, …,ak-2, ak-1, ak} if it contains a non-frequent (k-1) subset
  • 33.
    Example – Findingfrequent itemsets Dataset D TID Items T100 a1 a3 a4 T200 a2 a3 a5 T300 a1 a2 a3 a5 T400 a2 a5 1. scan D C1: a1:2, a2:3, a3:3, a4:1, a5:3  L1: a1:2, a2:3, a3:3, a5:3  C2: a1a2, a1a3, a1a5, a2a3, a2a5, a3a5 2. scan D  C2: a1a2:1, a1a3:2, a1a5:1, a2a3:2, a2a5:3, a3a5:2  L2: a1a3:2, a2a3:2, a2a5:3, a3a5:2  C3: a1a2a3, a2a3a5  Pruned C3: a1a2a3 3. scan D  L3: a2a3a5:2 minSup=0.5
  • 34.
    Order of itemscan make difference in porcess Dataset D TID Items T100 1 3 4 T200 2 3 5 T300 1 2 3 5 T400 2 5 minSup=0.5 1. scan D  C1: 1:2, 2:3, 3:3, 4:1, 5:3  L1: 1:2, 2:3, 3:3, 5:3  C2: 12, 13, 15, 23, 25, 35 2. scan D  C2: 12:1, 13:2, 15:1, 23:2, 25:3, 35:2 Suppose the order of items is: 5,4,3,2,1  L2: 31:2, 32:2, 52:3, 53:2  C3: 321, 532  Pruned C3: 532 3. scan D  L3: 532:2
  • 35.
    Generating Association Rules Fromfrequent itemsets • Procedure 1: • Let we have the list of frequent itemsets • Generate all nonempty subsets for each frequent itemset I • For I = {1,3,5}, all nonempty subsets are {1,3},{1,5},{3,5},{1},{3},{5} • For I = {2,3,5}, all nonempty subsets are {2,3},{2,5},{3,5},{2},{3},{5}
  • 36.
    • Procedure 2: •For every nonempty subset S of I, output the rule: S → (I - S) • If support_count(I)/support_count(s)>= min_conf where min_conf is minimum confidence threshold • Let us assume: • minimum confidence threshold is 60% Generating Association Rules From frequent itemsets
  • 37.
    Association Rules withconfidence • R1 : 1,3 -> 5 – Confidence = sc{1,3,5}/sc{1,3} = 2/3 = 66.66% (R1 is selected) • R2 : 1,5 -> 3 – Confidence = sc{1,5,3}/sc{1,5} = 2/2 = 100% (R2 is selected) • R3 : 3,5 -> 1 – Confidence = sc{3,5,1}/sc{3,5} = 2/3 = 66.66% (R3 is selected) • R4 : 1 -> 3,5 – Confidence = sc{1,3,5}/sc{1} = 2/3 = 66.66% (R4 is selected) • R5 : 3 -> 1,5 – Confidence = sc{3,1,5}/sc{3} = 2/4 = 50% (R5 is REJECTED) • R6 : 5 -> 1,3 – Confidence = sc{5,1,3}/sc{5} = 2/4 = 50% (R6 is REJECTED)
  • 38.
    How to efficientlygenerate rules? • In general, confidence does not have an anti-monotone property c(ABC→D) can be larger or smaller than c(AB →D) • But confidence of rules generated from the same itemset has an anti- monotone property • e.g., L= {A,B,C,D} c(ABC→D) ≥ c(AB→CD) ≥ c(A→BCD) Confidence is anti-monotone w.r.t number of items on the RHS of the rule.
  • 39.
    Rule generation forApriori Algorithm
  • 40.
    Rule generation forApriori Algorithm Pruned the Rule
  • 41.
    • Cdidate ruleis generated by merging two rules that share the same prefix in the rule consequent • join (CD=>AB, BD=>AC) would produce the candidate rule, D=>ABC • Prune rule D=>ABC if its subset AD=>BC does not have high confidence Rule generation for Apriori Algorithm