Apriori: implement join step of apriori-gen

The apriori-gen function described in section 2.1.1 of Apriori paper
has two steps; first, the join step looks for itemsets with the same
prefix, and creates new candidates by appending all pairs combinations
to this prefix. Here is pseudocode copied from paper:

  select p.1, p.2, ..., p.k-1, q.k-1
  from p in L(k-1), q in L(k-1)
  where p.1 = q.1, ..., p.k-2 = q.k-2, p.k-1 < q.k-1

The reason is that if a sequence q with the same prefix as p does not
belong to L(k-1), itemset p+(q.k-1,) cannot be frequent.
Before this commit, we were considering p+(q.k-1,) for any q.k-1 > p.k-1.

The second step of apriori-gen function is called prune step, it will
be implemented in a distinct commit.

See discussion in #644.

Author: dbarbier

mlxtend/frequent_patterns/apriori.py

@@ -30,8 +30,10 @@ def generate_new_combinations(old_combinations):
 
     Returns
     -----------
-    Generator of all combinations from the last step x items
-    from the previous step.
+    Generator of combinations based on the last state of Apriori algorithm.
+    In order to reduce number of candidates, this function implements the
+    join step of apriori-gen described in section 2.1.1 of Apriori paper.
+    Prune step is not yet implemented.
 
     Examples
     -----------
@@ -40,15 +42,17 @@ def generate_new_combinations(old_combinations):
 
     """
 
-    items_types_in_previous_step = np.unique(old_combinations.flatten())
-    for old_combination in old_combinations:
-        max_combination = old_combination[-1]
-        mask = items_types_in_previous_step > max_combination
-        valid_items = items_types_in_previous_step[mask]
-        old_tuple = tuple(old_combination)
-        for item in valid_items:
-            yield from old_tuple
-            yield item
+    length = len(old_combinations)
+    for i, old_combination in enumerate(old_combinations):
+        head_i = list(old_combination[:-1])
+        j = i + 1
+        while j < length:
+            *head_j, tail_j = old_combinations[j]
+            if head_i != head_j:
+                break
+            yield from old_combination
+            yield tail_j
+            j = j + 1
 
 
 def generate_new_combinations_low_memory(old_combinations, X, min_support,