The Data Mining Forum                             open-source data mining software data science journal data mining conferences high utility mining workshop
This forum is about data mining, data science and big data: algorithms, source code, datasets, implementations, optimizations, etc. You are welcome to post call for papers, data mining job ads, link to source code of data mining algorithms or anything else related to data mining. The forum is hosted by P. Fournier-Viger. No registration is required to use this forum!.  
SPMF : Implementation of MSApriori with Hashing to increase speed of execution
Posted by: sjp12012a
Date: July 05, 2018 10:03AM

Hi,

I am working on an CS assignment problem which was referred to use SPMF libraries. In order to increase the speed of computation of MSApriori, I used hashing technique in the first pass of the transaction data scan.

I am aware that few others have already used this technique before. Please let me know how can I discuss this further.

I can submit my analysis for review here.

Regards,
Srini



Edited 1 time(s). Last edit at 07/05/2018 10:04AM by sjp12012a.

Options: ReplyQuote
Re: SPMF : Implementation of MSApriori with Hashing to increase speed of execution
Date: July 06, 2018 05:35PM

Hello,

Sure, if you want to discuss this in the forum, you can share details. Is it improving the performance by a great amount? If so, your improvement could be integrated in the SPMF library and you could become a contributor.

Best regards,

Philippe

Options: ReplyQuote
Re: SPMF : Implementation of MSApriori with Hashing to increase speed of execution
Posted by: Srini
Date: July 08, 2018 08:09PM

Hi,

It improved quite a bit, from perpetual execution to supplying a result in 15 secs on the data set that I was using.

I think the key decision is to chose an appropriate Hash function for an arbitrary number of integers. I have used a Java Bitset object for doing the same in my assignment. Please put down your suggestions on this.

I am giving the URL here where I uploaded the assignment with analysis:

https://drive.google.com/drive/folders/1GIju0UZsQGjaDqRA_HUvrHA4CebJgVNq?usp=sharing

I could work on making this generic if approved and would like to contribute to SPMF code.

Regards,
Srini.

Options: ReplyQuote


This forum is powered by Phorum and provided by P. Fournier-Viger (© 2012).
Terms of use.