Adapting Quantum Clustering (QC) and Related Algorithms to Anomaly Detection in Big Data
Quantum clustering (QC) is a successful clustering algorithm. One version of it (DQC) has already proved its ability to detect anomalies in big data. We review these successes and propose to extend the algorithm, within a novel entropy formulation, to three different methods. We propose to adapt them to big data, thus allowing for their application to problems relevant to cyber security. We also propose to employ them within Deep Neural Networks as novel exploratory tools. We plan to test our methodology within various domains, including speech processing, financial fraud and malware.
Develop the Entropy analysis and clustering tools.
- Apply the algorithms to various scientific and technical problems to gain further insights into the strengths of the different schemes.
- Develop an approximation method, first discussed in ref. 5, in order to enable fast and accurate calculations of replica dynamics.
- Use the approximation for carrying out applications to big data.
- Search in big data for string-like structures of the type uncovered by DQC analyses.
Incorporate Clustering in Deep Neural Networks.
- Improve the performance of DNN on big data
- Explore for the existence of clusters with unexpected characteristics
Use different data sets such as:
- Speech processing (in collaboration with an expert team in Afeka college)
- Financial fraud data (possible connection with Citibank)
- Malware and fraud data (collaboration with an expert team in EMC)
- Israeli CERT data, as well as cyber security data from public resources such as https://www.predict.org.