Details

Enhanced Spark Cluster Recommendation Engine Powered by Generative AI

Tanvi S Hungund

Senior Manager, Dallas TX California State University Fullerton

26-32

Vol: 14, Issue: 1, 2024

Receiving Date: 2023-12-02 Acceptance Date:

2024-01-30

Publication Date:

2024-02-13

Download PDF

http://doi.org/10.37648/ijrst.v14i01.004

Abstract

Apache Spark, renowned for its proficiency in processing vast datasets, efficiently handles intricate processing tasks. It disperses these tasks across numerous computing instances autonomously or in conjunction with other distributed computing tools. As the volume of data burgeons and machine learning models advance, the imperative for swift and intricate feature engineering and model training intensifies. Clusters comprising multiple compute instances exhibit a noteworthy performance surge compared to individual cases, expediting data processing. However, leveraging such cluster configurations entails substantial costs due to the amalgamation of multiple compute instances (Worker Nodes) overseen by a Controller Node.

Keywords: Apache Spark; Artificial Intelligence; recommender systems

References

  1. J. Wen, B. Y. Chen, C. D. Wang, and Z. Tian, “PRGAN: Personalized Recommendation with Conditional Generative Adversarial Networks,” Proc. - IEEE Int. Conf. Data Mining, ICDM, vol. 2021-Decem, no. Icdm, pp. 729–738, 2021, https://doi.org/10.1109/ICDM51629.2021.00084
  2. J. R. Bock and A. Maewal, “Adversarial Learning for Product Recommendation,” Ai, vol. 1, no. 3, pp. 376–388, 2020, https://doi.org/10.3390/ai1030025
  3. A. Akbar, P. Agarwal, and A. J. Obaid, “Recommendation engines-neural embedding to graph-based: Techniques and evaluations,” Int. J. Nonlinear Anal. Appl, vol. 13, no. 1, pp. 2008–6822, 2022, [Online]. Available: http://dx.doi.org/10.22075/ijnaa.2022.5941
  4. G. Zhu, J. Cao, C. Li, and Z. Wu, “A recommendation engine for travel products based on topic sequential patterns,” Multimed. Tools Appl., vol. 76, no. 16, pp. 17595–17612, 2017, https://doi.org/10.1007/s11042-017-4406-6
  5. Q. Wang, Q. V. H. Nguyen, H. Yin, Z. Huang, H. Wang, and L. Cui, “Enhancing collaborative filtering with generative augmentation,” Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., pp. 548–556, 2019, https://doi.org/10.1145/3292500.3330873
  6. G. Adomavicius and A. Tuzhilin, “Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions,” IEEE Trans. Knowl. Data Eng., vol. 17, no. 6, pp. 734–749, 2005, https://doi.org/10.1109/TKDE.2005.99
  7. S. Panigrahi, R. K. Lenka, and A. Stitipragyan, “A Hybrid Distributed Collaborative Filtering Recommender Engine Using Apache Spark,” Procedia Comput. Sci., vol. 83, no. BigD2M, pp. 1000–1006, 2016, https://doi.org/10.1016/j.procs.2016.04.214
  8. D. Liu, G. P. Farajalla, and A. Boulenger, “BRec the Bank: Context-aware Self-attentive Encoder for Banking Products Recommendation,” Proc. Int. Jt. Conf. Neural Networks, vol. 2022-July, pp. 1–8, 2022, https://doi.org/10.1109/IJCNN55064.2022.9892130
  9. E. Lacić, D. Kowald, D. Parra, M. Kahr, and C. Trattner, “Towards a scalable social recommender engine for online marketplaces: The case of apache solr,” WWW 2014 Companion - Proc. 23rd Int. Conf. World Wide Web, pp. 817–822, 2014, https://doi.org/10.1145/2567948.2579245
  10. I. Goodfellow et al., “Generative adversarial networks,” Commun. ACM, vol. 63, no. 11, pp. 139–144, 2020, https://doi.org/10.1145/3422622
  11. Y. Zheng, Y. Zhang, and Z. Zheng, “Continuous Conditional Generative Adversarial Networks (cGAN) with Generator Regularization,” no. 2017, 2021, [Online]. Available: http://arxiv.org/abs/2103.14884
  12. C. Sun, H. Liu, M. Liu, Z. Ren, T. Gan, and L. Nie, “LarA: Attribute-to-feature adversarial learning for new-item recommendation,” WSDM 2020 - Proc. 13th Int. Conf. Web Search Data Min., pp. 582–590, 2020, https://doi.org/10.1145/3336191.3371805
  13. Q. Wu, Y. Liu, C. Miao, B. Zhao, Y. Zhao, and L.Guan, “PD-GAN: Adversarial learning for personalized diversity-promoting recommendation,” IJCAI Int. Jt. Conf. Artif. Intell., vol. 2019- Augus, pp. 3870–3876, 2019, https://doi.org/10.24963/ijcai.2019/537
  14. D. Vint, M. Anderson, Y. Yang, C. Ilioudis, G. Di Caterina, and C. Clemente, “Automatic target recognition for low resolution foliage penetrating SAR images using CNNS and GANS,” Remote Sens., vol. 13, no. 4, pp. 1–18, 2021, https://doi.org/10.3390/rs13040596
Back

Disclaimer: All papers published in IJRST will be indexed on Google Search Engine as per their policy.

We are one of the best in the field of watches and we take care of the needs of our customers and produce replica watches of very good quality as per their demands.