Tutorial: Evolutionary Computation for Feature Reduction
We are now in the era of big data, where vast amounts of high-dimensional data become ubiquitous in a variety of domains, such as social media, healthcare, and cybersecurity. When machine learning algorithms are applied to such high-dimensional data, they suffer from the curse of dimensionality, where the data becomes very sparse. Furthermore, the high-dimensional data might contain redundant and/or irrelevant features that blur useful information from relevant features. Feature reduction can address the above issues by building a smaller but more informative feature set.
Feature selection and feature construction are two main approaches of feature reduction. Feature selection aims to select a small subset of original (relevant) features. Feature construction aims to create a small set of new high-level (informative) features based on the original feature set. Although both approaches are essential data pre-processing steps, they are challenging due to their large search spaces and complex interactions between features. While exhaustive searches are impractical due to their intensive computation cost, traditional heuristic searches require less computational resources but can be trapped at local optima. Recently, evolutionary computation (EC) has been widely applied to achieve feature reduction because of its potential global search ability. Existing EC-based feature reduction approaches successfully reduce the data dimensionality while still improve the classification performance as well as the interpretability of the built models
This tutorial firstly describes a general framework of feature reduction followed by the applications of feature reduction in real-world scenarios. Then, we will show how EC techniques, particularly genetic algorithms, particle swarm optimisation, differential evolution, genetic programming, ant colony optimisation and evolutionary multi-objective optimisation (EMO) can be applied to address challenges in feature reduction. The effectiveness of EC-based feature reduction is illustrated through several applications including bioinformatics, image analysis and pattern classification, symbolic regression, and cybersecurity. The tutorial concludes with existing challenges for future research.
Potential audiences:The tutorial is designed as an introductory tutorial that provides the main concepts and the overall system of evolutionary feature reduction. At the same time, the attendees will also learn the pros/cons of applying different evolutionary algorithms to achieve feature reduction and state-of-the-art evolutionary feature reduction algorithms. Some potential future research directions are also explained in the tutorial.
The outline of the proposed tutorial is as follows:
- Introduction to feature reduction
- What is feature redunction?
- Why is feature reduction important?
- Feature Selection vs Feature Construction
- Feature selection
- Graph-based representation
- Tree-based representation
- Vector-based representation
- Feature construction
- Single-tree representation
- Multi-tree representation
- Hybridisation of feature selection and feature construction
- Real-world applications of feature reduction
- Existing challenges
Biography of the organisers:
Bach Nguyen is currently a Postdoctoral Research Fellow in School of Engineering and Computer Science at VUW. He has over 20 papers published in fully refereed international journals and conferences. His research focuses mainly on evolutionary computation, machine learning, classification, feature selection, transfer learning, and multi-objective optimisation.
Dr Nguyen is currently chairing the IEEE Task Force on Evolutionary Feature Selection and Construction and the Young Professionals of IEEE New Zealand Central Section. He is also a Vice-Chair of Data Mining and Big Data Analytics Technical Committee, IEEE Computational Intelligence Society.
Dr Nguyen has been serving as a program committee member for over 10 international conferences including AAAI, IJCAI, IEEE CEC, GECCO, and IEEE SSCI. He has been serving as a reviewer for over 10 international journals including IEEE Transactions on Evolutionary Computation and IEEE Transactions on Cybernetics.
Bing Xue is currently a Professor and Program Director of Science in School of Engineering and Computer Science at VUW. She has over 200 papers published in fully refereed international journals and conferences and her research focuses mainly on evolutionary computation, machine learning, classification, symbolic regression, feature selection, evolving deep neural networks, image analysis, transfer learning, multi-objective machine learning.
Dr Xue is currently the Vice-Chair of Evolutionary Computation Technical Committee, Chair of IEEE Task Force on Transfer Learning & Transfer Optimization, and Vice-Chair of IEEE Task Force on Evolutionary Feature Selection and Construction and of IEEE CIS Task Force on Evolutionary Deep Learning and Applications.
Prof Xue is the organiser of the special session on Evolutionary Feature Selection and Construction in IEEE Congress on Evolutionary Computation (CEC) 2015, 2016, 2017, 2018 2019, and 2020. A/Prof Xue has been a chair for a number of international conferences including the Chair of Women@GECCO 2018 and a co-Chair of the Evolutionary Machine Learning Track for GECCO 2019 and 2020. She is the Lead Chair of IEEE Symposium on Computational Intelligence in Feature Analysis, Selection, and Learning in Image and Pattern Recognition (FASLIP) at SSCI 2016, 2017,2018, 2019 and 2020, a Program Co-Chair of the 7th International Conference on Soft Computing and Pattern Recognition (SoCPaR2015), a Program Chair of the 31th Australasian Joint Conference on Artificial Intelligence (AI 2018), and Finance Chair for 2019 IEEE Congress on Evolutionary Computation.
She is an Associate Editor or Member of the Editorial Board for seven international journals, including IEEE Transactions of Evolutionary Computation, IEEE Computational Intelligence Magazine, and ACM Transactions on Evolutionary Learning and Optimisation.
Mengjie Zhang is a Fellow of Royal Society of New Zealand, a Fellow of IEEE, a Panel Member of the Marsden Fund (New Zealand Government Funding), and currently Professor of Computer Science at Victoria University of Wellington, where he heads the interdisciplinary Evolutionary Computation Research Group. He is a member of the University Academic Board, a member of the University Postgraduate Scholarships Committee, Associate Dean (Research and Innovation) in the Faculty of Engineering, and Chair of the Research Committee of the Faculty of Engineering and School of Engineering and Computer Science.
His research is mainly focused on evolutionary computation, particularly genetic programming, particle swarm optimisation and learning classifier systems with application areas of feature selection/construction and dimensionality reduction, computer vision and image processing, evolutionary deep learning and transfer learning, job shop scheduling, multi-objective optimisation, and clustering and classification with unbalanced and missing data. He is also interested in data mining, machine learning, and web information extraction. Prof Zhang has published over 500 research papers in refereed international journals and conferences in these areas.
He has been serving as an Associated Editor or Editorial Board Member for over 10 international journals including IEEE Transactions on Evolutionary Computation, IEEE Transactions on Cybernetics, the Evolutionary Computation Journal (MIT Press), ACM Transactions on Evolutionary Learning and Optimisation, Genetic Programming and Evolvable Machines (Springer), IEEE Transactions on Emergent Topics in Computational Intelligence, Applied Soft Computing, and Engineering Applications of Artificial Intelligence, and as a reviewer of over 30 international journals. He has been a major chair for 8 international conferences. He has also been serving as a steering committee member and a program committee member for over 80 international conferences including all major conferences in evolutionary computation. Since 2007, he has been listed in the top five world genetic programming researchers by the GP bibliography (http://www.cs.bham.ac.uk/~wbl/biblio/gp-html/index.html).
He is the Tutorial Chair for GECCO 2014, an AIS-BIO Track Chair for GECCO 2016, an EML Track Chair for GECCO 2017, and a GP Track Chair for GECCO 2020. Since 2012, he has been co-chairing several parts of IEEE CEC, SSCI, and EvoIASP/EvoApplications conference (he has been involving major EC conferences such as GECCO, CEC, EvoStar, SEAL). Since 2014, he has been co-organising and co-chairing the special session on evolutionary feature selection and construction at IEEE CEC and SEAL, and also delivered a keynote/plenary talk for IEEE CEC 2018, IEEE ICAVSS 2018, DOCSA 2019, IES 2017 and Chinese National Conference on AI in Law 2017.
Prof Zhang was the Chair of the IEEE CIS Intelligent Systems Applications, the IEEE CIS Emergent Technologies Technical Committee, and the IEEE CIS Evolutionary Computation Technical Committee; a Vice-Chair of the IEEE CIS Task Force on Evolutionary Feature Selection and Construction, the IEEE CIS Task Force on Evolutionary Computer Vision and Image Processing, and the IEEE CIS Task Force on Evolutionary Deep Learning and Applications; and also the founding chair of the IEEE Computational Intelligence Chapter in New Zealand.