Abstract—Feature selection, used as a preprocessing step, can reduce the dimensionality of data and thereby increase the efficiency, accuracy, and clarity of learning systems. However feature selection can be costly endeavour. This paper proposes two new feature selection algorithms, based on binary particle swarm optimisation, with the aim of reducing running time without affecting classification accuracy by combining filter and wrapper approaches. The first algorithm proceeds cautiously by only updating the pbest and gbest, two critical values, after the learning system has been consulted. The second algorithm performs more reckless updates with the aim of sacrificing some performance for speed. Some theoretical analysis is performed in order
…show more content…
r 1,i and r 2,i are random values uniformly distributed over [0, 1]. This description of PSO is applicable to real-valued search spaces. However feature selection, along with many other problems, occur in a discrete search space and require a modified algorithm. Binary Particle Swarm Optimisation (BPSO) [?] is just such an algorithm. In BPSO, the values of the components of all position vectors (x i , pbest i , and gbest i ) are restricted to 0 or 1. Equation (2) is still used to update the velocity, each component of which now indicates the probability of the corresponding component in the position vector being 1. A sigmoid function s(v i,d ) is used to transform the components of the velocity into a unit range. BPSO updates the position of each particle according to the following equation: B. Information Theory The tools of information theory [?] are the principal meth- ods to measure the information content of random variables, which can be used to reason about subsets of features. A core information measure is that of entropy, H(X), which measures the uncertainty of a discrete random variable X. It is defined as: A. Particle Swarm Optimisation PSO is an EC technique inspired by social behaviour pro- posed by Kennedy and
r 1,i and r 2,i are random values uniformly distributed over [0, 1]. This description of PSO is applicable to real-valued search spaces. However feature selection, along with many other problems, occur in a discrete search space and require a modified algorithm. Binary Particle Swarm Optimisation (BPSO) [?] is just such an algorithm. In BPSO, the values of the components of all position vectors (x i , pbest i , and gbest i ) are restricted to 0 or 1. Equation (2) is still used to update the velocity, each component of which now indicates the probability of the corresponding component in the position vector being 1. A sigmoid function s(v i,d ) is used to transform the components of the velocity into a unit range. BPSO updates the position of each particle according to the following equation: B. Information Theory The tools of information theory [?] are the principal meth- ods to measure the information content of random variables, which can be used to reason about subsets of features. A core information measure is that of entropy, H(X), which measures the uncertainty of a discrete random variable X. It is defined as: A. Particle Swarm Optimisation PSO is an EC technique inspired by social behaviour pro- posed by Kennedy and