2013-1 Machine Learning Lecture 07 - Michael Negnevitsky - Evolutionary Co…

 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 1
Lecture 10Lecture 10
Evolutionary Computation:Evolutionary Computation:
Evolution strategies and genetic programmingEvolution strategies and genetic programming
II Evolution strategiesEvolution strategies
II Genetic programmingGenetic programming
II SummarySummary

AnotherAnother approach to simulating naturalapproach to simulating natural
evolution was proposed in Germany in the earlyevolution was proposed in Germany in the early
1960s. Unlike genetic algorithms, this approach1960s. Unlike genetic algorithms, this approach
−− called ancalled an evolution strategyevolution strategy −− was designedwas designed
to solve technical optimisation problems.to solve technical optimisation problems.
Evolution StrategiesEvolution Strategies

II In 1963 two students of the TechnicalIn 1963 two students of the Technical
University of Berlin,University of Berlin, Ingo RechenbergIngo Rechenberg andand
HansHans--Paul SchwefelPaul Schwefel, were working on the, were working on the
search for the optimal shapes of bodies in asearch for the optimal shapes of bodies in a
flow. They decided to try random changes inflow. They decided to try random changes in
the parameters defining the shape following thethe parameters defining the shape following the
example of natural mutation. As a result, theexample of natural mutation. As a result, the
evolution strategyevolution strategy was born.was born.
II Evolution strategies were developed as anEvolution strategies were developed as an
alternative to the engineeralternative to the engineer’’s intuition.s intuition.
II UnlikeUnlike GAsGAs, evolution strategies use only a, evolution strategies use only a
mutation operator.mutation operator.

In its simplest form, termed as a (1In its simplest form, termed as a (1++1)1)--evolutionevolution
strategy, one parent generates one offspring perstrategy, one parent generates one offspring per
generation by applyinggeneration by applying normally distributednormally distributed
mutation. The (1mutation. The (1++1)1)--evolution strategy can beevolution strategy can be
implemented as follows:implemented as follows:
Step 1Step 1:: Choose the number of parametersChoose the number of parameters NN toto
represent the problem, and then determine a feasiblerepresent the problem, and then determine a feasible
range for each parameter:range for each parameter:
Define a standard deviation for each parameter andDefine a standard deviation for each parameter and
the function to be optimisedthe function to be optimised..
Basic evolutionBasic evolution strategiesstrategies
{ }axmmin xx 11 , , { }axmmin xx 22 , , . . . , { }axmNminN xx ,

Step 2Step 2:: Randomly select an initial value for eachRandomly select an initial value for each
parameter from the respective feasible range. Theparameter from the respective feasible range. The
set of these parameters will constitute the initialset of these parameters will constitute the initial
population of parent parameters:population of parent parameters:
xx11,, xx22, . . . ,, . . . , xxNN
Step 3Step 3:: Calculate the solution associated with theCalculate the solution associated with the
parent parameters:parent parameters:
X = fX = f ((xx11,, xx22, . . . ,, . . . , xxNN))

Step 4Step 4:: Create a new (offspring) parameter byCreate a new (offspring) parameter by
adding a normally distributed random variableadding a normally distributed random variable aa
with meanwith mean zero and prezero and pre--selected deviationselected deviation δδ to eachto each
parent parameterparent parameter::
ii = 1, 2, . . . ,= 1, 2, . . . , NN
NormallyNormally distributed mutations with mean zerodistributed mutations with mean zero
reflect the natural process of evolutionreflect the natural process of evolution (smaller(smaller
changes occur more frequently than largerchanges occur more frequently than larger ones).ones).
Step 5Step 5:: CalculateCalculate the solution associated with thethe solution associated with the
offspring parameters:offspring parameters:
( )δ,0axx ii +=′
( )NxxxfX ′′′=′ ,...,, 21

Step 6Step 6:: CompareCompare the solution associated with thethe solution associated with the
offspring parameters with the one associated withoffspring parameters with the one associated with
the parent parameters. If the solution for thethe parent parameters. If the solution for the
offspring is better than that for the parents, replaceoffspring is better than that for the parents, replace
the parent population with the offspringthe parent population with the offspring
population. Otherwise, keep the parentpopulation. Otherwise, keep the parent
parameters.parameters.
Step 7Step 7:: GoGo to Step 4, and repeat the process until ato Step 4, and repeat the process until a
satisfactory solution is reached, or a specifiedsatisfactory solution is reached, or a specified
number of generations is considerednumber of generations is considered..

II An evolution strategy reflects the nature of aAn evolution strategy reflects the nature of a
chromosome.chromosome.
II A single gene may simultaneously affect severalA single gene may simultaneously affect several
characteristics of the living organism.characteristics of the living organism.
II On the other hand, a single characteristic of anOn the other hand, a single characteristic of an
individual may be determined by the simultaneousindividual may be determined by the simultaneous
interactions of several genes.interactions of several genes.
II The natural selection acts on a collection of genes,The natural selection acts on a collection of genes,
not on a single gene in isolation.not on a single gene in isolation.

II One of the central problems in computer science isOne of the central problems in computer science is
how to make computers solve problems withouthow to make computers solve problems without
being explicitly programmed to do so.being explicitly programmed to do so.
II Genetic programming offers a solution through theGenetic programming offers a solution through the
evolution of computer programs by methods ofevolution of computer programs by methods of
natural selection.natural selection.
II InIn fact, genetic programming is an extension of thefact, genetic programming is an extension of the
conventional genetic algorithm, but the goal ofconventional genetic algorithm, but the goal of
genetic programming is not just to evolve a bitgenetic programming is not just to evolve a bit--
string representation of some problem but thestring representation of some problem but the
computer code that solves the problem.computer code that solves the problem.
GeneticGenetic programmingprogramming

II Genetic programming is a recent development inGenetic programming is a recent development in
the area of evolutionary computation. It wasthe area of evolutionary computation. It was
greatly stimulated in the 1990s bygreatly stimulated in the 1990s by John KozaJohn Koza..
II According to Koza, genetic programming searchesAccording to Koza, genetic programming searches
the space of possible computer programs for athe space of possible computer programs for a
program that is highly fit for solving the problem atprogram that is highly fit for solving the problem at
hand.hand.
II Any computer program is a sequence of operationsAny computer program is a sequence of operations
(functions) applied to values (arguments), but(functions) applied to values (arguments), but
different programming languages may includedifferent programming languages may include
different types of statements and operations, anddifferent types of statements and operations, and
have different syntactic restrictions.have different syntactic restrictions.

II Since genetic programming manipulates programsSince genetic programming manipulates programs
by applying genetic operators, a programmingby applying genetic operators, a programming
language should permit a computer program to belanguage should permit a computer program to be
manipulated as data and the newly created data tomanipulated as data and the newly created data to
be executed as a program. For these reasons,be executed as a program. For these reasons,
LISPLISP was chosen as the main language forwas chosen as the main language for
genetic programming.genetic programming.

LISP has a highly symbolLISP has a highly symbol--oriented structure. Itsoriented structure. Its
basic data structures arebasic data structures are atomsatoms andand listslists. An atom. An atom
is the smallest indivisible element of the LISPis the smallest indivisible element of the LISP
syntax. The numbersyntax. The number 2121, the symbol, the symbol XX and the stringand the string
““This is a stringThis is a string”” are examples of LISP atoms. Aare examples of LISP atoms. A
list is an object composed of atoms and/or otherlist is an object composed of atoms and/or other
lists. LISP lists are written as an ordered collectionlists. LISP lists are written as an ordered collection
of items inside a pair of parentheses.of items inside a pair of parentheses.
LISP structureLISP structure

For example, the listFor example, the list
((−− ((** A B) C)A B) C)
calls for the application of the subtraction functioncalls for the application of the subtraction function
((−−) to two arguments, namely the list (*A B) and) to two arguments, namely the list (*A B) and
the atom C. First, LISP applies the multiplicationthe atom C. First, LISP applies the multiplication
function (*) to the atoms A and B.function (*) to the atoms A and B.
Once the list (*A B) is evaluated, LISP applies theOnce the list (*A B) is evaluated, LISP applies the
subtraction function (subtraction function (−−) to the two arguments, and) to the two arguments, and
thus evaluates the entire listthus evaluates the entire list
((−− ((** A B) C).A B) C).
LISP structureLISP structure

Graphical representation of LISP SGraphical representation of LISP S--expressionsexpressions
II Both atoms and lists are called symbolicBoth atoms and lists are called symbolic
expressions orexpressions or SS--expressionsexpressions. In LISP, all data. In LISP, all data
and all programs are Sand all programs are S--expressions. This givesexpressions. This gives
LISP the ability to operate on programs as if theyLISP the ability to operate on programs as if they
were data. In other words, LISP programs canwere data. In other words, LISP programs can
modify themselves or even write other LISPmodify themselves or even write other LISP
programs. This remarkable property of LISPprograms. This remarkable property of LISP
makes it very attractive for genetic programmingmakes it very attractive for genetic programming..
II Any LISP SAny LISP S--expression can be depicted as a rootedexpression can be depicted as a rooted
pointpoint--labelled tree with ordered branches.labelled tree with ordered branches.

BA
* C
−
LISP SLISP S--expression (expression (−−−−−−−− (*A B) C)(*A B) C)

How do we apply genetic programmingHow do we apply genetic programming
to a problem?to a problem?
BeforeBefore applying genetic programming to a problem,applying genetic programming to a problem,
we must accomplishwe must accomplish five preparatory stepsfive preparatory steps::
1.1. Determine the set of terminals.Determine the set of terminals.
2.2. Select the set of primitive functions.Select the set of primitive functions.
3.3. Define the fitness function.Define the fitness function.
4.4. Decide on the parameters for controlling the run.Decide on the parameters for controlling the run.
5.5. Choose the method for designating a result ofChoose the method for designating a result of
the runthe run..

II The Pythagorean Theorem helps us to illustrateThe Pythagorean Theorem helps us to illustrate
these preparatory steps and demonstrate thethese preparatory steps and demonstrate the
potential of genetic programming. The theorempotential of genetic programming. The theorem
says that the hypotenuse,says that the hypotenuse, cc, of a right triangle with, of a right triangle with
short sidesshort sides aa andand bb is given byis given by
22
bac +=
II The aim of genetic programming is to discover aThe aim of genetic programming is to discover a
program that matches this function.program that matches this function.

II To measure the performance of the asTo measure the performance of the as--yetyet--
undiscovered computer program, we will use aundiscovered computer program, we will use a
number of differentnumber of different fitness casesfitness cases. The fitness. The fitness
cases for the Pythagorean Theorem arecases for the Pythagorean Theorem are
represented by the samples of right triangles inrepresented by the samples of right triangles in
Table. These fitness cases are chosen at randomTable. These fitness cases are chosen at random
over a range of values of variablesover a range of values of variables aa andand bb..
Side a Side b Hypotenuse c Side a Side b Hypotenuse c
3 5 5.830952 12 10 15.620499
8 14 16.124515 21 6 21.840330
18 2 18.110770 7 4 8.062258
32 11 33.837849 16 24 28.844410
4 3 5.000000 2 9 9.219545

Step 1Step 1:: Determine the set of terminals.Determine the set of terminals.
TheThe terminals correspond to the inputs of theterminals correspond to the inputs of the
computer program to be discovered. Ourcomputer program to be discovered. Our
program takes two inputs,program takes two inputs, aa andand bb..
Step 2Step 2:: SelectSelect the set of primitive functions.the set of primitive functions.
TheThe functions can be presented by standardfunctions can be presented by standard
arithmetic operations, standard programmingarithmetic operations, standard programming
operations, standard mathematical functions,operations, standard mathematical functions,
logical functions or domainlogical functions or domain--specific functions.specific functions.
Our program will use four standard arithmeticOur program will use four standard arithmetic
operations +,operations +, −−, * and, * and //, and one mathematical, and one mathematical
functionfunction sqrtsqrt..

Step 3Step 3:: Define the fitness function.Define the fitness function. A fitnessA fitness
function evaluates how well a particular computerfunction evaluates how well a particular computer
program can solve the problem. For our problem,program can solve the problem. For our problem,
the fitness of the computer program can bethe fitness of the computer program can be
measured by the error between the actual resultmeasured by the error between the actual result
produced by the program and the correct resultproduced by the program and the correct result
given by the fitness case. Typically, the error isgiven by the fitness case. Typically, the error is
not measured over just one fitness case, butnot measured over just one fitness case, but
instead calculated as a sum of the absolute errorsinstead calculated as a sum of the absolute errors
over a number of fitness cases. The closer thisover a number of fitness cases. The closer this
sum is to zero, the better the computer program.sum is to zero, the better the computer program.

Step 4Step 4:: Decide on the parameters for controllingDecide on the parameters for controlling
the run.the run. For controlling a run, geneticFor controlling a run, genetic
programming uses the same primary parametersprogramming uses the same primary parameters
as those used for GAs. They include theas those used for GAs. They include the
population size and the maximum number ofpopulation size and the maximum number of
generations to be run.generations to be run.
Step 5Step 5:: Choose the method for designating aChoose the method for designating a
result of the run.result of the run. It is common practice inIt is common practice in
genetic programming to designate the bestgenetic programming to designate the best--soso--farfar
generated program as the result of a run.generated program as the result of a run.

Once these five steps are complete, a run can beOnce these five steps are complete, a run can be
made. The run of genetic programming starts withmade. The run of genetic programming starts with
a random generation of an initial population ofa random generation of an initial population of
computer programs. Each programcomputer programs. Each program is composed ofis composed of
functions +,functions +, −−,, **,, // andand sqrtsqrt, and terminals, and terminals aa andand bb..
In the initial population, all computer programsIn the initial population, all computer programs
usually have poor fitness, but some individuals areusually have poor fitness, but some individuals are
more fit than others. Just as a fitter chromosome ismore fit than others. Just as a fitter chromosome is
more likely to be selected for reproduction, so amore likely to be selected for reproduction, so a
fitter computer program is more likely to survive byfitter computer program is more likely to survive by
copying itself into the next generation.copying itself into the next generation.

Crossover in genetic programming:Crossover in genetic programming:
Two parental STwo parental S--expressionsexpressions
a
*
a
+
a b
−
(/ (− (sqrt (+ (* a a) (− a b))) a) (* a b))
/
a b
*
sqrt
−
a
+
b
/
a
sqrt
b
−
−
(+ (− (sqrt (− (* b b) a)) b) (sqrt (/ a b)))
b
a*
b
sqrt

Crossover in genetic programming:Crossover in genetic programming:
Two offspring STwo offspring S--expressionsexpressions
a
*
a
+
a b
−
/
sqrt
−
a
+
b
/
a
sqrt−
b
a b
*
b
−
a
*
b
sqrt
(/ (− (sqrt (+ (* a a) (− a b))) a) (sqrt (− (* b b) a))) (+ (−(* a b) b) (sqrt (/ a b)))

Mutation in genetic programmingMutation in genetic programming
AA mutation operator can randomly change anymutation operator can randomly change any
function or any terminal in the LISP Sfunction or any terminal in the LISP S--expression.expression.
Under mutation, a function can only be replaced byUnder mutation, a function can only be replaced by
a function and a terminal can only be replaced by aa function and a terminal can only be replaced by a
terminal.terminal.

a
*
a
+
a b
−
(/ (− (sqrt (+ (* a a) (− a b))) a) (* a b))
/
a b
*
sqrt
−
a
+
b
/
a
sqrt
b
−
(+ (− (sqrt (− (* b b) a)) b) (sqrt (/ a b)))
a*
b
sqrt

−
b
Mutation in genetic programming:Mutation in genetic programming:
Original SOriginal S--expressionsexpressions

Mutation in genetic programming:Mutation in genetic programming:
Mutated SMutated S--expressionsexpressions
(/ (+ (sqrt (+ (* a a) (− a b))) a) (* a b))
/
a b
*
a
*
a
+
a b
−
sqrt
+
a
+
b
/
a
sqrt
b
−
(+ (− (sqrt (− (* b b) a)) a) (sqrt (/ a b)))
a*
b
sqrt
−
a

Step 1Step 1:: Assign the maximum number of generationsAssign the maximum number of generations
to be run and probabilities for cloning, crossoverto be run and probabilities for cloning, crossover
and mutation. Note that the sum of the probabilityand mutation. Note that the sum of the probability
of cloning, the probability of crossover and theof cloning, the probability of crossover and the
probability of mutation must be equal to one.probability of mutation must be equal to one.
StepStep 22:: GGenerateenerate an initial population of computeran initial population of computer
programs of sizeprograms of size NN by combining randomlyby combining randomly
selected functions and terminals.selected functions and terminals.
In summary, genetic programming creates computerIn summary, genetic programming creates computer
programs by executing the following steps:programs by executing the following steps:

Step 3Step 3:: Execute each computer program in theExecute each computer program in the
population and calculate its fitness with anpopulation and calculate its fitness with an
appropriate fitness function. Designate the bestappropriate fitness function. Designate the best--
soso--far individual as the result of the run.far individual as the result of the run.
Step 4Step 4:: With the assigned probabilities, select aWith the assigned probabilities, select a
genetic operator to perform cloning, crossover orgenetic operator to perform cloning, crossover or
mutation.mutation.

Step 5Step 5:: If the cloning operator is chosen, select oneIf the cloning operator is chosen, select one
computer program from the current population ofcomputer program from the current population of
programs and copy it into a new population.programs and copy it into a new population.
•• If the crossover operator is chosen, select a pairIf the crossover operator is chosen, select a pair
of computer programs from the currentof computer programs from the current
population, create a pair of offspring programspopulation, create a pair of offspring programs
and place them into the new population.and place them into the new population.
•• If the mutation operator is chosen, select oneIf the mutation operator is chosen, select one
computer program from the current population,computer program from the current population,
perform mutation and place the mutant into theperform mutation and place the mutant into the
new population.new population.

Step 6Step 6:: RepeatRepeat Step 4Step 4 until the size of the newuntil the size of the new
population of computer programs becomes equalpopulation of computer programs becomes equal
to the size of the initial population,to the size of the initial population, NN..
Step 7Step 7:: Replace the current (parent) populationReplace the current (parent) population
with the new (offspring) population.with the new (offspring) population.
Step 8Step 8:: Go toGo to Step 3Step 3 and repeat the process untiland repeat the process until
the termination criterion is satisfied.the termination criterion is satisfied.

Fitness history of the best SFitness history of the best S--expressionexpression
Fitness,%
0 1 2 3 4
0
20
40
60
80
100
a
*
a b b
*
sqrt
+
G e n e r a t i o n s B e s t o f g e n e r a t i o n

What are the main advantages of geneticWhat are the main advantages of genetic
programming compared to genetic algorithms?programming compared to genetic algorithms?
II Genetic programming applies the sameGenetic programming applies the same
evolutionaryevolutionary approach. However, geneticapproach. However, genetic
programming is no longer breeding bit strings thatprogramming is no longer breeding bit strings that
represent coded solutions but complete computerrepresent coded solutions but complete computer
programs that solve a particular problem.programs that solve a particular problem.
II TheThe fundamental difficulty offundamental difficulty of GAsGAs lies in thelies in the
problem representation, that is, in the fixedproblem representation, that is, in the fixed--lengthlength
coding. A poor representation limits the power ofcoding. A poor representation limits the power of
a GA, and even worse, may lead to a falsea GA, and even worse, may lead to a false
solution.solution.

II A fixedA fixed--length coding is rather artificial. As itlength coding is rather artificial. As it
cannot provide a dynamic variability in length,cannot provide a dynamic variability in length,
such a coding often causes considerablesuch a coding often causes considerable
redundancy and reduces the efficiency of geneticredundancy and reduces the efficiency of genetic
search. In contrast, genetic programming usessearch. In contrast, genetic programming uses
highhigh--level building blocks of variable length.level building blocks of variable length.
Their size and complexity can change duringTheir size and complexity can change during
breeding.breeding.
II Genetic programming works well in a largeGenetic programming works well in a large
number of different cases and has many potentialnumber of different cases and has many potential
applicationsapplications..

2013-1 Machine Learning Lecture 07 - Michael Negnevitsky - Evolutionary Co…

More Related Content

Viewers also liked

Similar to 2013-1 Machine Learning Lecture 07 - Michael Negnevitsky - Evolutionary Co…

More from Dongseo University

Recently uploaded

2013-1 Machine Learning Lecture 07 - Michael Negnevitsky - Evolutionary Co…