Article for Catholic Sabah column

I'm going to publish the article I written for the Catholic Sabah column in this post. So stay tuned!

MSC: HIS2011 Accepted Paper 1

Below are the comments I received from my reviewers of the paper, which I recently submitted to HIS 2011.

PAPER: 159
TITLE: Evolutionary Spiking Neural Networks as Racing Car Controllers
AUTHORS: Elias Yee and Jason Teo

REVIEW 1

OVERALL RATING: 1 (weak accept)
REVIEWER'S CONFIDENCE: 4 (expert)
How relevant is this manuscript for HIS 2011 attendees? Please explain under Public Comments:: 4 (good)
Is the manuscript technically sound? Please explain your answer under Public Comments:: 4 (good)
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic?: 3 (fair)
Please rate the readability of the manuscript. Explain your rating under Public Comments below:: 3 (fair)

This paper employs evolutionary programming to improve the control of open racing car simulator. It is interesting.

However, several revisions should be done. First, lots of cited references are lost in reference list, such as Figueira and Roque (2011), Vázquez and Garro (2011), Daun-Gruhn and Büschges (2011), Munoz, Gutierrez, and Sanchis (2010), and Jakobsen (2007).

Secondly, please clearly introduce of your models and provide some necessary figures or flowchart with some essential brief explanation vis-à-vis the text to understand the implementing procedure.

Finally, please provide some necessary comparison among the proposed model and other alternatives.


REVIEW 2

OVERALL RATING: 2 (accept)
REVIEWER'S CONFIDENCE: 2 (medium)
How relevant is this manuscript for HIS 2011 attendees? Please explain under Public Comments:: 4 (good)
Is the manuscript technically sound? Please explain your answer under Public Comments:: 4 (good)
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic?: 4 (good)
Please rate the readability of the manuscript. Explain your rating under Public Comments below:: 4 (good)

Interesting paper

MSC: Experiment running on all 5 PCs in Lab

Set up all 5 PCs to run experiments now. Will start Experiment on track 2 tonight or tomorrow morning.

Plan to meet Dr James tomorrow, Oct 7, Friday.

Write Paper 2!!

MSC: Text from - izhikevich and ffnn docx

Below is the copy-pasted text from the document "izhikevich and ffnn.docx"

Reason for pasting it here is so that I could delete that doc file and refer to the text here.

Abstract

The Izhikevich spiking neural network model is investigated as a method to develop controllers for a simple, but not trivial, car racing game, called TORCS. The controllers are evolved using the Evolutionary Strategy, and the performance of the best individuals are compared with the hand-coded controller included with the game.

racing competition is a wat wat wat, and since networks of spiking neurons have been gaining popularity in recent years, a great curiosity emerged to want to find out if a spiking neural network could perform as well or better than the other techniques used to train the driver. This paper investigates the performance of the Izhikevich spiking neural network model applied to the TORCS racing game, and trained using Evolutionary Strategy. The evolved driver is compared with the hand-coded driver included in the TORCS championship platform.

Introduction

Many methods have been used to optimize the TORCS game driver for the cig competition, curious if spiking neural network could also perform well in it. Hence, compared the network with the simple driver provided with the competition client.

Results – the spiking neural network does perform better.

Machine learning methods can be used to solve any simulated control problems. In this paper, an evolutionary optimization technique is used to evolve a controller to drive a simulated racing car. The car control is handled by an artificial neural network, using Evolutionary Strategies for the tuning of the weights. Different experiments are run in order to explore combinations of training parameters.

MSC: Submitted to HIS 2011

Dr J finally replied my email. What a relief!
So I continued on and submitted my 1st paper to HIS 2011 top cnference.

Thank You Jesus. Amen.

MSC: Paper ready for submission

Finish adding in the path driven and SNN spike train. Also revised ES to EP.
Revised some format to reflect the IEEE format for HIS2011


I finished revising it to how Dr J suggested, and sent him another email with the doc for a final review. Then, the paper will be sent to HIS2011.
Should be able to submit soon, before Sept 20.

EDIT: Registered with easychair, to submit paper to HIS2011 in Malacca

MSC: Paper Revised by Dr J

Dr J replied my email with an attached and revised document of the paper. He took out a few paragraphs and said to keep them for next paper.

Some things Dr J suggested include adding a chart of the SNN spike train, and revise ES to EP.

MSC: The evolved controllers beat me

there were some round, the evolved controller  had almost took over the leading car, but bangged on it and both the cars skidded, but the evolved controller could not recover from the crash and stayed there.

lap7: > 10sec behind

the controllers evolved on track 1 usually started slow, most likely due to the unoptimized automatic transmission, but would later overtake its oponents after one or two laps.

controllers evolved on track 2 were able to generalize on track 1, but could only complete one fifth of track 3.

controllers evolved on track 3 were able to generalize on both track 1 and 2.

controllers evolved on track 1 were not able to generalize on both track 2 and 3.



i lost to the evolved controller on both track 1 and 2, butnot track 3 despite the difficulty controlling the car.


the gear change played a very big impact

MSC: Suggested improvements and new discovery

Received a reply from Dr J yesterday.
Listed below are the suggested improvements and its state:

1. Use round figures to label the x-axis of your graphs (done)
2. Show more in-game screenshots of you SNN-driven car – although it is not of much scientific value here but nice pictures always gets attention
3. Think of plotting the path driven by the best evolved SNNs as they move across the track over the racing period
4. Avoid saying that your evolved controllers were so easily beaten by human and some of the included controllers.
5. Also, your explanation that humans cheat by flooring the accelerator is not convincing since if it is allowed in the game, then it is not cheating. What I meant by cheating is when the game AI controllers break rules such as achieving top speed immediately (as opposed to flooring the accelerator but gradual build-up of speed) or in the case of RTS, being able to see through fog-of-war, then that’s cheating. So I recommend that you amend the way you are currently explaining this point.
6. Please add in a detailed description of your fitness function including a mathematical formula for it. (done)

Discovered something new. In track 1, the controller is able to start by flooring the gas pedal, but it is still slower than the other car because of the gear change strategy. The evolved controller increased gear earlier than the static controller, hence picked up speed slower than it.

Drew track path, but...

Notice that
championship2011server uses a car1-trb1 and the others bot using the same car are:

• tita 3
• berniw 3
• olethros 3
• lliaw 3
• inferno 3
• bt 3

MSC: Paper 1 Finally Done

I'm finally done with paper 1! 4am in the morning, August 29, 2011. 2 days before the deadline of the IEEE SCOReD 2011 (9th SCOReD).

Sent email to Dr J to schedule meeting, attached with the document file of the paper.

MSC: Discussions are almost done

Given the fact that track 3 is much more difficult to race in, the fitness scores are considered quite high as compared to the ratio between this track and the other 2 tracks.

Low average speed, low speed = smaller distance raced,

An interesting observation to note, apart from spectating it drive on the track, is...

So.. quite a fruitful day today. but if I made use of the whole day, I could have done more. but it's ok. I'll try to do much more tomorrow.

Aiming to get it done by end of this week, which is on the August 27, latest by August 28.

MSC: Discussions on the way

Inserted all charts into paper, and also a table of the best lap times.

MSC: Fitness of controller on track 1 tells of sophisticated driving skills

These are just some thoughts and/or style of writing that unnecessarily be in the paper.

After examining the scores and the controller driving in real-time, we can tell that the controller evolved driving strategies. Although track 1 has only four turning corners, does not mean that the fitness of the controller will easily be that high, because there is high possibility that a less sophisticated driving technique would make the controller slow down while turning the corner.

MSC:

Formatted paper to mirror IEEE format requirement, for the paper due Aug 15.

Track 2
2 completed runs on my lab PC, and 1 ongoing run.
2 completed runs on the other PC on the far right, and 1 ongoing run.
2 completed runs on the PC on the right.
2 completed runs on home's PC, and 1 ongoing run.

Total: 11 runs

MSC: Spike rate encoding

Objectives for today:

1. Mention "spike rate coding" in paper and reference Floreano.
2. Mention the steps to perform the evolutionary process.
3. Download IEEE reference style for MS Word.
4. Refer Kar Bin's paper.
• Noticed that Kar Bin's paper only has 6 pages including references, with the references taking up the last page, and graphs taking up another page. Hence, he only used 4 pages.