HIS 2011 Extended Paper

About 2 weeks ago, I was invited to submit an extended paper of the HIS 2011 conference paper I recently published last December in Melacca, to be considered for submission in one of the various Journals. I decide to go for it, and today is the deadline for submission.

I added a new set of experiments conducted using the Generation 2 Sigmoid Neural Network, and discussed about it as a benchmark for the Spiking Neural Network (SNN). Results are very good, if you ask me, as the Sigmoid NN performed poorer than the SNN in terms of computation load and time taken to improve itself. As after 1000 generations, the SNN still performed better. Albeit not much difference in the easiest track, but showed significant difference in the more difficult track.

Gosh, it took me so long to format the paper, but I managed. I used the IEEE Trans. and Journal paper format for it. The final product is 8 pages long :D and I'm satisfied with it.

Now, off to start on my UMS Research & Innovation Award 2012 poster! Hope to get it done by next week. Wish me luck.
Oh and my brother is going to Kuala Lumpur for his Bachelor's degree. My mom will go with him to settle school fees, accommodation and other stuff, soit'll just be my dad and I this weekend.

MSC: HIS2011 Registration Payment Method

Note to self:  Payment method for HIS 2011


METHOD 3: Payment by wire transfer (For Local Participants Only):
Please send the payment instruction to BIMBMYKL by SWIFT with the following details:
Payment amount: Please mention the exact amount. Please pay all costs/bank transfer fees etc, so that the conference account gets the full amount.
Reimbursement instruction: HIS 2011 and paper ID

Beneficiary bank's name: Bank Islam Malaysia Berhad (Cawangan Bandar Melaka)
Beneficiary bank's address: No. 50, Jalan Cempaka 1, Taman Cempaka, Peringgit, 75400 Melaka, Malaysia.
Benificiary's account number: 04-015-01-004888-6
Benificiary's name: Universiti Teknikal Malaysia Melaka

MSC: TODO: Conference Registration

So, the registration fee for the HIS 2011 conference in Melacca is $400 ≈ RM1240

Sign Copyright form

Revise paper:
Add a flow chart the size of Fig.2
Take out unnecessary citations.
Put Fig 3 & 9 can be put together, and smaller.
Comparisons will be done in future work.

MSC: HIS2011 Accepted Paper 1

Below are the comments I received from my reviewers of the paper, which I recently submitted to HIS 2011.

PAPER: 159
TITLE: Evolutionary Spiking Neural Networks as Racing Car Controllers
AUTHORS: Elias Yee and Jason Teo

REVIEW 1

OVERALL RATING: 1 (weak accept)
REVIEWER'S CONFIDENCE: 4 (expert)
How relevant is this manuscript for HIS 2011 attendees? Please explain under Public Comments:: 4 (good)
Is the manuscript technically sound? Please explain your answer under Public Comments:: 4 (good)
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic?: 3 (fair)
Please rate the readability of the manuscript. Explain your rating under Public Comments below:: 3 (fair)

This paper employs evolutionary programming to improve the control of open racing car simulator. It is interesting.

However, several revisions should be done. First, lots of cited references are lost in reference list, such as Figueira and Roque (2011), Vázquez and Garro (2011), Daun-Gruhn and Büschges (2011), Munoz, Gutierrez, and Sanchis (2010), and Jakobsen (2007).

Secondly, please clearly introduce of your models and provide some necessary figures or flowchart with some essential brief explanation vis-à-vis the text to understand the implementing procedure.

Finally, please provide some necessary comparison among the proposed model and other alternatives.


REVIEW 2

OVERALL RATING: 2 (accept)
REVIEWER'S CONFIDENCE: 2 (medium)
How relevant is this manuscript for HIS 2011 attendees? Please explain under Public Comments:: 4 (good)
Is the manuscript technically sound? Please explain your answer under Public Comments:: 4 (good)
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic?: 4 (good)
Please rate the readability of the manuscript. Explain your rating under Public Comments below:: 4 (good)

Interesting paper

MSC: Experiment running on all 5 PCs in Lab

Set up all 5 PCs to run experiments now. Will start Experiment on track 2 tonight or tomorrow morning.

Plan to meet Dr James tomorrow, Oct 7, Friday.

Write Paper 2!!

MSC: Text from - izhikevich and ffnn docx

Below is the copy-pasted text from the document "izhikevich and ffnn.docx"

Reason for pasting it here is so that I could delete that doc file and refer to the text here.

Abstract

The Izhikevich spiking neural network model is investigated as a method to develop controllers for a simple, but not trivial, car racing game, called TORCS. The controllers are evolved using the Evolutionary Strategy, and the performance of the best individuals are compared with the hand-coded controller included with the game.

racing competition is a wat wat wat, and since networks of spiking neurons have been gaining popularity in recent years, a great curiosity emerged to want to find out if a spiking neural network could perform as well or better than the other techniques used to train the driver. This paper investigates the performance of the Izhikevich spiking neural network model applied to the TORCS racing game, and trained using Evolutionary Strategy. The evolved driver is compared with the hand-coded driver included in the TORCS championship platform.

Introduction

Many methods have been used to optimize the TORCS game driver for the cig competition, curious if spiking neural network could also perform well in it. Hence, compared the network with the simple driver provided with the competition client.

Results – the spiking neural network does perform better.

Machine learning methods can be used to solve any simulated control problems. In this paper, an evolutionary optimization technique is used to evolve a controller to drive a simulated racing car. The car control is handled by an artificial neural network, using Evolutionary Strategies for the tuning of the weights. Different experiments are run in order to explore combinations of training parameters.

MSC: Torcs Suprisingly Runs on Vista - Finally

Tried running the experiment of Sigmoidal ANN on PC2's native OS, Vista, and surprisingly returned no error. Will duplicate it and run the experiment on PC3 and PC5, which also runs Vista.

MSC: Gen2 ANN Experiment

Today start the experiment with the 2nd generation sigmoidal ANNs.

MSC: Submitted to HIS 2011

Dr J finally replied my email. What a relief!
So I continued on and submitted my 1st paper to HIS 2011 top cnference.

Thank You Jesus. Amen.

MSC: Paper ready for submission

Finish adding in the path driven and SNN spike train. Also revised ES to EP.
Revised some format to reflect the IEEE format for HIS2011


I finished revising it to how Dr J suggested, and sent him another email with the doc for a final review. Then, the paper will be sent to HIS2011.
Should be able to submit soon, before Sept 20.

EDIT: Registered with easychair, to submit paper to HIS2011 in Malacca

MSC: Paper Revised by Dr J

Dr J replied my email with an attached and revised document of the paper. He took out a few paragraphs and said to keep them for next paper.

Some things Dr J suggested include adding a chart of the SNN spike train, and revise ES to EP.

MSC: The evolved controllers beat me

there were some round, the evolved controller  had almost took over the leading car, but bangged on it and both the cars skidded, but the evolved controller could not recover from the crash and stayed there.

lap7: > 10sec behind

the controllers evolved on track 1 usually started slow, most likely due to the unoptimized automatic transmission, but would later overtake its oponents after one or two laps.

controllers evolved on track 2 were able to generalize on track 1, but could only complete one fifth of track 3.

controllers evolved on track 3 were able to generalize on both track 1 and 2.

controllers evolved on track 1 were not able to generalize on both track 2 and 3.



i lost to the evolved controller on both track 1 and 2, butnot track 3 despite the difficulty controlling the car.


the gear change played a very big impact

MSC: Suggested improvements and new discovery

Received a reply from Dr J yesterday.
Listed below are the suggested improvements and its state:

1. Use round figures to label the x-axis of your graphs (done)
2. Show more in-game screenshots of you SNN-driven car – although it is not of much scientific value here but nice pictures always gets attention
3. Think of plotting the path driven by the best evolved SNNs as they move across the track over the racing period
4. Avoid saying that your evolved controllers were so easily beaten by human and some of the included controllers.
5. Also, your explanation that humans cheat by flooring the accelerator is not convincing since if it is allowed in the game, then it is not cheating. What I meant by cheating is when the game AI controllers break rules such as achieving top speed immediately (as opposed to flooring the accelerator but gradual build-up of speed) or in the case of RTS, being able to see through fog-of-war, then that’s cheating. So I recommend that you amend the way you are currently explaining this point.
6. Please add in a detailed description of your fitness function including a mathematical formula for it. (done)

Discovered something new. In track 1, the controller is able to start by flooring the gas pedal, but it is still slower than the other car because of the gear change strategy. The evolved controller increased gear earlier than the static controller, hence picked up speed slower than it.

Drew track path, but...

Notice that
championship2011server uses a car1-trb1 and the others bot using the same car are:

• tita 3
• berniw 3
• olethros 3
• lliaw 3
• inferno 3
• bt 3

MSC: Paper 1 Finally Done

I'm finally done with paper 1! 4am in the morning, August 29, 2011. 2 days before the deadline of the IEEE SCOReD 2011 (9th SCOReD).

Sent email to Dr J to schedule meeting, attached with the document file of the paper.

MSC: Discussions are almost done

Given the fact that track 3 is much more difficult to race in, the fitness scores are considered quite high as compared to the ratio between this track and the other 2 tracks.

Low average speed, low speed = smaller distance raced,

An interesting observation to note, apart from spectating it drive on the track, is...

So.. quite a fruitful day today. but if I made use of the whole day, I could have done more. but it's ok. I'll try to do much more tomorrow.

Aiming to get it done by end of this week, which is on the August 27, latest by August 28.

MSC: Discussions on the way

Inserted all charts into paper, and also a table of the best lap times.

MSC: PC3 keeps crashing, another day to completely compile all results

Run on PC2 is done, but run on PC3 was interrupted by a PC crash at 4am, so I continued it at 10.30pm.
Transferred the results of a run from home PC to PC1, to be compiled together with the rest.

Archived a collection of 3 runs on track 1, the latest, "3_snn_1.csv" is guaranteed to run with the latest settings: RS parameters, best lap time, but there's 2 others, which i'm not quite sure. So I'm sending them home to test run it on home PC.

I compiled the results of the 2 latest completed runs, the one from home, and PC2, into "Track1.xlsx".

As of now, PC1 has completed around 650 generations. Will complete tomorrow.

Found out yesterday from the email Dr J sent us that there's gonna be an international conference in Bangkok. Submission due date is on September 10, 2011.

MSC: Fitness of controller on track 1 tells of sophisticated driving skills

These are just some thoughts and/or style of writing that unnecessarily be in the paper.

After examining the scores and the controller driving in real-time, we can tell that the controller evolved driving strategies. Although track 1 has only four turning corners, does not mean that the fitness of the controller will easily be that high, because there is high possibility that a less sophisticated driving technique would make the controller slow down while turning the corner.

MSC: My laptop is a snail

The run on laptop is running too slow. It's been a day already and it hasn't even completed 100 generations. Since the run on PC1 was done yesterday, I'm transferring the run on the laptop to continue on PC1.

The run on PC3 on the other hand has stopped, cos the computer crashed, around 2pm, so I continued it around 5pm. Run on PC2 is fine. and finishing today.

UPDATE: So I continued the run from my laptop on PC1, but I discovered that the results don't match, in fact, the results varies so much. One reason 1 observed, is because the settings I had on my laptop for the race is 1 lap! but how come?! Oh I remember, I duplicated the copy I had in my HDD, which probably has a setting of only 1 lap. Hence the flawed results. Hence I had taken to start a new run on PC1. What a waste of time.

Jessica's birthday 2011

Does it make sense to take the best controller of all runs: The best controller of all 10 runs, of the best controller of all 10 controllers of 1000 generations..?

Run on PC1 is done. inserted the results into the spreadsheet.
Arranged the best lap times for tracks 2 and 3.
Included chart of track 2 and 3 into doc file.
Going home to continue.

Produce analysis of results at home!

Today's Jessica's birthday!!