What are some interesting and relatively simple to implement projects to learn Neural Networks?
Optimisation problems. Anything where an near-optimal solution is good enough and where an appropriate fitness function can be formulated.
Basically, search and optimisation. A brief overview is at the wikipedia page
Life is using them for about 4 billion years with very interesting results.
Drawing the mona lisa
Two simple and clear examples:
Timetable: It was used to build the timetable of BCN’92 Olympic Games. If there is a change, you can have some new solution in short time.
Route: It can be used to improve the planning on a transport company. Imagine a company with 30 lorries in which the delivery route changes every day. Having the packets for tomorrow, they will put their specific GA to work at night so in the morning they will have a sub-optimal solution (in terms of distance, area, packet type or whatever they choose).
The good thing is that GA will give you a sub-optimal solution in a short time (seconds to hours, depending on the problem hardness) and every new iteration is a better one, so you stop it when you need it.
Training neural networks for example.
Google has quite a lot of results for this: http://lmgtfy.com/?q=training+neural+networks+with+genetic+algorithms
And I’ve actually used a .NET (also a Java version) library for doing just this: http://www.heatonresearch.com/encog
(They have a really nice demonstration on this by the way (look for the folder named “Chapter06GeneticTSP”). It solves the travelling salesman problem using a feed-forward neural network that is being trained by a genetic algorithm.)
I’m using GAs for model fitting, which is in some sense an optimization problem. Any problem that..
- .. cannot be stated in closed form..
- .. is not (at least once) continuously differentiable..
- .. is highly non-linear..
..is a good candidate for GAs.
The real trick is to come up with a good data representation and to tweak the genetic operators for fast convergence.
There are also a class of problems where GAs definitely is not a good choice:
- High-dimensional problems, which usually correlates with long chromosome length. Anything over 100 bit chromosome length, and you’re probably pushing it even as it is. Finding a good solution is mostly blind luck than real effort on your part.
- Anything that is readily solved by a modern gradient-based method. For almost anything that can be represent in closed form and differentiated, a tried-and-true MMA, BFGS or SLP (depending on circumstances) would work like a charm.
NASA has ‘evolved’ the design of an antenna.