Gradient Descent and Programming Neural Networks

linearly
separable
non-linearly
separable
problemsdifferences
perceptron
gradient descent
delta rule
stochastic
gradient descent

The rule for the perceptron (page 88) and the delta rule (page 93) appear similar. What is the fundamental difference?

Why will the gradient descent algorithm converge?

Why is there only one global minimum?


Introduction to neudl

FAQs from demo.doc

  1. What is NeuDL?

    NeuDL (pronounced noodle) is a Neural-Network Description Language that uses a C-like programming language interface to build, train, and run neural networks. It is currently limited to backpropagation neural networks, but it still shows the flexibility such an interface can give.

  2. Where can NeuDL be downloaded from?

    The University of Alabama: cs.ua.edu (130.160.44.1) in the file /pub/neudl/NeuDLver021.tar This file contains all of the source code, several example NeuDL programs, makefile, user manual, and a short paper describing NeuDL.

  3. What platform was NeuDL written for?

    It was written on an IBM RS/6000 with the xlC compiler, but it has been modified to compile with the GNU g++ compiler and it has been ported to DOS with no changes. It should compile with any good C++ compiler. If not, there should only be minor changes necessary.

  4. Can a neural network trained with NeuDL be used inside another program?

    Yess, NeuDL is simply an interface to a backpropagation neural network object written in C++. It is very easy to embed a network in a program, and then link it with the neural network object. If fact, if a NeuDL program is translated into C++ with the automatic translate feature of the interpreter, it simply creates a C++ program calling the network object.

  5. Is NeuDL an interpreter or a compiler?

    It is primarily an interpreter. It can execute the code directly. However, it has a feature that translate the NeuDL code into C++ which can be compiled with a C++ compiler and then executed. Translating and compiling the NeuDL code can sometime give a tremendous performance boost.

Guide to sample programs
test2.neuxor training
test3.neuxor manual training
test4.neulitho training and testing
test5.neulitho with argument input
test6.neudemo loops, arrays
test7.neuoperators
test8.neucompound statement
test9.neuif statement
test10.neuwhile, print, newline
test13.neuarray, keyboard input
test14.neucopy array
test15.neudump program state variables
test16.neucreate a network
test17.neumanually fully connect network
test18.neucreate training and test sets
test20.neuadd data after training
test21.neuif statement
test22.neufind high, low
test23.neunormalize data
test24.neumanual vs. automatic find high/low
test25.neulitho run, see outputs
test27.neuautomatically remove low weights during training
test28.neuiterate each weight