Information
|
- Instructor: Asst.Prof.Dr. Erol
Sahin
- Office hours: TBA (to be announced)
- Teaching Assistant: None.
- Textbook: [HKP] Introduction
to the Theory of
Neural Computation by Hertz, Krogh, and Palmer. Addison-Wesley,
1991. Available at the Reserve section of the library. The book
should also be available at the university bookstore.
- Course web page: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG569/
- Time: Mondays 13:40-16:30.
- Location: BMB4, Dept. of Computer Eng.
- Communication:
- If you have a specific question you can send
an e-mail to
me but make sure that the subject line starts with CENG569
[capital letters, and no spaces] to get faster reply. In any case,
questions that are general should be posted to the newsgroup.
|
Announcements
- Slides for week 5 and 6 are available
below.
- Readings for next week are available at the photocopy
room.
- Project 2 is now available!
|
Lectures
|
Introduction
to neural nets; the beginnings and the basics
Feb 6

|
- Topics
- Motivation and brief history of neural
networks
- McCulloch-Pitts neuron
- Hebb's learning rule
- Perceptron
- Adaline and the Widrow-Hoff rule
- Readings due:
|
- Slides
- Assignments
given:
|
Multi-layer
Perceptrons and Backpropagation
Feb 13
|
- Topics
- Multi-layer perceptron
- Back-propagation learning.
- Variations of the backpropagation algorithm
- Readings due:
- [HKP], chapter 1, 5.
- [KSJ], chapter 2 [Nerve cells and
behavior]
- [AR] Introductions of chapters 1,2 4,
8, and 10
|
- Slides
- Assignments given:
|
Learning,
generalization and applications of backpropagation
Feb 20

|
- Topics
- Generalization, over-training
- Benchmarking problems
- Applications
- Readings due:
- [HKP] chapter 5-5.4,
and 6.1-6.3.
- [AR] M. Minsky and
S. Papert. Prologue and Epilogue of the book Perceptrons.
- [AR] D. E. Rumelhart,
G. E. Hinton,
and R. J. Williams. Learning Internal Representations by
Error Propagation, volume 1, chapter 41, pages
318-362. in Neurocomputing: Foundations of Research, 1986. Parallel
Distributed Processing: Explorations in the Microstructures of
Cognition (in Neurocomputing: Foundations of Research / Chap. 41).
- [AR] D. E. Rumelhart,
G. E. Hinton,
and R. J. Williams.
|
- Slides
- Links
- Assignments given:
|
Hopfield
Model, Boltzmann Machines
and Simulated Annealing
Feb 27
|
- Topics
- Associative/Content Addressable Memory
- Hopfield model
- Simulated Annealing
- Boltzmann Machines
- Readings due:
|
|
Competitive
Learning and Kohonen Nets and RBF's
March 6

|
- Topics
- Unsupervised learning
- Competitive learning
- Kohonen's Self-Organizing Maps
- Radial Basis Functions
- Readings due:
|
|
Adaptive
Resonance Theory
March 13
|
- Topics
- ART-1
- ART-2 and Fuzzy ART
- ARTMAP
- Readings due:
|
|
Biological
Neurons and neural modeling
March 20

|
- Topics
- The biological neuron
- Ion channels
- Action potential generation
- Hodgkin-Huxley model of a neuron
- Shunting neuron model
- Feed-forward shunting networks
- Readings due:
- Self-organized formation of topologically
correct feature maps, Teuvo Kohonen. Biological Cybernetics, 4:59-69
- Fuzzy ART: Fast Stable Learning and
Categorization of Analog Patterns by an Adaptive Resonance System, Gail
A. Carpenter, Stephen Grossberg and David B. Rosen. Neural Networks,
Vol. 4, pp. 759-771. 1991
- ARTMAP: Supervised real time learning
and classification by a self-organizing neural network. G. A.
Carpenter, S. Grossberg, and J. H. Reynolds. Neural Networks,
4:565--588, 1991.
|
|
No
CLASS
|
|
|
Adaptive
Resonance Theory - II
|
|
|
Review
|
|
|
Student
presentations
-1
|
|
|
Student
presentations
-2
|
|
|
|
|
- Registration: The
size
of the class is limited and therefore, if necessary, I will try to make
the
class available to the most interested students.
- Objective: This course aims to familiarize the
student
with the neural network literature.
- Prerequisites: Consent of the instructor is
essential.
Calculus I and II (or equivalent level of math), Introduction to
Programming (CENG 210-220-230) and a Data Structures course.
- Grading:
- Class participation
(10%)
- 4 Projects
(60%)
- Expected project report quality: Check the following
reports as good examples.
- Paper presentation (10%)
- If you have done a specific project, then you will
present
your work.
- Otherwise, you will be asked to review a set of papers
and
make a 15 minute presentation in class.
- Homeworks (20%)
- All lectures have reading assignments, consisting of
some
chapters from the texbook as well as extra reading material (mostly
papers). Related to these readings, you will be given a short
list of questions to be answered in a prior lecture. The homeworks are
expected to be submitted by the beginning of each class. Late
submissions will not accepted. If
you are not able to read the readings, then do not attempt to cheat by
copying it from another one's homework.
-
Late assignments
- All assignments are due the start of a class. Assignments
submitted after 15 minutes of a class are considered late.
- Homeworks submitted late will not be accepted!
- Project reports submitted late are
eligible for a maximum of 80% credit. Projects
submitted after 2 weeks of their due date are worth at most 60% credit.
Any project not submitted within two weeks of its due date receive no
credit.
- Paper presentations have to be made
at the scheduled time. Re-scheduling [at least one week in
advance] without legal (e.g. health) excuses will cause the loss of 50%
of the credit.
- Presentation guidelines: You are given
one
or two papers to be presented. Your task is to prepare a 20 minute
presentation
about the paper[s]. You should fully understand the papers, and present
it.
The style of presentation is very important. Your slides should contain
the
main points that you will be talking [no full sentences!]. Try to make
your
slides "light" but informative. Resort to images/graphics wherever
possible.
Also rehearse your talk couple of times before your presentation.
During
and after the presentation you will be grilled by questions, and your
performance
in replying is also very important
- Additional material:
[AR]
Neurocomputing: Foundations of Research
Edited
by J. A. Anderson, E.Rosenfeld, MIT Press, Cambridge, 1988. |

|
[DHS] Pattern
Classification
R. O. Duda, P. E. Hart, and H. G. Stork, Wiley-Interscience, New York,
NY, 2 ed., 2000.
Available at the Reserve section of the library. |

|
[K]
Self-organization and associative
memory
by T. Kohonen, Springer-Verlag, 1988.
Available at the Reserve section of the library |

|
[MR] Parallel
Distributed Processing I and
II,
by J. McClelland and D. Rumelhart . MIT Press, Cambridge, MA, 1986. |
 
|
[G] Studies of
the Mind and Brain,
S. Grossberg, Reidel Press, Drodrecht, Holland, 1982. |
|
[CG] Pattern
Recognition by
Self-Organizing Neural
Networks,
Edited by G.A. Carpenter, and S. Grossberg,
Cambridge,
MA, MIT Press, 1994.
|

|
[KSJ] Principles
of Neural
Science,
E.R, Kandel, J.H. Schwartz, and T.M. Jessel,
Appleton
&Lange, 1991. |

|
| Also other complementary
articles that will be made
available |
|
|