METU
Middle East Technical University
Ankara, Turkey
CENG569
Neurocomputing
Spring 2005-2006
CENG
Department of Computer Engineering
Information
Lectures
Policies

Information

  • Instructor: Asst.Prof.Dr. Erol Sahin
  • Office hours:  TBA (to be announced)
  • Teaching Assistant: None.
  • Textbook: [HKP] Introduction to the Theory of Neural Computation by Hertz, Krogh, and Palmer. Addison-Wesley, 1991.  Available at the Reserve section of the library. The book should also be available at the university bookstore. 
  • Course web page: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG569/
  • Time: Mondays 13:40-16:30.
  • Location: BMB4, Dept. of Computer Eng.
  • Communication:
    • If you have a specific question you can  send an e-mail to me but make sure that the subject line starts with CENG569 [capital letters, and no spaces] to get faster reply. In any case, questions that are general should be posted to the newsgroup.
Announcements
  • Slides for week 5 and 6 are available below.
  • Readings for next week are available at the photocopy room.
  • Project 2 is now available!

Lectures

Introduction to neural nets; the beginnings and the basics

Feb 6
dam.gif
  • Topics
    • Motivation and brief history of neural networks
    • McCulloch-Pitts neuron
    • Hebb's learning rule
    • Perceptron
    • Adaline and the Widrow-Hoff rule
  • Readings due:
    • None.
Multi-layer Perceptrons and  Backpropagation

Feb 13
  • Topics
    • Multi-layer perceptron
    • Back-propagation learning. 
    • Variations of the backpropagation algorithm
  • Readings due:
    • [HKP], chapter 1, 5.
    • [KSJ], chapter 2 [Nerve cells and behavior]
    • [AR] Introductions of chapters 1,2 4, 8, and 10
Learning, generalization and applications of backpropagation

Feb 20

The image “http://www.cs.cmu.edu/Groups/ahs/images/navlab_1_5_images/navlab1.color.small.gif� cannot be displayed, because it contains errors.
  • Topics
    • Generalization, over-training
    • Benchmarking problems
    • Applications
  • Readings due:
    • [HKP]  chapter 5-5.4, and 6.1-6.3.
    • [AR] M. Minsky and S. Papert. Prologue and Epilogue of the book Perceptrons.
    • [AR]  D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning Internal Representations by Error Propagation, volume 1, chapter 41, pages 318-362. in Neurocomputing: Foundations of Research, 1986. Parallel Distributed Processing: Explorations in the Microstructures of Cognition (in Neurocomputing: Foundations of Research / Chap. 41).
    • [AR]  D. E. Rumelhart, G. E. Hinton, and R. J. Williams.
Hopfield Model, Boltzmann Machines and Simulated Annealing

Feb 27

 
Competitive Learning and Kohonen Nets and RBF's

March 6

The image “http://www.public.asu.edu/~droussi/ipm/Image81.gif� cannot be displayed, because it contains errors.
  • Topics
    • Unsupervised learning
    • Competitive learning
    • Kohonen's Self-Organizing Maps
    • Radial Basis Functions
  • Readings due:
    • None.
Adaptive Resonance Theory

March 13



 
  • Topics
    • ART-1
    • ART-2 and Fuzzy ART
    • ARTMAP
  • Readings due:


Biological Neurons and neural modeling

March 20


  • Topics
    • The biological neuron
    • Ion channels
    • Action potential generation
    • Hodgkin-Huxley model of a neuron
    • Shunting neuron model
    • Feed-forward shunting networks
  • Readings due:
    • Self-organized formation of topologically correct feature maps, Teuvo Kohonen. Biological Cybernetics, 4:59-69
    • Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System, Gail A. Carpenter, Stephen Grossberg and David B. Rosen. Neural Networks, Vol. 4, pp. 759-771. 1991
    • ARTMAP: Supervised real time learning and classification by a self-organizing neural network.  G. A. Carpenter, S. Grossberg, and J. H. Reynolds. Neural Networks, 4:565--588, 1991.

  • Slides

  • Links
No CLASS




Adaptive Resonance Theory - II


  • Topics

  • Readings due:

  • Slides

  • Links
Review
  • Topics

  • Readings due:

  • Slides

  • Links
Student presentations
-1
  • Topics

  • Readings:

  • Slides

  • Links
Student presentations
-2
  • Topics

  • Readings:

  • Slides

  • Links

Policies

  • Registration: The size of the class is limited and therefore, if necessary, I will try to make the class available to the most interested students.
  • Objective: This course aims to familiarize the student with the neural network literature.
  • Prerequisites: Consent of the instructor is essential.  Calculus I and II (or equivalent level of math), Introduction to Programming (CENG 210-220-230) and a Data Structures course.
  • Grading:
    • Class participation (10%)
    • 4 Projects (60%)
    • Paper presentation (10%)
      • If you have done a specific project, then you will present your work.
      • Otherwise, you will be asked to review a set of papers and make a 15 minute presentation in class.
    • Homeworks (20%)
      • All lectures have reading assignments, consisting of some chapters from the texbook as well as extra reading material (mostly papers).  Related to these readings, you will be given a short list of questions to be answered in a prior lecture. The homeworks are expected to be submitted by the beginning of each class. Late submissions will not accepted. If you are not able to read the readings, then do not attempt to cheat by copying it from another one's homework.
  • Late assignments

    • All assignments are due the start of a class. Assignments submitted after 15 minutes  of a class are considered late.
    • Homeworks submitted late will not be accepted!
    • Project reports submitted late are eligible for a maximum of 80% credit.  Projects submitted after 2 weeks of their due date are worth at most 60% credit. Any project not submitted within two weeks of its due date receive no credit.
    • Paper presentations have to be made at the scheduled time.  Re-scheduling [at least one week in advance] without legal (e.g. health) excuses will cause the loss of 50% of the credit. 
      • Presentation guidelines: You are given one or two papers to be presented. Your task is to prepare a 20 minute presentation about the paper[s]. You should fully understand the papers, and present it. The style of presentation is very important. Your slides should contain the main points that you will be talking [no full sentences!]. Try to make your slides "light" but informative. Resort to images/graphics wherever possible. Also rehearse your talk couple of times before your presentation. During and after the presentation you will be grilled by questions, and your performance in replying is also very important
  • Additional material:
[AR] Neurocomputing: Foundations of Research
Edited by J. A. Anderson, E.Rosenfeld,  MIT Press, Cambridge, 1988.

[DHS] Pattern Classification
R. O. Duda, P. E. Hart, and H. G. Stork, Wiley-Interscience, New York, NY, 2 ed., 2000.
Available at the Reserve section of the library.

[K] Self-organization and associative memory
by T. Kohonen, Springer-Verlag, 1988.
Available at the Reserve section of the library

[MR] Parallel Distributed Processing I and II,
by J. McClelland and D. Rumelhart . MIT Press, Cambridge, MA, 1986.

[G] Studies of the Mind and Brain,
S. Grossberg, Reidel Press, Drodrecht, Holland, 1982.

[CG] Pattern Recognition by Self-Organizing Neural Networks,
Edited by G.A. Carpenter, and S. Grossberg,  Cambridge, MA, MIT Press, 1994.

[KSJ] Principles of Neural Science,
E.R, Kandel, J.H. Schwartz, and T.M. Jessel, Appleton &Lange, 1991.

Also other complementary articles that will be made available