Instructor Information

Mark Fleischer

Course Information

Course Description

This course provides an introduction to concepts in neural networks and connectionist models. Topics include parallel distributed processing, learning algorithms, and applications. Specific networks discussed include Hopfield networks, bidirectional associative memories, perceptrons, feedforward networks with back propagation, and competitive learning networks, including self-organizing and Grossberg networks. Software for some networks is provided. Prerequisite(s): Multivariate calculus and linear algebra. Course Note(s): This course is the same as 625.638 Neural Networks.

Course Goal

The goal of this course is to enable the students to identify and describe the mathematical elements, characteristics and behaviors of different types of neural networks.This will involve gaining a thorough background into theoretical and computer programming considerations associated with neural networks.By the end of the course, the student will learn how to design, train, use and analyze neural networks for practical purposes.

Course Objectives

  • By the end of this course, students will be able to:

    ·Explain the biological and mathematical foundations of neural network models.

    ·Explain the different types of neural networks and different types of learning models.

    ·Develop mathematical competence for understanding neural networks,

    • Explain which types of neural networks are used for which purposes such as in discriminators, classifiers, computation, and a broad range of problems.
    • Explain how neural networks are implemented using training algorithms such as the feed-forward, back-propagation algorithm.
  • Use computational tools for experimentation leading to new theoretical insights.

    ·Design, build and train neural networks for practical purposes.

When This Course is Typically Offered

The course currently is available as a face-to-face class during the Fall at the Applied Physics Laboratory and available in an online format in the Spring.


  • What are ‘Neurons’ and what is their basic function? Math review
  • Mathematical Machinery and Review
  • Perceptrons and Logic: How and Why Perceptrons Can Compute Logic Statements
  • Training Perceptrons Using Supervised Learning Techniques
  • Training Multi-layer Neural Networks Using Supervised Learning Techniques
  • Other Optimization Techniques
  • Implementation and Performance Considerations
  • Recurrent Neural Networks and Unsupervised Learning
  • Variations on the Hopfield Network
  • A Stochastic Version of the Hopfield Network: The Boltzmann Machine
  • A Stochastic Version of the Binary Associative Memory: Restricted Boltzmann Machines
  • Competitive Learning and Self-Organizing Maps
  • Neural Network Modifications and Applications
  • Cellular Neural Networks and the Future of Massively Parallel Computation

Student Assessment Criteria

Homework assignments 25%
Project proposal 5%
Quizzes 10%
Short project documentation. 10%
Short project presentation 15%
Class Participation 10%
Exam 25%

Computer and Technical Requirements

Some programming capability is essential.  Some open-source neural network design software is available for download from various websites.   Familiarity with MATLAB is also desirable.

Some basic mathematics using matrix algebra, linear algebra and differential calculus will be used in this course and students should have some experience in these areas, even if it is long ago, in manipulating vectors and matrices and using basic elements of the calculus.  There will be some review of the necessary material.

Participation Expectations

An important component of learning and evaluation is the ability to ask good, incisive questions and students will be expected to do this.  I attempt to elicit good participation by asking good questions and provoking good questions from students, but students are also encouraged to ask what they might consider to be "dumb" questions for there are rarely, if ever, 'dumb' questions.  Questions that a student has are probably ones that other students have.  Reading materials and research will be discussed and provide some basis for class participation.

There will be homework assignments (approximately weekly), an exam, a short computer project and presentation and a final exam.  The homeworks usually consists of reading assignments and some problem sets.


Textbook information for this course is available online through the MBS Direct Virtual Bookstore.

Course Notes

There are notes for this course.

Final Words from the Instructor

The text book for this course is available free for download at

(Last Modified: 07/23/2014 04:08:16 PM)