This course provides an introduction to concepts in neural networks and connectionist models. Topics include parallel distributed processing, learning algorithms, and applications. Specific networks discussed include Hopfield networks, bidirectional associative memories, perceptrons, feedforward networks with back propagation, and competitive learning networks, including self-organizing and Grossberg networks. Software for some networks is provided. Prerequisite(s): Multivariate calculus and linear algebra. Course Note(s): This course is the same as EN.605.647 Neural Networks.
The goal of this course is to enable the students to identify and describe the mathematical elements, characteristics and behaviors of different types of neural networks.This will involve gaining a thorough background into theoretical and computer programming considerations associated with neural networks.By the end of the course, the student will learn how to design, train, use and analyze neural networks for practical purposes.
By the end of this course, students will be able to:
·Explain the biological and mathematical foundations of neural network models.
·Explain the different types of neural networks and different types of learning models.
·Develop mathematical competence for understanding neural networks,
·Explain which types of neural networks are used for which purposes such as in discriminators, classifiers, computation, and a broad range of problems.
·Explain how neural networks are implemented using training algorithms such as the feed-forward, back-propagation algorithm.
·Use computational tools for experimentation leading to new theoretical insights.
·Design, build and train neural networks for practical purposes.
When This Course is Typically Offered
The course currently is available as a face-to-face class during the Fall at the Applied Physics Laboratory and available in an online format in the Spring.
- What are ‘Neurons’ and what is their basic function?
- Mathematical Machinery and Review
- Perceptrons and Logic: How and Why Perceptrons Can Compute Logic Statements
- Training Perceptrons Using Supervised Learning Techniques
- Training Multi-layer Neural Networks Using Supervised Learning Techniques
- Other Optimization Techniques
- Implementation and Performance Considerations
- Recurrent Neural Networks and Unsupervised Learning
- Variations on the Hopfield Network
- A Stochastic Version of the Hopfield Network: The Boltzmann Machine
- A Stochastic Version of the Binary Associative Memory: Restricted Boltzmann Machines
- Competitive Learning and Self-Organizing Maps
- Neural Network Modifications and Applications
- Cellular Neural Networks and the Future of Massively Parallel Computation
Student Assessment Criteria
Computer and Technical Requirements
Some programming capability is desirable, but the capability to use software is essential. Some open-source neural network design software is available for download and students who wish to use such tools need to have some comfort-level downloading and using such software.
Some basic mathematics using matrix algebra will be covered and students should have some experience, even if it is long ago, in manipulating vectors and matrices and using basic calculus although there will be some review of this material.
Students will be expected to ask good questions based on their reading and homework assignments. An important component of learning and evaluation is the ability to ask good, incisive questions. While I attempt to elicit good participation by asking good questions myself and provoking good questions from students, students are also encouraged to ask what they might consider to be "dumb" questions for there are rarely, if ever, truly 'dumb' questions. Questions that a student has are probably ones that other students have.
There will be homework assignments (approximately weekly), an exam and a final project and presentation. The homeworks usually consists of reading assignments and some problem sets.
Textbook information for this course is available online through the MBS Direct Virtual Bookstore.
There are notes for this course.
Final Words from the Instructor
A textbook is available for download from
(Last Modified: 07/23/2014 06:14:21 PM)