605.647.8VL - Neural Networks

Course Structure

The course materials are divided into 14 modules each roughly corresponding to a week of study for the course. These modules can be accessed by clicking Modules on the course menu and will typically be comprised of several components. These components include

• Module Overview page with general descriptions of the module
• Video lectures/content
• Reading assignments and indications of required reading in the text
• Quizzes
• Discussions
  
• Assignments

You are encouraged to preview all sections of the module before starting. Most modules run for a period of seven (7) days, exceptions are noted on the Course Outline page. Students should regularly check the Calendar and Announcements for assignment due dates and any changes or modifications of the course.  Most assignments will become available on Wednesdays, the same day as the class meets.  Due dates are 10 days hence on a Saturday.  This leads to some overlap in the course modules and provides extra time to complete assignments.

Course Topics

Neurons and what is their basic functions
Mathematical Machinery
Perceptrons and Logic: How and why perceptrons can compute logic statements and approximate functions.
Training Perceptrons: Using Supervised Learning techniques
Training multi-layer neural networks using supervised learning techniques
Other Optimization Techniques
Implementation and Performance Considerations, Deep Learning and Convolution Neural Networks
Recurrent Neural Networks and Unsupervised Learning
Variations of the Hopfield Network: Binary Associative Memories
A Stochastic Version of the Hopfield Network: The Boltzmann Machine
A Stochastic Version of the Binary Associative Memory: Restricted Boltzmann Machines
Competitive Learning and Self-Organizing Maps
Neural Network Modifications and Applications
Cellular Neural Networks and the Future of Massively Parallel Computation

Course Goals

The goal of this course is to enable you to identify and describe the mathematical elements, characteristics and behaviors of different types of neural networks. This will involve gaining a thorough background into theoretical and computer programming considerations associated with neural networks. By the end of the course, you will be able to design, train, use and analyze neural networks for practical purposes.

Course Learning Outcomes (CLOs)

• Explain the biological and mathematical foundations of neural network models.
• Evaluate the different types of neural networks and learning models.
• Integrate mathematical competence in understanding neural networks.
• Classify neural networks according to their purposes such as in discriminators, classifiers, computation, and a broad range of problems.
• Implement neural networks using training algorithms such as the feed-forward, back-propagation algorithm.
• Apply computational tools for experimentation to gain new theoretical insights.
• Design, build, and train neural networks for practical purposes.

Textbooks

Rojas, R. (1996). Neural Networks - A Systematic Introduction. Downloadable from http://page.mi.fu-berlin.de/rojas/neural/

Required Software

Programming languages such as Java, C#, C++, C, Python or programming environments such as MATLAB, Octave, Scilab and FreeMat are also satisfactory. (Some lectures will be on Object Oriented Programming using the Python language.)

Student Coursework Requirements

The coursework requirements for this course entail the following components:

1. Completion of all module assignments. These are weekly assignments that test and challenge you on material covered in the module and require a little more thought and effort than the quizzes described below.
2. Completion of all quizzes. There will generally be an online quiz associated with each video lecture. These quizzes will typically involve just a few short answer or true/false questions to help you solidify the material associated with the correspond video lecture.  Answering these questions after watching the video should be quite easy and fun. Therefore, you should answer them immediately following the associated video lecture.
3. Completion of all exams. There will be two to three timed on-line exams. The link to them will be provided in the appropriate Course Module directory along with instructions for completing these exams.
4. Completion of all programming assignments. There will be several specified programming assignments. These programming assignments will be indicated in a course module as “Programming Assignment” which will include specific directions for implementing the particular assignment. A series of related questions in a separate section, Programming Assessment, will require you to execute your program in order to submit appropriate answers.
5. A group project will become available later in the semester and you will be assigned to a team with 1-3 collaborators (other students) to complete this project.  More information on the project will be available when that module component is published.
6. Discussion participation. Timely and thoughtful contributions, and active interaction with classmates in discussion forums.
   

Time Requirements and Expectations

It is expected that each module will take approximately 7–9 hours per week to complete. Here is an approximate breakdown:

• Reading the assigned sections of the texts as well as some outside reading (approximately 1–3 hours per week),
• Listening and viewing video slide presentations (approximately 1–2 hours per week),
• Taking online quizzes (approximately ½ - 1 hour per week)
• Module assignments (approximately 1–2 hours per week)
• Participation in online discussions (approximately ½ - 1 hours per week) Note the following guidelines as it pertains to each course requirement:
  

1. Discussions (5% of Final Grade Calculation)

Posting a response to the discussion question is part one of your grade for module discussions (i.e., Timeliness).

Part two of your grade for module discussion is your interaction (i.e., responding to classmate postings with thoughtful responses) with at least one other classmate (i.e., Critical Thinking). Just posting your response to a discussion question is not sufficient; I want you to interact with your classmates as this facilitates a ‘shared experience’ – an important element of learning. Be detailed in your postings and in your responses to your classmates' postings. Feel free to agree or disagree with your classmates. Please ensure that your postings are civil and constructive.

I will monitor and at times moderate module discussions and will respond to some of the discussions as discussions are posted.

Preparation and participation are evaluated by the following grading elements with the relative weighting indicated in parenthesis:

1. Timeliness (50%)
2. Critical Thinking (50%)

Preparation and participation is graded using a Likert scale of 1 to 5 noted as follows:

5—Timeliness [regularly participates; all required postings; early in discussion; throughout the discussion]; Critical Thinking [rich in content; full of thoughts, insight, and analysis].

4—Timeliness [frequently participates; all required postings; some not in time for others to read and respond]; Critical Thinking [substantial information; thought, insight, and analysis has taken place].

3—Timeliness [infrequently participates; all required postings; most at the last minute without allowing for response time]; Critical Thinking [generally competent; information is thin and commonplace].

2—Timeliness [rarely participates; many postings missing]; Critical Thinking [rudimentary and superficial; little or no analysis or insight is displayed].

1—Timeliness [rarely if ever participates; most, or all required postings missing]; Critical Thinking [rudimentary and superficial; no analysis or insight is displayed].

2. Module Assignments (20% of Final Grade Calculation)

Assignments will be given for each module and due the day before the start of the next module. The exercises include hand (“pencil and paper”) problems and computer-based assignments. For some answers that are not the best answers, partial credit will be given. These assignments only allow for a single attempt. You should therefore consider your answers carefully.

All assignments are due according to the dates indicated in the Calendar. Late assignments will be counted and marked as late in the Grade Center. Late assignments will be scored low depending on how late the assignment is and any extenuating circumstances.

Answers to qualitative questions are evaluated based on the following grading criteria, each according to a Likert scale of 1 – 5:

1. Each part of a question is answered.
2. Writing quality and technical accuracy (Writing is expected to meet or exceed accepted graduate-level English and scholarship standards. That is, all assignments will be graded on grammar and style as well as content.)
3. Rationale for answer is provided.
4. Examples are included to illustrate rationale (If a student does not have direct experience related to a particular question, then the student is to provide analogies versus examples.)
5. Outside references are included.

There will be approximately four programming assignments (with point values indicated) either in addition to or in lieu of Module Assignments.  These programming assignments will require you to implement a neural network design and execute your program to produce data that will help you answer questions. You will also be required to submit your code online so that it can be inspected. The code will, on occasion, be evaluated if your answers to the online questions are incorrect. This will allow for some partial credit to be given and also allow for a determination that your work was your own.

4. Quizzes (10% of Final Grade Calculation)

Brief quizzes will accompany all weekly lectures. Each question will have point values noted. The final grade will be based on the total points obtained and weighed accordingly.

5. Course Project (15% of Final Grade Calculation)

A course project involving a classification problem and the determination of measures of performance will be assigned a few weeks into the course. It may have two components equally weighted: a report and a presentation if time permits.  The presentation will be made via Zoom. This project will be completed in groups.

6. Exams (30% of Final Grade Calculation)

Midterm and final exams will be given and must be completed within 24 hours of beginning. You may refer to any legitimate source of information to complete the exams but may not consult other students or anybody else. If questions arise during the exam, you are advised to contact the instructor. If the question and answer is of a nature that it would benefit others, I will post the particular questions or concerns in the discussion forum for all to review.

Grading Policy

Student assignments are due according to the dates in the Calendar and Assignments items in the corresponding modules.

I generally do not directly grade spelling and grammar. However, egregious violations of the rules of the English language will be noted. Consistently poor performance in either spelling or grammar is taken as an indication of poor written communication ability that may detract from your grade.

Final grades will be determined using the following weighting:

Academic Policies

Deadlines for Adding, Dropping and Withdrawing from Courses

Students may add a course up to one week after the start of the term for that particular course. Students may drop courses according to the drop deadlines outlined in the EP academic calendar (https://ep.jhu.edu/student-services/academic-calendar/). Between the 6th week of the class and prior to the final withdrawal deadline, a student may withdraw from a course with a W on their academic record. A record of the course will remain on the academic record with a W appearing in the grade column to indicate that the student registered and withdrew from the course.

Academic Misconduct Policy

All students are required to read, know, and comply with the Johns Hopkins University Krieger School of Arts and Sciences (KSAS) / Whiting School of Engineering (WSE) Procedures for Handling Allegations of Misconduct by Full-Time and Part-Time Graduate Students.

This policy prohibits academic misconduct, including but not limited to the following: cheating or facilitating cheating; plagiarism; reuse of assignments; unauthorized collaboration; alteration of graded assignments; and unfair competition. Course materials (old assignments, texts, or examinations, etc.) should not be shared unless authorized by the course instructor. Any questions related to this policy should be directed to EP’s academic integrity officer at ep-academic-integrity@jhu.edu.

Students with Disabilities - Accommodations and Accessibility

Johns Hopkins University values diversity and inclusion. We are committed to providing welcoming, equitable, and accessible educational experiences for all students. Students with disabilities (including those with psychological conditions, medical conditions and temporary disabilities) can request accommodations for this course by providing an Accommodation Letter issued by Student Disability Services (SDS). Please request accommodations for this course as early as possible to provide time for effective communication and arrangements.

For further information or to start the process of requesting accommodations, please contact Student Disability Services at Engineering for Professionals, ep-disability-svcs@jhu.edu.

Student Conduct Code

The fundamental purpose of the JHU regulation of student conduct is to promote and to protect the health, safety, welfare, property, and rights of all members of the University community as well as to promote the orderly operation of the University and to safeguard its property and facilities. As members of the University community, students accept certain responsibilities which support the educational mission and create an environment in which all students are afforded the same opportunity to succeed academically. 

For a full description of the code please visit the following website: https://studentaffairs.jhu.edu/policies-guidelines/student-code/

Classroom Climate

JHU is committed to creating a classroom environment that values the diversity of experiences and perspectives that all students bring. Everyone has the right to be treated with dignity and respect. Fostering an inclusive climate is important. Research and experience show that students who interact with peers who are different from themselves learn new things and experience tangible educational outcomes. At no time in this learning process should someone be singled out or treated unequally on the basis of any seen or unseen part of their identity. 
 
If you have concerns in this course about harassment, discrimination, or any unequal treatment, or if you seek accommodations or resources, please reach out to the course instructor directly. Reporting will never impact your course grade. You may also share concerns with your program chair, the Assistant Dean for Diversity and Inclusion, or the Office of Institutional Equity. In handling reports, people will protect your privacy as much as possible, but faculty and staff are required to officially report information for some cases (e.g. sexual harassment).

Course Auditing

When a student enrolls in an EP course with “audit” status, the student must reach an understanding with the instructor as to what is required to earn the “audit.” If the student does not meet those expectations, the instructor must notify the EP Registration Team [EP-Registration@exchange.johnshopkins.edu] in order for the student to be retroactively dropped or withdrawn from the course (depending on when the "audit" was requested and in accordance with EP registration deadlines). All lecture content will remain accessible to auditing students, but access to all other course material is left to the discretion of the instructor.