605.742.81 - Deep Neural Networks

Computer Science
Fall 2025

Description

This hands-on course provides a practical introduction to deep neural networks (DNNs), designed to deepen students’ understanding of advanced deep learning (DL) techniques. Modeled after the brain’s architecture, DNNs drive powerful applications in natural language processing (NLP), computer vision (CV), and speech processing—especially with unstructured data like text, images, video, and audio.The course begins with a four-week refresher on machine learning (ML), focusing on model evaluation and feature engineering using Python and Scikit-Learn (SKL). Prior experience with Python and SKL-based ML models is expected. From there, we transition to TensorFlow and Keras, exploring key DNN architectures including convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM), attention mechanisms, generative adversarial networks (GANs), deep reinforcement learning (DRL), transfer learning, and more. Students will read seminal DL papers and collaborate in teams on weekly Kaggle challenges to apply their skills. Teamwork is central to the course, which concludes with a final presentation based on a public ML/DL competition.

Expanded Course Description

This course is taught by Oleg Melnikov (LinkedIn).

Relevant abbreviations/acronyms
ACC=accuracy, Adam=adaptive moment estimation, AE=autoencoder, AGI=artificial general intelligence, AI=artificial intelligence, ANN=artificial neural network, API=application programming interface, AUC=area under the (ROC) curve (AUC), AWS=Amazon Web Services, BASH=Bourne Again SHell, BERT=bidirectional encoder representations from transformers, BL=baseline, BN=batch normalization, BP=backpropagation, BS=bachelor of science, BTW=by the way, CC=carbon copy, CC=creative commons (copyright license), CLI=command line interface, CNN=convolutional neural network, CPU=central processing unit, CS=computer science, CV=computer vision, CV=cross validation, DAG=directed acyclic graph, DB=database, DB=decision boundary, DL=deep learning, DNA=deoxyribonucleic acid, DNN=deep neural network, Doc=document or documentation, DRL=deep RL, DT=decision tree, ed=edition, EDA=exploratory data analysis, ELU=exponential linear unit, ESL=Elements of Statistical Learning (textbook), ET=eastern time, ETL=extract transform load, FC=fully-connected, FFNN=feedforward neural network, FFT=fast Fourier transform, FNN=feedforward neural network, FNR=false negative rate, FP=final project, FPIP=FP interim presentation, FPR=false positive rate, FPVP=FP video presentation, FYI=for your information, GAN=generative adversarial network, GCE=Google Cloud Engine, GCP=Google Cloud Platform, GD=gradient descent, GMM=Gaussian mixture models, GNN=graph neural network, GPU=graphics processing unit, HC=hierarchical clustering, HOML=Hands on Machine Learning (textbook), HTML=hypertext markup language, hr=hour(s), I/O=input/output, ICA=independent component analysis, IDE=integrated development environment, ISL=Introduction to Statistical Learning (textbook), JHU=Johns Hopkins University, kNN=k-nearest neighbor, LB=leaderboard, LDA=linear discriminant analysis, LLE=locally linear embedding, LLM=large language model, LMS=learning management system, LOO=leave one out, LR=learning rate, LSTM=long short-term memory, M=Monday, MAE=mean absolute error, MAPE=mean absolute prediction error, MD=markdown, ML=machine learning, MLE=maximum likelihood estimation/or, MLP=multi-layer perceptron, mo=month(s), MS=master of science, MSE=mean squared error, MWF=Monday-Wednesday-Friday, NB=Naive Bayes, NLP=natural language processing, NLU=natural language understanding, NMF=nonnegative matrix factorization, NN=neural network, OOB=out-of-bag, PCA=principal component analysis, PCR=principal component regression, PhD=doctor of philosophy, PII=personally identifiable information, pt=point, QDA=quadratic discriminant analysis, R=high level programming language, R=Thursday, RAM=rapid access memory, ref=reference, ReLU=rectified linear unit, ResNet=residual network, RF=random forest, RGB=red green blue, RL=reinforcement learning, RMD=R markdown, RMSE=root MSE, RNN=recurrent neural network, ROC=receiver operating characteristic curve, RSS=residual sum of squares, RTL=runtime limit, sec=seconds, SGD=stochastic gradient descent, SKL=scikit-learn (library), SL=supervised learning, SOM=self-organizing maps, SVC=support vector classifier, SVD=singular value decomposition, SVM=support vector machine, T=Tuesday, TA=teaching assistant, TB=TensorBoard, TD=temporal difference, TF=TensorFlow (library), TLU=threshold logic unit, TMI=too much information, TNR=true negative rate, TOC=table of contents, TPR=true positive rate, TPU=tensor processing unit, TR=Tuesday and Thursday, t-SNE=t-distributed stochastic neighbor embedding, TTT=the teaching team, U=university, UL=unsupervised learning, URL=uniform resource locator, VAE=variational autoencoder, W=Wednesday,, wk=week(s), WSE=Whiting School of Engineering, YOLO=you only look once. 
More ML abbreviations (or ask LLM). Did we miss any? Be a ⭐! Let us know. 🥳

Instructor

Profile photo of Oleg Melnikov.

Oleg Melnikov

omelnik2@jh.edu

Course Structure

Course material is divided into weekly modules in Canvas. Each module runs for a period of seven (7) days and contains the following: 

  1. Viewing material: instructor recorded videos and content related YouTube videos
  2. Reading material: assigned chapter(s) for reading and related publication(s)
  3. Auto-graded quizzes on textbook material and related publications. 
  4. Team assignments and group final project.
Teamwork is a critical component of this course, designed not only to enhance your technical skills but also to develop your soft skills, which are equally important in real-world scenarios. The course includes 10 weekly competitions that account for 30% of your overall grade. These competitions are structured to encourage collaboration and problem-solving within a team setting.

You should regularly check the upcoming deadlines in the Canvas Calendar, recent Canvas Announcements, Discussion Board for assignment due dates. JHU Academic Calendar is here

All dates and times (for assignments, Zoom meetings, etc.) are in ET zone! Each module (a.k.a. "week") starts each week and lasts 1 week. The assignments are released weekly and, typically, have 2 week deadline on Sundays at 11:59 pm ET.

Course Topics

We will closely follow the topics in HOML textbook (2nd or 3rd edition): Géron, Aurélien. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. We will rapidly advance through HOML part I to cover classical ML, build up on terminology and Kaggle/Colab/Competition skills/grading. Weekly modules (topics):

  1. HOML Ch.1-2: Machine Learning Fundamentals
  2. HOML 3-4: Classification and Regression
  3. HOML 6-7: Decision Trees and Ensembles
  4. HOML 8-9: Dimension Reduction, Unsupervised Learning
  5. HOML 10: Artificial Neural Network (ANN) with Keras
  6. HOML 11: Training a Deep Neural Network (DNN)
  7. HOML 12: Custom Models and Training with TensorFlow (TF)
  8. HOML 13: Load and Preprocess Data with TF
  9. HOML 14: Deep Computer Vision (CV) and Convolutional Neural Networks (CNN)
  10. HOML 15: Process Sequences Using Recurrent Neural Networks (RNN) and CNNs
  11. HOML 16: NLP with RNNs and Attention
  12. HOML 17: Representation Learning and Generative Adversarial Networks (GAN)
  13. HOML 18: Reinforcement Learning
  14. HOML 19: Train and Deploy Scale
P.S.  We are updating our content to match HOML 2nd & 3ed and can provide limited support for 3ed.

Course Goals

To introduce the students to the variety of deep learning (DL) models, their construction, evaluation and application.
To enable students to tie Python programming code to mathematics/statistics in order to solve real world problems we face today and in the future.
To foster the development of teamwork and collaboration skills through weekly competitions, encouraging students to engage in problem-solving within a team setting, and preparing them for the collaborative nature of the modern workplace. 

Course Learning Outcomes (CLOs)

Textbooks

Required:

Optional (free):
  1. Deep learning, Ian Goodfellow, et. al. 2016. MIT Press
  2. Pattern recognition and machine learning. Svensén, M. and Bishop, C.M., 2007.
  3. Dive into deep learning. Zhang, A., Lipton, Z.C., Li, M. and Smola, A.J., 2021. arXiv:2106.11342.
  4. Foundations of machine learning. Mohri, M., Rostamizadeh, A. and Talwalkar, A., 2018. MIT press.
  5. An introduction to statistical learning (Vol. 112, p. 18). James, G., Witten, D., Hastie, T. and Tibshirani, R., 2013. New York: Springer.
  6. The elements of statistical learning (Vol. 1, No. 10). Friedman, J., Hastie, T. and Tibshirani, R., 2001. New York: Springer series in statistics.
  7. Applications of deep neural networks. Heaton, J., 2020. arXiv:2009.05673.
Prerequisite Material:
  1. Mathematics for machine learning. Deisenroth, M.P., Faisal, A.A. and Ong, C.S., 2020. Cambridge University Press. (Source)
  2. The Matrix Cookbook. Petersen, K.B. and Pedersen, M.S., 2008. Technical University of Denmark, 7(15), p.510. (Source)
  3. Beginner's Guide to Python and Python Tutorial. Python.org

Other Materials & Online Resources

Students need a Google Drive account (available for free with Gmail account) with a Google Colab plugin. Colab is a Jupyter Notebook-like interface, which provides a seamless access to Python, TensorFlow, visualizations and (free) CPU/GPU/TPU hardware from Google. Colab ensures identical programming environment among all participants of the course.

Note: This is an individual (not a team) assignment. Please don't post the copyrighted material in the public space outside of our course; the content takes many hundreds of hours to create and maintain. See JHU's Code of Conduct, Honor Code, and Academic Integrity Policy. Please report suspected cheating incidents to TTT. Do not post your solutions (coding or quizzes) in public space since this will hinder creativity of future students.

Required Software

We use Google Colab Notebooks for programming assignments, which is pre-configured with Python 3.x and most packages/versions already installed. Google frequently updates Colab's environment occasionally causing incompatible settings or packages, so please let us know if any of this comes up and you notice an error in Colab's code. We'll fix it for everyone.

Student Coursework Requirements

Final grades will be determined by the following weighting:

  1. 40%: Team-based competitions in Kaggle (graded on leaderboard performance, compliance with rules, documentation, team formations, etc.)
  2. 20%: Auto-graded quizzes
  3. 10%: Publication quizzes, occasional in-Canvas surveys. 
    1. We may extra-reward students which are active in Canvas discussions (for helping others and raising relevant insightful questions).
  4. 30%: Final project and presentation (e.g. Kaggle competition).
    1. All final projects are group projects. 
    2. The project is done in 2 stages - interim submission is 15% and final submission is 15% of the total grade
Grading Kaggle competitions.

Participating teams are graded generously, typically earning 80-100% (per assignment overall) if exceeded the baseline and adhered to the rules. Note: there is no "correct" solution in these competitions. Each week, we introduce new concepts and techniques, providing a range of approaches for you to explore/learn. Success in these competitions is not about being the best but about engaging with the process, teamworking, meeting the baseline criteria, and demonstrating effort, which can typically result in earning 80-90% per assignment. See grading rubric.

Grades for the Kaggle competition assignments are determined as follows (percentages are approximate, subject to change; see grading rubric):

The "baseline score" refers to the Starter Colab's LB performance provided to all students. It serves as the minimum standard that students are expected to meet or exceed in their competition submissions.

The competition assignment grades are adjusted based on several factors, including (see grading rubric):

In general, we try to objectively/systemtatically grade Kaggle competitions so that the grades are private (i.e. Leaderboard (LB) performance alone is insufficient to deduce the grade of other participants) and the grades remain withing 80-100% range in the competitions assignments category (again, see Syllabus), which are 40% of the course grade. 

Top LB scores can significantly drop if rules are not followed, while lower-performing submissions can receive higher grades if they adhere to the guidelines or excel in regards to speed, documentation, and references. Colab submissions are peer-reviewed after each competition ends.

Grading Policy

Score Range≥97≥94≥90≥87≥83≥80≥77≥73≥70≥67≥63≥0
Letter Grade A+AA-B+BB-C+CC-D+DF
.

Course Evaluation

Regrading:

  1. We aim to grade fairly, accurately, and timely. If you believe we made a crude grading error, make a private post in our discussion forum ASAP (within 1 week of grading) & tag a grader.
  2. To discourage frivolous appeals, we reserve the right to deduct a 2-5% of the grade, if your appeal lacks a strong justification or the benefit fails to exceed 2-5%. Be sure it is worth the mutual effort.
  3. If in doubt, always use the course textbook as the main source of truth. It has been well edited, provides full context, and has very few typos.
Plagiarism is a serious offense. Always reference your sources of code and ideas. We encourage learning from StockOverflow, GitHub, Kaggle Discussions, etc., but the key solution and contribution must be your own (and be clearly demonstrated).
  1. Rule 0: If ever in doubt, just ask!
  2. Every output (plot, table, metrics, ...) must have a purpose and an interpretation.
    1. We never produce outputs for the sake of filling the void.
  3. Output must be legible, readable, meaningfully organized, neat, clean, compact, precise, and concise.
  4. Avoid generic terms, such as data and model. Instead, be specific and use observationsfeatureslinear regressionlogistic regression, etc.
  5. No overplotting. No redundant plots. No data dumps (pages of table outputs).
  6. Annotate your programming code! Help your audience to understand your abstraction.
  7. All code must execute in Google Colab and Python 3.x without error/warning messages or local file dependencies.
  8. Always start your modeling with a simple baseline model (version 0), which you can do with minimal data munging.
    1. This is your Proof of Concept and benchmark for any future improvement.
  9. Start your work early! Never procrastinate. We are busy on assignment due days and may be inaccessible to answer last minute questions.
  10. Clearly label problems and order them as assigned (to help us avoid missing your solutions)
  11. IF late submissions are allowed, there will be 10% penalty per day. (Start early, manage your time, expect to get stuck!)
Quiz:
  1. Students are given 2 3 😉 quiz attempts with the highest grade recorded.
  2. Multi-choice questions are multi-answer. There are always 4 answer choices, each can be correct or not. The number of correct answer choices can vary from 0 to 4.
  3. The student can view the question's grade and their answer selections, but not correct/incorrect while the quiz is open. After the quiz closes, a full scoring is released. Thus, the grade of <100% indicates some missed or incorrect choices.
  4. Most questions show a hint if answered with <100%. Canvas has no direct way to post solutions after the quiz closes. If marking is unclear, please make a post in Q&A Discussion (after the quiz is closed), and we'll provide more clarity in reasoning.
  5. Incorrect response lowers your score. Correct response raises your score (to avoid guessing). Lowest grade is zero. Hence, it rewards partial credit.
    1. Here are examples with 2 correct answer choices (say, A,B), assuming 2 point value of the question (i.e. 1 point for each answer choice):
      1. Ex1: Student selects A & B, all correct answer choices only. Grade=max(0,1+1)=2
      2. Ex2: Student selects 1 correct only. Grade=max(0,1+0)=1 (out of 2 points or 50%)
      3. Ex3: Student selects 1 correct and 1 incorrect. Grade=max(0,1-1)=0
      4. Ex4: Student selects 2 correct and 1 incorrect. Grade=max(0,1+1-1)=1
      5. Ex5: Student selects 1 correct and 2 incorrect. Grade=max(0,1-1-1)=0
      6. Ex6: Student selects all answer choices. Grade=max(0,1+1-1-1)=0
      7. Ex7: Student leaves all answer choices unselected. Grade=max(0,-1-1)=0
  6. Bonus Points for Quiz Feedback. We reward bonus points for reporting unclear/confusing/misleading/incorrect quiz answer choices (with an explanation helpful in fixing these). Minor issues like typos or missing punctuation generally won’t qualify unless they cause confusion, but we do appreciate all feedback (for any part of the course organization and its content) and may reward for active participation. Points are awarded at the instructor’s discretion based on the relevance and impact of the feedback.
  7. Quiz question concerns: When emailing TTT directly about a seemingly-confusing or allegedly-wrong quiz question, please cite the quiz question and each answer choice - verbatim, state your answers and the corresponding rationales (cite HOML textbook or lecture notes/videos). This helps us understand your reasoning in the proper context. We will not share the answers while the quiz is open, but can validate the question phrasing, fix+regrade (if needed), or issue a hint (to the whole class).
  1. Grade disputes should be raised within 1 week of grade release. Please carefully monitor your grades in Canvas Gradebook. For example, Canvas may auto-apply last submission's late penalty to the highest grade for the assignment. We will not honor the correction, if it's noted outside of grade dispute interval.
  2. Quiz Solutions can be requested only after the quiz has closed (to prevent cheating and maintain fairness). The quiz bank is too large, so we will release only solutions to questions of concerns, since Canvas can't auto-release solutions on individual ("question-seen") basis. Just make a post in Q&A discussion and we'll reply with a solution (after the quiz closes).
  3. Answer hints: A single hint is displayed regardless of the selected answer choices after your 1st attempt and thereafter. Thus, if you accidentally-correctly guess an answer, you can still learn from the hint. Canvas does not allow attempt-specific hints (such as increasingly expanded hints with continued under-par attempts).

    The hints are provided regardless of your correct or incorrect answering of the question. However, Canvas displays the fraction of question's credit earned. If incomplete credit is earned for the question, then the student can consider the hint and each answer choice made. We leave this learning up to the students until the quiz is closed. Thereafter, we can help with the solution.
Participation.

There are 2 evaluations for each week's Kaggle assignment:

  1. Colab Eval (in Canvas as "Peer Review"): an opportunity to learn from the 2-3 shared Colabs and share improvements with peers who may be your teammates in the upcoming competitions.
    1. Ungraded in both directions
    2. Canvas's randomly assigns Colabs from anyone in the class, so you may see 
      1. a Colab from your teammate - be constructively frank in your eval
      2. the same Colab from several teammates of the same team - just copy-paste your eval
  2. Teammate Eval (in Google Form): measures your teammates' performance.
    1. Anonymous.
    2. Ungraded for reviewer, but aggregated evals can be used to adjust assignment/participation/course grade for the reviewee.
    3. It's the same Google Form for all evals, so you can re-edit it at anytime
Please complete the evals within a week to increase their effectiveness and to use fresh recalls.
FYI: in the past, we used Teammate Evals to boost course grades for 5-10 top students and leave the grades as they are for students with poor evals.

Course Policies

RE: Assignments affected by illness. If you caught a flu or some other short-term sickness, we can give you an extension on quizzes and other autograded assignments (please notify TTT of your situation, the affected assignments and the proposed deadline). It's a good idea to attach whatever you have completed at the moment to have it timestamped. We cannot pause or postpone deadlines for Kaggle competitions, however. Thankfully, you can coordinate and rebalance the workload with your teammates. Remember, the team is a single entity - you win together and you lose together - as one. Just in case, please familiarize yourselves with the JHU's policy on absences due to illness.

Academic Policies

Deadlines for Adding, Dropping and Withdrawing from Courses

Students may add a course up to one week after the start of the term for that particular course. Students may drop courses according to the drop deadlines outlined in the EP academic calendar (https://ep.jhu.edu/student-services/academic-calendar/). Between the 6th week of the class and prior to the final withdrawal deadline, a student may withdraw from a course with a W on their academic record. A record of the course will remain on the academic record with a W appearing in the grade column to indicate that the student registered and withdrew from the course.

Academic Misconduct Policy

All students are required to read, know, and comply with the Johns Hopkins University Krieger School of Arts and Sciences (KSAS) / Whiting School of Engineering (WSE) Procedures for Handling Allegations of Misconduct by Full-Time and Part-Time Graduate Students. This policy prohibits academic misconduct, including but not limited to the following: cheating or facilitating cheating; plagiarism; reuse of assignments; unauthorized collaboration; alteration of graded assignments; and unfair competition. Course materials (old assignments, texts, or examinations, etc.) should not be shared unless authorized by the course instructor. Any questions related to this policy should be directed to EP’s academic integrity officer at ep-academic-integrity@jhu.edu.

Students with Disabilities - Accommodations and Accessibility

Johns Hopkins University is committed to providing welcoming, equitable, and accessible educational experiences for all students. If disability accommodations are needed for this course, students should request accommodations through Student Disability Services (SDS) as early as possible to provide time for effective communication and arrangements.  For further information about this process, please refer to the SDS Website.

Student Conduct Code

The fundamental purpose of the JHU regulation of student conduct is to promote and to protect the health, safety, welfare, property, and rights of all members of the University community as well as to promote the orderly operation of the University and to safeguard its property and facilities. As members of the University community, students accept certain responsibilities which support the educational mission and create an environment in which all students are afforded the same opportunity to succeed academically.  For a full description of the code please visit the following website: https://studentaffairs.jhu.edu/policies-guidelines/student-code/

Classroom Climate

JHU is committed to creating a classroom environment that values the diversity of experiences and perspectives that all students bring. Everyone has the right to be treated with dignity and respect. Fostering an inclusive climate is important. Research and experience show that students who interact with peers who are different from themselves learn new things and experience tangible educational outcomes. At no time in this learning process should someone be singled out or treated unequally on the basis of any seen or unseen part of their identity.    If you have concerns in this course about harassment, discrimination, or any unequal treatment, or if you seek accommodations or resources, please reach out to the course instructor directly. Reporting will never impact your course grade. You may also share concerns with your program chair, the Assistant Dean for Diversity and Inclusion, or the Office of Institutional Equity. In handling reports, people will protect your privacy as much as possible, but faculty and staff are required to officially report information for some cases (e.g. sexual harassment).

Course Auditing

When a student enrolls in an EP course with “audit” status, the student must reach an understanding with the instructor as to what is required to earn the “audit.” If the student does not meet those expectations, the instructor must notify the EP Registration Team [EP-Registration@exchange.johnshopkins.edu] in order for the student to be retroactively dropped or withdrawn from the course (depending on when the "audit" was requested and in accordance with EP registration deadlines). All lecture content will remain accessible to auditing students, but access to all other course material is left to the discretion of the instructor.