525.721.81 - Advanced Digital Signal Processing

Expanded Course Description

The fundamentals of statistical signal analysis and the associated mathematical algorithms for the processing of wide-sense stationary random signals are presented in this course.  Topics include the mathematical structures of signal spaces, optimal linear filters including the Wiener filter, classical and modern spectral estimation, adaptive implementations of the Wiener filter, and optimal processing of sensor arrays.

Familiarity with linear algebra and multi-variable calculus and the ability to do sustained algebra is required.  Familiarity with basics of digital signal processing is highly recommended.

Course Structure

The course materials are divided into modules which can be accessed by clicking Course Modules on the left menu.   Each module may be divided into more than one video, and includes a homework assignment.  Officially, week begins on Tuesday and ends Monday midnight.  However, modules become available on Saturday mornings for those who wish to get an early start; effectively, there will be approximately 10 days to complete each module; assignments and Discussions are due Tuesdays at midnight.

Course Topics

• Mathematical structures of signal spaces:  Hilbert spaces, orthogonal projections, the approximation problem, prolate spheroidal functions, compressive sensing.
  
• Matrix factorization:  QR, Cholesky, SVD.
• Linear time invariant systems and transforms:  Laplace, Z, and Fourier transforms; phase and group delay response, the Hilbert transform and the analytic signal and applications.
• Least squares filters in frequency and time domains
• Random variables, estimation and information theory:  PCA, KL transformation, estimation of random variables and the Kalman filter, parameter estimation and the CRLB, ICA.
• Wide sense stationary random processes:  auto-correlation and the PSD, MUSIC and ESPRIT algorithms.
• Linear systems and stochastic inputs: filtered random processes, canonical factorization, the Wiener filter.
• PSD estimation and signal models:  periodogram, multi-taper method, MVD spectral estimate, ARMA modeling, AR and maximum entropy, Levinson-Durbin algorithm, AR model parameter estimation.
• Linear prediction:  forward and backward predictors and minimum phase, the Burg method, speech recognition.
• Adaptive filters:  LMS and RLS algorithms.
• Optimal processing of uniform linear arrays:  optimal beamforming, DOA estimation.
• Modern neural networks
  

Course Goals

Mastery of the underlying mathematical principles and algorithms of modern advanced signal processing and their implementation.

Course Learning Outcomes (CLOs)

• • Derive the mathematical structures of signal spaces and apply matrix factorizations to solve least squares problems.
• • Explain convolutional systems (continuous and discrete time), and the associated transforms (Laplace & Z), and characterize minimum phase systems.
• • Describe the Hilbert transform, the analytic signal, and the Wigner time-frequency distribution.
• • Solve for finite discrete time impulse response (FIR) filters using constrained quadratic minimization methods in both frequency and time domains (eigen-filters and least squares filters).
• • Describe stationary random signals, the auto-correlation function, the power spectral density (PSD), and implement algorithms related to the autocorrelation matrix to analyze discrete time data.
• • Derive the non-causal Wiener filter, and implement the classical techniques (Fourier transform based) of spectral estimation in order to use the Wiener filter in a multi-channel application to perform correlation cancellation for signal enhancement.
• • Describe the auto-regressive model for a stationary random signal and the Yule-Walker equations for model parameters, and solve for the parameters using using appropriate estimates for the auto-correlation function obtained from data and the Levinson algorithm; use the derived model to estimate the signal spectrum.
• • Derive the Yule-Walker equations for the discrete time Wiener filter, and analyze forward and backward prediction filters.
• • Derive adaptive LMS (Least Mean Square) and RLS (Recursive Least Square) Wiener filters, implement them, and analyze the performance of the algorithms on discrete time data.
• • Analyze signal and noise models for a uniform linear array of sensors and apply optimal beamforming methods to estimate signal direction of arrival.

Textbooks

"Advanced Signal Processing:  A Concise Guide", Amir-Homayoon Najmi and Todd K. Moon, McGraw Hill (2020)
ISBN: 978-1-260-45893-0

Required Software

Must be familiar with at least one scientific programming language.  For example, MATLAB, IDL, Python, Mathematica.

Student Coursework Requirements

This course will consist of two basic student requirements:

Weekly Assignment (85% of final grade)

Weekly assignments task you to apply the concepts that are covered in the weekly readings and lectures. There are 14 assignments, one each week. They are equally weighted toward your final grade.  Late assignments have their maximum reduced by 10% for the first 3 late days, 20% for the next 3, and 50% for later than 6 days.  If you are asked to resubmit to improve your grade then resubmission must occur no later than 3 days after the grade has been posted.

Discussion Forums (15% of final grade)

Each week, you are required to complete a discussion post explaining point(s) or concept(s) that challenged you during the week. These will help the instructor to offer guidance and feedback as the course progresses. All discussion forums are weighted equally.

Grading Policy

The course grading scale is the following:

100–98 = A+,  97–94 = A,  93–90 = A−,  89–87 = B+,  86–83 = B,  82–80 = B−,  79–70 = C,  <70 = F

Course Policies

Late assignments will lose 10% for the first 3 late days, 20% for the next 3 late days, and 50% for later than 6 days - extenuating circumstances must be communicated to the instructor.  In many cases you may be asked to resubmit an assignment within 3 days; re-submission maximum grad will be 90%. 

Academic Policies

Deadlines for Adding, Dropping and Withdrawing from Courses

Students may add a course up to one week after the start of the term for that particular course. Students may drop courses according to the drop deadlines outlined in the EP academic calendar (https://ep.jhu.edu/student-services/academic-calendar/). Between the 6th week of the class and prior to the final withdrawal deadline, a student may withdraw from a course with a W on their academic record. A record of the course will remain on the academic record with a W appearing in the grade column to indicate that the student registered and withdrew from the course.

Academic Misconduct Policy

All students are required to read, know, and comply with the Johns Hopkins University Krieger School of Arts and Sciences (KSAS) / Whiting School of Engineering (WSE) Procedures for Handling Allegations of Misconduct by Full-Time and Part-Time Graduate Students.

This policy prohibits academic misconduct, including but not limited to the following: cheating or facilitating cheating; plagiarism; reuse of assignments; unauthorized collaboration; alteration of graded assignments; and unfair competition. Course materials (old assignments, texts, or examinations, etc.) should not be shared unless authorized by the course instructor. Any questions related to this policy should be directed to EP’s academic integrity officer at ep-academic-integrity@jhu.edu.

Students with Disabilities - Accommodations and Accessibility

Johns Hopkins University values diversity and inclusion. We are committed to providing welcoming, equitable, and accessible educational experiences for all students. Students with disabilities (including those with psychological conditions, medical conditions and temporary disabilities) can request accommodations for this course by providing an Accommodation Letter issued by Student Disability Services (SDS). Please request accommodations for this course as early as possible to provide time for effective communication and arrangements.

For further information or to start the process of requesting accommodations, please contact Student Disability Services at Engineering for Professionals, ep-disability-svcs@jhu.edu.

Student Conduct Code

The fundamental purpose of the JHU regulation of student conduct is to promote and to protect the health, safety, welfare, property, and rights of all members of the University community as well as to promote the orderly operation of the University and to safeguard its property and facilities. As members of the University community, students accept certain responsibilities which support the educational mission and create an environment in which all students are afforded the same opportunity to succeed academically. 

For a full description of the code please visit the following website: https://studentaffairs.jhu.edu/policies-guidelines/student-code/

Classroom Climate

JHU is committed to creating a classroom environment that values the diversity of experiences and perspectives that all students bring. Everyone has the right to be treated with dignity and respect. Fostering an inclusive climate is important. Research and experience show that students who interact with peers who are different from themselves learn new things and experience tangible educational outcomes. At no time in this learning process should someone be singled out or treated unequally on the basis of any seen or unseen part of their identity. 
 
If you have concerns in this course about harassment, discrimination, or any unequal treatment, or if you seek accommodations or resources, please reach out to the course instructor directly. Reporting will never impact your course grade. You may also share concerns with your program chair, the Assistant Dean for Diversity and Inclusion, or the Office of Institutional Equity. In handling reports, people will protect your privacy as much as possible, but faculty and staff are required to officially report information for some cases (e.g. sexual harassment).

Course Auditing

When a student enrolls in an EP course with “audit” status, the student must reach an understanding with the instructor as to what is required to earn the “audit.” If the student does not meet those expectations, the instructor must notify the EP Registration Team [EP-Registration@exchange.johnshopkins.edu] in order for the student to be retroactively dropped or withdrawn from the course (depending on when the "audit" was requested and in accordance with EP registration deadlines). All lecture content will remain accessible to auditing students, but access to all other course material is left to the discretion of the instructor.