535.742.81 - Applied Machine Learning for Mechanical Engineers

Expanded Course Description

Machine Learning (ML) is a core technology in developing intelligent systems and has been a focus of substantial research in the past two decades. ML is a tool for predictions, estimations, feature extractions, knowledge discovery, dimensionality reduction, and automation. ML application varies from engineering to medicine. In this course, students will learn some fundamental ML concepts in addition to popular ML and Neural Network (NN) models. This course includes real-life ML examples, including Mechanical Engineering-related examples, through extensive Python 3+ programming and literature reviews, particularly research papers. Those students who are not familiar (mid-level to high-level) with Python 3+, but are familiar (mid-level to high-level) with at least one computational programming language such as MATLAB, C, R, etc., will be given conditional access (free and/or limited) to the instructor's Python 3+ tutorials at www.python3h.com. Mid-level expertise in a computational programming language means that you know, say, data types and how to create if-else (logical) statements, functions, and for-loops. As a mid-level programmer, you are familiar with arrays, variable indexing and variable assignments, and some object-oriented programming concepts. In that case, the Python course would help you, gives you the "minimum" skills required to start and understand the programming lectures/assignments in this course. However, you would need to work hard, particularly on the programming assignments, to catch up. This course would be an opportunity for you to become a professional "Pythonist."

The course will start with the definition of formal optimization, the universal approximation theorem, the data required for training the ML algorithms, and the training and testing procedures in ML problems. Next, supervised and unsupervised algorithms will be discussed with a focus on popular ML and NN models such as support vector machines and probabilistic NNs. Furthermore, some deep learning algorithms such as deep Boltzmann machine and Convolutional NNs will be discussed. In the end, the broad topic of smart systems and the development of multi-paradigm computational models, along with limited supercomputing techniques, are discussed. Through this course, students will learn some general information about these ML models with a focus on the application of them in the field of engineering, especially Mechanical Engineering, with some theoretical and mathematical concepts behind their algorithms.

Prerequisites

Instructor permission or courses that include (1) probability, (2) conditional probability, (3) random variables, (4) expectation, (5) hypothesis testing, (6) statistical characteristics (e.g., means, variances, residuals, p-value, z- score) (7) Bayesian and likelihood (8) covariance and correlation (9) distributions (10) linear algebra, (11) linear and nonlinear transformations, and (12) Python 3+ or at least one computational programming language such as MATLAB, C, C++, R, etc. with medium proficiency. At least one course in list A and one course in list B are recommended:

List A:

• 1) 553.291: Linear Algebra and Differential Equations.
• 2) 553.385: Scientific Computing: Linear Algebra.
• 3) 553.792: Matrix Analysis and Linear Algebra.
• 4) 625.602: Modern Algebra.

List B:

• 1) 553.310: Probability & Statistics for the Physical Sciences & Engineering.
• 2) 553.420: Introduction to Probability.
• 3) 553.720: Probability Theory.
• 4) 553.810: Probability & Statistics.

Course Structure

The course materials are divided into modules, which can be accessed by clicking Modules on the course menu. A module will have several sections, including the overview, content, readings, discussions, and assignments. You are encouraged to preview all sections of the module before starting. Most modules run for seven (7) days; exceptions are noted in the Course Outline. You should regularly check the Calendar and Announcements for assignment due dates.

Course Topics

• Optimization
• Machine Learning Fundamentals
• Popular Supervised Machine Learning Techniques
• Unsupervised Machine Learning Techniques
• Multi-Paradigm Machine Learning Models
• Machine Learning of Dislocation-Induced Stress Fields and Interaction Forces
• Machine Learning Using Supercomputers

Course Goals

Describe some fundamental and advanced Machine Learning (ML) and Neural Network (NN) concepts and apply popular and advanced ML and NN models on popular, standard, and/or mechanical engineering-related examples through extensive Python 3+ programming using Google Colab GPU and CPU resources and MARCC supercomputer.

Course Learning Outcomes (CLOs)

• Describe the basic concepts of optimization, Machine Learning (ML), and Neural Network (NN) techniques.
• Apply ML- and NN-based computational solutions for prediction, estimation, feature extraction, data generation, and dimensionality reduction in Mechanical Engineering problems.
• Design multi-paradigm computational solutions using clever integrations of popular optimization, ML, and NN techniques for knowledge discovery, decision-making, and automation problems in Mechanical Engineering.
• Develop Python 3+ programming scripts to implement several ML- and NN- based computational models.

Textbooks

There are no required textbooks.

Other Materials & Online Resources

Course Readings (see eReserves in Canvas)

1. Adeli, H., & Park, H. S. (1995). A neural dynamics model for structural optimization—theory. Computers & structures, 57(3), 383-390.
2. Ahmadlou, , & Adeli, H. (2010). Enhanced probabilistic Neural Network with local decision circles: A robust classifier. Integrated Computer-Aided Engineering, 17(3), 197-210.
3. Arora, K. (2015). Optimization: algorithms and applications. Chapman and Hall/CRC.
4. Bartholomew-Biggs, (2008). Nonlinear optimization with engineering applications (Vol. 19). Springer Science & Business Media. [LOs 1.2-1.9]
5. Cortes, , & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273-297
6. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial In Advances in neural information processing systems (pp. 2672-2680).
7. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with Neural Networks. science, 313(5786), 504-507.
8. Huang, B., Wang, D. H., & Lan, Y. (2011). Extreme learning machines: a survey. International journal of Machine Learning and cybernetics, 2(2), 107-122.
9. Kingma, P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
10. Rafiei, H., & Adeli, H. (2017). A new neural dynamic classification algorithm. IEEE transactions on Neural Networks and learning systems, 28(12), 3074-3083.
11. Rafiei, M. H., & Adeli, H. (2017). NEEWS: A novel earthquake early warning model using neural dynamic classification and neural dynamic Soil Dynamics and Earthquake Engineering, 100, 417-427.
12. Rafiei, M. H., & Adeli, H. (2018). A novel unsupervised deep learning model for global and local health condition assessment of Engineering Structures, 156, 598-607.
13. Rafiei, H., Kelly, K. M., Borstad, A. L., Adeli, H., & Gauthier, L. V. (2019). Predicting improved daily use of the more affected arm poststroke following constraint-induced movement therapy. Physical Therapy, 99(12), 1667-1678.
14. Rafiei, H., Khushefati, W. H., Demirboga, R., & Adeli, H. (2017). Novel Approach for Concrete Mixture Design Using Neural Dynamics Model and Virtual Lab Concept. ACI Materials Journal, 114(1).
15. Hull, , & Bacon, D. J. (2011). Introduction to dislocations (Vol. 37). Elsevier.
16. Rafiei, M. H., Gu, Y., & El-Awady, J. A. (2020). Machine Learning of Dislocation-Induced Stress Fields and Interaction JOM, 1-13. [11.1 – 11.11]
17. Siddique, , & Adeli, H. (2017). Nature-inspired chemical reaction optimisation algorithms. Cognitive computation, 9(4), 411-422.

Required Software

A web browser, especially Google Chrome, which is free as of January 2021, and a Gmail, which is free as of January 2021.

Student Coursework Requirements

It is expected that each module will take approximately 7 - 10 hours per week to complete. Here is an approximate breakdown: reading the journal papers (approximately 3 - 4 hours per week) as well as some outside reading, listening to the audio annotated slide presentations (approximately 1 - 2 hours per week), and writing/programming assignments (approximately 3 - 4 hours per week).

For assessment, please refer to the relevant Rubric.

Timely feedback on students' performance is an established learning tool, so we will endeavor to grade and return to you, as quickly as possible, all material that you submit. Homework will normally be graded and returned via the website within a week. If you do not receive a grade on homework that you have turned in, please ask of its whereabouts; it may need to be resubmitted.

Students are divided into groups of at least two students. This course will consist of the following basic requirements:

Discussions (After Each Odd Module - By Each Individual Student - 10% of Final Grade Calculation)

Students will be assigned a set of brief topic questions, referred to as module discussions.

Weekly Homework (After Each Module - By Each Individual Student - 35% of Final Grade Calculation)

After each odd module (i.e., modules 1, 3, …, and 13), students will be assigned to read one or more scientific papers/webpage or watch one or more videos and provide short and high-level summaries. After each even module (i.e., modules 2, 4, …, and 14), students will receive a programming assignment to (1) design an ML-based computational solution for mostly a mechanical engineering-based problem, and (2) develop corresponding programming scripts as a ".ipynb" file in Python 3+, and (3) provide short and high-level summaries of the problems, computational solutions, programming scripts, and results.

Midterm Presentation (Midterm - By Each Group - 7.5% of Final Grade Calculation)

Students present a relevant topic of their choice in conjunction with modules 1 to 7 around the midterm to assess students' achievement of the learning objectives.

Midterm Project (Midterm - By Each Group - 20% of Final Grade Calculation)

Students will be assigned an ML-based mechanical engineering-related problem that requires the first seven Modules' knowledge. Through this project, students will (1) design an ML-based computational solution for mostly a mechanical engineering-based problem, and (2) develop corresponding programming scripts as a ".ipynb" file in Python 3+, and (3) provide short and high-level summaries of the problems, computational solutions, programming scripts, and results. The midterm project is mandatory for passing the course.

Final Presentation (Final - By Each Group - 7.5% of Final Grade Calculation)

Students present a relevant topic of their choice in conjunction with modules 8 to 14 around the final to assess students' achievement of the learning objectives.

Final Project (Final – By Each Group - 20% of Final Grade Calculation)

Students will be assigned an ML-based mechanical engineering-related problem that requires all Modules' knowledge. Through this project, students will (1) design an ML-based computational solution for mostly a mechanical engineering-based problem, and (2) develop corresponding programming scripts as a ".ipynb" file in Python 3+, and (3) provide short and high-level summaries of the problems, computational solutions, programming scripts, and results. The final project is mandatory for passing the course.

Office Hours Participation and Critical Thinking (After Each Module - By Each Individual Student – Up to 20% Bonus in the Final Grade Calculation)

Participation, asking questions, showing at office hours, error detection, modification suggestions, interaction with other students over the discussions and presentations, and critical thinking will be appreciated by up to 20% bonus in the Final Grade Calculation.

Grading Policy

Assignments are due according to the dates posted on your Canvas course site. You may check these due dates in the Course Calendar or the Assignments in the corresponding modules. We will post grades one week after assignment due dates.

We generally do not directly grade spelling and grammar. However, egregious violations of the rules of the English language will be noted without comment. Consistently poor performance in either spelling or grammar is taken to indicate poorly written communication ability that may detract from your grade.

A grade of A indicates consistent excellence and distinction throughout the course; that is, conspicuous excellence in all aspects of assignments and discussion every week.

A grade of B indicates work that meets all course requirements on a level appropriate for graduate academic work. These criteria apply to both undergraduates and graduate students taking the course.

EP uses a +/- grading system (see "Grading System," Graduate Programs catalog, p. 10).

The following weighting will determine final grades:

Academic Policies

Deadlines for Adding, Dropping and Withdrawing from Courses

Students may add a course up to one week after the start of the term for that particular course. Students may drop courses according to the drop deadlines outlined in the EP academic calendar (https://ep.jhu.edu/student-services/academic-calendar/). Between the 6th week of the class and prior to the final withdrawal deadline, a student may withdraw from a course with a W on their academic record. A record of the course will remain on the academic record with a W appearing in the grade column to indicate that the student registered and withdrew from the course.

Academic Misconduct Policy

All students are required to read, know, and comply with the Johns Hopkins University Krieger School of Arts and Sciences (KSAS) / Whiting School of Engineering (WSE) Procedures for Handling Allegations of Misconduct by Full-Time and Part-Time Graduate Students.

This policy prohibits academic misconduct, including but not limited to the following: cheating or facilitating cheating; plagiarism; reuse of assignments; unauthorized collaboration; alteration of graded assignments; and unfair competition. Course materials (old assignments, texts, or examinations, etc.) should not be shared unless authorized by the course instructor. Any questions related to this policy should be directed to EP’s academic integrity officer at ep-academic-integrity@jhu.edu.

Students with Disabilities - Accommodations and Accessibility

Johns Hopkins University values diversity and inclusion. We are committed to providing welcoming, equitable, and accessible educational experiences for all students. Students with disabilities (including those with psychological conditions, medical conditions and temporary disabilities) can request accommodations for this course by providing an Accommodation Letter issued by Student Disability Services (SDS). Please request accommodations for this course as early as possible to provide time for effective communication and arrangements.

For further information or to start the process of requesting accommodations, please contact Student Disability Services at Engineering for Professionals, ep-disability-svcs@jhu.edu.

Student Conduct Code

The fundamental purpose of the JHU regulation of student conduct is to promote and to protect the health, safety, welfare, property, and rights of all members of the University community as well as to promote the orderly operation of the University and to safeguard its property and facilities. As members of the University community, students accept certain responsibilities which support the educational mission and create an environment in which all students are afforded the same opportunity to succeed academically. 

For a full description of the code please visit the following website: https://studentaffairs.jhu.edu/policies-guidelines/student-code/

Classroom Climate

JHU is committed to creating a classroom environment that values the diversity of experiences and perspectives that all students bring. Everyone has the right to be treated with dignity and respect. Fostering an inclusive climate is important. Research and experience show that students who interact with peers who are different from themselves learn new things and experience tangible educational outcomes. At no time in this learning process should someone be singled out or treated unequally on the basis of any seen or unseen part of their identity. 
 
If you have concerns in this course about harassment, discrimination, or any unequal treatment, or if you seek accommodations or resources, please reach out to the course instructor directly. Reporting will never impact your course grade. You may also share concerns with your program chair, the Assistant Dean for Diversity and Inclusion, or the Office of Institutional Equity. In handling reports, people will protect your privacy as much as possible, but faculty and staff are required to officially report information for some cases (e.g. sexual harassment).

Course Auditing

When a student enrolls in an EP course with “audit” status, the student must reach an understanding with the instructor as to what is required to earn the “audit.” If the student does not meet those expectations, the instructor must notify the EP Registration Team [EP-Registration@exchange.johnshopkins.edu] in order for the student to be retroactively dropped or withdrawn from the course (depending on when the "audit" was requested and in accordance with EP registration deadlines). All lecture content will remain accessible to auditing students, but access to all other course material is left to the discretion of the instructor.