📘 Syllabus (CSE473 - Theory)

• Unit 1: Core Concepts of Machine Learning
  • A. What is Machine Learning? The ML mindset; ML problem framing (common ML problems, use cases, identifying good vs hard ML problems); ML applications (image & speech recognition, medical diagnosis, statistical arbitrage, learning associations); standard learning tasks (classification, regression, ranking, clustering, dimensionality reduction / manifold learning)
  • B. Learning stages (features, labels, hyperparameters, validation & test samples, loss function, hypothesis tests); learning scenarios (supervised, unsupervised, semi-supervised, transductive inference, online, reinforcement, active learning); generalization
  • C. Data preparation & feature engineering; data preprocessing & data quality; data cleaning (missing values, noisy data); data integration; data reduction (attribute subset selection, histograms, sampling, aggregation); data transformation & discretization (normalization, binning, concept hierarchy)
• Unit 2: Supervised Learning Algorithms - Part One
  • A. How supervised learning algorithms work; bias–variance trade-off; function complexity vs amount of training data; dimensionality of the input space; noise in output values; data heterogeneity, redundancy, interactions and non-linearities
  • B. Linear Regression: model representation, simple linear regression, ordinary least squares, gradient descent; regularization / shrinkage (Lasso & Ridge), overfitting; cost function, feature scaling, normalization, learning rate, convergence test
  • C. Logistic Regression: the logistic model, odds, odds ratio & logit; model fitting (IRLS, goodness of fit, limitations); Linear Discriminant Analysis (LDA); applications (bankruptcy prediction, face recognition, marketing, biomedical studies)
• Unit 3: Supervised Learning Algorithms - Part Two
  • A. Support Vector Machines: linear SVM (hard-margin, soft-margin), non-linear classification, computing the SVM classifier (primal, dual, kernel trick), sub-gradient & coordinate descent, empirical risk minimization, hinge loss, parameter selection
  • B. Artificial Neural Networks: feed-forward network functions, weight-space symmetries; network training & parameter optimization; gradient descent optimization; error backpropagation and its efficiency
  • C. Decision Tree Learning: tree representation, ID3 algorithm, entropy, information gain, overfitting & validation, pruning; Random Forests: bagging, from bagging to random forests, extra trees, variable importance
• Unit 4: Unsupervised Learning
  • A. What is unsupervised learning?; clustering methods (Euclidean-distance based, probability based, hierarchical clustering)
  • B. k-Means clustering algorithm (naive k-means, initialization methods); applications (vector quantization, cluster analysis, feature learning); Gaussian Mixture Models; Expectation–Maximization method
  • C. Principal Component Analysis (first & further components, covariances, dimensionality reduction, SVD); properties & limitations of PCA; computing PCA via the covariance method; typical applications
• Unit 5: Parameter Estimation, Model Evaluation and Ensemble Methods
  • A. Parameter Estimation: point estimation, Maximum Likelihood Estimation, unbiased estimation, confidence intervals (one mean, two means, variances)
  • B. Model Evaluation: holdout set validation, cross-validation, leave-one-out, random subsampling, teach-and-test, bootstrapping; the ROC curve
  • C. Ensemble Methods: ensemble theory & size; voting and averaging based methods; boosting, weighted average, stacking, bagging and bootstrap aggregating

💻 Syllabus (CSP473 - Lab)

• Unit 1: Core Concepts of Machine Learning
  • 1. Understanding problem solving through Machine Learning and problem identification. [CO1]
  • 2. State-of-the-art review of tools used in Machine Learning experiments and project development. [CO1]
  • 3. Write a program to load and view a dataset file. [Iris dataset] [CO1]
  • 4. Implement simple linear regression using the housing price prediction problem. [California Housing dataset] [CO1]
  • 5. Implement binary logistic regression using the cancer identification problem. [Breast Cancer Wisconsin dataset] [CO2]
• Unit 2: Supervised Learning Algorithms - Part One
  • 6. Implement the Gradient Descent algorithm to optimize model parameters and visualize convergence of the cost function. [California Housing dataset] [CO2]
  • 7. Investigate the effect of learning rate and number of iterations on Gradient Descent convergence. [California Housing dataset] [CO2]
  • 8. Implement Ridge and Lasso regularized linear regression and compare their performance with standard linear regression. [California Housing & Breast Cancer Wisconsin datasets] [CO2]
  • 9. Develop a regularized logistic regression model and analyze the impact of regularization strength on classification accuracy. [California Housing & Breast Cancer Wisconsin datasets] [CO2]
• Unit 3: Supervised Learning Algorithms - Part Two
  • 10. Implement a Support Vector Regression (SVR) model, compare different kernel functions, and evaluate using MAE, MSE and R² score. [California Housing / UCI Diabetes dataset] [CO3]
  • 11. Design and train an Artificial Neural Network using the Backpropagation algorithm; analyze the effect of network architecture, activation functions and learning rate. [MNIST / Fashion-MNIST dataset] [CO3]
  • 12. Demonstrate the working of the decision tree based ID3 algorithm and classify a new sample. [Play Tennis / Iris dataset] [CO3]
  • 13. Demonstrate the working of the Random Forest algorithm and classify a new sample. [Wine / Breast Cancer dataset] [CO3]
• Unit 4: Unsupervised Learning
  • 14. Implement the K-Means clustering algorithm using an appropriate dataset. [Iris / Mall Customers dataset] [CO4]
• Unit 5: Model Evaluation, Ensemble Methods & Course Project
  • 15. Split data into training, cross-validation and testing sets; evaluate model performance using Hold-Out and k-Fold Cross-Validation. [Iris dataset] [CO5]
  • 16. Implement an Ensemble Learning approach (Bagging, Boosting or Stacking) by combining models to solve a time-series prediction problem. [California Housing / UCI Bike Sharing Demand dataset] [CO5]
  • 17. Perform statistical hypothesis testing (t-test, Chi-square, ANOVA or Mann-Whitney U) and interpret the results using Python statistical libraries. [Tips / Titanic / Iris dataset] [CO5]
  • 18. Course Project Evaluation — three reviews (Problem Statement, Design Specification, Development & Results). [CO6]

📚 Lectures

Theory lecture materials will be uploaded unit-wise as the course progresses. Please go through and review all the lecture content and additional materials carefully. If you have any doubts, post them using the “Post Doubt” option.