Monday, January 05, 2015

Machine Learning Course Summary (Part 3)

 

Summary from the Stanford's Machine learning class by Andrew Ng


  • Part 1
    • Supervised vs. Unsupervised learning, Linear Regression, Logistic Regression, Gradient Descent
  • Part 2
    • Regularization, Neural Networks
  • Part 3
    • Debugging and Diagnostic, Machine Learning System Design
  • Part 4
    • Support Vector Machine, Kernels
  • Part 5
    • K-means algorithm, Principal Component Analysis (PCA) algorithm
  • Part 6
    • Anomaly detection, Multivariate Gaussian distribution
  • Part 7
    • Recommender Systems, Collaborative filtering algorithm, Mean normalization
  • Part 8
    • Stochastic gradient descent, Mini batch gradient descent, Map-reduce and data parallelism

Debugging & Diagnostics

  • Evaluating a Hypothesis
    • What if the hypothesis fails to generalize to new examples not in training set?
    • Divide the dataset into Train set, Cross Validation st and Test set

image

image

image

  • Bias vs. Variance
    • Underfit, Just Right and Overfit graphs.

image

    • Error graph comparing Cross validation error and Training error

image

    • How to identify if it is a bias problem or a variance problem?

image

  • Regularization and bias/variance
    • Large, Intermediate and Small lamda values.

image

    • Choosing the regularization parameter “lamda”:

image

image

  • Learning Curves

image

image

  • Debugging 
    • Suppose you have implemented regularized linear regression to predict housing prices.
      However, when you test your hypothesis on a new set of houses, you find that it makes unacceptably large errors in its predictions. What should you try next?
      • Get more training examples ?  Fixes High Variance 
      • Try smaller sets of features ?  Fixes High Variance 
      • Try getting additional features ?  Fixes High Bias
      • Try adding polynomial features ?  Fixes High Bias
      • Try decreasing lamda ?  Fixes High Bias
      • Try increasing lamda ? Fixes High Variance 

  • Neural networks and overfitting

image

Machine Learning System Design

  • Prioritize What to work on
    • Spend your time to make it have low error
      • Collect lots of data (e.g. “honeypot” project)
      • Develop sophisticated features
      • Develop sophisticated algorithms
    • Error Analysis
      • Start with simple algorithm that you can implement quickly.
      • Plot learning curves to decide if more data, more feature etc. is likely to help.
      • Manually examine the examples in Cross Validation set, which our algorithm made errors on. See if we can find any systematic trend.
  • Error Metrics for Skewed classes
    • Use this when you have more number of positive examples than negative examples.
    • Precision / Recall
      • A classifier with high precision and high recall is good, even if we have very skewed classes.

image

image

    • F1 Score (F Score)

image

  • Large Data Rationale
    • “It’s not who has the best algorithm that wins. It’s who has the most data.”

image

No comments:

AddIn