Term Comments

Algorithm Set of rules to be followed while solving a problem

Artificial Intelligence use of computer to mimic the cognitive functions of humans
Machine Learning Subset of AI, focuses on ability of machines to receive a set of data & learn for themselves,
changing algorithms as they learn more about the information they are processing. 3 types
- Supervised: We have correct answers for the training cases. Eg. Predict price of home.
Actual sales price
- Unsupervised: No correct answer for training cases. Goal is exploratory. Eg. Detect unusual
events, discover new consumer segments
- Reinforcement: No correct answer. Eg ranking of outcomes. Noisy feedback. Eg. Chess,
puzzles
Neural Networks Computers are trained to think like humans using neural networks. NN are a series of
algorithms modeled after human brain. Eg. Text Recognition, facial recognition. Supervised
Learning approach
Deep Learning Subset of machine learning. Many layers involved while NN only has one layer. Exposure to
millions of data points.
Reducible Error
Irreducible Error
Prediction Setting
Inference Setting
Parametric We make assumptions about the form of f – usually linear. Then we use data to fit the
model. So we estimate parameters(coefficients in the equation) Usually least sq method.
Basically work is to estimate parameters
Non-Parametric No explicit assumptions about functional form of f. seek to estimate an f that is as close to
datapoints as possible without being too wiggly.
Over -Fitting When a model follows the data too closely including all its errors & noise
Euclidian Distance
Lasso
Spline
Generalized Additive
models
Bagging
Boosting
Support vector
Machines
Supervised learning There exists an associated y for every set of x we use. Models to predict well or to better
understand the relationships
Unsupervised No y for set of x. so can’t fit a linear model. Working blind. Unsupervised cos we lack results
learning that can supervise our response. Eg. clustering
Semi-Supervised Learn from supervised & apply on unsupervised. i.e. use of small amount of labeled data
Learning with large amounts of unlabeled data
Linear regression
Logistic regression
Cluster Analysis
Regression problem Dealing with a quantitative response
Classification problem Dealing with a qualitative/categorical response
Error rate Usually for classification problems. No. of errors/ total classifications
Indicator variable From above.. Is 0 if actual = predicted. Is 1 if actual not same as predicted
Bayes classifier Conditional probability that Y belongs to class j, given x. basically assigns each observation to
its most likely class given its predictors values. Can be used to minimize test error rate
Bayes Decision The point where the points have equal possibilities to belong to any of the given classes. i.e.
Boundary just as likely to fall into any class.
Bayes Error rate Bayes classifier produces the lowest possible test error rate which is called Bayes error rate
k-NN Given +ve int k, & test observation x0, identifies k points in training set nearest to x0 clalled
N0, & then estimates the conditional probability for class j as a fraction of points in N0 whose
response values equal j. Then applies bayes rule & classifies test obs to class with largest
probability
Leverage Point
Outlier
Leverage Point vs
Outlier
Loss Function
Interaction term
AIC, AICC?
AIC Akaike Information Criterion. This is an asymptotic approximation to a “distance” measure
between the data and the model
Laplace Correction
Model Generality