Algorithm Input Output Learns by Complexity Control Notes
Naive Bayes boolean feature vector discrete estimating conditional Assumes independent LaPlace Correction, XOR probabilities (counting) features basic Decision Tree boolean feature vector discrete minimizing average parametric: leaf size, entropy at the branches minimum entropy Continuous-Valued Decision real feature vector discrete minimizing average Tree entropy at the branches K-Nearest Neighbor real feature vector discrete memorizing all points parametric: K Scaling Perceptron real feature vector discrete maximizing margin limited to linear separator guarantees separator if it (weight space search) exists SVM real feature vector discrete maximizing margin maximizes margin in error (quadratic function programming) Neural Net real feature vector discrete gradient descent (weight architecture Architecture, Scaling space search) Neural Net Regression real feature vector real gradient descent (weight architecture Architecture, Scaling space search) Regression Trees real feature vector real minimizing variance at Kernel functions (not SVM the branches kernels) Nearest Neighbor Regression real feature vector real memorizing all points Kernel functions (not SVM kernels)