O1 Octave Programming - Linear Regression & Linear Regression With Multiple Variables - Tutorials For IoT-Makers

5/3/2018 o1 Octave Programming: Linear Regression & Linear regression with multiple variables – Tutorials For IoT-Makers
Tutorials For IoT-Makers

(Philosophy : Maker = Hacker )
o1 Octave Programming: Linear Regression

& Linear regression with multiple variables
[ o1 Octave Programming: Linear Regression & Linear regression with multiple

variables ]
Next Contents
01 Lecture Note 02 Lecture Note
[ Files ]
ex1.m
1 %% Machine Learning Online Class - Exercise 1: Linear Regression

2
3 % Instructions
4 % ------------
5 %
6 % This file contains code that helps you get started on the
7 % linear exercise. You will need to complete the following function
8 % in this exericse:
9 %
10 % warmUpExercise.m
11 % plotData.m
12 % gradientDescent.m
13 % computeCost.m
14 % gradientDescentMulti.m
15 % computeCostMulti.m
16 % featureNormalize.m
17 % normalEqn.m
18 %
https://iotmakerblog.wordpress.com/2016/06/17/o1-octave-programming-linear-regression-linear-regression-with-multiple-variables/ 1/24
19 % For this exercise, you will not need to change any code in this f
20 % or any other files other than those mentioned above.
21 %
22 % x refers to the population size in 10,000s
23 % y refers to the profit in $10,000s
24 %
25
26 %% Initialization
27 clear ; close all; clc
28
29 %% ==================== Part 1: Basic Function ====================
30 % Complete warmUpExercise.m
31 fprintf('Running warmUpExercise ... \n');
32 fprintf('5x5 Identity Matrix: \n');
33 warmUpExercise()
34
35 fprintf('Program paused. Press enter to continue.\n');
36 pause;
37
38 %% ======================= Part 2: Plotting =======================
39 fprintf('Plotting Data ...\n')
40 data = load('ex1data1.txt');
41 X = data(:, 1); y = data(:, 2);
42 m = length(y); % number of training examples
43
44 % Plot Data
45 % Note: You have to complete the code in plotData.m
46 plotData(X, y);
47
49 pause;
50
51 %% =================== Part 3: Gradient descent ===================
52 fprintf('Running Gradient Descent ...\n')
53
54 X = [ones(m, 1), data(:,1)]; % Add a column of ones to x
55 theta = zeros(2, 1); % initialize fitting parameters
56
57 % Some gradient descent settings
58 iterations = 1500;
59 alpha = 0.01;
60
61 % compute and display initial cost
62 computeCost(X, y, theta)
63
64 % run gradient descent
65 theta = gradientDescent(X, y, theta, alpha, iterations);
66
67 % print theta to screen
68 fprintf('Theta found by gradient descent: ');
69 fprintf('%f %f \n', theta(1), theta(2));
70
71 % Plot the linear fit
72 hold on; % keep previous plot visible
73 plot(X(:,2), X*theta, '-')
74 legend('Training data', 'Linear regression')
75 hold off % don't overlay any more plots on this figure
76
77 % Predict values for population sizes of 35,000 and 70,000
78 predict1 = [1, 3.5] *theta;
79 fprintf('For population = 35,000, we predict a profit of %f\n',...
80 predict1*10000);
81 predict2 = [1, 7] * theta;
82 fprintf('For population = 70,000, we predict a profit of %f\n',...
83 predict2*10000);
84
86 pause;
87
88 %% ============= Part 4: Visualizing J(theta_0, theta_1) ===========
89 fprintf('Visualizing J(theta_0, theta_1) ...\n')
90
91 % Grid over which we will calculate J
92 theta0_vals = linspace(-10, 10, 100);
93 theta1_vals = linspace(-1, 4, 100);
94
95 % initialize J_vals to a matrix of 0's
96 J_vals = zeros(length(theta0_vals), length(theta1_vals));
97
98 % Fill out J_vals
99 for i = 1:length(theta0_vals)
100 for j = 1:length(theta1_vals)
101 t = [theta0_vals(i); theta1_vals(j)];
102 J_vals(i,j) = computeCost(X, y, t);
103 end
104 end
105
106 % Because of the way meshgrids work in the surf command, we need to
107 % transpose J_vals before calling surf, or else the axes will be fli
108 J_vals = J_vals';
109 % Surface plot
110 figure;
111 surf(theta0_vals, theta1_vals, J_vals)
112 xlabel('\theta_0'); ylabel('\theta_1');
113
114 % Contour plot
115 figure;
116 % Plot J_vals as 15 contours spaced logarithmically between 0.01 and
117 contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 3, 20))
118 xlabel('\theta_0'); ylabel('\theta_1');
119 hold on;
120 plot(theta(1), theta(2), 'rx', 'MarkerSize', 10, 'LineWidth', 2);
ex1_multi.m
1 %% Machine Learning Online Class

2 % Exercise 1: Linear regression with multiple variables
3 %
4 % Instructions
5 % ------------
6 %
7 % This file contains code that helps you get started on the
8 % linear regression exercise.

9 %
10 % You will need to complete the following functions in this
11 % exericse:
12 %
13 % warmUpExercise.m
14 % plotData.m
15 % gradientDescent.m
16 % computeCost.m
17 % gradientDescentMulti.m
18 % computeCostMulti.m
19 % featureNormalize.m
20 % normalEqn.m
21 %
22 % For this part of the exercise, you will need to change some
23 % parts of the code below for various experiments (e.g., changing
24 % learning rates).
25 %
26
27 %% Initialization
28
29 %% ================ Part 1: Feature Normalization ================
30
31 %% Clear and Close Figures
32 clear ; close all; clc
33
34 fprintf('Loading data ...\n');
35
36 %% Load Data
37 data = load('ex1data2.txt');
38 X = data(:, 1:2);
39 y = data(:, 3);
40 m = length(y);
41
42 % Print out some data points
43 fprintf('First 10 examples from the dataset: \n');
44 fprintf(' x = [%.0f %.0f], y = %.0f \n', [X(1:10,:) y(1:10,:)]');
45
47 pause;
48
49 % Scale features and set them to zero mean
50 fprintf('Normalizing Features ...\n');
51
52 [X mu sigma] = featureNormalize(X);
53
54 % Add intercept term to X
55 X = [ones(m, 1) X];
56
57 %% ================ Part 2: Gradient Descent ================
58
59 % ====================== YOUR CODE HERE ======================
60 % Instructions: We have provided you with the following starter
61 % code that runs gradient descent with a particular
62 % learning rate (alpha).
63 %
64 % Your task is to first make sure that your functions
65 % computeCost and gradientDescent already work with

66 % this starter code and support multiple variables.
67 %
68 % After that, try running gradient descent with
69 % different values of alpha and see which one gives
70 % you the best result.
71 %
72 % Finally, you should complete the code at the end
73 % to predict the price of a 1650 sq-ft, 3 br house.
74 %
75 % Hint: By using the 'hold on' command, you can plot multiple
76 % graphs on the same figure.
77 %
78 % Hint: At prediction, make sure you do the same feature normalizati
79 %
80
81 fprintf('Running gradient descent ...\n');
82
83 % Choose some alpha value
84 alpha = 0.01;
85 num_iters = 400;
86
87 % Init Theta and Run Gradient Descent
88 theta = zeros(3, 1);
89 [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_it
90
91 % Plot the convergence graph
92 figure;
93 plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2);
94 xlabel('Number of iterations');
95 ylabel('Cost J');
96
97 % Display gradient descent's result
98 fprintf('Theta computed from gradient descent: \n');
99 fprintf(' %f \n', theta);
100 fprintf('\n');
101
102 % Estimate the price of a 1650 sq-ft, 3 br house
103 % ====================== YOUR CODE HERE ======================
104 % Recall that the first column of X is all-ones. Thus, it does
105 % not need to be normalized.
106 price = 0; % You should change this
107
108 % ============================================================
109
110 fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
111 '(using gradient descent):\n $%f\n'], price);
112
114 pause;
115
116 %% ================ Part 3: Normal Equations ================
117
118 fprintf('Solving with normal equations...\n');
119
120 % ====================== YOUR CODE HERE ======================
121 % Instructions: The following code computes the closed form
122 % solution for linear regression using the normal

123 % equations. You should complete the code in
124 % normalEqn.m
125 %
126 % After doing so, you should complete this code
127 % to predict the price of a 1650 sq-ft, 3 br house.
128 %
129
130 %% Load Data
131 data = csvread('ex1data2.txt');
132 X = data(:, 1:2);
133 y = data(:, 3);
134 m = length(y);
135
136 % Add intercept term to X
137 X = [ones(m, 1) X];
138
139 % Calculate the parameters from the normal equation
140 theta = normalEqn(X, y);
141
142 % Display normal equation's result
143 fprintf('Theta computed from the normal equations: \n');
144 fprintf(' %f \n', theta);
145 fprintf('\n');
146
147 % Estimate the price of a 1650 sq-ft, 3 br house
148 % ====================== YOUR CODE HERE ======================
149 price = 0; % You should change this
150
151 % ============================================================
152
153 fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
154 '(using normal equations):\n $%f\n'], price);
warmUpExercise.m
1 function A = warmUpExercise()
2 %WARMUPEXERCISE Example function in octave
3 % A = WARMUPEXERCISE() is an example function that returns the 5x5
4
5 A = [];
6 % ============= YOUR CODE HERE ==============
7 % Instructions: Return the 5x5 identity matrix
8 % In octave, we return values by defining which variabl
9 % represent the return values (at the top of the file)
10 % and then set them accordingly.
11
12 A = eye(5);
13
14 % ===========================================
15
16 end
plotData.m
1 function plotData(x, y)
2 %PLOTDATA Plots the data points x and y into a new figure
3 % PLOTDATA(x,y) plots the data points and gives the figure axes lab
4 % population and profit.
5
6 % ====================== YOUR CODE HERE ======================
7 % Instructions: Plot the training data into a figure using the
8 % &amp;amp;quot;figure&amp;amp;quot; an
9 % the &amp;amp;quot;xlabel&amp;amp;quot
10 % population and revenue data have been passed in
11 % as the x and y arguments of this function.
12 %
13 % Hint: You can use the 'rx' option with plot to have the markers
14 % appear as red crosses. Furthermore, you can make the
15 % markers larger by using plot(..., 'rx', 'MarkerSize', 10);
16
17 figure; % open a new figure window
18
19 plot(x, y, 'rx', 'MarkerSize', 10); % Plot the data
20 ylabel('Profit in $10,000s'); % Set the y?axis label
21 xlabel('Population of City in 10,000s'); % Set the x?axis label
22
23 % ============================================================
24
25 end
gradientDescent.m
1 function [theta, J_history] = gradientDescent(X, y, theta, alpha, num

2 %GRADIENTDESCENT Performs gradient descent to learn theta
3 % theta = GRADIENTDESENT(X, y, theta, alpha, num_iters) updates the
4 % taking num_iters gradient steps with learning rate alpha
5
6 % Initialize some useful values
8 J_history = zeros(num_iters, 1);
9
10 for iter = 1:num_iters
11
12 % ====================== YOUR CODE HERE ======================
13 % Instructions: Perform a single gradient step on the parameter v
14 % theta.
15 %
16 % Hint: While debugging, it can be useful to print out the values
17 % of the cost function (computeCost) and gradient here.
18 %
19
20 % http://stackoverflow.com/questions/10479353/gradient-descent-se
21
22 % Non-vectorized:
23 % theta_1 = theta(1) - alpha * (1/m) * sum((X*theta-y).*X(:,1))

25 % theta(1) = theta_1;
27
28 % vectorized:
29 theta = theta - (alpha .* (X * theta - y)'*X ./m)';
30
31 % ============================================================
32
33 % Save the cost J in every iteration
34 J_history(iter) = computeCost(X, y, theta);
35
36 end
37
38 end
computeCost.m
1 function J = computeCost(X, y, theta)

2 %COMPUTECOST Compute cost for linear regression
3 % J = COMPUTECOST(X, y, theta) computes the cost of using theta as
4 % parameter for linear regression to fit the data points in X and y
5
8
9 % You need to return the following variables correctly
10 J = 0;
11
12 % ====================== YOUR CODE HERE ======================
13 % Instructions: Compute the cost of a particular choice of theta
14 % You should set J to the cost.
15
16 J = sum((X * theta - y).^2)/(2*length(X));
17
18 % ===================================================================
19
20 end
computeCostMulti.m
1 function J = computeCostMulti(X, y, theta)

2 %COMPUTECOSTMULTI Compute cost for linear regression with multiple va
3 % J = COMPUTECOSTMULTI(X, y, theta) computes the cost of using thet
4 % parameter for linear regression to fit the data points in X and y
5
8
9 % You need to return the following variables correctly
10 J = 0;
11
12 % ====================== YOUR CODE HERE ======================

13 % Instructions: Compute the cost of a particular choice of theta
14 % You should set J to the cost.
15
16 J = sum((X * theta - y).^2)/(2*length(X));
17
18 % ===================================================================
19
20 end
gradientDescent.m
1 function [theta, J_history] = gradientDescent(X, y, theta, alpha, num

2 %GRADIENTDESCENT Performs gradient descent to learn theta
3 % theta = GRADIENTDESENT(X, y, theta, alpha, num_iters) updates the
5
9
11
12 % ====================== YOUR CODE HERE ======================
14 % theta.
15 %
17 % of the cost function (computeCost) and gradient here.
18 %
19
20 % http://stackoverflow.com/questions/10479353/gradient-descent-se
21
22 % Non-vectorized:
27
28 % vectorized:
29 theta = theta - (alpha .* (X * theta - y)'*X ./m)';
30
31 % ============================================================
32
34 J_history(iter) = computeCost(X, y, theta);
35
36 end
37
38 end
gradientDescentMulti.m
1 function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha

2 %GRADIENTDESCENTMULTI Performs gradient descent to learn theta
3 % theta = GRADIENTDESCENTMULTI(x, y, theta, alpha, num_iters) updat
5
9
11
12 % ====================== YOUR CODE HERE ======================
14 % theta.
15 %
17 % of the cost function (computeCostMulti) and gradient here
18 %
19
20 theta = theta - (alpha .* X'*(X * theta - y) ./m);
21
22 % ============================================================
23
25 J_history(iter) = computeCostMulti(X, y, theta);
26
27 end
28
29 end
featureNormalize.m
1 function [X_norm, mu, sigma] = featureNormalize(X)

2 %FEATURENORMALIZE Normalizes the features in X
3 % FEATURENORMALIZE(X) returns a normalized version of X where
4 % the mean value of each feature is 0 and the standard deviation
5 % is 1. This is often a good preprocessing step to do when
6 % working with learning algorithms.
7
8 % You need to set these values correctly
9 X_norm = X;
10 mu = zeros(1, size(X, 2));
11 sigma = zeros(1, size(X, 2));
12
13 % ====================== YOUR CODE HERE ======================
14 % Instructions: First, for each feature dimension, compute the mean
15 % of the feature and subtract it from the dataset,
16 % storing the mean value in mu. Next, compute the
17 % standard deviation of each feature and divide
18 % each feature by it's standard deviation, storing
19 % the standard deviation in sigma.
20 %
21 % Note that X is a matrix where each column is a
22 % feature and each row is an example. You need
23 % to perform the normalization separately for
24 % each feature.
25 %
26 % Hint: You might find the 'mean' and 'std' functions useful.
27 %
28
29 mn = mean(X);
30 sd = std(X);
31 X_norm = bsxfun(@minus,X_norm,mn);
32 X_norm = bsxfun(@rdivide,X_norm,sd);
33
34 % ============================================================
35
36 end
normalEqn.m
1 function [theta] = normalEqn(X, y)

2 %NORMALEQN Computes the closed-form solution to linear regression
3 % NORMALEQN(X,y) computes the closed-form solution to linear
4 % regression using the normal equations.
5
6 theta = zeros(size(X, 2), 1);
7
8 % ====================== YOUR CODE HERE ======================
9 % Instructions: Complete the code to compute the closed form solution
10 % to linear regression and put the result in theta.
11 %
12
13 % ---------------------- Sample Solution ----------------------
14 theta = inv(X'*X)*X'*y;
15
16 % -------------------------------------------------------------
17
18 % ============================================================
19
20 end
submit.m
1 function submit()
2 addpath('./lib');
3
4 conf.assignmentSlug = 'linear-regression';
5 conf.itemName = 'Linear Regression with Multiple Variables';
6 conf.partArrays = { ...
7 { ...
8 '1', ...
9 { 'warmUpExercise.m' }, ...
10 'Warm-up Exercise', ...
11 }, ...
12 { ...
13 '2', ...
14 { 'computeCost.m' }, ...
15 'Computing Cost (for One Variable)', ...
16 }, ...
17 { ...
18 '3', ...
19 { 'gradientDescent.m' }, ...
20 'Gradient Descent (for One Variable)', ...
21 }, ...
22 { ...
23 '4', ...
24 { 'featureNormalize.m' }, ...
25 'Feature Normalization', ...
26 }, ...
27 { ...
28 '5', ...
29 { 'computeCostMulti.m' }, ...
30 'Computing Cost (for Multiple Variables)', ...
31 }, ...
32 { ...
33 '6', ...
34 { 'gradientDescentMulti.m' }, ...
35 'Gradient Descent (for Multiple Variables)', ...
36 }, ...
37 { ...
38 '7', ...
39 { 'normalEqn.m' }, ...
40 'Normal Equations', ...
41 }, ...
42 };
43 conf.output = @output;
44
45 submitWithConfiguration(conf);
46 end
47
48 function out = output(partId)
49 % Random Test Cases
50 X1 = [ones(20,1) (exp(1) + exp(2) * (0.1:0.1:2))'];
51 Y1 = X1(:,2) + sin(X1(:,1)) + cos(X1(:,2));
52 X2 = [X1 X1(:,2).^0.5 X1(:,2).^0.25];
53 Y2 = Y1.^0.5 + Y1;
54 if partId == '1'
55 out = sprintf('%0.5f ', warmUpExercise());
56 elseif partId == '2'
57 out = sprintf('%0.5f ', computeCost(X1, Y1, [0.5 -0.5]'));
59 out = sprintf('%0.5f ', gradientDescent(X1, Y1, [0.5 -0.5]', 0.01
61 out = sprintf('%0.5f ', featureNormalize(X2(:,2:4)));
63 out = sprintf('%0.5f ', computeCostMulti(X2, Y2, [0.1 0.2 0.3 0.4
65 out = sprintf('%0.5f ', gradientDescentMulti(X2, Y2, [-0.1 -0.2 -
67 out = sprintf('%0.5f ', normalEqn(X2, Y2));
68 end
69 end
ex1data1.txt
1 6.1101,17.592
2 5.5277,9.1302
3 8.5186,13.662
4 7.0032,11.854
5 5.8598,6.8233
6 8.3829,11.886
7 7.4764,4.3483
8 8.5781,12
9 6.4862,6.5987
10 5.0546,3.8166
11 5.7107,3.2522
12 14.164,15.505
13 5.734,3.1551
14 8.4084,7.2258
15 5.6407,0.71618
16 5.3794,3.5129
17 6.3654,5.3048
18 5.1301,0.56077
19 6.4296,3.6518
20 7.0708,5.3893
21 6.1891,3.1386
22 20.27,21.767
23 5.4901,4.263
24 6.3261,5.1875
25 5.5649,3.0825
26 18.945,22.638
27 12.828,13.501
28 10.957,7.0467
29 13.176,14.692
30 22.203,24.147
31 5.2524,-1.22
32 6.5894,5.9966
33 9.2482,12.134
34 5.8918,1.8495
35 8.2111,6.5426
36 7.9334,4.5623
37 8.0959,4.1164
38 5.6063,3.3928
39 12.836,10.117
40 6.3534,5.4974
41 5.4069,0.55657
42 6.8825,3.9115
43 11.708,5.3854
44 5.7737,2.4406
45 7.8247,6.7318
46 7.0931,1.0463
47 5.0702,5.1337
48 5.8014,1.844
49 11.7,8.0043
50 5.5416,1.0179
51 7.5402,6.7504
52 5.3077,1.8396
53 7.4239,4.2885
54 7.6031,4.9981
55 6.3328,1.4233
56 6.3589,-1.4211
57 6.2742,2.4756
58 5.6397,4.6042
59 9.3102,3.9624
60 9.4536,5.4141
61 8.8254,5.1694
62 5.1793,-0.74279
63 21.279,17.929
64 14.908,12.054
65 18.959,17.054
66 7.2182,4.8852
67 8.2951,5.7442
68 10.236,7.7754
69 5.4994,1.0173
70 20.341,20.992
71 10.136,6.6799
72 7.3345,4.0259
73 6.0062,1.2784
74 7.2259,3.3411
75 5.0269,-2.6807
76 6.5479,0.29678
77 7.5386,3.8845
78 5.0365,5.7014
79 10.274,6.7526
80 5.1077,2.0576
81 5.7292,0.47953
82 5.1884,0.20421
83 6.3557,0.67861
84 9.7687,7.5435
85 6.5159,5.3436
86 8.5172,4.2415
87 9.1802,6.7981
88 6.002,0.92695
89 5.5204,0.152
90 5.0594,2.8214
91 5.7077,1.8451
92 7.6366,4.2959
93 5.8707,7.2029
94 5.3054,1.9869
95 8.2934,0.14454
96 13.394,9.0551
97 5.4369,0.61705
ex1data2.txt
1 2104,3,399900
2 1600,3,329900
3 2400,3,369000
4 1416,2,232000
5 3000,4,539900
6 1985,4,299900
7 1534,3,314900
8 1427,3,198999
9 1380,3,212000
10 1494,3,242500
11 1940,4,239999
12 2000,3,347000
13 1890,3,329999
14 4478,5,699900
15 1268,3,259900
16 2300,4,449900
17 1320,2,299900
18 1236,3,199900
19 2609,4,499998
20 3031,4,599000
21 1767,3,252900
22 1888,2,255000
23 1604,3,242900
24 1962,4,259900
25 3890,3,573900
26 1100,3,249900
27 1458,3,464500
28 2526,3,469000
29 2200,3,475000
30 2637,3,299900
31 1839,2,349900
32 1000,1,169900
33 2040,4,314900
34 3137,3,579900
35 1811,4,285900
36 1437,3,249900
37 1239,3,229900
38 2132,4,345000
39 4215,4,549000
40 2162,4,287000
41 1664,2,368500
42 2238,3,329900
43 2567,4,314000
44 1200,3,299000
45 852,2,179900
46 1852,4,299900
47 1203,3,239500
[ ex1.m Output ]
1 Running warmUpExercise ...

2 5x5 Identity Matrix:
3
4 ans =
5
6 1 0 0 0 0
7 0 1 0 0 0
8 0 0 1 0 0
9 0 0 0 1 0
10 0 0 0 0 1
11
12 Program paused. Press enter to continue.
Plotting data
1 Plotting Data ...

Figure 1
1 Running Gradient Descent ...

2
3 ans =
4
5 32.0727
6
7 Theta found by gradient descent: -3.630291 1.166362
8 For population = 35,000, we predict a profit of 4519.767868
9 For population = 70,000, we predict a profit of 45342.450129
1 Visualizing J(theta_0, theta_1) ...
Figure 2
Figure 3
[ Lib ]
Files
makeValidFieldName.m
1 function str = makeValidFieldName(str)

2 % From MATLAB doc: field names must begin with a letter, which may be
3 % followed by any combination of letters, digits, and underscores.
4 % Invalid characters will be converted to underscores, and the prefix
5 % &amp;quot;x0x[Hex code]_&amp;quot; will be added if
6 isoct=exist('OCTAVE_VERSION','builtin');
7 pos=regexp(str,'^[Â-Za-z]','once');
8 if(~isempty(pos))
9 if(~isoct)
10 str=regexprep(str,'^([Â-Za-z])','x0x${sprintf(''%X'',uni
11 else
12 str=sprintf('x0x%X_%s',char(str(1)),str(2:end));
13 end
14 end
15 if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end
16 if(~isoct)
17 str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unic
18 else
19 pos=regexp(str,'[^0-9A-Za-z_]');
20 if(isempty(pos)) return; end
21 str0=str;
22 pos0=[0 pos(:)' length(str)];
23 str='';
24 for i=1:length(pos)
25 str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(p
26 end
27 if(pos(end)~=length(str))
28 str=[str str0(pos0(end-1)+1:pos0(end))];
29 end
30 end
submitWithCon guration.m
1 function str = makeValidFieldName(str)

2 % From MATLAB doc: field names must begin with a letter, which may be
3 % followed by any combination of letters, digits, and underscores.
4 % Invalid characters will be converted to underscores, and the prefix
5 % &amp;quot;x0x[Hex code]_&amp;quot; will be added if
6 isoct=exist('OCTAVE_VERSION','builtin');
7 pos=regexp(str,'^[Â-Za-z]','once');
8 if(~isempty(pos))
9 if(~isoct)
10 str=regexprep(str,'^([Â-Za-z])','x0x${sprintf(''%X'',uni
11 else
12 str=sprintf('x0x%X_%s',char(str(1)),str(2:end));
13 end
14 end
15 if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end
16 if(~isoct)
17 str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unic
18 else
19 pos=regexp(str,'[^0-9A-Za-z_]');
20 if(isempty(pos)) return; end
21 str0=str;
22 pos0=[0 pos(:)' length(str)];
23 str='';
24 for i=1:length(pos)
25 str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(p
26 end
27 if(pos(end)~=length(str))
28 str=[str str0(pos0(end-1)+1:pos0(end))];
29 end
30 end
[ Unit Testing ]
computeCost:
1 computeCost( [1 2; 1 3; 1 4; 1 5], [7;6;5;4], [0.1;0.2] )

2
3 ans =
4
5 11.9450
6
7 computeCost( [1 2 3; 1 3 4; 1 4 5; 1 5 6], [7;6;5;4], [0.1;0.2;0.3])
8
9 ans =
10
11 7.0175
gradientDescent
Test Case 1:
1 >>[theta J_hist] = gradientDescent([1 5; 1 2; 1 4; 1 5],[1 6 4

2
3 % then type in these variable names, to display the final results
4 >>theta
5 theta =
6 5.2148
7 -0.5733
8 >>J_hist(1)
9 ans = 5.9794
10 >>J_hist(1000)
11 ans = 0.85426
For debugging, here are the rst few theta values computed in the gradientDescent()
for-loop for this test case:
1 % first iteration
2 theta =
3 0.032500
4 0.107500
5 % second iteration
6 theta =
7 0.060375
8 0.194887
9 % third iteration
10 theta =
11 0.084476
12 0.265867
13 % fourth iteration
14 theta =
15 0.10550
16 0.32346
he values can be inspected by adding the “keyboard” command within your for-
loop. This exits the code to the debugger, where you can inspect the values. Use the
“return” command to resume execution.
Test Case 2:
This test case is similar, but uses a non-zero initial theta value.
1 >> [theta J_hist] = gradientDescent([1 5; 1 2],[1 6]',[.5 .5]',

2 >> theta
3 theta =
4 1.70986
5 0.19229
6
7 >> J_hist
8 J_hist =
9 5.8853
10 5.7139
11 5.5475
12 5.3861
13 5.2294
14 5.0773
15 4.9295
16 4.7861
17 4.6469
18 4.5117
featureNormalize():
1 [Xn mu sigma] = featureNormalize([1 2 3])

2
3 % result
4
5 Xn =
6 NaN NaN NaN
7 mu =
8 1 2 3
9 sigma =
10 0 0 0
11
12 % ---------------
13 [Xn mu sigma] = featureNormalize([1 ; 2 ; 3])
14
15 % result
16
17 Xn =
18 -1
19 0
20 1
21
22 mu = 2
23 sigma = 1
24
25 %----------------
26 [Xn mu sigma] = featureNormalize(magic(3))
27
28 % result
29
30 Xn =
31 1.13389 -1.00000 0.37796
32 -0.75593 0.00000 0.75593
33 -0.37796 1.00000 -1.13389
34
35 mu =
36 5 5 5
37 sigma =
38 2.6458 4.0000 2.6458
39
40 %--------------
41 [Xn mu sigma] = featureNormalize([-ones(1,3); magic(3)])
42
43 % results
44
45 Xn =
46 -1.21725 -1.01472 -1.21725
47 1.21725 -0.56373 0.67625
48 -0.13525 0.33824 0.94675
49 0.13525 1.24022 -0.40575
50
51 mu =
52 3.5000 3.5000 3.5000
53
54 sigma =
55 3.6968 4.4347 3.6968
computeCostMulti and gradientDescentMulti:

1 >> X = [ 2 1 3; 7 1 9; 1 8 1; 3 7 4 ];
2 >> computeCostMulti( X, [ 2; 5; 5; 6 ], [ 0.4; 0.8; 0.8 ] )
3 ans = 7.5500
4
5 >>gradientDescentMulti([3 5 6; 1 2 3; 9 4 2],[1 6 4]',[0 0 0]',0
6 ans =
7 1.2123
8 -2.9458
9 2.3219
Advertisements
Pianoforall - Pianoforall -
Incredible New Incredible New
Way To Learn Way To Learn
Piano & Keyboard Piano & Keyboard
US$199 US$199
US$14.99 US$14.99
View This Course View This Course
Report this ad Report this ad
Share this:
 Twitter  Facebook  Google
Like
Be the first to like this.
Author: iotmaker
I am interested in IoT, robot, gures & leadership. Also, I have spent almost every day of the
past 15 years making robots or electronic inventions or computer programs.
View all posts by iotmaker
iotmaker / June 17, 2016 / Machine Learning, Uncategorized
Tutorials For IoT-Makers /


O1 Octave Programming - Linear Regression & Linear Regression With Multiple Variables - Tutorials For IoT-Makers

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

O1 Octave Programming - Linear Regression & Linear Regression With Multiple Variables - Tutorials For IoT-Makers

Uploaded by

Copyright:

Available Formats

5/3/2018 o1 Octave Programming: Linear Regression & Linear regression with multiple variables – Tutorials For IoT-Makers

Tutorials For IoT-Makers

o1 Octave Programming: Linear Regression

[ o1 Octave Programming: Linear Regression & Linear regression with multiple

01 Lecture Note 02 Lecture Note

1 %% Machine Learning Online Class - Exercise 1: Linear Regression

1 %% Machine Learning Online Class

8 % linear regression exercise.

65 % computeCost and gradientDescent already work with

122 % solution for linear regression using the normal

1 function [theta, J_history] = gradientDescent(X, y, theta, alpha, num

24 % theta_2 = theta(2) - alpha * (1/m) * sum((X*theta-y).*X(:,2))

1 function J = computeCost(X, y, theta)

1 function J = computeCostMulti(X, y, theta)

12 % ====================== YOUR CODE HERE ======================

1 function [theta, J_history] = gradientDescent(X, y, theta, alpha, num

1 function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha

1 function [X_norm, mu, sigma] = featureNormalize(X)

1 function [theta] = normalEqn(X, y)

1 Running warmUpExercise ...

1 Plotting Data ...

1 Running Gradient Descent ...

1 Visualizing J(theta_0, theta_1) ...

1 function str = makeValidFieldName(str)

1 function str = makeValidFieldName(str)

1 computeCost( [1 2; 1 3; 1 4; 1 5], [7;6;5;4], [0.1;0.2] )

1 &gt;&gt;[theta J_hist] = gradientDescent([1 5; 1 2; 1 4; 1 5],[1 6 4

1 &gt;&gt; [theta J_hist] = gradientDescent([1 5; 1 2],[1 6]',[.5 .5]',

1 [Xn mu sigma] = featureNormalize([1 2 3])

computeCostMulti and gradientDescentMulti:

View This Course View This Course

Report this ad Report this ad

 Twitter  Facebook  Google

iotmaker / June 17, 2016 / Machine Learning, Uncategorized

Tutorials For IoT-Makers /

You might also like

24 % theta_2 = theta(2) - alpha * (1/m) * sum((Xtheta-y).X(:,2))

1 >>[theta J_hist] = gradientDescent([1 5; 1 2; 1 4; 1 5],[1 6 4

1 >> [theta J_hist] = gradientDescent([1 5; 1 2],[1 6]',[.5 .5]',