API tools faq
paste
View difference between Paste ID: P0u6bHkL and 86QddmT2
SHOW: | | - or go back to the newest paste.
1-
// This may or may not help you understand what's going
1+
// This may or may not help you understand what's going
2-
// on in week one of the Stanford Machine Learning course
2+
// on in week one of the Stanford Machine Learning course
3-
// particulary if you're coming from Javascript programming.
3+
// particulary if you're coming from Javascript programming.
4-
4+
5-
// The lecture deals with linear regression. You may remember
5+
// The lecture deals with linear regression. You may remember

6-
// a concept called "line of best fit" from middle school.
6+
// a concept called "line of best fit" from middle school.

7-
// A refresher: you have a set of points with x and y co-ordinates
7+
// A refresher: you have a set of points with x and y co-ordinates

8-
// and you have to draw a straight line through them. It doesn't
8+
// and you have to draw a straight line through them. It doesn't

9-
// have to intersect all (or any) of the points, but it does need
9+
// have to intersect all (or any) of the points, but it does need

10-
// to represent the general layout of your dataset. The idea is
10+
// to represent the general layout of your dataset. The idea is

11-
// that this will let you make guesses for the y position of a
11+
// that this will let you make guesses for the y position of a

12-
// new point when you only know the x or vice versa. It often
12+
// new point when you only know the x or vice versa. It often

13-
// won't be 100% accurate, but it'll give you a good idea, and a
13+
// won't be 100% accurate, but it'll give you a good idea, and a

14-
// good understanding of the range of possible error it could have
14+
// good understanding of the range of possible error it could have

15-
// made.
15+
// made.

16-
16+
17-
// A linear regression algorithm is just a way to make a computer
17+
// A linear regression algorithm is just a way to make a computer

18-
// figure out what the line of best fit for a given dataset is.
18+
// figure out what the line of best fit for a given dataset is.

19-
// The linear regression algorithm I describe below comes from
19+
// The linear regression algorithm I describe below comes from

20-
// the Stanford Machine Learning course on Coursera. I found it
20+
// the Stanford Machine Learning course on Coursera. I found it

21-
// to be a bit too mathsy, so I'm writing it out in code, which
21+
// to be a bit too mathsy, so I'm writing it out in code, which

22-
// may help some people understand.
22+
// may help some people understand.

23-
23+
24-
// Linear regression algorithms return a function that takes an
24+
// Linear regression algorithms return a function that takes an

25-
// x and returns a guess for the corresponding y value, based on
25+
// x and returns a guess for the corresponding y value, based on

26-
// the *specific* dataset you trained against. This function is
26+
// the *specific* dataset you trained against. This function is

27-
// called a "model". There are many other kinds of models for
27+
// called a "model". There are many other kinds of models for

28-
// various other kinds of machine learning algorithms, but because
28+
// various other kinds of machine learning algorithms, but because

29-
// we're focusing on linear regression, this model will be the
29+
// we're focusing on linear regression, this model will be the

30-
// function of a line. You may remember this as `y=mx+c` where
30+
// function of a line. You may remember this as `y=mx+c` where

31-
// `m` is the slope of the line, and c is the horizontal offset of
31+
// `m` is the slope of the line, and c is the horizontal offset of

32-
// the line, which you can think about as 'the value of the equation
32+
// the line, which you can think about as 'the value of the equation

33-
// when x is 0'. In the course, this is written as hθ1=θ1+θ2*x.
33+
// when x is 0'. In the course, this is written as hθ1=θ1+θ2*x.

34-
34+
35-
(()=>{
35+
(()=>{

36-
  
36+
  

37-
  // You'll need to make lots of linear functions, so
37+
  // You'll need to make lots of linear functions, so

38-
  // lets make that easy.
38+
  // lets make that easy.

39-
  function prepareLinearFunction (theta1, theta2) {
39+
  function prepareLinearFunction (theta1, theta2) {

40-
    return x => theta1 + theta2 * x;
40+
    return x => theta1 + theta2 * x;

41-
  }
41+
  }

42-
  
42+
  

43-
  // We're assuming in this example that your dataset contains
43+
  // We're assuming in this example that your dataset contains

44-
  // two columns, and therefore we have two thetas. You could
44+
  // two columns, and therefore we have two thetas. You could

45-
  // generalise the above function to take more columns into
45+
  // generalise the above function to take more columns into

46-
  // account, but for now, lets focus on just two columns.
46+
  // account, but for now, lets focus on just two columns.

47-
  
47+
  

48-
  // Lets define a dataset to learn from. It is possible to
48+
  // Lets define a dataset to learn from. It is possible to

49-
  // accurately model this dataset with a line, so we should
49+
  // accurately model this dataset with a line, so we should

50-
  // end up without any errors.
50+
  // end up without any errors.

51-
  const dataset = [
51+
  const dataset = [

52-
    [0,1],
52+
    [0,1],

53-
    [1,3],
53+
    [1,3],

54-
    [2,5],
54+
    [2,5],

55-
    [3,7]
55+
    [3,7]

56-
  ];
56+
  ];

57-
  // You'll notice that the 0th element of each inner array is
57+
  // You'll notice that the 0th element of each inner array is

58-
  // the index of that element. I could just use the index,
58+
  // the index of that element. I could just use the index,

59-
  // but I wanted to make it more obvious.
59+
  // but I wanted to make it more obvious.

60-
  
60+
  

61-
  // Next you'll need a "cost function", which is just a
61+
  // Next you'll need a "cost function", which is just a

62-
  // fancy name for a function that measures how well a
62+
  // fancy name for a function that measures how well a

63-
  // model is performing at it's task. To calculate this,
63+
  // model is performing at it's task. To calculate this,

64-
  // the function needs both of the current thetas, and
64+
  // the function needs both of the current thetas, and

65-
  // the dataset it's training against
65+
  // the dataset it's training against

66-
  function costFunction(theta1, theta2, dataset) {
66+
  function costFunction(theta1, theta2, dataset) {

67-
    // Here we generate our model using the function we
67+
    // Here we generate our model using the function we

68-
    // wrote earlier, giving it our current theta values.
68+
    // wrote earlier, giving it our current theta values.

69-
    const model = prepareLinearFunction(theta1,theta2);
69+
    const model = prepareLinearFunction(theta1,theta2);

70-
    
70+
    

71-
    // We apply a procedure to our model for each row
71+
    // We apply a procedure to our model for each row

72-
    // in the dataset.
72+
    // in the dataset.

73-
    return dataset.map((x)=>{
73+
    return dataset.map((x)=>{

74-
      // First we get the difference between what the
74+
      // First we get the difference between what the

75-
      // data row says about the mapping and what our
75+
      // data row says about the mapping and what our

76-
      // model says.
76+
      // model says.

77-
      const difference = model(x[0])-x[1];
77+
      const difference = model(x[0])-x[1];

78-
      
78+
      

79-
      // We then return the square of that difference.
79+
      // We then return the square of that difference.

80-
      return Math.pow(difference,2);
80+
      return Math.pow(difference,2);

81-
    }) // Now we have an array of the square of the difference
81+
    }) // Now we have an array of the square of the difference

82-
    // between each of the examples and what we predicted for
82+
    // between each of the examples and what we predicted for

83-
    // their x value. The following line sums that array, which
83+
    // their x value. The following line sums that array, which

84-
    // we divide by the length of the dataset, and return.
84+
    // we divide by the length of the dataset, and return.

85-
    .reduce((x,y)=>{return x+y},0)/dataset.length*2;
85+
    .reduce((x,y)=>{return x+y},0)/dataset.length*2;

86-
  }
86+
  }

87-
  
87+
  

88-
  // Ok, now we have a cost function. Next up, we implement
88+
  // Ok, now we have a cost function. Next up, we implement

89-
  // "gradient descent", which is just a prodedure for modifying
89+
  // "gradient descent", which is just a prodedure for modifying

90-
  // our theta values in a way that will decrease the overall "cost"
90+
  // our theta values in a way that will decrease the overall "cost"

91-
  // (otherwise known as "error") of our model.
91+
  // (otherwise known as "error") of our model.

92-
  
92+
  

93-
  function gradientDescent(learningRate, thetaOne, thetaTwo, dataset) {
93+
  function gradientDescent(learningRate, thetaOne, thetaTwo, dataset) {

94-
    // First, we calculate the error of the model that our current
94+
    // First, we calculate the error of the model that our current

95-
    // values of theta creates
95+
    // values of theta creates

96-
    const error = costFunction(thetaOne, thetaTwo, dataset);
96+
    const error = costFunction(thetaOne, thetaTwo, dataset);

97-
    
97+
    

98-
    // Then we use the error, the learning rate to update each theta
98+
    // Then we use the error, the learning rate to update each theta

99-
    // thetas must be updated simultaneously - don't get into a
99+
    // thetas must be updated simultaneously - don't get into a

100-
    // situation where you update one and you use that new theta
100+
    // situation where you update one and you use that new theta

101-
    // to update your next theta. It would probably work, but it's
101+
    // to update your next theta. It would probably work, but it's

102-
    // not proper gradient descent.
102+
    // not proper gradient descent.

103-
    /*const newThetas = [thetaOne, thetaTwo].map(function(theta){
103+
    /*const newThetas = [thetaOne, thetaTwo].map(function(theta){

104-
      // Note - I don't know how to calculate derivative of the cost
104+
      // Note - I don't know how to calculate derivative of the cost

105-
      // function yet. It wasn't covered in this lecture. It's a
105+
      // function yet. It wasn't covered in this lecture. It's a

106-
      // measure of how far away your thetas are from a "local minimum"
106+
      // measure of how far away your thetas are from a "local minimum"

107-
      // in error, it's what guides your thetas towards finding their
107+
      // in error, it's what guides your thetas towards finding their

108-
      // correct values.
108+
      // correct values.

109-
      theta * learningRate * derivitave * error;
109+
      theta * learningRate * derivitave * error;

110-
    });*/
110+
    });*/

111-
    //please excuse this hackery nonsense
111+
    //please excuse this hackery nonsense

112-
112+
113-
    const model = prepareLinearFunction(thetaOne,thetaTwo);
113+
    const model = prepareLinearFunction(thetaOne,thetaTwo);

114-
114+
115-
    const a = dataset.map((x) => {
115+
    const a = dataset.map((x) => {

116-
      return model(x[0])-x[1];
116+
      return model(x[0])-x[1];

117-
    }).reduce((x,y) => {return x + y}, 0)/dataset.length;
117+
    }).reduce((x,y) => {return x + y}, 0)/dataset.length;

118-
118+
119-
    const b = dataset.map((x) => {
119+
    const b = dataset.map((x) => {

120-
      return (model(x[0])-x[1])*x[0];
120+
      return (model(x[0])-x[1])*x[0];

121-
    }).reduce((x,y) => {return x + y}, 0)/dataset.length;
121+
    }).reduce((x,y) => {return x + y}, 0)/dataset.length;

122-
122+
123-
    const newThetas = [
123+
    const newThetas = [

124-
      thetaOne - learningRate * a,
124+
      thetaOne - learningRate * a,

125-
      thetaTwo - learningRate * b
125+
      thetaTwo - learningRate * b

126-
    ];
126+
    ];

127-
debugger;
127+
debugger;

128-
    return { newThetas, error };
128+
    return { newThetas, error };

129-
  }
129+
  }

130-
  
130+
  

131-
  // Lets run through the whole training process
131+
  // Lets run through the whole training process

132-
  (function linearRegress(dataset){
132+
  (function linearRegress(dataset){

133-
    // Initialize thetas to zero. It's not important that
133+
    // Initialize thetas to zero. It's not important that

134-
    // they're zero, they could start out as any number,
134+
    // they're zero, they could start out as any number,

135-
    // but zero works just fine.
135+
    // but zero works just fine.

136-
    const initialThetaOne = 0;
136+
    const initialThetaOne = 0;

137-
    const initialThetaTwo = 0;
137+
    const initialThetaTwo = 0;

138-
    const learningRate = 0.2;
138+
    const learningRate = 0.2;

139-
    
139+
    

140-
    // Create a named function so that you can recurse
140+
    // Create a named function so that you can recurse

141-
    // through the values of theta that gradient descent gives
141+
    // through the values of theta that gradient descent gives

142-
    // you until you converge.
142+
    // you until you converge.

143-
    (function reg(learningRate, thetaOne, thetaTwo, dataset) {
143+
    (function reg(learningRate, thetaOne, thetaTwo, dataset) {

144-
      console.log(`Current thetas: ${thetaOne}, ${thetaTwo}`);
144+
      console.log(`Current thetas: ${thetaOne}, ${thetaTwo}`);

145-
      
145+
      

146-
      // Run gradient descent
146+
      // Run gradient descent

147-
      const {error,newThetas} = gradientDescent(learningRate, thetaOne, thetaTwo, dataset);
147+
      const {error,newThetas} = gradientDescent(learningRate, thetaOne, thetaTwo, dataset);

148-
      
148+
      

149-
      console.log(`Those thetas gave us an error of ${error}.`);
149+
      console.log(`Those thetas gave us an error of ${error}.`);

150-
      
150+
      

151-
      if (error < 0.00001) {
151+
      if (error < 0.00001) {

152-
        // Our error is zero! That means our thetas have
152+
        // Our error is zero! That means our thetas have

153-
        // "converged", or "settled" on values that make sense
153+
        // "converged", or "settled" on values that make sense

154-
        // for the dataset. Note that because our dataset fits
154+
        // for the dataset. Note that because our dataset fits

155-
        // perfectly on a line, we can check that our thetas
155+
        // perfectly on a line, we can check that our thetas

156-
        // match exactly. When your dataset is always going to
156+
        // match exactly. When your dataset is always going to

157-
        // have a certain amount of error in it (i.e. when the
157+
        // have a certain amount of error in it (i.e. when the

158-
        // data doesn't exactly match any straight line), you'll
158+
        // data doesn't exactly match any straight line), you'll

159-
        // have to check tolerate some difference.
159+
        // have to check tolerate some difference.

160-
        console.log(`Convergeance! Line of best fit is: y = ${thetaOne} + ${thetaTwo}x.`);
160+
        console.log(`Convergeance! Line of best fit is: y = ${thetaOne} + ${thetaTwo}x.`);

161-
        process.exit();
161+
        process.exit();

162-
      }
162+
      }

163-
      
163+
      

164-
      // Recursively call this function with updated thetas until
164+
      // Recursively call this function with updated thetas until

165-
      // convergeance happens and the condition above is met
165+
      // convergeance happens and the condition above is met

166-
      console.log("Running gradient descent again to reduce the error.")
166+
      console.log("Running gradient descent again to reduce the error.")

167-
      reg(learningRate, newThetas[0], newThetas[1], dataset);
167+
      reg(learningRate, newThetas[0], newThetas[1], dataset);

168-
    })(learningRate, initialThetaOne, initialThetaTwo, dataset);
168+
    })(learningRate, initialThetaOne, initialThetaTwo, dataset);

169-
  })(dataset);
169+
  })(dataset);

170
})();
        

    


            


                                


        

            



                
    

        Public Pastes
    

    
            

    

                    

        

    





    


    
    
    
    
    
    
    
    




    


    



                

            We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.             OK, I Understand
        

    
                

            

                
                    
                
            

            

                Not a member of Pastebin yet?

                Sign Up, it unlocks many cool features!