API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Feb 19th, 2017
79
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 37.95 KB | None | 0 0
raw download clone embed print report
  1. {
  2. "cells": [
  3. {
  4. "cell_type": "markdown",
  5. "metadata": {},
  6. "source": [
  7. "### Class 7- Starter code\n"
  8. ]
  9. },
  10. {
  11. "cell_type": "code",
  12. "execution_count": 1,
  13. "metadata": {
  14. "collapsed": false
  15. },
  16. "outputs": [],
  17. "source": [
  18. "import numpy as np\n",
  19. "import pandas as pd\n",
  20. "from sklearn import linear_model, metrics\n"
  21. ]
  22. },
  23. {
  24. "cell_type": "markdown",
  25. "metadata": {},
  26. "source": [
  27. "### Create sample data and fit a model"
  28. ]
  29. },
  30. {
  31. "cell_type": "code",
  32. "execution_count": 2,
  33. "metadata": {
  34. "collapsed": false
  35. },
  36. "outputs": [],
  37. "source": [
  38. "df = pd.DataFrame({'x': range(100), 'y': range(100)})\n",
  39. "biased_df = df.copy()\n",
  40. "biased_df.loc[:20, 'x'] = 1\n",
  41. "biased_df.loc[:20, 'y'] = 1\n",
  42. "\n",
  43. "def append_jitter(series):\n",
  44. " jitter = np.random.random_sample(size=100)\n",
  45. " return series + jitter\n",
  46. "\n",
  47. "df['x'] = append_jitter(df.x)\n",
  48. "df['y'] = append_jitter(df.y)\n",
  49. "\n",
  50. "biased_df['x'] = append_jitter(biased_df.x)\n",
  51. "biased_df['y'] = append_jitter(biased_df.y)\n"
  52. ]
  53. },
  54. {
  55. "cell_type": "code",
  56. "execution_count": 3,
  57. "metadata": {
  58. "collapsed": false
  59. },
  60. "outputs": [
  61. {
  62. "name": "stdout",
  63. "output_type": "stream",
  64. "text": [
  65. "0.168113519476\n"
  66. ]
  67. }
  68. ],
  69. "source": [
  70. "## fit\n",
  71. "lm = linear_model.LinearRegression().fit(df[['x']], df['y'])\n",
  72. "print metrics.mean_squared_error(df['y'], lm.predict(df[['x']]))\n"
  73. ]
  74. },
  75. {
  76. "cell_type": "code",
  77. "execution_count": 4,
  78. "metadata": {
  79. "collapsed": false
  80. },
  81. "outputs": [
  82. {
  83. "name": "stdout",
  84. "output_type": "stream",
  85. "text": [
  86. "0.17157553965\n"
  87. ]
  88. }
  89. ],
  90. "source": [
  91. "## biased fit\n",
  92. "lm = linear_model.LinearRegression().fit(biased_df[['x']], biased_df['y'])\n",
  93. "print metrics.mean_squared_error(df['y'], lm.predict(df[['x']]))"
  94. ]
  95. },
  96. {
  97. "cell_type": "markdown",
  98. "metadata": {},
  99. "source": [
  100. "## Cross validation\n",
  101. "#### Intro to cross validation with bike share data from last time. We will be modeling casual ridership. "
  102. ]
  103. },
  104. {
  105. "cell_type": "code",
  106. "execution_count": 5,
  107. "metadata": {
  108. "collapsed": false
  109. },
  110. "outputs": [],
  111. "source": [
  112. "from sklearn import cross_validation\n",
  113. "wd = '../../assets/dataset/'\n",
  114. "bikeshare = pd.read_csv(wd + 'bikeshare.csv')"
  115. ]
  116. },
  117. {
  118. "cell_type": "markdown",
  119. "metadata": {},
  120. "source": [
  121. "#### Create dummy variables and set outcome (dependent) variable"
  122. ]
  123. },
  124. {
  125. "cell_type": "code",
  126. "execution_count": 6,
  127. "metadata": {
  128. "collapsed": true
  129. },
  130. "outputs": [],
  131. "source": [
  132. "weather = pd.get_dummies(bikeshare.weathersit, prefix='weather')\n",
  133. "modeldata = bikeshare[['temp', 'hum']].join(weather[['weather_1', 'weather_2', 'weather_3']])\n",
  134. "y = bikeshare.casual "
  135. ]
  136. },
  137. {
  138. "cell_type": "markdown",
  139. "metadata": {},
  140. "source": [
  141. "#### Create a cross valiation with 5 folds"
  142. ]
  143. },
  144. {
  145. "cell_type": "code",
  146. "execution_count": 7,
  147. "metadata": {
  148. "collapsed": true
  149. },
  150. "outputs": [],
  151. "source": [
  152. "kf = cross_validation.KFold(len(modeldata), n_folds=5, shuffle=True)"
  153. ]
  154. },
  155. {
  156. "cell_type": "code",
  157. "execution_count": 8,
  158. "metadata": {
  159. "collapsed": false
  160. },
  161. "outputs": [
  162. {
  163. "name": "stdout",
  164. "output_type": "stream",
  165. "text": [
  166. "~~~~ CROSS VALIDATION each fold ~~~~\n",
  167. "Model 1\n",
  168. "MSE: 1808.27525551\n",
  169. "R2: 0.311880442375\n",
  170. "Model 2\n",
  171. "MSE: 1675.17498999\n",
  172. "R2: 0.311926484007\n",
  173. "Model 3\n",
  174. "MSE: 1641.19957929\n",
  175. "R2: 0.311884067469\n",
  176. "Model 4\n",
  177. "MSE: 1698.31203534\n",
  178. "R2: 0.311899972438\n",
  179. "Model 5\n",
  180. "MSE: 1544.00046427\n",
  181. "R2: 0.311895007988\n",
  182. "~~~~ SUMMARY OF CROSS VALIDATION ~~~~\n",
  183. "Mean of MSE for all folds: 1673.39246488\n",
  184. "Mean of R2 for all folds: 0.311897194855\n"
  185. ]
  186. }
  187. ],
  188. "source": [
  189. "mse_values = []\n",
  190. "scores = []\n",
  191. "n= 0\n",
  192. "print \"~~~~ CROSS VALIDATION each fold ~~~~\"\n",
  193. "for train_index, test_index in kf:\n",
  194. " lm = linear_model.LinearRegression().fit(modeldata.iloc[train_index], y.iloc[train_index])\n",
  195. " mse_values.append(metrics.mean_squared_error(y.iloc[test_index], lm.predict(modeldata.iloc[test_index])))\n",
  196. " scores.append(lm.score(modeldata, y))\n",
  197. " n+=1\n",
  198. " print 'Model', n\n",
  199. " print 'MSE:', mse_values[n-1]\n",
  200. " print 'R2:', scores[n-1]\n",
  201. "\n",
  202. "\n",
  203. "print \"~~~~ SUMMARY OF CROSS VALIDATION ~~~~\"\n",
  204. "print 'Mean of MSE for all folds:', np.mean(mse_values)\n",
  205. "print 'Mean of R2 for all folds:', np.mean(scores)"
  206. ]
  207. },
  208. {
  209. "cell_type": "code",
  210. "execution_count": 9,
  211. "metadata": {
  212. "collapsed": false
  213. },
  214. "outputs": [
  215. {
  216. "name": "stdout",
  217. "output_type": "stream",
  218. "text": [
  219. "~~~~ Single Model ~~~~\n",
  220. "MSE of single model: 1672.58110765\n",
  221. "R2: 0.311934605989\n"
  222. ]
  223. }
  224. ],
  225. "source": [
  226. "lm = linear_model.LinearRegression().fit(modeldata, y)\n",
  227. "print \"~~~~ Single Model ~~~~\"\n",
  228. "print 'MSE of single model:', metrics.mean_squared_error(y, lm.predict(modeldata))\n",
  229. "print 'R2: ', lm.score(modeldata, y)"
  230. ]
  231. },
  232. {
  233. "cell_type": "markdown",
  234. "metadata": {},
  235. "source": [
  236. "### Check\n",
  237. "While the cross validated approach here generated more overall error, which of the two approaches would predict new data more accurately: the single model or the cross validated, averaged one? Why?\n"
  238. ]
  239. },
  240. {
  241. "cell_type": "markdown",
  242. "metadata": {},
  243. "source": [
  244. "Cross-validation would predict new data more accurately, as it swaps bias error for generalized error (variance). We would rather describe future trends well enough than past trends (or training data?) perfectly. "
  245. ]
  246. },
  247. {
  248. "cell_type": "markdown",
  249. "metadata": {},
  250. "source": [
  251. "### There are ways to improve our model with regularization. \n",
  252. "Let's check out the effects on MSE and R2"
  253. ]
  254. },
  255. {
  256. "cell_type": "code",
  257. "execution_count": 10,
  258. "metadata": {
  259. "collapsed": false
  260. },
  261. "outputs": [
  262. {
  263. "name": "stdout",
  264. "output_type": "stream",
  265. "text": [
  266. "~~~ OLS ~~~\n",
  267. "OLS MSE: 1672.58110765\n",
  268. "OLS R2: 0.311934605989\n",
  269. "~~~ Lasso ~~~\n",
  270. "Lasso MSE: 1725.41581608\n",
  271. "Lasso R2: 0.290199495922\n",
  272. "~~~ Ridge ~~~\n",
  273. "Ridge MSE: 1672.60490113\n",
  274. "Ridge R2: 0.311924817843\n"
  275. ]
  276. }
  277. ],
  278. "source": [
  279. "lm = linear_model.LinearRegression().fit(modeldata, y)\n",
  280. "print \"~~~ OLS ~~~\"\n",
  281. "print 'OLS MSE: ', metrics.mean_squared_error(y, lm.predict(modeldata))\n",
  282. "print 'OLS R2:', lm.score(modeldata, y)\n",
  283. "\n",
  284. "lm = linear_model.Lasso().fit(modeldata, y)\n",
  285. "print \"~~~ Lasso ~~~\"\n",
  286. "print 'Lasso MSE: ', metrics.mean_squared_error(y, lm.predict(modeldata))\n",
  287. "print 'Lasso R2:', lm.score(modeldata, y)\n",
  288. "\n",
  289. "lm = linear_model.Ridge().fit(modeldata, y)\n",
  290. "print \"~~~ Ridge ~~~\"\n",
  291. "print 'Ridge MSE: ', metrics.mean_squared_error(y, lm.predict(modeldata))\n",
  292. "print 'Ridge R2:', lm.score(modeldata, y)"
  293. ]
  294. },
  295. {
  296. "cell_type": "markdown",
  297. "metadata": {},
  298. "source": [
  299. "### Figuring out the alphas can be done by \"hand\""
  300. ]
  301. },
  302. {
  303. "cell_type": "code",
  304. "execution_count": 11,
  305. "metadata": {
  306. "collapsed": false
  307. },
  308. "outputs": [
  309. {
  310. "name": "stdout",
  311. "output_type": "stream",
  312. "text": [
  313. "Alpha: 1e-10\n",
  314. "[ 112.68901765 -84.01121684 -24.68489063 -21.00314493 -21.71893628]\n",
  315. "1672.58110765\n",
  316. "Alpha: 1e-09\n",
  317. "[ 112.68901765 -84.01121684 -24.6848906 -21.00314491 -21.71893626]\n",
  318. "1672.58110765\n",
  319. "Alpha: 1e-08\n",
  320. "[ 112.68901765 -84.01121684 -24.6848904 -21.00314471 -21.71893606]\n",
  321. "1672.58110765\n",
  322. "Alpha: 1e-07\n",
  323. "[ 112.68901763 -84.01121682 -24.68488837 -21.00314268 -21.71893403]\n",
  324. "1672.58110765\n",
  325. "Alpha: 1e-06\n",
  326. "[ 112.68901745 -84.01121667 -24.68486804 -21.00312237 -21.71891373]\n",
  327. "1672.58110765\n",
  328. "Alpha: 1e-05\n",
  329. "[ 112.68901562 -84.01121509 -24.68466472 -21.00291929 -21.71871079]\n",
  330. "1672.58110765\n",
  331. "Alpha: 0.0001\n",
  332. "[ 112.68899732 -84.01119938 -24.68263174 -21.00088873 -21.71668161]\n",
  333. "1672.58110765\n",
  334. "Alpha: 0.001\n",
  335. "[ 112.68881437 -84.01104228 -24.66232204 -20.98060316 -21.69640993]\n",
  336. "1672.58110774\n",
  337. "Alpha: 0.01\n",
  338. "[ 112.68698753 -84.00947323 -24.46121539 -20.77973778 -21.49568404]\n",
  339. "1672.58111645\n",
  340. "Alpha: 0.1\n",
  341. "[ 112.66896732 -83.99396383 -22.63109556 -18.95202277 -19.66942371]\n",
  342. "1672.58185208\n",
  343. "Alpha: 1.0\n",
  344. "[ 112.50129738 -83.84805622 -13.38214934 -9.72671278 -10.46162477]\n",
  345. "1672.60490113\n",
  346. "Alpha: 10.0\n",
  347. "[ 110.96062533 -82.49604961 -3.94431741 -0.51765034 -1.45024412]\n",
  348. "1672.83347262\n",
  349. "Alpha: 100.0\n",
  350. "[ 97.69060562 -71.17602377 -0.31585194 1.18284675 -1.33281591]\n",
  351. "1686.31830362\n",
  352. "Alpha: 1000.0\n",
  353. "[ 44.59923075 -30.85843772 5.07876321 0.05369643 -5.107457 ]\n",
  354. "1937.81576044\n",
  355. "Alpha: 10000.0\n",
  356. "[ 7.03007064 -5.07733082 3.29039029 -1.2136063 -2.06842808]\n",
  357. "2314.83675678\n",
  358. "Alpha: 100000.0\n",
  359. "[ 0.75195708 -0.56490872 0.52067881 -0.25075496 -0.26895254]\n",
  360. "2415.77806566\n",
  361. "Alpha: 1000000.0\n",
  362. "[ 0.07576571 -0.05727511 0.05520142 -0.0273591 -0.02774349]\n",
  363. "2429.28026459\n",
  364. "Alpha: 10000000.0\n",
  365. "[ 0.00758239 -0.00573569 0.0055535 -0.00276043 -0.00278317]\n",
  366. "2430.68891798\n",
  367. "Alpha: 100000000.0\n",
  368. "[ 0.0007583 -0.00057365 0.00055569 -0.00027629 -0.00027841]\n",
  369. "2430.83041212\n",
  370. "Alpha: 1000000000.0\n",
  371. "[ 7.58303020e-05 -5.73659720e-05 5.55719458e-05 -2.76314619e-05\n",
  372. " -2.78414555e-05]\n",
  373. "2430.84456787\n",
  374. "Alpha: 10000000000.0\n",
  375. "[ 7.58303603e-06 -5.73660542e-06 5.55722818e-06 -2.76317091e-06\n",
  376. " -2.78415441e-06]\n",
  377. "2430.84598351\n"
  378. ]
  379. }
  380. ],
  381. "source": [
  382. "alphas = np.logspace(-10, 10, 21)\n",
  383. "for a in alphas:\n",
  384. " print 'Alpha:', a\n",
  385. " lm = linear_model.Ridge(alpha=a)\n",
  386. " lm.fit(modeldata, y)\n",
  387. " print lm.coef_\n",
  388. " print metrics.mean_squared_error(y, lm.predict(modeldata))"
  389. ]
  390. },
  391. {
  392. "cell_type": "markdown",
  393. "metadata": {},
  394. "source": [
  395. "### Or we can use grid search to make this faster"
  396. ]
  397. },
  398. {
  399. "cell_type": "code",
  400. "execution_count": 12,
  401. "metadata": {
  402. "collapsed": false
  403. },
  404. "outputs": [
  405. {
  406. "data": {
  407. "text/plain": [
  408. "GridSearchCV(cv=None, error_score='raise',\n",
  409. " estimator=Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,\n",
  410. " normalize=False, random_state=None, solver='auto', tol=0.001),\n",
  411. " fit_params={}, iid=True, n_jobs=1,\n",
  412. " param_grid={'alpha': array([ 1.00000e-10, 1.00000e-09, 1.00000e-08, 1.00000e-07,\n",
  413. " 1.00000e-06, 1.00000e-05, 1.00000e-04, 1.00000e-03,\n",
  414. " 1.00000e-02, 1.00000e-01, 1.00000e+00, 1.00000e+01,\n",
  415. " 1.00000e+02, 1.00000e+03, 1.00000e+04, 1.00000e+05,\n",
  416. " 1.00000e+06, 1.00000e+07, 1.00000e+08, 1.00000e+09,\n",
  417. " 1.00000e+10])},\n",
  418. " pre_dispatch='2*n_jobs', refit=True, scoring='mean_squared_error',\n",
  419. " verbose=0)"
  420. ]
  421. },
  422. "execution_count": 12,
  423. "metadata": {},
  424. "output_type": "execute_result"
  425. }
  426. ],
  427. "source": [
  428. "from sklearn import grid_search\n",
  429. "\n",
  430. "alphas = np.logspace(-10, 10, 21)\n",
  431. "gs = grid_search.GridSearchCV(\n",
  432. " estimator=linear_model.Ridge(),\n",
  433. " param_grid={'alpha': alphas},\n",
  434. " scoring='mean_squared_error')\n",
  435. "\n",
  436. "gs.fit(modeldata, y)\n"
  437. ]
  438. },
  439. {
  440. "cell_type": "markdown",
  441. "metadata": {},
  442. "source": [
  443. "##### Best score "
  444. ]
  445. },
  446. {
  447. "cell_type": "code",
  448. "execution_count": 13,
  449. "metadata": {
  450. "collapsed": false
  451. },
  452. "outputs": [
  453. {
  454. "data": {
  455. "text/plain": [
  456. "-1814.0936913337957"
  457. ]
  458. },
  459. "execution_count": 13,
  460. "metadata": {},
  461. "output_type": "execute_result"
  462. }
  463. ],
  464. "source": [
  465. "gs.best_score_ "
  466. ]
  467. },
  468. {
  469. "cell_type": "markdown",
  470. "metadata": {},
  471. "source": [
  472. "##### mean squared error here comes in negative, so let's make it positive."
  473. ]
  474. },
  475. {
  476. "cell_type": "code",
  477. "execution_count": 14,
  478. "metadata": {
  479. "collapsed": false
  480. },
  481. "outputs": [
  482. {
  483. "data": {
  484. "text/plain": [
  485. "1814.0936913337957"
  486. ]
  487. },
  488. "execution_count": 14,
  489. "metadata": {},
  490. "output_type": "execute_result"
  491. }
  492. ],
  493. "source": [
  494. "-gs.best_score_ "
  495. ]
  496. },
  497. {
  498. "cell_type": "markdown",
  499. "metadata": {},
  500. "source": [
  501. "##### explains which grid_search setup worked best"
  502. ]
  503. },
  504. {
  505. "cell_type": "code",
  506. "execution_count": 15,
  507. "metadata": {
  508. "collapsed": false
  509. },
  510. "outputs": [
  511. {
  512. "data": {
  513. "text/plain": [
  514. "Ridge(alpha=10.0, copy_X=True, fit_intercept=True, max_iter=None,\n",
  515. " normalize=False, random_state=None, solver='auto', tol=0.001)"
  516. ]
  517. },
  518. "execution_count": 15,
  519. "metadata": {},
  520. "output_type": "execute_result"
  521. }
  522. ],
  523. "source": [
  524. "gs.best_estimator_ "
  525. ]
  526. },
  527. {
  528. "cell_type": "markdown",
  529. "metadata": {},
  530. "source": [
  531. "##### shows all the grid pairings and their performances."
  532. ]
  533. },
  534. {
  535. "cell_type": "code",
  536. "execution_count": 16,
  537. "metadata": {
  538. "collapsed": false
  539. },
  540. "outputs": [
  541. {
  542. "name": "stdout",
  543. "output_type": "stream",
  544. "text": [
  545. "mean: -1817.58711, std: 542.14315, params: {'alpha': 1e-10}\n",
  546. "mean: -1817.58711, std: 542.14315, params: {'alpha': 1.0000000000000001e-09}\n",
  547. "mean: -1817.58711, std: 542.14315, params: {'alpha': 1e-08}\n",
  548. "mean: -1817.58711, std: 542.14315, params: {'alpha': 9.9999999999999995e-08}\n",
  549. "mean: -1817.58711, std: 542.14315, params: {'alpha': 9.9999999999999995e-07}\n",
  550. "mean: -1817.58711, std: 542.14317, params: {'alpha': 1.0000000000000001e-05}\n",
  551. "mean: -1817.58707, std: 542.14331, params: {'alpha': 0.0001}\n",
  552. "mean: -1817.58663, std: 542.14477, params: {'alpha': 0.001}\n",
  553. "mean: -1817.58230, std: 542.15933, params: {'alpha': 0.01}\n",
  554. "mean: -1817.54318, std: 542.30102, params: {'alpha': 0.10000000000000001}\n",
  555. "mean: -1817.20111, std: 543.63587, params: {'alpha': 1.0}\n",
  556. "mean: -1814.09369, std: 556.35563, params: {'alpha': 10.0}\n",
  557. "mean: -1818.51694, std: 653.68607, params: {'alpha': 100.0}\n",
  558. "mean: -2125.58777, std: 872.45270, params: {'alpha': 1000.0}\n",
  559. "mean: -2458.08836, std: 951.30428, params: {'alpha': 10000.0}\n",
  560. "mean: -2532.21151, std: 962.80083, params: {'alpha': 100000.0}\n",
  561. "mean: -2541.38479, std: 963.98339, params: {'alpha': 1000000.0}\n",
  562. "mean: -2542.32833, std: 964.10141, params: {'alpha': 10000000.0}\n",
  563. "mean: -2542.42296, std: 964.11321, params: {'alpha': 100000000.0}\n",
  564. "mean: -2542.43242, std: 964.11439, params: {'alpha': 1000000000.0}\n",
  565. "mean: -2542.43337, std: 964.11450, params: {'alpha': 10000000000.0}\n"
  566. ]
  567. }
  568. ],
  569. "source": [
  570. "for s in gs.grid_scores_:\n",
  571. " print s"
  572. ]
  573. },
  574. {
  575. "cell_type": "markdown",
  576. "metadata": {},
  577. "source": [
  578. "### ACTIVITY"
  579. ]
  580. },
  581. {
  582. "cell_type": "markdown",
  583. "metadata": {},
  584. "source": [
  585. "## Gradient Descent"
  586. ]
  587. },
  588. {
  589. "cell_type": "code",
  590. "execution_count": 17,
  591. "metadata": {
  592. "collapsed": false,
  593. "scrolled": true
  594. },
  595. "outputs": [
  596. {
  597. "name": "stdout",
  598. "output_type": "stream",
  599. "text": [
  600. "5.2 is better than 6.2\n",
  601. "found better solution! using 5.2\n",
  602. "4.2 is better than 5.2\n",
  603. "found better solution! using 4.2\n",
  604. "3.2 is better than 4.2\n",
  605. "found better solution! using 3.2\n",
  606. "2.2 is better than 3.2\n",
  607. "found better solution! using 2.2\n",
  608. "1.2 is better than 2.2\n",
  609. "found better solution! using 1.2\n",
  610. "0.2 is better than 1.2\n",
  611. "found better solution! using 0.2\n",
  612. "6.0 is closest to 6.2\n"
  613. ]
  614. }
  615. ],
  616. "source": [
  617. "num_to_approach, start, steps, optimized = 6.2, 0., [-1, 1], False\n",
  618. "\n",
  619. "while not optimized:\n",
  620. " current_distance = num_to_approach - start\n",
  621. " got_better = False\n",
  622. " next_steps = [start + i for i in steps]\n",
  623. " for n in next_steps:\n",
  624. " distance = np.abs(num_to_approach - n)\n",
  625. " if distance < current_distance:\n",
  626. " got_better = True\n",
  627. " print distance, 'is better than', current_distance\n",
  628. " current_distance = distance\n",
  629. " start = n\n",
  630. " if got_better:\n",
  631. " print 'found better solution! using', current_distance\n",
  632. " else:\n",
  633. " optimized = True\n",
  634. " print start, 'is closest to', num_to_approach"
  635. ]
  636. },
  637. {
  638. "cell_type": "markdown",
  639. "metadata": {},
  640. "source": [
  641. "### Bonus: \n",
  642. "implement a stopping point, similar to what n_iter would do in gradient descent when we've reached \"good enough\""
  643. ]
  644. },
  645. {
  646. "cell_type": "code",
  647. "execution_count": 18,
  648. "metadata": {
  649. "collapsed": false
  650. },
  651. "outputs": [
  652. {
  653. "name": "stdout",
  654. "output_type": "stream",
  655. "text": [
  656. "15.2 is better than 16.2\n",
  657. "found better solution! using 15.2\n",
  658. "14.2 is better than 15.2\n",
  659. "found better solution! using 14.2\n",
  660. "13.2 is better than 14.2\n",
  661. "found better solution! using 13.2\n",
  662. "12.2 is better than 13.2\n",
  663. "found better solution! using 12.2\n",
  664. "11.2 is better than 12.2\n",
  665. "found better solution! using 11.2\n",
  666. "10.2 is better than 11.2\n",
  667. "found better solution! using 10.2\n",
  668. "9.2 is better than 10.2\n",
  669. "found better solution! using 9.2\n",
  670. "8.2 is better than 9.2\n",
  671. "found better solution! using 8.2\n",
  672. "7.2 is better than 8.2\n",
  673. "found better solution! using 7.2\n",
  674. "6.2 is better than 7.2\n",
  675. "found better solution! using 6.2\n",
  676. "5.2 is better than 6.2\n",
  677. "found better solution! using 5.2\n",
  678. "\n",
  679. "Exceeded maximum iterations!\n",
  680. "Stopping gradient descent...\n"
  681. ]
  682. }
  683. ],
  684. "source": [
  685. "num_to_approach, start, steps, optimized, max_iter = 6.2, -10., [-1, 1], False, 10\n",
  686. "n_iter = 0\n",
  687. "\n",
  688. "while not optimized:\n",
  689. " current_distance = num_to_approach - start\n",
  690. " got_better = False\n",
  691. " next_steps = [start + i for i in steps]\n",
  692. " for n in next_steps:\n",
  693. " distance = np.abs(num_to_approach - n)\n",
  694. " if distance < current_distance:\n",
  695. " got_better = True\n",
  696. " print distance, 'is better than', current_distance\n",
  697. " current_distance = distance\n",
  698. " start = n\n",
  699. " if got_better:\n",
  700. " print 'found better solution! using', current_distance\n",
  701. " else:\n",
  702. " optimized = True\n",
  703. " print start, 'is closest to', num_to_approach\n",
  704. " \n",
  705. " n_iter += 1\n",
  706. " \n",
  707. " if n_iter > max_iter:\n",
  708. " print \"\\nExceeded maximum iterations!\\nStopping gradient descent...\"\n",
  709. " optimized = True\n",
  710. "\n"
  711. ]
  712. },
  713. {
  714. "cell_type": "markdown",
  715. "metadata": {},
  716. "source": [
  717. "## Demo: Application of Gradient Descent "
  718. ]
  719. },
  720. {
  721. "cell_type": "code",
  722. "execution_count": 19,
  723. "metadata": {
  724. "collapsed": false
  725. },
  726. "outputs": [
  727. {
  728. "name": "stdout",
  729. "output_type": "stream",
  730. "text": [
  731. "-- Epoch 1\n",
  732. "Norm: 73.53, NNZs: 5, Bias: 14.490061, T: 17379, Avg. loss: 1005.162964\n",
  733. "Total training time: 0.01 seconds.\n",
  734. "-- Epoch 2\n",
  735. "Norm: 98.87, NNZs: 5, Bias: 17.287780, T: 34758, Avg. loss: 948.414396\n",
  736. "Total training time: 0.01 seconds.\n",
  737. "-- Epoch 3\n",
  738. "Norm: 112.96, NNZs: 5, Bias: 18.692687, T: 52137, Avg. loss: 918.970590\n",
  739. "Total training time: 0.01 seconds.\n",
  740. "-- Epoch 4\n",
  741. "Norm: 121.51, NNZs: 5, Bias: 19.512785, T: 69516, Avg. loss: 901.276514\n",
  742. "Total training time: 0.01 seconds.\n",
  743. "-- Epoch 5\n",
  744. "Norm: 127.03, NNZs: 5, Bias: 20.075823, T: 86895, Avg. loss: 889.599148\n",
  745. "Total training time: 0.02 seconds.\n",
  746. "\n",
  747. "Gradient Descent R2: 0.308305395581\n",
  748. "Gradient Descent MSE: 1681.40315977\n"
  749. ]
  750. }
  751. ],
  752. "source": [
  753. "lm = linear_model.SGDRegressor(verbose=1)\n",
  754. "lm.fit(modeldata, y)\n",
  755. "print \"\\nGradient Descent R2:\", lm.score(modeldata, y)\n",
  756. "print \"Gradient Descent MSE:\", metrics.mean_squared_error(y, lm.predict(modeldata))"
  757. ]
  758. },
  759. {
  760. "cell_type": "markdown",
  761. "metadata": {},
  762. "source": [
  763. "### Check: Untuned, how well did gradient descent perform compared to OLS?"
  764. ]
  765. },
  766. {
  767. "cell_type": "markdown",
  768. "metadata": {},
  769. "source": [
  770. "Comparing the two:\n",
  771. "\n",
  772. " OLS MSE: 1672.58110765\n",
  773. " OLS R2: 0.311934605989\n",
  774. "\n",
  775. " Gradient Descent R2: 0.307785992311\n",
  776. " Gradient Descent MSE: 1682.6657492\n",
  777. "\n",
  778. "Untuned gradient descent performed slightly worse in both MSE and R-squared."
  779. ]
  780. },
  781. {
  782. "cell_type": "markdown",
  783. "metadata": {},
  784. "source": [
  785. "# Independent Practice: Bike data revisited\n",
  786. "\n",
  787. "There are tons of ways to approach a regression problem. The regularization techniques appended to ordinary least squares optimizes the size of coefficients to best account for error. Gradient Descent also introduces learning rate (how aggressively do we solve the problem), epsilon (at what point do we say the error margin is acceptable), and iterations (when should we stop no matter what?)\n",
  788. "\n",
  789. "For this deliverable, our goals are to:\n",
  790. "\n",
  791. "- implement the gradient descent approach to our bike-share modeling problem,\n",
  792. "- show how gradient descent solves and optimizes the solution,\n",
  793. "- demonstrate the grid_search module!\n",
  794. "\n",
  795. "While exploring the Gradient Descent regressor object, you'll build a grid search using the stochastic gradient descent estimator for the bike-share data set. Continue with either the model you evaluated last class or the simpler one from today. In particular, be sure to implement the \"param_grid\" in the grid search to get answers for the following questions:\n",
  796. "\n",
  797. "- With a set of alpha values between 10^-10 and 10^-1, how does the mean squared error change?\n",
  798. "- Based on the data, we know when to properly use l1 vs l2 regularization. By using a grid search with l1_ratios between 0 and 1 (increasing every 0.05), does that statement hold true? If not, did gradient descent have enough iterations?\n",
  799. "- How do these results change when you alter the learning rate (eta0)?\n",
  800. "\n",
  801. "**Bonus**: Can you see the advantages and disadvantages of using gradient descent after finishing this exercise?"
  802. ]
  803. },
  804. {
  805. "cell_type": "markdown",
  806. "metadata": {},
  807. "source": [
  808. "### Starter Code\n",
  809. "\n",
  810. "- With a set of alpha values between 10^-10 and 10^-1, how does the mean squared error change?"
  811. ]
  812. },
  813. {
  814. "cell_type": "code",
  815. "execution_count": 20,
  816. "metadata": {
  817. "collapsed": false
  818. },
  819. "outputs": [
  820. {
  821. "name": "stdout",
  822. "output_type": "stream",
  823. "text": [
  824. "BEST ESTIMATOR\n",
  825. "1687.88462245\n",
  826. "SGDRegressor(alpha=1.0000000000000001e-05, average=False, epsilon=0.1,\n",
  827. " eta0=0.01, fit_intercept=True, l1_ratio=0.15,\n",
  828. " learning_rate='invscaling', loss='squared_loss', n_iter=5,\n",
  829. " penalty='l2', power_t=0.25, random_state=None, shuffle=True,\n",
  830. " verbose=0, warm_start=False)\n",
  831. "ALL ESTIMATORS\n",
  832. "[mean: -1690.00089, std: 83.71968, params: {'alpha': 1e-10}, mean: -1690.52355, std: 86.15990, params: {'alpha': 1.0000000000000001e-09}, mean: -1689.36280, std: 87.97990, params: {'alpha': 1e-08}, mean: -1689.67661, std: 86.58292, params: {'alpha': 9.9999999999999995e-08}, mean: -1688.35663, std: 86.38886, params: {'alpha': 9.9999999999999995e-07}, mean: -1687.88462, std: 85.34930, params: {'alpha': 1.0000000000000001e-05}, mean: -1690.12154, std: 85.66560, params: {'alpha': 0.0001}, mean: -1693.91600, std: 87.55442, params: {'alpha': 0.001}, mean: -1728.73018, std: 89.25997, params: {'alpha': 0.01}, mean: -2053.85245, std: 99.39294, params: {'alpha': 0.10000000000000001}]\n"
  833. ]
  834. }
  835. ],
  836. "source": [
  837. "alphas = np.logspace(-10, -1, 10)\n",
  838. "params = {'alpha': alphas}\n",
  839. "\n",
  840. "gs = grid_search.GridSearchCV(\n",
  841. " estimator=linear_model.SGDRegressor(),\n",
  842. " cv=cross_validation.KFold(len(modeldata), n_folds=5, shuffle=True),\n",
  843. " param_grid=params,\n",
  844. " scoring='mean_squared_error',\n",
  845. " )\n",
  846. "\n",
  847. "gs.fit(modeldata, y)\n",
  848. "\n",
  849. "print 'BEST ESTIMATOR'\n",
  850. "print -gs.best_score_\n",
  851. "print gs.best_estimator_\n",
  852. "print 'ALL ESTIMATORS'\n",
  853. "print gs.grid_scores_"
  854. ]
  855. },
  856. {
  857. "cell_type": "markdown",
  858. "metadata": {},
  859. "source": [
  860. "Changing the alphas between 10^-10 and 10^-1, we get a lowest MSE at 10^-8 of 1687.88. At 10^-10 MSE was 1690.00 and at 10^-1 the MSE was the highest value of 2053.85. Clearly picking the selecting the correct regularization parameter is important. Will set alpha to 10^-08 going forward.\n",
  861. "\n",
  862. "Next we will look at l1 vs l2:\n",
  863. "- Based on the data, we know when to properly use l1 vs l2 regularization. By using a grid search with l1_ratios between 0 and 1 (increasing every 0.05), does that statement hold true? If not, did gradient descent have enough iterations?\n"
  864. ]
  865. },
  866. {
  867. "cell_type": "code",
  868. "execution_count": 21,
  869. "metadata": {
  870. "collapsed": false
  871. },
  872. "outputs": [
  873. {
  874. "name": "stdout",
  875. "output_type": "stream",
  876. "text": [
  877. "BEST ESTIMATOR\n",
  878. "1688.43060467\n",
  879. "SGDRegressor(alpha=1e-08, average=False, epsilon=0.1, eta0=0.01,\n",
  880. " fit_intercept=True, l1_ratio=0.75, learning_rate='invscaling',\n",
  881. " loss='squared_loss', n_iter=5, penalty='l2', power_t=0.25,\n",
  882. " random_state=None, shuffle=True, verbose=0, warm_start=False)\n",
  883. "\n",
  884. "ALL ESTIMATORS\n",
  885. "[mean: -1689.50994, std: 56.84150, params: {'l1_ratio': 0.0}, mean: -1689.78229, std: 55.62580, params: {'l1_ratio': 0.050000000000000003}, mean: -1689.11004, std: 57.74534, params: {'l1_ratio': 0.10000000000000001}, mean: -1690.72398, std: 58.51519, params: {'l1_ratio': 0.15000000000000002}, mean: -1688.77832, std: 57.24667, params: {'l1_ratio': 0.20000000000000001}, mean: -1691.31072, std: 58.06187, params: {'l1_ratio': 0.25}, mean: -1690.51925, std: 59.51318, params: {'l1_ratio': 0.30000000000000004}, mean: -1690.12908, std: 56.28294, params: {'l1_ratio': 0.35000000000000003}, mean: -1689.26135, std: 58.15524, params: {'l1_ratio': 0.40000000000000002}, mean: -1688.75290, std: 57.99243, params: {'l1_ratio': 0.45000000000000001}, mean: -1689.45895, std: 57.81882, params: {'l1_ratio': 0.5}, mean: -1688.79888, std: 57.59656, params: {'l1_ratio': 0.55000000000000004}, mean: -1689.48729, std: 57.10687, params: {'l1_ratio': 0.60000000000000009}, mean: -1689.19351, std: 57.64491, params: {'l1_ratio': 0.65000000000000002}, mean: -1689.78649, std: 57.38352, params: {'l1_ratio': 0.70000000000000007}, mean: -1688.43060, std: 57.29220, params: {'l1_ratio': 0.75}, mean: -1689.41114, std: 57.81888, params: {'l1_ratio': 0.80000000000000004}, mean: -1689.36202, std: 57.49003, params: {'l1_ratio': 0.85000000000000009}, mean: -1689.15929, std: 56.94886, params: {'l1_ratio': 0.90000000000000002}, mean: -1689.29226, std: 57.48814, params: {'l1_ratio': 0.95000000000000007}, mean: -1689.21732, std: 57.47393, params: {'l1_ratio': 1.0}]\n"
  886. ]
  887. }
  888. ],
  889. "source": [
  890. "l1s = np.linspace(0, 1, num=21)\n",
  891. "params = {'l1_ratio': l1s}\n",
  892. "\n",
  893. "gs = grid_search.GridSearchCV(\n",
  894. " estimator=linear_model.SGDRegressor(alpha=0.00000001),\n",
  895. " cv=cross_validation.KFold(len(modeldata), n_folds=5, shuffle=True),\n",
  896. " param_grid=params,\n",
  897. " scoring='mean_squared_error',\n",
  898. " )\n",
  899. "\n",
  900. "gs.fit(modeldata, y)\n",
  901. "\n",
  902. "print 'BEST ESTIMATOR'\n",
  903. "print -gs.best_score_\n",
  904. "print gs.best_estimator_\n",
  905. "print '\\nALL ESTIMATORS'\n",
  906. "print gs.grid_scores_"
  907. ]
  908. },
  909. {
  910. "cell_type": "markdown",
  911. "metadata": {},
  912. "source": [
  913. "Running the above a number of times resulted in l1_ratios ranging from 0.15 to 0.85, at the default 5 iterations. Will now try to increase the number of iterations to attempt to find a stable l1_ratio:"
  914. ]
  915. },
  916. {
  917. "cell_type": "code",
  918. "execution_count": 22,
  919. "metadata": {
  920. "collapsed": false
  921. },
  922. "outputs": [
  923. {
  924. "name": "stdout",
  925. "output_type": "stream",
  926. "text": [
  927. "BEST ESTIMATOR\n",
  928. "1673.20055258\n",
  929. "SGDRegressor(alpha=1e-08, average=False, epsilon=0.1, eta0=0.01,\n",
  930. " fit_intercept=True, l1_ratio=0.65000000000000002,\n",
  931. " learning_rate='invscaling', loss='squared_loss', n_iter=20,\n",
  932. " penalty='l2', power_t=0.25, random_state=None, shuffle=True,\n",
  933. " verbose=0, warm_start=False)\n"
  934. ]
  935. }
  936. ],
  937. "source": [
  938. "l1s = np.linspace(0, 1, num=21)\n",
  939. "n_iters = (10, 20, 50, 100)\n",
  940. "params = {'l1_ratio': l1s, 'n_iter': n_iters}\n",
  941. "\n",
  942. "gs = grid_search.GridSearchCV(\n",
  943. " estimator=linear_model.SGDRegressor(alpha=0.00000001),\n",
  944. " cv=cross_validation.KFold(len(modeldata), n_folds=5, shuffle=True),\n",
  945. " param_grid=params,\n",
  946. " scoring='mean_squared_error',\n",
  947. " )\n",
  948. "\n",
  949. "gs.fit(modeldata, y)\n",
  950. "\n",
  951. "print 'BEST ESTIMATOR'\n",
  952. "print -gs.best_score_\n",
  953. "print gs.best_estimator_\n",
  954. "#print 'ALL ESTIMATORS'\n",
  955. "#print gs.grid_scores_"
  956. ]
  957. },
  958. {
  959. "cell_type": "markdown",
  960. "metadata": {
  961. "collapsed": true
  962. },
  963. "source": [
  964. "Running the above in an attempt to find a stable l1_ratio was not successful. All runs resulted in a MSE of about 1673, with l1_ratios varying anywhere from 0.1 to 0.95 and number of iterations from 20 to 100. Given that the l1_ratio appears to be dependent on the particular shuffling from a given run, it does not look like further effort should be expended in optimizing the l1_ratio, particularly with similiar MSE results. I'll set n_iters at 20 as that was the minimum value I saw for the Best Estimator and higher iterations did not markedly change results.\n",
  965. "\n",
  966. "Next will look at learning rates:"
  967. ]
  968. },
  969. {
  970. "cell_type": "code",
  971. "execution_count": 23,
  972. "metadata": {
  973. "collapsed": false
  974. },
  975. "outputs": [
  976. {
  977. "name": "stdout",
  978. "output_type": "stream",
  979. "text": [
  980. "BEST ESTIMATOR\n",
  981. "1673.89830475\n",
  982. "SGDRegressor(alpha=1e-08, average=False, epsilon=0.1, eta0=0.01,\n",
  983. " fit_intercept=True, l1_ratio=0.15, learning_rate='invscaling',\n",
  984. " loss='squared_loss', n_iter=20, penalty='l2', power_t=0.25,\n",
  985. " random_state=None, shuffle=True, verbose=0, warm_start=False)\n",
  986. "ALL ESTIMATORS\n",
  987. "[mean: -1934.35399, std: 400.13078, params: {'learning_rate': 'constant', 'eta0': 0.1}, mean: -26352639739801690830857568256.00000, std: 16865968182123518421972811776.00000, params: {'learning_rate': 'optimal', 'eta0': 0.1}, mean: -1681.45153, std: 77.83622, params: {'learning_rate': 'invscaling', 'eta0': 0.1}, mean: -1976.93762, std: 377.64384, params: {'learning_rate': 'constant', 'eta0': 0.05}, mean: -22804907109609210721452687360.00000, std: 11842527670805083618562539520.00000, params: {'learning_rate': 'optimal', 'eta0': 0.05}, mean: -1675.15116, std: 78.44786, params: {'learning_rate': 'invscaling', 'eta0': 0.05}, mean: -1694.36804, std: 83.75146, params: {'learning_rate': 'constant', 'eta0': 0.01}, mean: -12861899041372012389230182400.00000, std: 10163577213498355691307401216.00000, params: {'learning_rate': 'optimal', 'eta0': 0.01}, mean: -1673.89830, std: 77.48118, params: {'learning_rate': 'invscaling', 'eta0': 0.01}]\n"
  988. ]
  989. }
  990. ],
  991. "source": [
  992. "learning_rates = ('constant', 'optimal', 'invscaling')\n",
  993. "eta0s = (0.1, 0.05, 0.01)\n",
  994. "\n",
  995. "params = {'learning_rate': learning_rates, 'eta0': eta0s}\n",
  996. "\n",
  997. "gs = grid_search.GridSearchCV(\n",
  998. " estimator=linear_model.SGDRegressor(alpha=0.00000001, n_iter=20),\n",
  999. " cv=cross_validation.KFold(len(modeldata), n_folds=5, shuffle=True),\n",
  1000. " param_grid=params,\n",
  1001. " scoring='mean_squared_error',\n",
  1002. " )\n",
  1003. "\n",
  1004. "gs.fit(modeldata, y)\n",
  1005. "\n",
  1006. "print 'BEST ESTIMATOR'\n",
  1007. "print -gs.best_score_\n",
  1008. "print gs.best_estimator_\n",
  1009. "print 'ALL ESTIMATORS'\n",
  1010. "print gs.grid_scores_"
  1011. ]
  1012. },
  1013. {
  1014. "cell_type": "markdown",
  1015. "metadata": {
  1016. "collapsed": true
  1017. },
  1018. "source": [
  1019. "To answer the question: How do these results change when you alter the learning rate (eta0)?\n",
  1020. "\n",
  1021. "Varying learning rates between 0.1, 0.05, and .01 along with selecting between constant, optimal, and invscaling, resulted in MSE ranging from about from expected values in the range of 1673 to around 1700 for various eta0s and constant or invscaling. But absurdly high MSEs were found with optimal learning rate, in the range of 10^30! Obviously, something with the 'optimal' selection is not playing well with the selection of eta0. It may be that optimal has a default fixed eta0 and attempt to vary it leads to these non-useful results.\n",
  1022. "\n",
  1023. "In any case, repeated runs suggest that an invscaling selection along with eta0 of 0.01 resulted in the lowest MSE.\n",
  1024. "\n",
  1025. "Putting this all together for a final model:"
  1026. ]
  1027. },
  1028. {
  1029. "cell_type": "code",
  1030. "execution_count": 24,
  1031. "metadata": {
  1032. "collapsed": false
  1033. },
  1034. "outputs": [
  1035. {
  1036. "name": "stdout",
  1037. "output_type": "stream",
  1038. "text": [
  1039. "BEST ESTIMATOR\n",
  1040. "1674.35011068\n",
  1041. "SGDRegressor(alpha=1e-08, average=False, epsilon=0.1, eta0=0.01,\n",
  1042. " fit_intercept=True, l1_ratio=0.15, learning_rate='invscaling',\n",
  1043. " loss='squared_loss', n_iter=20, penalty='l2', power_t=0.25,\n",
  1044. " random_state=None, shuffle=True, verbose=0, warm_start=False)\n",
  1045. "ALL ESTIMATORS\n",
  1046. "[mean: -1785.90070, std: 97.78826, params: {'learning_rate': 'constant', 'eta0': 0.1}, mean: -11277800250012638309148262400.00000, std: 6777965403180558106704740352.00000, params: {'learning_rate': 'optimal', 'eta0': 0.1}, mean: -1687.49211, std: 109.20730, params: {'learning_rate': 'invscaling', 'eta0': 0.1}, mean: -1744.31843, std: 109.50611, params: {'learning_rate': 'constant', 'eta0': 0.05}, mean: -11050721264616227253038088192.00000, std: 8124596001136890987856003072.00000, params: {'learning_rate': 'optimal', 'eta0': 0.05}, mean: -1676.31982, std: 104.01003, params: {'learning_rate': 'invscaling', 'eta0': 0.05}, mean: -1704.08559, std: 127.64098, params: {'learning_rate': 'constant', 'eta0': 0.01}, mean: -19492282714118071886198865920.00000, std: 10900841562942524212643889152.00000, params: {'learning_rate': 'optimal', 'eta0': 0.01}, mean: -1674.35011, std: 103.74864, params: {'learning_rate': 'invscaling', 'eta0': 0.01}]\n"
  1047. ]
  1048. }
  1049. ],
  1050. "source": [
  1051. "gs = grid_search.GridSearchCV(\n",
  1052. " estimator=linear_model.SGDRegressor(alpha=0.00000001, n_iter=20, learning_rate='invscaling', eta0=0.01),\n",
  1053. " cv=cross_validation.KFold(len(modeldata), n_folds=5, shuffle=True),\n",
  1054. " param_grid=params,\n",
  1055. " scoring='mean_squared_error',\n",
  1056. " )\n",
  1057. "\n",
  1058. "gs.fit(modeldata, y)\n",
  1059. "\n",
  1060. "print 'BEST ESTIMATOR'\n",
  1061. "print -gs.best_score_\n",
  1062. "print gs.best_estimator_\n",
  1063. "print 'ALL ESTIMATORS'\n",
  1064. "print gs.grid_scores_"
  1065. ]
  1066. },
  1067. {
  1068. "cell_type": "markdown",
  1069. "metadata": {},
  1070. "source": [
  1071. "Repeated runs resulted in an MSE ranging from 1673 to 1675.\n",
  1072. "\n",
  1073. "### Observations and Conclusions\n",
  1074. "\n",
  1075. "One notable result is that the optimizations converged on a MSE of about 1673 once we started varying the number of iterations and the l1 ratio. Futher optimization with learning rates did not improve the error, although the error could be made markedly worse with the 'optimal' learning rate selection.\n",
  1076. "\n",
  1077. "I suspect that the area of diminishing returns has been reached: additional effort on optimization is unlikely to result in further significant decreases in MSE. This is likely due to the fundamental nature of Gradient Descent: it seeks to minimize the error in a step-wise fashion, converging on a solution of a given set of data. If there is no perfect solution, than the result of Gradient Descent will have some inherent level of error. In this case, a linear regression is able to fit the data well but not perfectly, hence the results we saw with the Gradient Descent."
  1078. ]
  1079. }
  1080. ],
  1081. "metadata": {
  1082. "anaconda-cloud": {},
  1083. "kernelspec": {
  1084. "display_name": "Python [default]",
  1085. "language": "python",
  1086. "name": "python2"
  1087. },
  1088. "language_info": {
  1089. "codemirror_mode": {
  1090. "name": "ipython",
  1091. "version": 2
  1092. },
  1093. "file_extension": ".py",
  1094. "mimetype": "text/x-python",
  1095. "name": "python",
  1096. "nbconvert_exporter": "python",
  1097. "pygments_lexer": "ipython2",
  1098. "version": "2.7.12"
  1099. }
  1100. },
  1101. "nbformat": 4,
  1102. "nbformat_minor": 0
  1103. }
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!