API tools faq
paste
Advertisement
tomdraug

kaggle issue

tomdraug
Jan 13th, 2019
96
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 25.92 KB | None | 0 0
raw download clone embed print report
  1.  
  2. Histopathologic Cancer Detection
  3.  
  4. https://www.kaggle.com/c/histopathologic-cancer-detection
  5.  
  6. I use fast.ai library for pytorch library
  7.  
  8. %reload_ext autoreload
  9. %autoreload 2
  10. %matplotlib inline
  11.  
  12. !pip install fastai==0.7.0
  13. #!pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cu90/torch_nightly.htmlfrom fastai.imports import *
  14. !pip install torch==0.3.1
  15. !pip install torch_imports
  16. from fastai.torch_imports import *
  17. from fastai.transforms import *
  18. from fastai.conv_learner import *
  19. from fastai.model import *
  20. from fastai.dataset import *
  21. from fastai.sgdr import *
  22. from fastai.plots import *
  23.  
  24. Collecting fastai==0.7.0
  25. Downloading https://files.pythonhosted.org/packages/50/6d/9d0d6e17a78b0598d5e8c49a0d03ffc7ff265ae62eca3e2345fab14edb9b/fastai-0.7.0-py3-none-any.whl (112kB)
  26. 100% |████████████████████████████████| 122kB 6.4MB/s
  27. Requirement already satisfied: Pygments in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.2.0)
  28. Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.1.0)
  29. Requirement already satisfied: entrypoints in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.2.3)
  30. Requirement already satisfied: bleach in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.1.3)
  31. Requirement already satisfied: ipywidgets in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (7.2.1)
  32. Requirement already satisfied: matplotlib in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.2.3)
  33. Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.2.1)
  34. Requirement already satisfied: html5lib in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.0.1)
  35. Requirement already satisfied: ptyprocess in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.5.2)
  36. Requirement already satisfied: jsonschema in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.6.0)
  37. Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.15.4)
  38. Requirement already satisfied: pyzmq in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (17.0.0)
  39. Requirement already satisfied: certifi in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2018.11.29)
  40. Requirement already satisfied: ipython-genutils in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.2.0)
  41. Requirement already satisfied: pickleshare in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.7.4)
  42. Requirement already satisfied: opencv-python in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (3.4.5.20)
  43. Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.6.0)
  44. Collecting torch<0.4 (from fastai==0.7.0)
  45. Downloading https://files.pythonhosted.org/packages/5b/a5/e8b50b55b1abac9f1e3346c4242f1e42a82d368a8442cbd50c532922f6c4/torch-0.3.1-cp36-cp36m-manylinux1_x86_64.whl (496.4MB)
  46. 100% |████████████████████████████████| 496.4MB 77kB/s
  47. Requirement already satisfied: cycler in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.10.0)
  48. Requirement already satisfied: MarkupSafe in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.0)
  49. Requirement already satisfied: simplegeneric in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.8.1)
  50. Requirement already satisfied: testpath in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.3.1)
  51. Requirement already satisfied: PyYAML in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (3.12)
  52. Requirement already satisfied: torchtext in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.3.1)
  53. Requirement already satisfied: sklearn-pandas in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.8.0)
  54. Requirement already satisfied: feather-format in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.4.0)
  55. Requirement already satisfied: tornado in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (5.0.2)
  56. Requirement already satisfied: pyparsing in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.2.0)
  57. Requirement already satisfied: Pillow in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (5.1.0)
  58. Requirement already satisfied: pandas in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.23.4)
  59. Requirement already satisfied: ipython in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (6.4.0)
  60. Requirement already satisfied: graphviz in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.8.4)
  61. Requirement already satisfied: plotnine in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.4.0)
  62. Requirement already satisfied: ipykernel in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (4.8.2)
  63. Requirement already satisfied: pandas-summary in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.0.5)
  64. Requirement already satisfied: webencodings in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.5.1)
  65. Requirement already satisfied: traitlets in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (4.3.2)
  66. Requirement already satisfied: widgetsnbextension in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (3.2.1)
  67. Requirement already satisfied: jupyter in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.0.0)
  68. Requirement already satisfied: wcwidth in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.1.7)
  69. Requirement already satisfied: jedi in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.12.0)
  70. Requirement already satisfied: tqdm in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (4.28.1)
  71. Requirement already satisfied: decorator in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (4.3.0)
  72. Requirement already satisfied: Jinja2 in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2.10)
  73. Requirement already satisfied: pytz in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (2018.4)
  74. Requirement already satisfied: isoweek in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.3.3)
  75. Requirement already satisfied: seaborn in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (0.9.0)
  76. Requirement already satisfied: bcolz in /opt/conda/lib/python3.6/site-packages (from fastai==0.7.0) (1.2.1)
  77. Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from bleach->fastai==0.7.0) (1.11.0)
  78. Requirement already satisfied: nbformat>=4.2.0 in /opt/conda/lib/python3.6/site-packages (from ipywidgets->fastai==0.7.0) (4.4.0)
  79. Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==0.7.0) (1.0.1)
  80. Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from torchtext->fastai==0.7.0) (2.21.0)
  81. Requirement already satisfied: scikit-learn>=0.15.0 in /opt/conda/lib/python3.6/site-packages (from sklearn-pandas->fastai==0.7.0) (0.20.2)
  82. Requirement already satisfied: pyarrow>=0.4.0 in /opt/conda/lib/python3.6/site-packages (from feather-format->fastai==0.7.0) (0.10.0)
  83. Requirement already satisfied: setuptools>=18.5 in /opt/conda/lib/python3.6/site-packages (from ipython->fastai==0.7.0) (39.1.0)
  84. Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.15 in /opt/conda/lib/python3.6/site-packages (from ipython->fastai==0.7.0) (1.0.15)
  85. Requirement already satisfied: pexpect; sys_platform != "win32" in /opt/conda/lib/python3.6/site-packages (from ipython->fastai==0.7.0) (4.5.0)
  86. Requirement already satisfied: backcall in /opt/conda/lib/python3.6/site-packages (from ipython->fastai==0.7.0) (0.1.0)
  87. Requirement already satisfied: descartes>=1.1.0 in /opt/conda/lib/python3.6/site-packages (from plotnine->fastai==0.7.0) (1.1.0)
  88. Requirement already satisfied: geopandas>=0.3.0 in /opt/conda/lib/python3.6/site-packages (from plotnine->fastai==0.7.0) (0.4.0)
  89. Requirement already satisfied: mizani>=0.4.5 in /opt/conda/lib/python3.6/site-packages (from plotnine->fastai==0.7.0) (0.5.3)
  90. Requirement already satisfied: patsy>=0.4.1 in /opt/conda/lib/python3.6/site-packages (from plotnine->fastai==0.7.0) (0.5.0)
  91. Requirement already satisfied: statsmodels>=0.8.0 in /opt/conda/lib/python3.6/site-packages (from plotnine->fastai==0.7.0) (0.9.0)
  92. Requirement already satisfied: jupyter_client in /opt/conda/lib/python3.6/site-packages (from ipykernel->fastai==0.7.0) (5.2.3)
  93. Requirement already satisfied: notebook>=4.4.1 in /opt/conda/lib/python3.6/site-packages (from widgetsnbextension->fastai==0.7.0) (5.5.0)
  94. Requirement already satisfied: qtconsole in /opt/conda/lib/python3.6/site-packages (from jupyter->fastai==0.7.0) (4.3.1)
  95. Requirement already satisfied: jupyter-console in /opt/conda/lib/python3.6/site-packages (from jupyter->fastai==0.7.0) (5.2.0)
  96. Requirement already satisfied: nbconvert in /opt/conda/lib/python3.6/site-packages (from jupyter->fastai==0.7.0) (5.3.1)
  97. Requirement already satisfied: parso>=0.2.0 in /opt/conda/lib/python3.6/site-packages (from jedi->fastai==0.7.0) (0.2.0)
  98. Requirement already satisfied: jupyter_core in /opt/conda/lib/python3.6/site-packages (from nbformat>=4.2.0->ipywidgets->fastai==0.7.0) (4.4.0)
  99. Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->torchtext->fastai==0.7.0) (3.0.4)
  100. Requirement already satisfied: urllib3<1.25,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->torchtext->fastai==0.7.0) (1.22)
  101. Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->torchtext->fastai==0.7.0) (2.6)
  102. Requirement already satisfied: fiona in /opt/conda/lib/python3.6/site-packages (from geopandas>=0.3.0->plotnine->fastai==0.7.0) (1.8.4)
  103. Requirement already satisfied: shapely in /opt/conda/lib/python3.6/site-packages (from geopandas>=0.3.0->plotnine->fastai==0.7.0) (1.6.4.post2)
  104. Requirement already satisfied: pyproj in /opt/conda/lib/python3.6/site-packages (from geopandas>=0.3.0->plotnine->fastai==0.7.0) (1.9.6)
  105. Requirement already satisfied: palettable in /opt/conda/lib/python3.6/site-packages (from mizani>=0.4.5->plotnine->fastai==0.7.0) (3.1.1)
  106. Requirement already satisfied: terminado>=0.8.1 in /opt/conda/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension->fastai==0.7.0) (0.8.1)
  107. Requirement already satisfied: Send2Trash in /opt/conda/lib/python3.6/site-packages (from notebook>=4.4.1->widgetsnbextension->fastai==0.7.0) (1.5.0)
  108. Requirement already satisfied: mistune>=0.7.4 in /opt/conda/lib/python3.6/site-packages (from nbconvert->jupyter->fastai==0.7.0) (0.8.3)
  109. Requirement already satisfied: pandocfilters>=1.4.1 in /opt/conda/lib/python3.6/site-packages (from nbconvert->jupyter->fastai==0.7.0) (1.4.2)
  110. Requirement already satisfied: attrs>=17 in /opt/conda/lib/python3.6/site-packages (from fiona->geopandas>=0.3.0->plotnine->fastai==0.7.0) (18.1.0)
  111. Requirement already satisfied: click<8,>=4.0 in /opt/conda/lib/python3.6/site-packages (from fiona->geopandas>=0.3.0->plotnine->fastai==0.7.0) (7.0)
  112. Requirement already satisfied: munch in /opt/conda/lib/python3.6/site-packages (from fiona->geopandas>=0.3.0->plotnine->fastai==0.7.0) (2.3.2)
  113. Requirement already satisfied: cligj>=0.5 in /opt/conda/lib/python3.6/site-packages (from fiona->geopandas>=0.3.0->plotnine->fastai==0.7.0) (0.5.0)
  114. Requirement already satisfied: click-plugins>=1.0 in /opt/conda/lib/python3.6/site-packages (from fiona->geopandas>=0.3.0->plotnine->fastai==0.7.0) (1.0.4)
  115. Installing collected packages: torch, fastai
  116. Found existing installation: torch 1.0.0
  117. Uninstalling torch-1.0.0:
  118. Successfully uninstalled torch-1.0.0
  119. Found existing installation: fastai 1.0.39
  120. Uninstalling fastai-1.0.39:
  121. Successfully uninstalled fastai-1.0.39
  122. Successfully installed fastai-0.7.0 torch-0.3.1
  123. Requirement already satisfied: torch==0.3.1 in /opt/conda/lib/python3.6/site-packages (0.3.1)
  124. Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from torch==0.3.1) (1.15.4)
  125. Requirement already satisfied: pyyaml in /opt/conda/lib/python3.6/site-packages (from torch==0.3.1) (3.12)
  126. Collecting torch_imports
  127. Could not find a version that satisfies the requirement torch_imports (from versions: )
  128. No matching distribution found for torch_imports
  129.  
  130. ---------------------------------------------------------------------------
  131. AttributeError Traceback (most recent call last)
  132. <ipython-input-2-c9fce4d7e207> in <module>()
  133. 3 get_ipython().system('pip install torch==0.3.1')
  134. 4 get_ipython().system('pip install torch_imports')
  135. ----> 5 from fastai.torch_imports import *
  136. 6 from fastai.transforms import *
  137. 7 from fastai.conv_learner import *
  138.  
  139. /opt/conda/lib/python3.6/site-packages/fastai/torch_imports.py in <module>()
  140. 1 import os
  141. ----> 2 import torch, torchvision, torchtext
  142. 3 from torch import nn, cuda, backends, FloatTensor, LongTensor, optim
  143. 4 import torch.nn.functional as F
  144. 5 from torch.autograd import Variable
  145.  
  146. /opt/conda/lib/python3.6/site-packages/torchtext/__init__.py in <module>()
  147. ----> 1 from . import data
  148. 2 from . import datasets
  149. 3 from . import utils
  150. 4 from . import vocab
  151. 5
  152.  
  153. /opt/conda/lib/python3.6/site-packages/torchtext/data/__init__.py in <module>()
  154. 2 from .dataset import Dataset, TabularDataset
  155. 3 from .example import Example
  156. ----> 4 from .field import RawField, Field, ReversibleField, SubwordField, NestedField, LabelField
  157. 5 from .iterator import (batch, BucketIterator, Iterator, BPTTIterator,
  158. 6 pool)
  159.  
  160. /opt/conda/lib/python3.6/site-packages/torchtext/data/field.py in <module>()
  161. 59
  162. 60
  163. ---> 61 class Field(RawField):
  164. 62 """Defines a datatype together with instructions for converting to Tensor.
  165. 63
  166.  
  167. /opt/conda/lib/python3.6/site-packages/torchtext/data/field.py in Field()
  168. 116 # numeric type.
  169. 117 dtypes = {
  170. --> 118 torch.float32: float,
  171. 119 torch.float: float,
  172. 120 torch.float64: float,
  173.  
  174. AttributeError: module 'torch' has no attribute 'float32'
  175.  
  176. torch.cuda.set_device(0)
  177.  
  178. ---------------------------------------------------------------------------
  179. NameError Traceback (most recent call last)
  180. <ipython-input-3-95c0021f53ac> in <module>()
  181. ----> 1 torch.cuda.set_device(0)
  182.  
  183. NameError: name 'torch' is not defined
  184.  
  185. Kaggle Histopathologic Cancer Detection. Get data from https://www.kaggle.com/c/histopathologic-cancer-detection/data
  186.  
  187. PATH = "/kaggle/input/"
  188. sz = 96
  189. arch = resnext50
  190. bs = 112
  191.  
  192. ---------------------------------------------------------------------------
  193. NameError Traceback (most recent call last)
  194. <ipython-input-4-36203de4b1bb> in <module>()
  195. 1 PATH = "/kaggle/input/"
  196. 2 sz = 96
  197. ----> 3 arch = resnext50
  198. 4 bs = 112
  199.  
  200. NameError: name 'resnext50' is not defined
  201.  
  202. label_csv = f'{PATH}labels.csv'
  203. n = len(list(open(label_csv))) - 1 # header is not counted (-1)
  204. val_idxs = get_cv_idxs(n) # random 20% data for validation set
  205.  
  206. ---------------------------------------------------------------------------
  207. FileNotFoundError Traceback (most recent call last)
  208. <ipython-input-5-0dfefc2b5633> in <module>()
  209. 1 label_csv = f'{PATH}labels.csv'
  210. ----> 2 n = len(list(open(label_csv))) - 1 # header is not counted (-1)
  211. 3 val_idxs = get_cv_idxs(n) # random 20% data for validation set
  212.  
  213. FileNotFoundError: [Errno 2] No such file or directory: '/kaggle/input/labels.csv'
  214.  
  215. n
  216.  
  217. ---------------------------------------------------------------------------
  218. NameError Traceback (most recent call last)
  219. <ipython-input-6-ab0680a89434> in <module>()
  220. ----> 1 n
  221.  
  222. NameError: name 'n' is not defined
  223.  
  224. len(val_idxs)
  225.  
  226. ---------------------------------------------------------------------------
  227. NameError Traceback (most recent call last)
  228. <ipython-input-7-bdfc40b28c26> in <module>()
  229. ----> 1 len(val_idxs)
  230.  
  231. NameError: name 'val_idxs' is not defined
  232.  
  233. # If you haven't downloaded weights.tgz yet, download the file.
  234. # http://forums.fast.ai/t/error-when-trying-to-use-resnext50/7555
  235. # http://forums.fast.ai/t/lesson-2-in-class-discussion/7452/222
  236. #!wget -O fastai/weights.tgz http://files.fast.ai/models/weights.tgz
  237.  
  238. #!tar xvfz fastai/weights.tgz -C fastai
  239.  
  240. Initial exploration
  241.  
  242. !ls {PATH}
  243.  
  244. sample_submission.csv test train train_labels.csv
  245.  
  246. label_df = pd.read_csv(label_csv)
  247.  
  248. ---------------------------------------------------------------------------
  249. NameError Traceback (most recent call last)
  250. <ipython-input-10-650961dbaaa8> in <module>()
  251. ----> 1 label_df = pd.read_csv(label_csv)
  252.  
  253. NameError: name 'pd' is not defined
  254.  
  255. label_df.pivot_table(index="id", aggfunc=len).sort_values('id', ascending=False)
  256.  
  257. ---------------------------------------------------------------------------
  258. NameError Traceback (most recent call last)
  259. <ipython-input-11-169ab50e0f5f> in <module>()
  260. ----> 1 label_df.pivot_table(index="id", aggfunc=len).sort_values('id', ascending=False)
  261.  
  262. NameError: name 'label_df' is not defined
  263.  
  264. tfms = tfms_from_model(arch, sz, aug_tfms=transforms_top_down, max_zoom=1.1) #n_aug=12 aug_tfms=transforms_side_on
  265. data = ImageClassifierData.from_csv(PATH, 'train', f'{PATH}labels.csv', test_name='test', # we need to specify where the test set is if you want to submit to Kaggle competitions
  266. val_idxs=val_idxs, suffix='.tif', tfms=tfms, bs=bs)
  267.  
  268. ---------------------------------------------------------------------------
  269. NameError Traceback (most recent call last)
  270. <ipython-input-12-fac883bf3a3b> in <module>()
  271. ----> 1 tfms = tfms_from_model(arch, sz, aug_tfms=transforms_top_down, max_zoom=1.1) #n_aug=12 aug_tfms=transforms_side_on
  272. 2 data = ImageClassifierData.from_csv(PATH, 'train', f'{PATH}labels.csv', test_name='test', # we need to specify where the test set is if you want to submit to Kaggle competitions
  273. 3 val_idxs=val_idxs, suffix='.tif', tfms=tfms, bs=bs)
  274.  
  275. NameError: name 'tfms_from_model' is not defined
  276.  
  277. fn = PATH + data.trn_ds.fnames[0]; fn
  278.  
  279. ---------------------------------------------------------------------------
  280. NameError Traceback (most recent call last)
  281. <ipython-input-13-cee5c9114d60> in <module>()
  282. ----> 1 fn = PATH + data.trn_ds.fnames[0]; fn
  283.  
  284. NameError: name 'data' is not defined
  285.  
  286. img = PIL.Image.open(fn); img
  287.  
  288. ---------------------------------------------------------------------------
  289. NameError Traceback (most recent call last)
  290. <ipython-input-14-de231e723b9b> in <module>()
  291. ----> 1 img = PIL.Image.open(fn); img
  292.  
  293. NameError: name 'PIL' is not defined
  294.  
  295. img.size
  296.  
  297. ---------------------------------------------------------------------------
  298. NameError Traceback (most recent call last)
  299. <ipython-input-15-859141e1bca2> in <module>()
  300. ----> 1 img.size
  301.  
  302. NameError: name 'img' is not defined
  303.  
  304. size_d = {k: PIL.Image.open(PATH + k).size for k in data.trn_ds.fnames}
  305.  
  306. ---------------------------------------------------------------------------
  307. NameError Traceback (most recent call last)
  308. <ipython-input-16-4af52d784b63> in <module>()
  309. ----> 1 size_d = {k: PIL.Image.open(PATH + k).size for k in data.trn_ds.fnames}
  310.  
  311. NameError: name 'data' is not defined
  312.  
  313. row_sz, col_sz = list(zip(*size_d.values()))
  314.  
  315. ---------------------------------------------------------------------------
  316. NameError Traceback (most recent call last)
  317. <ipython-input-17-a65493d42e82> in <module>()
  318. ----> 1 row_sz, col_sz = list(zip(*size_d.values()))
  319.  
  320. NameError: name 'size_d' is not defined
  321.  
  322. row_sz = np.array(row_sz); col_sz = np.array(col_sz)
  323.  
  324. ---------------------------------------------------------------------------
  325. NameError Traceback (most recent call last)
  326. <ipython-input-18-1dda77d38aed> in <module>()
  327. ----> 1 row_sz = np.array(row_sz); col_sz = np.array(col_sz)
  328.  
  329. NameError: name 'np' is not defined
  330.  
  331. row_sz[:5]
  332.  
  333. ---------------------------------------------------------------------------
  334. NameError Traceback (most recent call last)
  335. <ipython-input-19-5a9ca5cf4b06> in <module>()
  336. ----> 1 row_sz[:5]
  337.  
  338. NameError: name 'row_sz' is not defined
  339.  
  340. len(data.trn_ds), len(data.test_ds)
  341.  
  342. ---------------------------------------------------------------------------
  343. NameError Traceback (most recent call last)
  344. <ipython-input-20-e3d82542a653> in <module>()
  345. ----> 1 len(data.trn_ds), len(data.test_ds)
  346.  
  347. NameError: name 'data' is not defined
  348.  
  349. len(data.classes), data.classes[:5]
  350.  
  351. ---------------------------------------------------------------------------
  352. NameError Traceback (most recent call last)
  353. <ipython-input-21-8aa559a05419> in <module>()
  354. ----> 1 len(data.classes), data.classes[:5]
  355.  
  356. NameError: name 'data' is not defined
  357.  
  358. Initial model
  359.  
  360. learn = ConvLearner.pretrained(arch, data, precompute=True)
  361.  
  362. ---------------------------------------------------------------------------
  363. NameError Traceback (most recent call last)
  364. <ipython-input-22-bfd96ca26d21> in <module>()
  365. ----> 1 learn = ConvLearner.pretrained(arch, data, precompute=True)
  366.  
  367. NameError: name 'ConvLearner' is not defined
  368.  
  369. learn.fit(0.03, 8)
  370.  
  371. ---------------------------------------------------------------------------
  372. NameError Traceback (most recent call last)
  373. <ipython-input-23-1d2a5863ca42> in <module>()
  374. ----> 1 learn.fit(0.03, 8)
  375.  
  376. NameError: name 'learn' is not defined
  377.  
  378. learn.save('96_pre')
  379. learn.load('96_pre')
  380.  
  381. ---------------------------------------------------------------------------
  382. NameError Traceback (most recent call last)
  383. <ipython-input-24-95e148c6c779> in <module>()
  384. ----> 1 learn.save('96_pre')
  385. 2 learn.load('96_pre')
  386.  
  387. NameError: name 'learn' is not defined
  388.  
  389. Improving
  390.  
  391. learn.precompute=False
  392.  
  393. ---------------------------------------------------------------------------
  394. NameError Traceback (most recent call last)
  395. <ipython-input-25-13e63b337452> in <module>()
  396. ----> 1 learn.precompute=False
  397.  
  398. NameError: name 'learn' is not defined
  399.  
  400. learn.fit(1e-2, 3, cycle_len=3)
  401.  
  402. ---------------------------------------------------------------------------
  403. NameError Traceback (most recent call last)
  404. <ipython-input-26-2a284dbb70eb> in <module>()
  405. ----> 1 learn.fit(1e-2, 3, cycle_len=3)
  406.  
  407. NameError: name 'learn' is not defined
  408.  
  409. learn.sched.plot_lr()
  410.  
  411. ---------------------------------------------------------------------------
  412. NameError Traceback (most recent call last)
  413. <ipython-input-27-b31956846713> in <module>()
  414. ----> 1 learn.sched.plot_lr()
  415.  
  416. NameError: name 'learn' is not defined
  417.  
  418. Unfreezing all layers to slightly modify them
  419.  
  420. learn.unfreeze()
  421. #learn.bn_freeze(True)
  422. lr=np.array([1e-5,5e-4,1e-3])
  423.  
  424. ---------------------------------------------------------------------------
  425. NameError Traceback (most recent call last)
  426. <ipython-input-28-6e92771670f3> in <module>()
  427. ----> 1 learn.unfreeze()
  428. 2 #learn.bn_freeze(True)
  429. 3 lr=np.array([1e-5,5e-4,1e-3])
  430.  
  431. NameError: name 'learn' is not defined
  432.  
  433. learn.fit(lr, 6, cycle_len=3)
  434.  
  435. ---------------------------------------------------------------------------
  436. NameError Traceback (most recent call last)
  437. <ipython-input-29-e7b82bc83593> in <module>()
  438. ----> 1 learn.fit(lr, 6, cycle_len=3)
  439.  
  440. NameError: name 'learn' is not defined
  441.  
  442. TTA
  443.  
  444. # Use a fair number of iterations to cover different combinations of flips and rotations.
  445. # The predictions are then averaged.
  446. preds_t,y_t = learn.TTA(is_test=True)
  447.  
  448. ---------------------------------------------------------------------------
  449. NameError Traceback (most recent call last)
  450. <ipython-input-30-6031a82d557f> in <module>()
  451. 1 # Use a fair number of iterations to cover different combinations of flips and rotations.
  452. 2 # The predictions are then averaged.
  453. ----> 3 preds_t,y_t = learn.TTA(is_test=True)
  454.  
  455. NameError: name 'learn' is not defined
  456.  
  457. preds_t = np.stack(preds_t, axis=-1)
  458. preds_t = np.exp(preds_t)
  459. preds_t = preds_t.mean(axis=-1)[:,1]
  460.  
  461. ---------------------------------------------------------------------------
  462. NameError Traceback (most recent call last)
  463. <ipython-input-31-9fa76da8f98a> in <module>()
  464. ----> 1 preds_t = np.stack(preds_t, axis=-1)
  465. 2 preds_t = np.exp(preds_t)
  466. 3 preds_t = preds_t.mean(axis=-1)[:,1]
  467.  
  468. NameError: name 'np' is not defined
  469.  
  470. Submission - based on https://www.kaggle.com/qitvision/a-complete-ml-pipeline-fast-ai/notebook
  471.  
  472. sample_df = pd.read_csv(PATH + 'sample_submission.csv')
  473. sub=pd.read_csv(PATH + 'sample_submission.csv').set_index('id')
  474. names=np.vectorize(lambda img_name: str(img_name).split('/')[-1][:-4])
  475. file_names= names(data.test_ds.fnames).astype(str)
  476. sub.loc[file_names,'label']=preds_t
  477. sub.to_csv(PATH + 'submission_4.csv')
  478.  
  479. ---------------------------------------------------------------------------
  480. NameError Traceback (most recent call last)
  481. <ipython-input-32-24889291a4ae> in <module>()
  482. ----> 1 sample_df = pd.read_csv(PATH + 'sample_submission.csv')
  483. 2 sub=pd.read_csv(PATH + 'sample_submission.csv').set_index('id')
  484. 3 names=np.vectorize(lambda img_name: str(img_name).split('/')[-1][:-4])
  485. 4 file_names= names(data.test_ds.fnames).astype(str)
  486. 5 sub.loc[file_names,'label']=preds_t
  487.  
  488. NameError: name 'pd' is not defined
  489.  
  490. The End
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!