# Deep Learning

## Machine Learning Basics

### Learning

Experience E —> Performance P

> (Mitchell, 97) “A computer program is said to learn from experience E w.r.t. some class of tasks T and performance measure P, if its performance at tasks in T, as measuredby P, improves with experience E.”

### Learning Tasks

- Classification
- Regression
- Transcription (e.g. OCR)
- Machine translation
- Structured output (e.g. natural language -> tree)
- Anomaly detection
- Synthesis and sampling
- Imputation of missing values
- Denoising
- Estimation of pdf or pmf

### Experience

- Supervised Learning
  - Get output data from input data
- Unsupervised Learning
  - Data mining with only input data
- Semi-supervised Learning
  - Always has input but only has output data at the very first time for labeling
- Reinforcement Learning
  - ex) games
  - Agent give action to environment and get rewards

### Statistical Learning

- Train and test points are...
  - idependently sampled
  - identically distributed

### Supervised Learning

**Dataset **: $\{(x_i,y_i)\}^{m}_{i=1}, x_i \in \mathbb{R}^n$

- Classification : $y_i$ is categorical e.g. $y_i \in \{-1, +1\}$
- Regression : $y_i \in \mathbb{R}^n$