API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Aug 29th, 2016
67
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 15.77 KB | None | 0 0
raw download clone embed print report
  1. {
  2. "cells": [
  3. {
  4. "cell_type": "markdown",
  5. "metadata": {},
  6. "source": [
  7. "<h3>Structural Time Series</h3>\n",
  8. "\n",
  9. "<p>The structural time series is a different theoretic framework from Box-Jenkins modeling framework.Although they do have a good deal in common that they are both based on statistical models and are normally handled by classical procedures, the structural time series starts from a somewhat different point and in doing so challenges some of the underlying assumptions of the Box-Jenkins approach, such as the most fundamental one: stationarity.\n",
  10. "\n",
  11. "<p>The structural time series is set up in terms of components which have a direct interpretation, and we analysis the components we are interested. The structural time series provides a clear link with regression models in their technical formulation, it also has a clear link with regression models. A structural model can not only provide forecasts, but can also, through estimates of the components, present a set of stylised facts (Harvey and Jaeger 1999). A thorough discussion of the methodological and technical ideas underlying Structural Time Series models is contained in the monographs by Harvey (1989) and WWest and harrison (1989)\n",
  12. "\n"
  13. ]
  14. },
  15. {
  16. "cell_type": "markdown",
  17. "metadata": {},
  18. "source": [
  19. "<h3>Introduction to State Space Form</h3>\n",
  20. "\n",
  21. "<p>The State Space Form (SSF) is the methodology or a tool to handeling structural time series,all the linear time series models can be written into a linear state space form.It is a powerful tool rooted and originally from the engineering field's control theory. The state space form opened a new way to study the time series, or we can say that State Space Form generalizes the time series theories. \n",
  22. "\n",
  23. "<p>The estimation of state space form is known as linear quadratic estimation, which is better known as the Kalman Filter named after the Hungarian mathematician Rudolf Kalman. One of the great advantages of the Kalman Filter algorithm is that does not require any assumptions on the errors. However, the filter yields the exact conditional probability estimate in the special case that all errors are Gaussian-distributed.\n",
  24. "\n",
  25. "<p>Parameters in the state space form are called hyperparameters. S.J Koopman (2001) showed the ML estimation of hyperparameters can be calculated by a routine application of the kalman Filter, and the Kalman Filter will yield the minimum variance linear unbiased stimation under the Gaussian assumtion, and will yield a minimum variance linear unbiased estimation without the Gaussian assumption."
  26. ]
  27. },
  28. {
  29. "cell_type": "markdown",
  30. "metadata": {},
  31. "source": [
  32. "<h3>The general linear Gaussian State Space Form</h3>\n",
  33. "\n",
  34. "<p> Since the linear state space form is a very general form to represent all linear time series, there are many different ways to write the linear state space form. With the Gaussian assumptions, we will adopt the Linear Gaussian State Space Form as:\n",
  35. "\n",
  36. "$$\n",
  37. "\\begin{align}\n",
  38. "& y_t = Z_t \\alpha_t +\\epsilon_t,\\qquad \\epsilon_t\\sim iid.\\mathcal{N}(0,H_t),\\quad t=1,\\dotsc,T \\\\\n",
  39. "& \\alpha_t = T_t \\alpha_t + R_t \\eta_t,\\quad \\eta_t \\sim iid. \\mathcal{N}(0,Q_t),\\quad \\alpha_1 \\sim N(a_1,P_1) \\\\\n",
  40. "\\end{align}\n",
  41. "$$\n",
  42. "\n",
  43. "Where the first equation is $measurement \\, equation$ or $observation \\, equation$, the second equation is $transition \\, equation$ or $state \\, equation$. \n",
  44. "The $a_1$ and $P_1$ are assumped to be given. The determination of $N(a_1,P_1)$ is initial value problem and it will be discussed independently in a separate section. Both of the two initial values are independent of the two error terms, while the two error terms also mutually independent for all periods. namely:\n",
  45. "\n",
  46. "$$\n",
  47. "\\left\\{ a_1 \\right\\} \\wedge \\left\\{ P_1 \\right\\} \\bot \\left\\{ \\epsilon_t \\right\\} \\bot \\left\\{ \\eta_t \\right\\},\\quad \\forall t\\in \\left[1,T \\right].\n",
  48. "$$\n",
  49. "\n",
  50. "The hyperparameters are either known or unknown that need to be estimated, so they are all non-random elements under frequencist's scenario.\n",
  51. "If the state space form contains multiple time series, both of the two error covariance matrices are semi-positive definite diagonal matrices and higher dimention arrangements need to be assumed. Besides, the coefficients matrices $Z_t$, $T_t$, $R_t$, $H_t$, $Q_t$, are $system\\, matrices$.\n",
  52. "\n",
  53. "<p>The vectors in a multiple time series state space form follow these dimentions settings: $ \\forall \\, \\left\\{p,m,r \\right\\} \\in \\mathbb{Z}^+ $\n",
  54. "<ul>\n",
  55. " <li> $ y_t\\sim p\\times 1 \\qquad \\alpha_t \\sim m \\times 1 $ </li>\n",
  56. " <li> $ \\epsilon_t \\sim p \\times 1 \\qquad \\eta_t \\sim r \\times 1 \\quad r \\leqslant m. $ </li>\n",
  57. "</ul>\n",
  58. "\n",
  59. "<p>The hyperparameters in the state space form have dimentions:\n",
  60. "<ul>\n",
  61. " <li> $ Z_t \\sim p \\times m \\qquad T_t \\sim m \\times m \\qquad R_t \\sim m \\times r $ </li>\n",
  62. " <li> $ H_t \\sim p \\times p \\qquad Q_t \\sim r \\times r. $ </li>\n",
  63. "</ul>\n",
  64. "\n",
  65. "\n"
  66. ]
  67. },
  68. {
  69. "cell_type": "markdown",
  70. "metadata": {},
  71. "source": [
  72. "<h3>The autoregressive integrated moving average models</h3>\n",
  73. "\n",
  74. "<p>The frequently used phrase ARIMA stands for Autoregressive integrated Moving Average. It is a time series model system developed by Box and Jenkins in their pathbreaking book (1970). In stead of researching the different components independently and separately, Box and Jenkins regarded a sequence $y_t$ as made up of many different components, such as trend, seasonal, cycle and irregular components. The main methodology is to eliminate certain components to reach the fundamental assumption stationarity. For instance, eliminating the trend and seasonal components by differencing operation. This is because of the statistical characteristic properties maintains invariant through time under stationarity.\n",
  75. "\n",
  76. "<p> Given a discrete sequence $\\left\\{ y_t \\right\\}_{t\\in T}$, a transformed sequence is defined as:\n",
  77. "\n",
  78. "$$\n",
  79. "\\left\\{ y_{t}^* = \\Delta^{d}\\Delta_{s}^{D}y_t\\right\\}_{t\\in T} \\qquad d,D \\in \\left\\{0\\wedge Z^+ \\right\\}.\n",
  80. "$$\n",
  81. "\n",
  82. "Where $\\Delta y_t = y_t - y_{t-1}$, $\\Delta^2 y_t = \\Delta (\\Delta y_t) $, $\\Delta_{s}y_t = y_t - y_{t-s}$ and $\\Delta_{s}^2 y_t = \\Delta_{s}(\\Delta_{s}y_t) $\n",
  83. "\n",
  84. "\n",
  85. "<p>Thus, denoting $r=max\\,(p,q+1)$ and $D=0$, a General $ARIMA\\,(p,d,q)$ model system can be written as:\n",
  86. "\n",
  87. "$$\n",
  88. "y_{t}^* = \\sum_{j=1}^{r}\\phi_j y_{t-j}^* + \\zeta_t +\\sum_{j=1}^{r-1}\\theta_j \\zeta_{t-j}, \\qquad \\zeta_t \\sim iid. \\mathcal{N}(0,\\sigma_{\\zeta}^2), \\qquad t \\in [1,T].\n",
  89. "$$\n",
  90. "\n",
  91. "The $y_t^*$ in this equation can be replaced by $y_t$ if $d=D=0$.\n",
  92. "\n",
  93. "<p>$\\,$\n",
  94. "\n",
  95. "<p>Going one more step further to generalize this model, exogenous variables are added and the $D=0$ constraint is dropped, we can get the $SARIMAX\\,(p,d,q,D)$ model, where $\"S\"$ stands for seasonal and $\"X\"$ stands for the exogenous variables. The model system written as:\n",
  96. "\n",
  97. "$$\n",
  98. "y_{t}^* = \\sum_{j=1}^{r}\\phi_j y_{t-j}^* +\\sum_{j=1}^k \\beta_j x_{jt}+ \\zeta_t +\\sum_{j=1}^{r-1}\\theta_j \\zeta_{t-j}, \\qquad \\zeta_t \\sim iid. \\mathcal{N}(0,\\sigma_{\\zeta}^2), \\qquad t \\in [1,T].\n",
  99. "$$\n"
  100. ]
  101. },
  102. {
  103. "cell_type": "markdown",
  104. "metadata": {
  105. "collapsed": true
  106. },
  107. "source": [
  108. "<h3>Write the $ARMA\\,(p,q)$ model system in State Space Form</h3>\n",
  109. "\n",
  110. "<p>To write the $ARMA\\,(p,q)$ model system in a state space form, first we define the $system\\,matrices$ that contains hyperparameters:\n",
  111. "\n",
  112. "$$\n",
  113. "\\begin{align}\n",
  114. "Z_t &= [1,0,0,\\dotsb \\dotsb,0],\\\\\n",
  115. "\\,\\\\\n",
  116. "T_t &= T = \n",
  117. " \\begin{bmatrix}\n",
  118. " \\phi_1 & 1 & \\, & 0 \\\\\n",
  119. " \\vdots & \\, & \\ddots & \\,\\\\\n",
  120. " \\phi_{r-1} & 0 & \\, & 1 \\\\\n",
  121. " \\phi_r & 0 & \\dotsb & 0\n",
  122. " \\end{bmatrix}, \\\\\n",
  123. "\\,\\\\\n",
  124. "R_t &= R = \\left[\n",
  125. " \\begin{array}{c}\n",
  126. " 1\\\\ \\theta_1\\\\ \\vdots\\\\ \\theta_{r-1}\n",
  127. " \\end{array}\n",
  128. " \\right]\n",
  129. "\\end{align}\n",
  130. "$$\n",
  131. "\n",
  132. "<p>The state vector in transition equation is defined as:\n",
  133. "\n",
  134. "$$\n",
  135. "\\alpha_t = \\left(\n",
  136. " \\begin{array}{c}\n",
  137. " y_t\\\\ \n",
  138. " \\phi_2 y_{t-1} + \\dotsb + \\phi_r y_{t-r+1} + \\theta_1 \\zeta_t + \\dotsb + \\theta_{r-1} \\zeta_{t-r+2}\\\\\n",
  139. " \\phi_3 y_{t-1} + \\dotsb + \\phi_r y_{t-r+2} + \\theta_2 \\zeta_t + \\dotsb + \\theta_{r-1} \\zeta_{t-r+3}\\\\\n",
  140. " \\vdots\\\\ \n",
  141. " \\phi_r y_{t-1} + \\theta_{r-1} \\zeta_t\n",
  142. " \\end{array}\n",
  143. " \\right)\n",
  144. "$$\n",
  145. "\n",
  146. "<p> With the state vector defined, the $transition\\,equation$ can be easily written in a level equation by defining the error term $\\zeta_{t+1} = \\eta_t$ ($\\Rightarrow \\sigma_{\\eta}^2 = \\sigma_{\\zeta}^2$). The state space form of $ARMA\\,(p,q)$ is\n",
  147. "\n",
  148. "\n",
  149. "$$\n",
  150. "\\begin{align}\n",
  151. "&y_t = Z_t \\alpha_t,\\\\\n",
  152. "&\\alpha_{t+1} =T \\alpha_t + R \\eta_{t}, \\qquad \\eta_t \\sim iid.\\mathcal{N}(0,\\sigma_{\\eta}^2), \\qquad t\\in [1,T]\n",
  153. "\\end{align}\n",
  154. "$$"
  155. ]
  156. },
  157. {
  158. "cell_type": "markdown",
  159. "metadata": {},
  160. "source": [
  161. "<h3>Write the $ARIMA\\,(p,d,q)$ model system in State Space Form</h3>\n",
  162. "\n",
  163. "<p> The generalization process from $ARMA\\,(p,q)$ system to $ARIMA\\,(p,d,q)$ system is not as intuitive as replacing the sequence $\\left\\{y_t\\right\\}_{t\\in T}$ by the sequence $\\left\\{y_t^*\\right\\}_{t\\in T}$. It is even not easy to write in a compact representation. However, the $SSF$ of $ARIMA\\,(p,d,q)$ does have a rule can be observed, and it can be easily concluded by illustrating the $d=1$ and $d=2$ cases.\n",
  164. "\n",
  165. "<p>For the $system\\,matrix$ $Z_t$:\n",
  166. "$$\n",
  167. "\\begin{align}\n",
  168. "&Z_t = [1,1,0,0,\\dotsb,0] \\qquad d=1\\\\\n",
  169. "&\\,\\\\\n",
  170. "&Z_t = [1,1,1,0,\\dotsb,0] \\qquad d=2\\\\\n",
  171. "&\\vdots\n",
  172. "\\end{align}\n",
  173. "$$\n",
  174. "\n",
  175. "<p>For the $system\\,matrix$ $T_t$:\n",
  176. "$$\n",
  177. "\\begin{align}\n",
  178. "T_t &= T = \n",
  179. " \\begin{bmatrix}\n",
  180. " 1 & 1 & 0 & 0 & 0\\\\\n",
  181. " 0 & \\phi_1 & 1 & \\, & 0 \\\\\n",
  182. " 0 & \\vdots & \\, & \\ddots & \\,\\\\\n",
  183. " 0 & \\phi_{r-1} & 0 & \\, & 1 \\\\\n",
  184. " 0 & \\phi_r & 0 & \\dotsb & 0\n",
  185. " \\end{bmatrix} \\qquad &d=1 \\\\\n",
  186. "\\,\\\\\n",
  187. "T_t &= T = \n",
  188. " \\begin{bmatrix}\n",
  189. " 1 & 1 & 1 & 0 & 0 & 0 \\\\\n",
  190. " 0 & 1 & 1 & 0 & 0 & 0\\\\\n",
  191. " 0 & 0 & \\phi_1 & 1 & \\, & 0 \\\\\n",
  192. " 0 & 0 & \\vdots & \\, & \\ddots & \\,\\\\\n",
  193. " 0 & 0 & \\phi_{r-1} & 0 & \\, & 1 \\\\\n",
  194. " 0 & 0 & \\phi_r & 0 & \\dotsb & 0\n",
  195. " \\end{bmatrix} \\qquad &d=2\n",
  196. "\\end{align}\n",
  197. "$$\n",
  198. "\n",
  199. "<p>For the $system\\,matrix$ $R_t$:\n",
  200. "$$\n",
  201. "\\begin{align}\n",
  202. "R_t = R &= [0, 1, \\theta_1, \\dotsb, \\theta_{r-1}]^T \\qquad &d=1 \\\\\n",
  203. "\\,\\\\\n",
  204. "R_t = R &= [0, 0, 1, \\theta_1, \\dotsb, \\theta_{r-1}]^T \\qquad &d=2\n",
  205. "\\end{align}\n",
  206. "$$\n",
  207. "\n",
  208. "<p>For the $state\\,vector$ $\\alpha_t$ in the $transition\\,equation$:\n",
  209. "$$\n",
  210. "\\begin{align}\n",
  211. "\\alpha_t &= \\left(\n",
  212. " \\begin{array}{c}\n",
  213. " y_{t-1}\\\\ \n",
  214. " y_t^*\\\\ \n",
  215. " \\phi_2 y_{t-1}^* + \\dotsb + \\phi_r y_{t-r+1}^* + \\theta_1 \\zeta_t + \\dotsb + \\theta_{r-1} \\zeta_{t-r+2}\\\\\n",
  216. " \\phi_3 y_{t-1}^* + \\dotsb + \\phi_r y_{t-r+2}^* + \\theta_2 \\zeta_t + \\dotsb + \\theta_{r-1} \\zeta_{t-r+3}\\\\\n",
  217. " \\vdots\\\\ \n",
  218. " \\phi_r y_{t-1}^* + \\theta_{r-1} \\zeta_t\n",
  219. " \\end{array}\n",
  220. " \\right) \\qquad &d=1\\\\\n",
  221. "\\,\\\\\n",
  222. "\\alpha_t &= \\left(\n",
  223. " \\begin{array}{c}\n",
  224. " y_{t-1}\\\\\n",
  225. " \\Delta y_{t-1}\\\\\n",
  226. " y_t^*\\\\ \n",
  227. " \\phi_2 y_{t-1}^* + \\dotsb + \\phi_r y_{t-r+1}^* + \\theta_1 \\zeta_t + \\dotsb + \\theta_{r-1} \\zeta_{t-r+2}\\\\\n",
  228. " \\phi_3 y_{t-1}^* + \\dotsb + \\phi_r y_{t-r+2}^* + \\theta_2 \\zeta_t + \\dotsb + \\theta_{r-1} \\zeta_{t-r+3}\\\\\n",
  229. " \\vdots\\\\ \n",
  230. " \\phi_r y_{t-1}^* + \\theta_{r-1} \\zeta_t\n",
  231. " \\end{array}\n",
  232. " \\right) \\qquad &d=2\n",
  233. "\\end{align}\n",
  234. "$$\n",
  235. "\n",
  236. "<p>The error term $\\zeta_{t+1} = \\eta_t$ stays the same. The $SSF$ have no change from the Non-differencing case. However, it should be emphasized the sequences $\\left\\{y_t^* | \\,d_1\\right\\}_{t\\in T}$ and the sequence $\\left\\{y_t^* | \\,d_2\\right\\}_{t\\in T}$ are totally two different sequences, remember $y_t^* = \\Delta^d y_t$."
  237. ]
  238. },
  239. {
  240. "cell_type": "markdown",
  241. "metadata": {},
  242. "source": [
  243. "<h3>Write the $ARIMAX\\,(p,d,q)$ model system in State Space Form</h3>\n",
  244. "\n",
  245. "<p>The State Space Form for the $ARIMAX\\,(p,d,q)$ is established on the existing model identification:\n",
  246. "\n",
  247. "$$\n",
  248. "y_{t}^* = \\sum_{j=1}^{r}\\phi_j y_{t-j}^* +\\sum_{j=1}^k \\beta_j x_{jt}+ \\zeta_t +\\sum_{j=1}^{r-1}\\theta_j \\zeta_{t-j}, \\qquad \\zeta_t \\sim iid. \\mathcal{N}(0,\\sigma_{\\zeta}^2), \\qquad t \\in [1,T].\n",
  249. "$$\n",
  250. "\n",
  251. "and the existing $ARIMA\\,(p,d,q)$ $SSF$:\n",
  252. "\n",
  253. "\n",
  254. "$$\n",
  255. "\\begin{align}\n",
  256. "&y_t = Z_t \\alpha_t,\\\\\n",
  257. "&\\alpha_{t+1} =T \\alpha_t + R \\eta_{t}, \\qquad \\eta_t \\sim iid.\\mathcal{N}(0,\\sigma_{\\eta}^2), \\qquad t\\in [1,T]\n",
  258. "\\end{align}\n",
  259. "$$\n",
  260. "\n",
  261. "<p>If we write the exogenous variable term into vector form, we have $\\sum_{j-1}^k \\beta_j x_{jt} = X_t \\beta$, where $\\beta$ is a $k \\times 1$ vector, and $X_t$ is a $1 \\times k$ vector. Now redefine the new $state\\,vector$ and $system\\,matrices$ for the state space form:\n",
  262. "$$\\,$$\n",
  263. "$$\n",
  264. "\\alpha_t^*= \\begin{bmatrix} \\beta \\\\ \\alpha_t \\end{bmatrix}; \\qquad \\qquad\n",
  265. "T^* = \\begin{bmatrix} I_k & 0 \\\\ 0 & T \\end{bmatrix}, \\quad\n",
  266. "R^* = \\begin{bmatrix} 0 \\\\ R \\end{bmatrix}, \\quad\n",
  267. "Z_t^* = [X_t,Z]\n",
  268. "$$\n",
  269. "\n",
  270. "<p>The $SSF$ of $ARIMAX\\,(p,d,q)$ model system is finally obtained:\n",
  271. "\n",
  272. "$$\n",
  273. "\\begin{align}\n",
  274. "&y_t = Z_t^* \\alpha_t^*,\\\\\n",
  275. "&\\alpha_{t+1}^* =T^* \\alpha_t^* + R^* \\eta_{t}, \\qquad \\eta_t \\sim iid.\\mathcal{N}(0,\\sigma_{\\eta}^2), \\qquad t\\in [1,T]\n",
  276. "\\end{align}\n",
  277. "$$\n"
  278. ]
  279. },
  280. {
  281. "cell_type": "code",
  282. "execution_count": null,
  283. "metadata": {
  284. "collapsed": true
  285. },
  286. "outputs": [],
  287. "source": []
  288. }
  289. ],
  290. "metadata": {
  291. "anaconda-cloud": {},
  292. "kernelspec": {
  293. "display_name": "Python [Root]",
  294. "language": "python",
  295. "name": "Python [Root]"
  296. },
  297. "language_info": {
  298. "codemirror_mode": {
  299. "name": "ipython",
  300. "version": 3
  301. },
  302. "file_extension": ".py",
  303. "mimetype": "text/x-python",
  304. "name": "python",
  305. "nbconvert_exporter": "python",
  306. "pygments_lexer": "ipython3",
  307. "version": "3.5.2"
  308. }
  309. },
  310. "nbformat": 4,
  311. "nbformat_minor": 0
  312. }
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!