Structural changes#

What is structural change?#

Applications#

  • Data Monitoring

  • Mining Natural experiments as suggested in [He et al., 2022]

  • Feature engineering in ML, or retraining model

Example: Detecting trend changes in time series#

Models#

Model 1 (Structural change in both intercept and slope)#

Design Matrix#
\[\begin{split}X = \begin{bmatrix} 1 & 1 & 0 & 0 \\ 1 & 2 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots \\ 1 & n & 0 & 0 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 1 & 2 \\ \vdots & \vdots & \vdots & \vdots \\ 0 & 0 & 1 & m \\ \end{bmatrix}\end{split}\]

Model 2 (Structural change in slope)#

Model 3 (Structural change in intercept)#

Hypothesis testing#

Wald test#

Robust inference#

Example with model 1#

We are going

\[ \begin{align}\begin{aligned}\begin{split}X^{t}X = \begin{bmatrix} A(n) & 0_{2,2} \\ 0_{2,2} & A(m) \end{bmatrix}\end{split}\\\text{where}\\\begin{split}A_{n} =& \begin{bmatrix} \sum_{i=1}^{n} 1 & \sum_{i=1}^{n} i \\ \sum_{i=1}^{n} i & \sum_{i=1}^{n} i^{2} \end{bmatrix} =& \begin{bmatrix} n & \frac{n(n+1)}{2} \\ \frac{n(n+1)}{2} & \frac{n(n+1)(2n+1)}{6} \end{bmatrix}\end{split}\end{aligned}\end{align} \]

Since:

  1. \(\sum_{i=1}^{n} i = \frac{n(n+1)}{2}\)

  2. \(\sum_{i=1}^{n} i^{2} = \frac{n(n+1)(2n+1)}{6}\)

  3. \(n (\sum_{i=1}^{n} i^{2}) - (\sum_{i=1}^{n} i)^{2} = \frac{n^{2}(n+1)(n-1)}{12} = \sum_{i=1}^{n} (i -\frac{n+1}{2})^{2}\)

We get:

\[\begin{split}A(n)^{-1} =& \begin{bmatrix} \sum_{i=1}^{n} i^{2} & - \sum_{i=1}^{n} i \\ - \sum_{i=1}^{n} i & \sum_{i=1}^{n} 1 \end{bmatrix} \times \frac{1}{\sum_{i=1}^{n} (i -\frac{n+1}{2})^{2}} =& \begin{bmatrix} \frac{4n + 2}{n(n-1)} & - \frac{6}{n(n-1)} \\ - \frac{6}{n(n-1)} & - \frac{12}{n(n+1)(n-1)} 1 \end{bmatrix}\end{split}\]

Please note that

Proof

You can make up your own admonition too.

References#

AM07

Ryan Prescott Adams and David JC MacKay. Bayesian online changepoint detection. arXiv preprint arXiv:0710.3742, 2007.

Gun22

Gregory Gundersen. Bayesian online changepoint detection. 2022. URL: https://gregorygundersen.com/blog/2019/08/13/bocd/.

HBL22

Yuzi He, Keith A Burghardt, and Kristina Lerman. Leveraging change point detection to discover natural experiments in data. EPJ Data Science, 11(1):49, 2022.

KC15

Taehoon Kim and Jaesik Choi. Reading documents for bayesian online change point detection. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1610–1619. 2015.

Mur12

Kevin P Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.

PY09

Pierre Perron and Tomoyoshi Yabu. Testing for shifts in trend with an integrated or stationary noise component. Journal of Business & Economic Statistics, 27(3):369–396, 2009.

ZLSZ19

Bin Zuo, Jianping Li, Cheng Sun, and Xin Zhou. A new statistical method for detecting trend turning. Theoretical and Applied Climatology, 138(1):201–213, 2019.