Unit 5 — Advanced Topics

Topics Covered

Syllabus topics: Introduction to Time Series Analysis in R (ARIMA basics) — concept of time series (trend, seasonality, noise), time series objects in R (ts, zoo, xts), plotting and decomposing time series, stationarity and differencing, autocorrelation and partial autocorrelation (ACF/PACF), AR, MA, ARIMA model basics, forecasting using the forecast package. Creating interactive visualizations with plotly — converting ggplot2 plots to interactive plots, animations and sliders. R Shiny: building interactive web applications — introduction to the Shiny framework, UI and server functions, reactive expressions and reactivity, input and output widgets, layouts and dashboard design.

This file is split into three independent parts. Study them separately.

Part A — Time Series Analysis

A.1 What makes a time series different

The four components

Seasonality vs cyclic is a favourite two-mark question: seasonality has a fixed, known period (12 months, 7 days); cycles do not.

Decomposition models

Plot the series first. If the peaks get taller as the series rises, it is multiplicative — and taking logs converts it to additive, since log(T × S × I) = log T + log S + log I.

data(AirPassengers)
plot(AirPassengers)                          # classic multiplicative example
decompose(AirPassengers, type = "multiplicative")
stl(log(AirPassengers), s.window = "periodic")   # more robust

A.2 Time series objects in R

sales <- ts(c(112, 118, 132, 129, 121, 135),
            start = c(2024, 1), frequency = 12)   # monthly from Jan 2024

start(sales); end(sales); frequency(sales)
window(sales, start = c(2024, 3), end = c(2024, 5))

frequency is observations per cycle, not per year: 12 = monthly, 4 = quarterly, 7 = daily-with-weekly-seasonality, 1 = annual. Getting this wrong silently disables all seasonal modelling — the commonest error in this part of the course.

A.3 Stationarity

Testing for it

library(tseries)
adf.test(series)      # Augmented Dickey-Fuller
kpss.test(series)     # KPSS

The two tests have opposite null hypotheses. Reading ADF's p-value as though it were KPSS's gives exactly the wrong conclusion, and it is a standard exam trap. Write the null down before interpreting either.

Differencing

diff(series)                    # first difference -- d = 1
diff(series, differences = 2)   # second difference -- d = 2
diff(series, lag = 12)          # seasonal difference -- D = 1
ndiffs(series); nsdiffs(series) # how many are needed

Rarely difference more than twice. Over-differencing adds artificial negative autocorrelation and makes the model worse, not better. If ndiffs() says 1, use 1.

A.4 ACF and PACF

acf(series); pacf(series)

Reading them to identify the model

The mnemonic: PACF for AR, ACF for MA. PACF gives p, ACF gives q. Both alphabetically inverted, which is oddly the easiest way to remember it.

A.5 AR, MA and ARIMA

SARIMA(p,d,q)(P,D,Q)[s] adds the seasonal counterparts, where s is the period.

Special cases worth naming

Fitting and forecasting

library(forecast)

fit <- auto.arima(series)          # searches (p,d,q) by AIC
summary(fit); checkresiduals(fit)

fc <- forecast(fit, h = 12)        # 12 periods ahead
plot(fc); accuracy(fit)

checkresiduals(fit)

Accuracy measures

Part B — Interactive Visualisation with plotly

B.1 From ggplot2 to interactive, in one line

library(plotly)

p <- ggplot(students, aes(x = hours, y = marks, colour = section)) +
       geom_point()

ggplotly(p)                # done -- hover, zoom, pan all work

ggplotly() is the whole trick for most purposes. Any ggplot2 plot from Unit 3 becomes interactive without rewriting it.

B.2 Native plotly

plot_ly(students, x = ~hours, y = ~marks, color = ~section,
        type = "scatter", mode = "markers",
        text = ~paste("Name:", name, "<br>Marks:", marks),
        hoverinfo = "text") %>%
  layout(title = "Marks against study hours",
         xaxis = list(title = "Hours"),
         yaxis = list(title = "Marks"))

Layers chain with %>%, not + — the reverse of ggplot2. Mixing the two is the most common plotly error.

B.3 Animations and sliders

plot_ly(gapminder, x = ~gdpPercap, y = ~lifeExp,
        size = ~pop, color = ~continent,
        frame = ~year,                       # <- this creates the animation
        type = "scatter", mode = "markers") %>%
  layout(xaxis = list(type = "log")) %>%
  animation_opts(frame = 1000, transition = 500, redraw = FALSE)

frame = ~year is all an animation needs — plotly adds the play button and slider automatically. This reproduces the famous Hans Rosling Gapminder chart in about six lines.

For a manual slider, use layout(sliders = list(...)) or rangeslider() for time series.

B.4 When interactivity helps and when it does not

Interactivity is not automatically better. A static chart with a clear title often communicates more than an interactive one the reader must explore.

Part C — R Shiny

C.1 What Shiny is

Shiny turns an R script into a web application — with no HTML, CSS or JavaScript required. You will meet those properly in Web Technologies; Shiny lets you skip them.

C.2 The two halves of every Shiny app

library(shiny)

ui <- fluidPage(
  titlePanel("Student Marks Explorer"),
  sidebarLayout(
    sidebarPanel(
      selectInput("section", "Choose a section:",
                  choices = c("A", "B", "C")),
      sliderInput("minmarks", "Minimum marks:",
                  min = 0, max = 100, value = 40)
    ),
    mainPanel(
      plotOutput("marksPlot"),
      tableOutput("summaryTable")
    )
  )
)

server <- function(input, output, session) {
  filtered <- reactive({
    students %>%
      filter(section == input$section, marks >= input$minmarks)
  })

  output$marksPlot <- renderPlot({
    ggplot(filtered(), aes(x = marks)) + geom_histogram(bins = 10)
  })

  output$summaryTable <- renderTable({
    filtered() %>% summarise(n = n(), avg = mean(marks))
  })
}

shinyApp(ui = ui, server = server)

C.3 Reactivity — the concept the whole framework rests on

The three reactive constructs

Two rules that cause most Shiny bugs:

1. Call a reactive with parentheses. filtered(), not filtered. Without them you are referring to the function, not its value.

2. input$x can only be read inside a reactive context — a reactive(), observe() or render*(). Reading it at the top level of server throws "operation not allowed without an active reactive context".

C.4 Widgets

Every output ID must match: plotOutput("marksPlot") in the UI pairs with output$marksPlot in the server. A typo produces a blank panel and no error message — check the spelling first when something does not appear.

C.5 Layout and deployment

fluidPage()  sidebarLayout()  fluidRow() / column(width = 6)
tabsetPanel(tabPanel("Plot", ...), tabPanel("Data", ...))
navbarPage()

shinydashboard and bs4Dash give proper dashboard chrome. Bootstrap's grid is 12 columns wide, so column(6) is half the page.

Deployment: shinyapps.io (free tier), Shiny Server, or Posit Connect.

📝 Practice problems

fit <- auto.arima(log(sales))
fc  <- forecast(fit, h = 12)
exp(fc$mean)        # back to the original units

d1 <- diff(series)
adf.test(d1)        # expect a small p-value now
acf(d1); pacf(d1)   # now read p and q from these
ndiffs(series)      # confirms how many differences are needed

server <- function(input, output) {
  output$plot <- renderPlot({
    d <- students %>% filter(section == input$section)   # filter #1
    ggplot(d, aes(marks)) + geom_histogram()
  })
  output$table <- renderTable({
    d <- students %>% filter(section == input$section)   # filter #2 -- duplicated
    summarise(d, n = n(), avg = mean(marks))
  })
}

server <- function(input, output) {
  filtered <- reactive({
    students %>% filter(section == input$section)
  })

  output$plot <- renderPlot({
    ggplot(filtered(), aes(marks)) + geom_histogram()    # note the ()
  })
  output$table <- renderTable({
    summarise(filtered(), n = n(), avg = mean(marks))
  })
}

Exam questions from this unit

Two marks

1. Distinguish seasonality from a cycle.
2. What does the d in ARIMA(p,d,q) represent?
3. State the null hypothesis of the ADF test.
4. Which plot gives p, and which gives q?
5. What are the two halves of a Shiny app?
6. Why must a reactive be called with parentheses?

Five marks

1. Explain the components of a time series and the two decomposition models.
2. Explain stationarity, why it matters, and how to achieve it.
3. Explain how ACF and PACF identify AR and MA orders.
4. Explain reactivity in Shiny with an example.
5. Explain ggplotly() and native plotly, with the differences.

Ten marks

1. Explain ARIMA modelling end to end — stationarity testing, differencing, order identification, fitting, residual checking and forecasting.

2. Build and explain a complete Shiny application with inputs, reactive expressions and multiple outputs.

Mistakes that cost marks