Seminar: Learning from non-stationary distributions

I attended this seminar presentation by Mr. Geoff Webb from Faculty of IT at Monash University in April 2016. The seminar focussed on the theoretical tools for analysing non-stationary distributions and discusses insights that they provide. The underlying idea of the presentation was that, “The world is dynamic – in a constant state of flux – but most learned models are static. Models learned from historical data are likely to decline in accuracy over time.”

Date: Wednesday 13 April, 2016
Time: 2:00pm – 3:00pm
Venue: Clayton CL_26_G12A, VC to 7.84 Caulfield

My PhD project deals with non-stationary data from learning management system and the notion of “concept-drift” is highly prevalent in these environments. The seminar highlighted that models getting out of sync and/or out of date is a serious problem in real world. Google’s Flu Trend prediction system was one such famous project that had to succumb to the reality of concept drift.

Important Dimensions of Concept Drift

  1. Re-learning (preferred in practice) and/or Incremental Learning (most researched)
  2. Window / Ageing – notion of decay (used in practice)
  3. Detention and Fixed Schedule (when to update the model when drift is detected)

Summarising the presentation (as per the presenters conclusion):

  • Concept drift is a critical problem
  • Much current work is ad-hoc
  • Quantitative models provide rigorous theoretical basis for
    • mechanisms to detect, characterise and resolve drift.
    • understand best forms of drift best handled by each mechanism
    • develop synthetic drift data generators.
    • evaluating stream mining algorithms and
    • designing incremental learning techniques that are robust to a diversity of situations
  • Need different windows in different attribute subspaces.
  • Need different bias-variance profiles for different windows.
  • Useful to exploit cycles
  • Useful to map drift
  • Do we need high-dimensional forecasting ???

Overall, the seminar was very enlightening and will contribute positively towards my development as a research student.