## FORECASTING : dataset of a single time series assigned in a miniature competition between the algoriths.

# FORECASTING : dataset of a single time series assigned in a miniature competition between the algoriths.

## Coursework Information & Submission

This is an individual assignment weighted 100%. Coursework deadline is April 28th 2020, 10:00am. Standard

departmental penalties will apply for late work unless you have been given an extension for exceptional reasons from

the course administrator. All submissions will be checked by the plagiarism software. Coursework must be submitted

online on Moodle. Submit your report PLUS R scripts in the appendix through in Moodle.

## Task: Forecast 1 Real-Word Time Series – 100%

You are required to forecast a dataset of a single time series individually assigned to you in a miniature competition

between the algorithms. Your objective is to:

(a) Develop the most accurate statistical forecasting models, demonstrating your modelling skills!

• Obtain your time series from the respective competition dataset

o Download the datasets from the Moodle website

o Select the 1 time series allocated to you from the competition file

• Construct forecasts, and for each of the time series

o conduct a thorough exploratory data analysis (using graphs, statistical test, etc. and a verbal analysis)

o build multiple potentially suitable forecasting models, including a suitable Exponential Smoothing,

ARIMA, and Time Series Regression model to predict the 14 next values (2 weeks ahead)

o Choose what you assume to be the “best” model from these models for a final submission, by

assessing errors and comparing errors against a Naive and a Seasonal Naive benchmark models.

(b) Document your forecasting process in a technical research report

• Write a technical report to document your model building skills & to justify your choice of model

• Critically discuss some findings and choices

Marking Scheme & Hints

20% of points – Data Exploration

– Explore the regular components and the irregular components of the time series using graphs, statistical

summaries and statistical tests.

– NOTE:

o Important tests relate to stationarity (e.g. the ADF test) and patterns (e.g. seasonality, trend, ACF &

PACF analysis) of the time series to support your visual analysis

o Document your findings comprehensively, making adequate use of (readable and correctly labelled)

graphs which you also discuss verbally to support your arguments.

o Excessive evidence (e.g. the complete information from statistical tests) may be placed in the

appendix, but must be referenced directly at the corresponding place in the main the text, else it does

not count.

o Conclude by recommending one (or multiple) suitable model forms for forecasting.

60% of points – Forecast Model Building

– Build multiple potential contender models (each one suitable for the identified data properties), leading to

o 1 good manually built Exponential Smoothing Model – in comparison to 1 automatically built

o 1 good manually built ARIMA model – in comparison to 1 automatically built

o 1 good manually built Time Series Regression model – in comparison to 1 automatically built

– Document the specification of each model

o Document the specification process of model building, i.e. the iterative steps to build this final model,

including the analysis of residuals of intermediate models that have led to better ones

o Document the final model form and parameters to allow a complete replication of your experiments

(i.e. specify what model forms or lags you selected, which transformations in which order & which

parameters were used so that others could replicate what you did exactly only from your document).

– NOTE:

o For each contender model, one or more different model forms may be feasible to produce forecasts

(i.e. for a seasonal time series with slight trend in Exponential Smoothing you may use multiplicative

or additive seasonality, with or without trend, so 4 candidates that could perform well; for ARIMA you

may consider models with seasonal differences or first differences or both; for regression models you

may consider to capture seasonality as dummy variables, or as a seasonal autoregressive lag etc.).

Where applicable, you should always consider multiple plausible candidate models, and must justify

your choice of candidates in comparison to other potential models in each class of models (feel free to

explicitly rule out implausible ones). To get high marks it will not be sufficient to build a single

Exponential Smoothing model using an auto specification ZZZ or a single automatic ARIMA model

and a single Regression model with stepwise, but rather require the development of a subset of

potentially useful models which you compare and accept/reject. Base your justification on evidence

and document your ITERATIVE modelling process throughout.

o We recommend using R, but you are free to use any external software but report the software used.

10% of points – Discussion of expected Accuracy (Errors)

– Determine a suitable error metric and compute the expected in-sample and out-of sample errors of your

recommended models and compare them. Comment on the suitability of the chosen metric for this task.

– Select one “best” model to be used for forecasting each of the time series across all algorithm families, and

provide your final out of sample forecasts.

– NOTE:

o To assess the future forecast accuracy of your models before the final future values become

available, consider to create a hold-out dataset of equal or better larger size to the forecasting horizon

and assess both in-sample and (quasi) out-of-sample errors

o Different error metrics are feasible. You should use at least two suitable error metrics for the

assessment, and justify the use of each error metrics you are using

o To show improvements in accuracy of your methods to some objective benchmark, you should

compare all to the Naive Level and a Naive Seasonal Model. Use tables to provide a suitable

overview of different methods’ accuracy and their uplfit of accuracy on the 2 Naïve models.

o Comment on the accuracy and suitability of each of the methods for each of the time series. Which

method would you recommend to use for each of the time series? Comment on why you think some

methods perform better than others? Comment on the available data and number of origins for

forecast evaluations on the withheld test set and given your fixed forecasting horizon.

10% of points – General report writing skills

– General report writing skills such as critical discussion of findings, thoroughness of documentation, clarity of

arguments, structure of the report, readability of the report (i.e. lack of spelling and grammatical mistakes etc.)

will also be considered in marking each section. Please see next page for some more technical considerations

on report writing.

General suggestions on writing a report for Task II

The coursework requires you to document your analysis and critically discuss your chosen experimental design,

modelling approaches and the results in a technical report. This technical report should be written as if tailored to an

OR specialist (e.g. who has an MSc in Management Science from Lancaster University and has taken the MSCI 523

course, and who wants to evaluate your results AND your decision making process to determine your skills in

modelling and whether you have missed anything). This means that you are not required to write a general description

(i.e. Exponential Smoothing is … ) as an OR expert would be aware of this! Consequently the report should document

the process of modelling, and allow an understanding of your choices and a replication of your experiments.

The report should contain an introduction and a summary with conclusions on your findings, numbered headings, list

of figures and tables and an executive summary (tailored to senior management) indicating the most relevant findings.

The report should display a logical and concise structure, be generally “readable” and support your argument using

plots of time series, forecasts and /or accuracy. Make adequate use of graphs to show time series, model fit /

predictions and residuals to support your arguments (for this graphs must be completely readable and with labels), as

well as tables to compare results.

The page limit for the report is 18 pages (note this is a maximum to make your life easier – you can produce shorter

reports! pages count only for main text incl. graphs and tables, but not for the cover sheet, executive summary,

contents sheet or appendices). Reports of excessive length will be penalised by deducting 10 marks (i.e. 10% of 100)

but only if they are including un-necessary material. For formatting, use single spacing, format normal text in times

new roman font size 12, text in tables, figure and table headings in font size 10, and leave 2cm of margin left and right.

Include any technical details and hardcopies that support your arguments in a set of appendices (i.e. the printouts

from ADF tests in the appendix, with only the conclusion of significance / insignificance at a probability in the main

text), which will not count towards the page limit. You must ensure the main text is readable and that your argument is

coherent without needing to consult the appendices. All parts of the text supported by an appendix must crossreference directly to the relevant part.

Non-disclosure clause: these datasets and the coursework task is subject to copyright © by Sven Crone, all rights reserved. In

downloading the documents and submitting the assignment for assessment the copyright agreement is deemed accepted. Any

publication of the dataset, the coursework task, or its solution (e.g. on a coursework website or a social network site), or a part thereof,

will be considered a violation of copyright. The person breaking the copyright may be held liable for damages by international law suit.

Furthermore, the publication will count the assignment as a plagiarism – even in retrospect after receiving the MSc degree – leading to a

mark of zero, with the usual right to appeal to university court in official hearing,