Time Series Analysis

Author

Yair Mau

about

Welcome to Time Series Analysis for Environmental Sciences (71606) at the Hebrew University of Jerusalem. This is Yair Mau, your host for today. I am a senior lecturer at the Institute of Environmental Sciences, at the Faculty of Agriculture, Food and Environment, in Rehovot, Israel.

This website contains (almost) all the material you’ll need for the course. If you find any mistakes, or have any comments, please email me.

disclaimer

The material here is not comprehensive and does not constitute a stand alone course in Time Series Analysis. This is only the support material for the actual presential course I give.

what, who, when and where?

Course number 71606, 3 academic points
Yair Mau (lecturer), Erez Feuer (TA)
Tuesdays, from 14:15 to 17:00
Computer classroom #18
Office hours: upon request

syllabus

course description

Data analysis of time series, with practical examples from environmental sciences.

course aims

This course aims at giving the students a broad overview of the main steps involved in the analysis of time series: data management, data wrangling, visualization, analysis, and forecast. The course will provide a hands-on approach, where students will actively engage with real-life datasets from the field of environmental science.

learning outcomes

On successful completion of this module,students should be able to:

  • Explore a time-series dataset, while formulating interesting questions.
  • Choose the appropriate tools to attack the problem and answer the questions.
  • Communicate their findings and the methods they used to achieve them, using graphs, statistics, text, and a well-documented code.

course content

  • Data wrangling: organization, cleaning, merging, filling gaps, excluding outliers, smoothing, resampling.
  • Visualization: best practices for graph making using leading python libraries.
  • Analysis: stationarity, seasonality, (auto)correlations, lags, derivatives, spectral analysis.
  • Forecast: ARIMA
  • Data management: how to plan ahead and best organize large quantities of data. If there is enough time, we will build a simple time-series database.

books and other sources

Click here.

course evaluation

There will be assignments during the semester (totaling 50% of the final grade), and one final project (50%).

Evaluation policy

  • Individual Work: While we support helping your peers, it’s important to remember that all assignments must be completed individually. This means that your submissions should be your own unique work and not contain code or text that is identical to someone else’s.

  • Zero Plagiarism: Do not copy text verbatim from any source. Always express ideas in your own words.

  • On-Time Submission: Assignments must be turned in by the specified deadline. Late submissions will receive a grade of 0. If you require an extension, requests will only be considered if made at least 24 hours before the due date.

  • Non-Compliance Consequence: Assignments that do not adhere to these guidelines will automatically receive a grade of 0.

weekly program

This year’s course will be a bit different that planned due to the shortening of the academic semester. The information below is NOT up to date. Ask Yair what is relevant this year.

week 1

  • Lecture: Course overview, setting of expectations. Introduction, basic concepts, continuous vs discrete time series, sampling, aliasing
  • Exercise: Loading csv file into python, basic time series manipulation with pandas and plotting

week 2

  • Lecture: Filling gaps, removing outliers
  • Exercise: Practice the same topics learned during the lecture. Data: air temperature and relative humidity

week 3

  • Lecture: Interpolation, resampling, binning statistics
  • Exercise: Practice the same topics learned during the lecture. Data: air temperature and relative humidity, precipitation

week 4

  • Lecture: Time series plotting: best practices. Dos and don’ts and maybes
  • Exercise: Practice with Seaborn, Plotly, Pandas, Matplotlib
Project 1
Basic data wrangling, using real data (temperature, relative humidity, precipitation) downloaded from USGS. 25% of the final grade

week 5

  • Lecture: Smoothing, running averages, convolution
  • Exercise: Practice the same topics learned during the lecture. Data: sap flow, evapotranspiration

week 6

  • Lecture: Strong and weak stationarity, stochastic processes, auto-correlation
  • Exercise: Practice the same topics learned during the lecture. Data: temperature and wind speed

week 7

  • Lecture: Correlation between signals. Pearson correlation, time-lagged cross-correlations, dynamic time warping
  • Exercise: Practice the same topics learned during the lecture. Data: temperature, solar radiation, relative humidity, soil moisture, evapotranspiration

week 8

Same as lecture 7 above

week 9

  • Lecture: Download data from repositories, using API, merging, documentation
  • Exercise: Download data from USGS, NOAA, Fluxnet, Israel Meteorological Service
Project 2
Students will study a Fluxnet site of their choosing. How do gas fluxes (CO2, H2O) depend on environmental conditions? 25% of the final grade

week 10

  • Lecture: Fourier decomposition, filtering, Nyquist–Shannon sampling theorem
  • Exercise: Practice the same topics learned during the lecture. Data: dendrometer data

week 11

  • Lecture: Seasonality, seasonal decomposition (trend, seasonal, residue), Hilbert transform
  • Exercise: Practice the same topics learned during the lecture. Data: monthly atmospheric CO2 concentration, hourly air temperature

week 12

  • Lecture: Derivatives, differencing
  • Exercise: Practice the same topics learned during the lecture. Data: dendrometer data

week 13

  • Lecture: Forecasting. ARIMA
  • Exercise: Practice the same topics learned during the lecture. Data: vegetation variables (sap flow, ET, DBH, etc)

Final Project
In consultation with the lecturer, students will ask a specific scientific question about a site of their choosing (from NOAA, USGS, Fluxnet), and answer it using the tools learned during the semester. The report will be written in Jupyter Notebook, combining in one document all the calculations, documentation, figures, analysis, and discussion. 50% of the final grade.