hctsa is a software package for running highly comparative time-series analysis using Matlab (full support for versions R2014b or later; for use in python cf. pyopy).
The software provides a code framework that allows thousands of time-series analysis features to be extracted from time series (or a time-series dataset), as well as tools for normalizing and clustering the data, producing low-dimensional representations of the data, identifying discriminating features between different classes of time series, learning multivariate classification models using large sets of time-series features, finding nearest matches to a time series of interest, and a range of other visualizations and analyses.
If you use this software, please read and cite these (open access) articles:
- B.D. Fulcher and N.S. Jones. hctsa: A computational framework for automated time-series phenotyping using massive feature extraction. Cell Systems 5, 527 (2017).
- B.D. Fulcher, M.A. Little, N.S. Jones Highly comparative time-series analysis: the empirical structure of time series and their methods. J. Roy. Soc. Interface 10, 83 (2013).
Any feedback, as email, github issues or pull requests, is much appreciated.
📖 📖 Comprehensive documentation 📖 📖 for hctsa is on gitbook, which can be read online or downloaded as a pdf, epub, or mobi file.
For users unfamiliar with git, the current version of the repository can be downloaded by simply clicking the green Download .zip button.
It is recommended to use the repository with git. For this, please make a fork of it, clone it to your local machine, and then set an upstream remote to keep it synchronized with the main repository e.g., using the following code:
git remote add upstream git://github.com/benfulcher/hctsa.git
(make sure that you have generated an ssh key and associated it with your github account).
You can then update to the latest stable version of the repository by pulling the master branch to your local repository:
git pull upstream master
For analyzing specific datasets, we recommend working outside of the repository so that incremental updates can be pulled from the upstream repository. Details on how to merge the latest version of the repository with the local changes in your fork can be found here.
There are two licenses applied to the core parts of the repository:
-
Sections of the repository required to compute features from time-series data is licensed as GNU General Public License version 3.
-
Sections implementing the framework for running hctsa analyses and visualizations is licensed as the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
To use hctsa for commercial applications, please contact Ben Fulcher.
A range of external code packages are provided in the Toolboxes directory of the repository, and each have their own associated license (as outlined below).
The following Matlab toolboxes are used by hctsa and are required for full functionality of the software. In the case that some toolboxes are unavailable, the hctsa software can still be used, but only a reduced set of time-series features will be computed.
- Statistics Toolbox
- Signal Processing Toolbox
- Curve Fitting Toolbox
- System Identification Toolbox
- Wavelet Toolbox
- Econometrics Toolbox
The following time-series analysis packages are provided with the software (in the Toolboxes directory), and are used by our main feature extraction algorithms to compute meaningful structural features from time series:
- TISEAN package for nonlinear time-series analysis, version 3.0.1 (GPL license).
- TSTOOL package for nonlinear time-series analysis, version 1.2 (GPL license).
- Joseph T. Lizier's Java Information Dynamics Toolkit (JIDT) for studying information-theoretic measures of computation in complex systems, version 1.3 (GPL license).
- Time-series analysis code developed by Michael Small (unlicensed).
- Max Little's Time-series analysis code (GPL license).
- Sample Entropy code from Physionet (GPL license).
- ARFIT Toolbox for AR model estimation (unlicensed).
- gpml Toolbox for Gaussian Process regression model estimation, version 3.5 (FreeBSD license).
- Danilo P. Mandic's delay vector variance code (GPL license).
- Cross Recurrence Plot Toolbox (GPL license)
- Zoubin Ghahramani's Hidden Markov Model (HMM) code (MIT license).
- Danny Kaplan's Code for embedding statistics (GPL license).
- Two-dimensional histogram code from Matlab Central (BSD license).
- Various histogram and entropy code by Rudy Moddemeijer (unlicensed).
See the following publications for examples of hctsa use:
- Implementation paper introducing the hctsa package, with applications to high throughput phenotyping of C. Elegans and Drosophila movement time series 📗 : B. D. Fulcher & N. S. Jones. hctsa: A Computational Framework for Automated Time-Series Phenotyping Using Massive Feature Extraction. Cell Systems 5, 527 (2017). Link.
- Introduction to feature-based time-series analysis 📗 : B. D. Fulcher. Feature-based time-series analysis. arXiv 1709.08055 (2017) Link.
- Application to fMRI data 📗 : S. S. Sethi, V. Zerbi, N. Wenderoth, A. Fornito, B. D. Fulcher. Structural connectome topology relates to regional BOLD signal dynamics in the mouse brain. Chaos 27, 047405 (2017). Link, preprint.
- Application to time-series data mining 📗 : B. D. Fulcher & N. S. Jones. Highly comparative feature-based time-series classification. IEEE Trans. Knowl. Data Eng. 26, 3026 (2014). Link.
- Application to fetal heart rate time series 📗 : B. D. Fulcher, A. E. Georgieva, C. W. G. Redman, N. S. Jones. Highly comparative fetal heart rate analysis. 34th Ann. Int. Conf. IEEE EMBC 3135 (2012). Link.
- Original paper, showing that the behavior of thousands of time-series methods on thousands of different time series can provide structure to the interdisciplinary time-series analysis literature 📗 : B. D. Fulcher, M. A. Little, N. S. Jones. Highly comparative time-series analysis: the empirical structure of time series and their methods. J. Roy. Soc. Interface 10, 20130048 (2013). Link.
Many thanks go to Romesh Abeysuriya for helping with the mySQL database set-up and install scripts, and Santi Villalba for lots of helpful feedback and advice on the software.
There are a range of open datasets with pre-computed hctsa features. (If you have data to share and host, let me know and I'll add it to this list):
- 1000 empirical time series
- C. elegans movement speed data and associated analysis code.
- Drosophila movement speed and associated analysis code.
An accompanying web resource for this project is Comp-Engine Time Series, which allows users to compare thousands of diverse types of time-series analysis code and time-series data. This website also allows you to download large volumes of univariate time-series data. Note that the code files on Comp-Engine Time Series are based on an early implementation and rarely match with the updated features and functions contained in this repository.
This excellent repository allows users to run hctsa software from within python: pyopy.
Native python time-series code to extract hundreds of time-series features, with in-built feature filtering, is tsfresh.
These R packages are by Rob Hyndman.
The first, tscompdata, makes available existing collections of time-series data for analysis.
The second, tsfeatures, includes implementations of a range of time-series features.
A python-based nonlinear time-series analysis and complex systems code package, pyunicorn.