Search

Sequence modeling faces challenges in capturing long-range dependencies across diverse tasks. Recent linear and transformer-based forecasters have shown superior performance in time series forecasting. However, they are constrained by their inherent inability to effectively address long-range dependencies in time series data, primarily due to using fixed-size inputs for prediction. Furthermore, they typically sacrifice essential temporal correlation among consecutive training samples by shuffling them into mini-batches. To overcome these limitations, we introduce a fast and effective Spectral Attention mechanism, which preserves temporal correlations among samples and facilitates the handling of long-range information while maintaining the base model structure. Spectral Attention preserves long-period trends through a low-pass filter and facilitates gradient to flow between samples. Spectral Attention can be seamlessly integrated into most sequence models, allowing models with fixed-sized look-back windows to capture long-range dependencies over thousands of steps. Through extensive experiments on 11 real-world time series datasets using 7 recent forecasting models, we consistently demonstrate the efficacy of our Spectral Attention mechanism, achieving state-of-the-art results., Comment: Co-first Author: Bong Gyun Kang, Dongjun Lee. NeurIPS 2024 (Conference on Neural Information Processing Systems)

Time series forecasting is a critical task that provides key information for decision-making across various fields. Recently, various fundamental deep learning architectures such as MLPs, CNNs, RNNs, and GNNs have been developed and applied to solve time series forecasting problems. However, the structural limitations caused by the inductive biases of each deep learning architecture constrained their performance. Transformer models, which excel at handling long-term dependencies, have become significant architectural components for time series forecasting. However, recent research has shown that alternatives such as simple linear layers can outperform Transformers. These findings have opened up new possibilities for using diverse architectures. In this context of exploration into various models, the architectural modeling of time series forecasting has now entered a renaissance. This survey not only provides a historical context for time series forecasting but also offers comprehensive and timely analysis of the movement toward architectural diversification. By comparing and re-examining various deep learning models, we uncover new perspectives and presents the latest trends in time series forecasting, including the emergence of hybrid models, diffusion models, Mamba models, and foundation models. By focusing on the inherent characteristics of time series data, we also address open challenges that have gained attention in time series forecasting, such as channel dependency, distribution shift, causality, and feature extraction. This survey explores vital elements that can enhance forecasting performance through diverse approaches. These contributions lead to lowering the entry barriers for newcomers to the field of time series forecasting, while also offering seasoned researchers broad perspectives, new opportunities, and deep insights., Comment: Submitted to the Artificial Intelligence Review on October 10, 2024

We present the star formation rate (SFR) measurements based on the spectral energy distribution (SED) analysis with new sub-mm fluxes combined with archival multi-wavelength data for a sample of 52 AGN host galaxies at z $<0.2$. We carried out sub-mm observations using the SCUBA-2 camera at the James Clerk Maxwell Telescope, and obtained flux or an upper limit at 450 and 850 $\mu$m for each target. By experimenting the effect of the AGN dust component in the SED fit, we find that dust luminosity can be overestimated if AGN contribution is ignored. While the SFR based on 4000{\AA} break shows a significant offset compared to dust luminosity based SFR, the SFR obtained by the artificial neural network (Ellison et al. 2016) generally shows consistency albeit with a large scatter. We find that SFR correlates with AGN outflow strength manifested by the [OIII] $\lambda 5007$ emission line, and that AGNs with higher Eddington ratios and stronger outflows are in general hosted by galaxies with higher SFR, which is consistent with the correlation reported by Woo et al. (2020). This suggests no instantaneous quenching of star formation due to AGN feedback., Comment: Accepted for publication in ApJ (Feb. 9th, 2022). 23 pages, 15 figures