Accurate forecasting of ocean observation data plays a critical role in environmental monitoring and climate research. There are various types of marine environmental variables with complex interactions among them, and accurately capturing these relationships is the key to improving the accuracy of the forecast. However, existing deep learning methods fail to effectively utilize covariates to enhance target variable forecast due to the inconsistency in the number and types of variables observed at different marine monitoring stations, making it difficult to effectively deploy models in practical applications. This paper presents a novel pre-trained model specifically designed for ocean observation data that employs self-supervised learning to capture data distribution patterns from massive marine datasets. The proposed model introduces an innovative token construction method at the granularity level of the timestamp that integrates offset delay features and covariate information, enabling precise capture of real-time changes and long-term trends in ocean time series data. The model is pre-trained on large scale data, supporting zero-shot and few-shot inference, thereby reducing the reliance on extensive historical data from observation location. Furthermore, the model demonstrates strong performance on both inner-dataset and intra-dataset. The experimental results demonstrate that the model exhibits excellent performance in predicting various marine parameters, providing a reliable solution for the forecasting of marine data. As far as we know, this is the first marine-oriented pre-trained model for time series data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Ocean-Llama: A Self-supervised Pre-trained Deep Learning Model for Ocean Observation Data

  • Hao Wang,
  • Xiang Li,
  • Xi Fu,
  • Zhigang Zhao,
  • Chunxiao Wang,
  • Liting Geng,
  • Jian Zhang

摘要

Accurate forecasting of ocean observation data plays a critical role in environmental monitoring and climate research. There are various types of marine environmental variables with complex interactions among them, and accurately capturing these relationships is the key to improving the accuracy of the forecast. However, existing deep learning methods fail to effectively utilize covariates to enhance target variable forecast due to the inconsistency in the number and types of variables observed at different marine monitoring stations, making it difficult to effectively deploy models in practical applications. This paper presents a novel pre-trained model specifically designed for ocean observation data that employs self-supervised learning to capture data distribution patterns from massive marine datasets. The proposed model introduces an innovative token construction method at the granularity level of the timestamp that integrates offset delay features and covariate information, enabling precise capture of real-time changes and long-term trends in ocean time series data. The model is pre-trained on large scale data, supporting zero-shot and few-shot inference, thereby reducing the reliance on extensive historical data from observation location. Furthermore, the model demonstrates strong performance on both inner-dataset and intra-dataset. The experimental results demonstrate that the model exhibits excellent performance in predicting various marine parameters, providing a reliable solution for the forecasting of marine data. As far as we know, this is the first marine-oriented pre-trained model for time series data.