Research to Operations (R2O) and S2S forecast and verification products development

Last modified by s2s_wiki on 2023/12/23 06:55

Membership of Research to Operations (R2O) and S2S forecast and verification products development sub-project

Caio Coelho (CPTEC/INPE, Brazil)
Andrew Robertson (IRI, USA)
Arun Kumar (NOAA, USA)
Yuhei Takaya (JMA, Japan)
Anca Brookshaw (ECMWF)
Debra Hudson (BoM, Australia)
Angel Muñoz (IRI, USA)
Joanne Robbins (UKMO, UK)

1) Scientific and Operational Objectives

Pursue research for testing and developing methodologies for calibration, multi-model combination, verification and generation of forecast products.
Coordinate with the relevant WMO technical commissions to define the standards and protocols for operational implementation and exchange of S2S forecasts such that by the end of the Phase II of the S2S, the infrastructure related to the data exchange to support research can be transitioned into the operational domain.

The R2O and S2S forecast and verification products activities plan and report provides additional information about this sub-project, which builds on the work of the previous (Phase I) S2S Verification and Products sub-project.

2) Linkages with WMO activities

On the research side this sub-project has linkages with the Joint Working Group on Forecast Verification Research (JWGFVR), a WMO joint working group of the Working Group on Numerical Experimentation (WGNE) and the World Weather Research Programme (WWRP). This sub-project also has synergies with the World Climate Research Programme (WCRP) Working Group on Subseasonal to Interdecadal Prediction (WGSIP) through the Climate forecast information for decision making (I4D) project.

On the operational side this sub-project has linkages with the Expert Team on Operational Climate Prediction System (ET-OCPS) , a WMO team of the Infrastructure Commission (INFCOM). This sub-project also has links with the Expert Team on Climate Services Information System Operations (ET-CSISO) , a WMO team of the Services Commission (SERCOM).

3) Proposed questions to be addressed

The World Weather Research Programme (WWRP) has flagged improving forecasts of precipitation over land as an important area for S2S to focus research and services development efforts. In order to help advance scientific knowledge and the development of forecast and verification products in this priority area this sub-project invites the S2S research and operational communities to address the following questions:

What is the current performance level of sub-seasonal precipitation forecasts over land? Over which continental regions can these forecasts be best trusted? How performance levels vary through the seasons of the year?
What is the current capability of S2S models in anticipating the occurrence of extreme precipitation events over land (periods of deficit or excess precipitation)?
How well the main patterns of precipitation variability on the sub-seasonal time scale over various continental regions are represented in S2S prediction models?
How best to combine and calibrate sub-seasonal precipitation forecasts over land in order to produce improved, combined and well-calibrated products and services?
Are there identifiable opportunities for producing sub-seasonal precipitation forecasts over land with improved quality? For example, are forecasts produced during Madden and Julian Oscillation (MJO) and/or El Niño Southern Oscillation (ENSO) events more skilful than when neutral conditions are present? Are forecasts for active and break rainfall phases and dry/wet spells (or other quantities of interest) of adequate quality for developing forecast products for use in application sectors?

In order to address these questions the research and operational communities are encouraged to explore existing and develop novel methodologies for forecast calibration, combination and verification. Following the S2S verification chapter produced by the JWGFVR for the recent S2S book, it is particularly encouraged the identification of the most relevant forecast quality attributes for the target audiences (e.g. model and forecast developers, and various application sectors) in order to choose appropriate scores and metrics to be able to adequately address clearly and previously defined verification questions of interest. This practice helps performing a thorough assessment of sub-seasonal forecasts from both the probabilistic and deterministic points of view.

4) Current work of S2S operational and research communities on calibration, multi-model combination, verification and forecast products generation

4.1) Under development sub-seasonal multi-model ensemble (MME) forecasts and verification products at the WMO LC-SSFMME

ECMWF has been designated in 2023 by WMO as the first Global Producing Centre for Sub-Seasonal Forecasts (GPC-SSF) and the Lead Centre for Sub-Seasonal Forecast Multi-Model Ensemble (LC-SSFMME). This website is being developed to provide sub-seasonal multi-model ensemble forecasts and verification products.

4.2) Pilot real-time sub-seasonal multi-model ensemble (MME) forecasts and verification products at the WMO LC-LRFMME

The WMO Lead Centre for Long Range Forecast Multi-Model Ensemble (LC-LRFMME) has been developing a pilot system for real-time multi-model subseasonal forecasts using real-time forecasts (and hindcasts) from a subset of models contributing to the WWRP/WCRP S2S research project accessible via ECMWF data archive. Following this link the S2S research and operational communities have the opportunity to see the characteristics of the pilot real-time sub-seasonal MME prediction system developed by the LC-LRFMME, which includes forecast and verification products. These additional slides provide examples of products developed in support of future global, regional and national operational activities performed by WMO members. Subseasonal models from eight Global Producing Centers (GPCs) are currently used: Beijing, ECMWF, Exeter, Melbourne, Montreal, Seuol, Tokyo and Washington. A range of forecast products has been developed including probabilities for tercile categories of weekly/fortnightly averages of 2m temperature and precipitation as well as the MJO and BSISO indices. Hindcast verification has also been generated using ROC curves and scores, reliability diagrams, root mean square error and correlation between hindcast and observed anomalies.

4.3) IRI initiative on investigating the seasonality of subseasonal rainfall and temperature global prediction skill

This document summarizes the methods followed to conduct a global predictive skill assessment for uncalibrated rainfall and 2-m temperature forecasts, produced using the ECMWF’s IFS model, available through the WWRP/WCRP S2S Prediction Project Database via the International Research Institute for Climate and Society (IRI) Data Library.

4.4) The Australian Bureau of Meteorology adaptable framework for development and real time production of experimental sub-seasonal to seasonal forecast products

This document describes the new post-processing pipeline developed to add value to sub-seasonal and seasonal forecasts produced by the Australian Bureau of Meteorology.

4.5) Projects and networks dealing with S2S predictions

ACToday: The Adapting Agriculture to Climate Today, for Tomorrow project.

African SWIFT: Science for Weather Information and Forecasting Techniques.

CLIMAX: Climate Services Through Knowledge Co-Production: A Euro-South American Initiative for Strengthening Societal Adaptation Response to Extreme Events.

CSSP Brazil: Climate Science for Service Partnership Brazil

CSSP China: Climate Science for Service Partnership China

SNAP: Stratospheric Network for the Assessment of Predictability. See verification plans on page 6 of this presentation.

WCSSP India: Weather and Climate Science for Service Partnership India

4.6) Software tools

PyCPT and PyWR of the International Research Institute for Climate and Society (IRI)

WRtool of the Institute of Atmospheric Sciences and Climate (ISAC—CNR, Bologna, Italy)

MJO Diagnostics tools of the climpred development team

4.7) Web portals

IRI Sub-seasonal forecasts map room

CLIMAX project forecast maps: Precipitation, 2 meter temperature, 200 hPa Geopotential height and Outgoing Longwave Radiation.

BoM Sub-seasonal forecasts

CPTEC Sub-seasonal forecasts

JMA Sub-seasonal forecasts

HMC Sub-seasonal forecasts

ECMWF Extended-range forecasts

5) Publications

Below is a list of publications recently produced by the international S2S research community on forecast verification, calibration and multi-model combination, including prediction quality assessment and methodological studies. Additional references including books and technical reports on methods relevant for S2S verification are available in section 4 of the phase I S2S verification and products sub-project wiki page. A more comprehensive list of publications produced by the international community including various other S2S research aspects is available in the main S2S project website.

2023

· Zhang, Z., DeFlorio, M. J., Delle Monache, L., Subramanian, A. C., Ralph, F. M., Waliser, D. E., et al. (2023) 'Multi-model subseasonal prediction skill assessment of water vapor transport associated with atmospheric rivers over the western U.S.', Journal of Geophysical Research: Atmospheres. 128, e2022JD037608. https://doi.org/10.1029/2022JD037608

· Li, X., Tang, Y., Shen, Z., & Li, Y. (2023) 'Spatial variations in seamless predictability of subseasonal precipitation over Asian summer monsoon region in S2S models.', Journal of Geophysical Research: Atmospheres,. 128, e2023JD038480. https://doi.org/10.1029/2023JD038480

· Inatsu, M., M. Matsueda, N. Nakano, and S. Kawazoe, (2023) 'Prediction skill and practical predictability depending on the initial atmospheric states in S2S forecasts.', J. Atmos. Sci.. https://doi.org/10.1175/JAS-D-22-0262.1

· Chen, D., Pan, C., Qiao, S., Zhi, R., Tang, S., Yang, J., Feng, G., & Dong, W. (2023) 'Evolution and prediction of the extreme rainstorm event in July 2021 in Henan province, China.', Atmospheric Science Letters. e1156. https://doi.org/10.1002/asl.1156

· Ma, R., and Yuan, X. (2023) 'Sub-seasonal ensemble prediction of flash droughts over China.', J. Hydrometeor.. https://doi.org/10.1175/JHM-D-22-0150.1.

· Yan, Y., Zhu, C., & Liu, B. (2023) 'Subseasonal predictability of the July 2021 extreme rainfall event over Henan China in S2S operational models.', Journal of Geophysical Research: Atmospheres,.128, e2022JD037879. https://doi.org/10.1029/2022JD037879

· Xie, J., Hsu, P., Hu, Y., Ye, M., & Yu, J. (2023) 'Skilful Extended-Range Forecast of Rainfall and Extreme Events in East China Based on Deep Learning,.', Weather and Forecasting.

2022

· Stan, C. (2022) 'The forecast skill of the Northern Hemisphere middle latitudes seasonal oscillation and its impact on the surface air temperature.', Geophysical Research Letters. 49, e2021GL095543. https://doi.org/10.1029/2021GL.

· Amaya, D. J., Jacox, M. G., Dias, J., Alexander, M. A., Karnauskas, K. B., Scott, J. D., & Gehne, M. (2022) 'Subseasonal-to-seasonal forecast skill in the California Current System and its connection to coastal Kelvin waves', Journal of Geophysical Research: Oceans. 127, e2021JC017892. https://doi.org/10.1029/2021JC017892.

· Vitart, F., Robertson, A.W., Spring, A., Pinault, F., Roškar, R., Cao, W., Bech, S., Bienkowski, A., Caltabiano, N., De Coning, E., Denis, B., Dirkson, A., Dramsch, J., Dueben, P., Gierschendorf, J., Kim, H. S., Nowak, K., Landry, D., Lledó, L., Palma, L., Rasp, S., & Zhou, S. (2022) 'Outcomes of the WMO Prize Challenge to Improve Sub-Seasonal to Seasonal Predictions Using Artificial Intelligence', Bulletin of the American Meteorological Society.

· Gonzalez, P.L., Howard, E., Ferrett, S., Frame, T.H., Martínez-Alvarado, O., Methven, J. and Woolnough, S.J. (2022) 'Weather Patterns in Southeast Asia: Enhancing high-impact weather sub-seasonal forecast skill', Q J R Meteorol Soc..

· Li, Y., Wu, Z., He, H., and Yin, H. (2022) 'Probabilistic subseasonal precipitation forecasts using preceding atmospheric intraseasonal signals in a Bayesian perspective', Hydrol. Earth Syst. Sci.. 26, 4975–4994.

· Garfinkel, C. I., Chen, W., Li, Y., Schwartz, C., Yadav, P., & Domeisen, D. (2022) 'The winter North Pacific teleconnection in response to ENSO and the MJO in operational subseasonal forecasting models is too weak', Journal of Climate.

· Wang, X.; Li, S.; Liu, L.; Bai, H.; Feng, G. (2022) 'The Performance of S2S Models on Predicting the 21.7 Extreme Rainfall Event in Henan China', Atmosphere. 13, 1516.

· Jia, Z., Zheng, Z., Zhu, Y. et al. (2022) 'Predictable patterns of midsummer surface air temperature over Eastern China and their corresponding signal sources in ECMWF subseasonal forecasts', Clim Dyn.

· Sun, L., Hoerling, M. P., Richter, J. H., Hoell, A., Kumar, A., & Hurrell, J. W. (2022) 'Attribution of North American Subseasonal Precipitation Prediction Skill', Weather and Forecasting.

· Wu, J., Li, J., Zhu, Z. et al. (2022) 'Factors determining the subseasonal prediction skill of summer extreme rainfall over southern China', Clim Dyn.

· Qin J, Zhou L, Li B and Meng Z (2022) 'Prediction of the Central Indian Ocean Mode in S2S Models', Front. Mar. Sci. 9:880469. doi: 10.3389/fmars.2022.880469

· Graham, R. M., Browell, J., Bertram, D., & White, C. J. (2022) 'The application of sub-seasonal to seasonal (S2S) predictions for hydropower forecasting', Meteorological Applications. 29( 1), e2047.

· Chevuturi, A., Klingaman, N.P., Guo, L., Holloway, C.E., Guimarães, B.S., Coelho, C.A.S., Kubota, P.Y., Young, M., Black, E., Baker, J.C.A., Vidale, P.L. (2022) 'Subseasonal prediction performance for South American land–atmosphere coupling in extended austral summer', Climate Resilience and Sustainability, 1, e28.

· Fischer, C., Fink, A. H., Schömer, E., van der Linden, R., Maier-Gerber, M., Rautenhaus, M., and Riemer, M. (2022) 'A novel method for objective identification of 3-D potential vorticity anomalies', Geosci. Model Dev., 15, 4447–4468.

· Zeqing Huang, Tongtiegang Zhao, Weixin Xu, Huayang Cai, Jiabiao Wang, Yongyong Zhang, Zhiyong Liu, Yu Tian, Denghua Yan, Xiaohong Chen (2022) 'A seven-parameter Bernoulli-Gamma-Gaussian model to calibrate subseasonal to seasonal precipitation forecasts', Journal of Hydrology. ISSN 0022-1694.

· Goutham, N., Plougonven, R., Omrani, H., Parey, S., Tankov, P., Tantet, A., Hitchcock, P., & Drobinski, P. (2022) 'How skillful are the European sub-seasonal forecasts of wind speed and surface temperature?', Monthly Weather.

· Domeisen, D. I. V. et al. (2022) 'Advances in the subseasonal prediction of extreme events: Relevant case studies across the globe', Bulletin of the American Meteorological Society. American Meteorological Society, 1(aop). doi: 10.1175/BAMS-D-20-0221.1.

· Becker, E. J. et al. (2022) 'A Decade of the North American Multimodel Ensemble (NMME): Research, Application, and Future Directions', Bulletin of the American Meteorological Society. American Meteorological Society, 103(3), pp. E973–E995. doi: 10.1175/BAMS-D-20-0327.1.

· Specq, D., & Batté, L. (2022). Do subseasonal forecasts take advantage of Madden–Julian oscillation windows of opportunity? Atmospheric Science Letters.

· Lin, H., Mo, R., & Vitart, F. (2022). The 2021 western North American heatwave and its subseasonal predictions. Geophysical Research Letters, 49.

· Parker, D. J., Blyth, A. M., Woolnough, S. J., Dougill, A. J., Bain, C. L., de Coning, E., Diop-Kane, M., Kamga Foamouhoue, A., Lamptey, B., Ndiaye, O., Ruti, P., Adefisan, E. A., Amekudzi, L. K., Antwi-Agyei, P., Birch, C. E., Cafaro, C., Carr, H., Chanzu, B., Clarke, S. J., Coskeran, H., Danuor, S. K., de Andrade, F. M., Diakaria, K., Dione, C., Diop, C. A., Fletcher, J. K., Gaye, A. T., Groves, J. L., Gudoshava, M., Hartley, A. J., Hirons, L. C., Ibrahim, I., James, T. D., Lawal, K. A., Marsham, J. H., Mutemi, J. N., Okogbue, E. C., Olaniyan, E., Omotosho, J. B., Portuphy, J., Roberts, A. J., Schwendike, J., Segele, Z. T., Stein, T. H. M., Taylor, A. L., Taylor, C. M., Warnaars, T. A., Webster, S., Woodhams, B. J., & Youds, L. (2022). The African SWIFT Project: Growing Science Capability to Bring about a Revolution in Weather Prediction, Bulletin of the American Meteorological Society, 103(2), E349-E369.

· Wulff, C.O., Vitart, F. and Domeisen, D.I.V. (2022), Influence of Trends on Subseasonal Temperature Prediction Skill. Q J R Meteorol Soc. Accepted Author Manuscript.

· Stan, C., Zheng, C., Chang, E. K., Domeisen, D. I., Garfinkel, C. I., Jenney, A. M., Kim, H., Lim, Y., Lin, H., Robertson, A., Schwartz, C., Vitart, F., Wang, J., & Yadav, P. (2022). Advances in the prediction of MJO-Teleconnections in the S2S forecast systems, Bulletin of the American Meteorological Society (published online ahead of print 2022).

· Stan, C. (2022). The forecast skill of the Northern Hemisphere middle latitudes seasonal oscillation and its impact on the surface air temperature. Geophysical Research Letters, 49.

· Amaya, D. J., Jacox, M. G., Dias, J., Alexander, M. A., Karnauskas, K. B., Scott, J. D., & Gehne, M. (2022). Subseasonal-to-seasonal forecast skill in the California Current System and its connection to coastal Kelvin waves. Journal of Geophysical Research: Oceans, 127, e2021JC017892. https://doi.org/10.1029/2021JC017892 Pham-Thanh, H., Phan-Van, T., van der Linden, R., & Fink, A. H. (2022). The Performance of ECMWF Subseasonal Forecasts to Predict the Rainy Season Onset Dates in Vietnam, Weather and Forecasting, 37(1), 113-124.

· Zhang, M., Yang, X.-Y., & Huang, Y. (2022). Impacts of sudden stratospheric warming on extreme cold events in early 2021: An ensemble-based sensitivity analysis. Geophysical Research Letters, 49, e2021GL096840.

· Pham-Thanh, H., Phan-Van, T., van der Linden, R., & Fink, A. H. (2022). The Performance of ECMWF Subseasonal Forecasts to Predict the Rainy Season Onset Dates in Vietnam, Weather and Forecasting, 37(1), 113-124.

2021

· White, C. J., Domeisen, D. I. V., Acharya, N., Adefisan, E. A., Anderson, M. L., Aura, S., Balogun, A. A., Bertram, D., Bluhm, S., Brayshaw, D. J., Browell, J., Büeler, D., Charlton-Perez, A., Chourio, X., Christel, I., Coelho, C. A. S., DeFlorio, M. J., Delle Monache, L., Di Giuseppe, F., García-Solórzano, A. M., Gibson, P. B., Goddard, L., González Romero, C., Graham, R. J., Graham, R. M., Grams, C. M., Halford, A., Katty Huang, W. T., Jensen, K., Kilavi, M., Lawal, K. A., Lee, R. W., MacLeod, D., Manrique-Suñén, A., Martins, E. S. P. R., Maxwell, C. J., Merryfield, W. J., Muñoz, Á. G., Olaniyan, E., Otieno, G., Oyedepo, J. A., Palma, L., Pechlivanidis, I. G., Pons, D., Ralph, F. M., Reis, D. S., Jr., Remenyi, T. A., Risbey, J. S., Robertson, D. J. C., Robertson, A. W., Smith, S., Soret, A., Sun, T., Todd, M. C., Tozer, C. R., Vasconcelos, F. C., Jr., Vigo, I., Waliser, D. E., Wetterhall, F., & Wilson, R. G. (2021). Advances in the application and utility of subseasonal-to-seasonal predictions, BAMS.

· Chevuturi, A., Klingaman, N. P., Guo, L., Holloway,C. E., Guimarães, B. S., Coelho, C. A. S. Kubota, P. Y., Young, M., Black, E. Baker, J. C.A., Vidale, P. L., 2021. Subseasonal prediction performance for South American land-atmosphere coupling in extended austral summer. Climate Resilience and Sustainability. http://dx.doi.org/10.1002/cli2.28.

· Hayman, P and Hudson, D. 2021. Forewarned is forearmed – Exploring the value of new forecast products from the BOM to enable more informed decisions on profit and risk on grain farms, Grains Research and Development Corporation (GRDC) Update Paper: https://grdc.com.au/resources-and-publications/grdc-update-papers/tab-content/grdc-update-papers/2021/05/forewarned-is-forearmed-exploring-the-value-of-new-forecast-products-from-the-bom-to-enable-more-informed-decisions-on-profit-and-risk-on-grain-farms

· de Burgh-Day, C. and Dillon, F. 2021. A hybrid parametrisation for precipitation probability of exceedance data. Bureau Research Report No. 052, Bureau of Meteorology Australia

· Lim, Hudson, Wheeler, et al. 2021. Why Australia was Not Wet during Spring 2020 despite La Niña. Scientific Reports. www.nature.com/articles/s41598-021-97690-w.

· Marshall, A.G., H.H. Hendon, and D. Hudson. 2021. Influence of the Madden-Julian Oscillation on Multiweek Prediction of Australian Rainfall Extremes using the ACCESS-S1 Prediction System. J. Southern Hem. Earth Sys. Sci., https://doi.org/10.1071/ES21001.

· Marshall AG, Gregory PA, de Burgh-Day CO, and Griffiths M, 2021: Subseasonal drivers of extreme fire weather in Australia and its prediction in ACCESS-S1 during spring and summer. Climate Dynamics. https://doi.org/10.1007/s00382-021-05920-8.

· Lim E, Hendon HH, Shi L, de Burgh-Day C, Hudson D, King A, Trewin B, Griffiths M, Marshall A 2021. Tropical forcing of Australian extreme low minimum temperatures in September 2019, Climate Dynamics, https://doi.org/10.1007/s00382-021-05661-8.

· Lim, E-P., H.H. Hendon and co-authors, 2021: The 2019 Southern Hemisphere polar stratospheric warming and its impacts. Bulletin of the American Meteorological Society, https://doi.org/10.1175/BAMS-D-20-0112.1.

· Cowan, Tim and Wheeler, Matthew C. and Sharmila, S. Narsey, Sugata and de Burgh-Day, Catherine 2021. Forecasting Northern Australian Summer Rainfall Bursts Using a Seasonal Prediction System. Weather and Forecasting, 37 (1). pp. 23-44. ISSN 0882-8156

· Pons, D. et al. (2021) 'A Coffee Yield Next-Generation Forecast System for Rain-fed Plantations: the Case of the Samalá Watershed in Guatemala', Weather and Forecasting, 36(6), pp. 2021–2038. doi: 10.1175/WAF-D-20-0133.1.

· DeMott, C. et al. (2021) 'The Benefits of Better Ocean Weather Forecasting', Eos. American Geophysical Union (AGU), 102. doi: 10.1029/2021eo210601.

· Cavalcanti, I. F. A., Barreto, N. J. C., Alvarez, M. S., Osman, M., & Coelho, C. A. S. (2021). Teleconnection patterns in the Southern Hemisphere represented by ECMWF and NCEP S2S project models and influences on South America precipitation. Meteorological Applications, 28( 4), e2011.

· Zhou Y and Wang Y (2021) Influence of the Madden–Julian Oscillation on the Arctic Oscillation Prediction in S2S Operational Models. Front. Earth Sci. 9:787680. doi: 10.3389/feart.2021.787680

· Ardilouze, C., Specq, D., Batté, L., and Cassou, C.: Flow dependence of wintertime subseasonal prediction skill over Europe, Weather Clim. Dynam., 2, 1033–1049.

· Cui, J., Yang, S. & Li, T. Intraseasonal Variability of Summertime Surface Air Temperature over Mid-High-Latitude Eurasia and Its Prediction Skill in S2S Models. J Meteorol Res 35, 815–830 (2021).

· Li, Y.;Wu, Z.; He, H.; Lu, G. Deterministic and Probabilistic Evaluation of Sub-Seasonal Precipitation Forecasts at Various Spatiotemporal Scales over China during the Boreal Summer Monsoon. Atmosphere 2021, 12, 1049.

· Bloomfield, H. C., Brayshaw, D. J., Gonzalez, P. L. M., & Charlton-Perez, A. (2021). Pattern-based conditioning enhances sub-seasonal prediction skill of European national energy variables. Meteorological Applications, 28( 4), e2018.

· MacLeod, D. A., Dankers, R., Graham, R., Guigma, K., Jenkins, L., Todd, M. C., Kiptum, A., Kilavi, M., Njogu, A., & Mwangi, E. (2021). Drivers and Subseasonal Predictability of Heavy Rainfall in Equatorial East Africa and Relationship with Flood Risk, Journal of Hydrometeorology, 22(4), 887-903.

· Rao, J., Garfinkel, C. I., Wu, T., Lu, Y., Lu, Q., & Liang, Z. (2021). The January 2021 sudden stratospheric warming and its prediction in subseasonal to seasonal models. Journal of Geophysical Research: Atmospheres, 126, e2021JD035057.

· Lim, Y., Peings, Y., & Magnusdottir, G. (2021). The role of atmospheric drivers in a sudden transition of California precipitation in the 2012/13 winter. Journal of Geophysical Research: Atmospheres, 126, e2021JD035028.

· Zhou Y and Wang Y (2021) Influence of the Madden–Julian Oscillation on the Arctic Oscillation Prediction in S2S Operational Models. Front. Earth Sci. 9:787680.

· Gonzalez, P.L.M., Brayshaw, D.J. & Ziel, F.(2021) A new approach to extended-range multimodel forecasting: Sequential learning algorithms. Q J R Meteorol Soc, 1– 14.

· Wandel, J., Quinting, J. F., & Grams, C. M. (2021). Toward a systematic evaluation of warm conveyor belts in numerical weather prediction and climate models. Part II: Verification of operational reforecasts, Journal of the Atmospheric Sciences.

· Ham, S., Jeong, Y. Characteristics of Subseasonal Winter Prediction Skill Assessment of GloSea5 for East Asia. Atmosphere 2021, 12, 1311.

· Zhang, K. Y., J. Li, Z. W. Zhu, and T. Li, 2021: Implications from subseasonal prediction skills of the prolonged heavy snow event over southern China in early 2008. Adv. Atmos. Sci., 38(11), 1873−1888.

· Lin, H., Huang, Z., Hendon, H., & Brunet, G. (2021). NAO Influence on the MJO and its Prediction Skill in the Subseasonal-to-Seasonal Prediction Models, Journal of Climate.

· Büeler, D., Ferranti, L., Magnusson, L., Quinting, J.F. and Grams, C.M. (2021), Year-round sub-seasonal forecast skill for Atlantic-European weather regimes. Q J R Meteorol Soc.

· Dai, G., Mu, M., Li, C., Han, Z., & Wang, L. (2021). Evaluation of the forecast performance for extreme cold events in East Asia with subseasonalto-seasonal data sets from ECMWF. Journal of Geophysical Research: Atmospheres, 126, 2020JD033860.

· Kolstad, E. W., MacLeod, D., & Demissie, T. D. (2021). Drivers of subseasonal forecast errors of the East African short rains. Geophysical Research Letters, 48, e2021GL093292.

· Yuan Li, Zhiyong Wu, Hai He, Quan J. Wang, Huating Xu, Guihua Lu, Post-processing sub-seasonal precipitation forecasts at various spatiotemporal scales across China during boreal summer monsoon, Journal of Hydrology, Volume 598, 2021, 125742, ISSN 0022-1694.

· L.A.D.B. Bandurathna, L. Wang, X. Zhou et al., Intraseasonal oscillation of the southwest monsoon over Sri Lanka and evaluation of its subseasonal forecast skill, Atmospheric and Oceanic Science Letters

· Tsai,W.Y.-H.; Lu, M.-M.; Sui, C.-H.; Cho, Y.-M. Subseasonal Forecasts of the Northern Queensland Floods of February 2019: Causes and Forecast Evaluation. Atmosphere 2021, 12, 758.

· Kim, H., Ham, Y.G., Joo, Y.S. et al. Deep learning for bias correction of MJO prediction. Nat Commun 12, 3087 (2021). https://doi.org/10.1038/s41467-021-23406-3.

· Albers, J. R., Butler, A. H., Breeden, M. L., Langford, A. O., and Kiladis, G. N.: Subseasonal prediction of springtime Pacific–North American transport using upper-level wind forecasts. Weather Clim. Dynam., 2, 433–452.

· Moron V, Robertson AW. Relationships between subseasonal-to-seasonal predictability and spatial scales in tropical rainfall. Int J Climatol. 2021;1–29.

· Endris, H. S., Hirons, L., Segele, Z. T., Gudoshava, M., Woolnough, S., & Artan, G. A. (2021). Evaluation of the Skill of Monthly Precipitation Forecasts from Global Prediction Systems over the Greater Horn of Africa. Weather and Forecasting.

· Li, X., & Tang, Y. (2021). Predictable Mode of Tropical Intraseasonal Variability in Boreal Summer, Journal of Climate, 34(9), 3355-3366.

· John R Albers, Matthew Newman (2021). Subseasonal predictability of the North Atlantic Oscillation, Environ. Res. Lett., 16, 044024

· Quinting, J. F., & Grams, C. M. (2021). Toward a Systematic Evaluation of Warm Conveyor Belts in Numerical Weather Prediction and Climate Models. Part I: Predictor Selection and Logistic Regression Model, Journal of the Atmospheric Sciences, 78(5), 1465-1485.

· Abdolghafoorian, A., & Dirmeyer, P. A. (2021). Validating the Land-Atmosphere Coupling Behavior in Weather and Climate Models Using Observationally-Based Global Products, Journal of Hydrometeorology,.

· Feng, P., Lin, H., Derome, J., & Merlis, T. M. (2021). Forecast Skill of the NAO in the Subseasonal to-Seasonal Prediction Models, Journal of Climate, 1-50.

· Wu, J., & Jin, F.-F. (2021). Improving the MJO forecast of S2S operation models by correcting their biases in linear dynamics. Geophysical Research Letters, 48.

· Liu, Y., Bogaardt, L., Attema, J., & Hazeleger, W. (2021). Extended Range Arctic Sea Ice Forecast with Convolutional Long-Short Term Memory Networks, Monthly Weather Review.

· Musonda, B., Jing, Y., Nyasulu, M. et al. Evaluation of sub-seasonal to seasonal rainfall forecast over Zambia. J Earth Syst Sci 130, 47 (2021).

· Ma, Y., Li, J., Zhang, S. et al. A multi-model study of atmosphere predictability in coupled ocean–atmosphere systems. Clim Dyn (2021).

· Cui, J., Yang, S. & Li, T. How well do the S2S models predict intraseasonal wintertime surface air temperature over mid-high-latitude Eurasia?. Clim Dyn (2021).

· Deoras, A., Hunt, K. M. R., & Turner, A. G. (2021). Comparison of the prediction of Indian monsoon low-pressure systems by Subseasonal-to-Seasonal prediction models, Weather and Forecasting.

· Grimm, A.M., Hakoyama, L.R. & Scheibe, L.A. Active and break phases of the South American summer monsoon: MJO influence and subseasonal prediction. Clim Dyn (2021).

· MacLeod, D. A., Dankers, R., Graham, R., Guigma, K., Jenkins, L., Todd, M. C., Kiptum, A., Kilavi, M., Njogu, A., & Mwangi, E. (2021). Drivers and subseasonal predictability of heavy rainfall in equatorial East Africa and relationship with flood risk, Journal of Hydrometeorology.

· Richardson, D., Black, A. S., Monselesan, D. P., Moore II, T. S., Risbey, J. S., Schepen, A., Squire, D. T., & Tozer, C. R. (2021). Identifying Periods of Forecast Model Confidence for Improved Subseasonal Prediction of Precipitation, Journal of Hydrometeorology, 22(2), 371-385.

· de Andrade, F. M., Young, M. P., MacLeod, D., Hirons, L. C., Woolnough, S. J., & Black, E. (2021). Subseasonal Precipitation Prediction for Africa: Forecast Evaluation and Sources of Predictability, Weather and Forecasting, 36(1), 265-284.

· Engelbrecht, C. J., Phakula, S., Landman, W. A., & Engelbrecht, F. A. (2021). Subseasonal Deterministic Prediction Skill of Low-Level Geopotential Height Affecting Southern Africa, Weather and Forecasting, 36(1), 195-205.

· Klingaman, N. P., Young, M., Chevuturi, A., Guimaraes, B., Guo, L., Woolnough, S. J., Coelho, C. A. S., Kubota, P. Y., & Holloway, C. E. (2021). Subseasonal Prediction Performance for Austral Summer South American Rainfall, Weather and Forecasting, 36(1), 147-169

· Guimarães, B.S., Coelho, C.A.S., Woolnough, S.J. et al. (2021). An inter-comparison performance assessment of a Brazilian global sub-seasonal prediction model against four sub-seasonal to seasonal (S2S) prediction project models, Climate Dynamics.

· Zhu, S., Zhi, X., Ge, F., Fan, Y., Zhang, L., & Gao, J. (2021). Subseasonal Forecast of Surface Air Temperature Using Superensemble Approaches: Experiments over Northeast Asia for 2018, Weather and Forecasting, 36(1), 39-51.

· Zheng, C., Kar-Man Chang, E., Kim, H., Zhang, M., & Wang, W. (2021). Subseasonal Prediction of Wintertime Northern Hemisphere Extratropical Cyclone Activity by SubX and S2S Models, Weather and Forecasting, 36(1), 75-89.

2020

· de Burgh-Day C, Griffiths M, Yan H, Young G, Hudson D, Alves O, 2020: An adaptable framework for development and real time production of experimental sub-seasonal to seasonal forecast products, Bureau Research Report, No. 42. Bureau of Meteorology Australia. (http://www.bom.gov.au/research/publications/researchreports/BRR-042.pdf)

· Cowan, Tim and Stone, Roger and Wheeler, Matthew C. and Griffiths, Morwenna 2020. Improving the seasonal prediction of Northern Australian rainfall onset to help with grazing management decisions. Climate Services, 19:100182. pp. 1-14.

· Wang, G. and Hendon, H.H. 2020. Impacts of the Madden–Julian Oscillation on wintertime Australian minimum temperatures and Southern Hemisphere circulation. Climate Dynamics, https://doi.org/10.1007/s00382-020-05432-x.

· Hendon, H. H., Lim, E.‐P., & Abhik, S. 2020. Impact of interannual ozone variations on the downward coupling of the 2002 Southern Hemisphere stratospheric warming. Journal of Geophysical Research: Atmospheres, 125, e2020JD032952. https://doi.org/10.1029/2020JD032952.

· Muñoz, Á. G. et al. (2020) 'AeDES: a next-generation monitoring and forecasting system for environmental suitability of Aedes-borne disease transmission', Scientific Reports. Nature Publishing Group, 10(1), p. 12640. doi: 10.1038/s41598-020-69625-4.

· Gonzalez Romero, C. et al. (2020) 'When Rainfall Meets Hunger: Towards an Early-Action System for Acute Undernutrition in Guatemala', AGUFM, 2020, pp. GC051-0012. Available at: https://ui.adsabs.harvard.edu/abs/2020AGUFMGC0510012G/abstract

· Zhu, S., Zhi, X., Ge, F., Fan, Y., Zhang, L., & Gao, J. (2020). Subseasonal Forecast of Surface Air Temperature Using Superensemble Approaches: Experiments over Northeast Asia for 2018. Weather and Forecasting, 1-44.

· Yamagami, A., & Matsueda, M. (2020). Sub‐seasonal Forecast Skill for Weekly Mean Atmospheric Variability over the Northern Hemisphere in Winter and its Relationship to Mid‐Latitude Teleconnections. Geophysical Research Letters, e2020GL088508.

· Taguchi, M. (2020). A Study of False Alarms of a Major Sudden Stratospheric Warming by Real-Time Subseasonal-to-Seasonal Forecasts for the 2017/2018 Northern Winter. Atmosphere, 11(8), 875.

· Lledó, L., & Doblas-Reyes, F. J. (2020). Predicting daily mean wind speed in Europe weeks ahead from MJO status. Monthly Weather Review, 148(8), 3413-3426.

· Specq, D., Batté, L. (2020). Improving subseasonal precipitation forecasts through a statistical–dynamical approach : application to the southwest tropical Pacific. Clim. Dyn. https://doi.org/10.1007/s00382-020-05355-7

· Quedi, E. S., & Fan, F. M. (2020). Sub seasonal streamflow forecast assessment at large-scale basins. Journal of Hydrology, 584, 124635.

· Merryfield, W. J., Baehr, J., Batté, L., Becker, E. J., Butler, A. H., Coelho, C. A., ... & Ferranti, L. (2020). Current and emerging developments in subseasonal to decadal prediction. Bulletin of the American Meteorological Society, 101(6), E869-E896.

· Robertson, A. W., Vitart, F., & Camargo, S. J. (2020). Subseasonal to Seasonal Prediction of Weather to Climate with Application to Tropical Cyclones. Journal of Geophysical Research: Atmospheres, 125(6), e2018JD029375.

· Minami, A., & Takaya, Y. Enhanced Northern Hemisphere correlation skill of subseasonal predictions in the strong negative phase of the Arctic Oscillation. Journal of Geophysical Research: Atmospheres, e2019JD031268.

· Mayer, K. J., & Barnes, E. A. (2020). Subseasonal midlatitude prediction skill following Quasi-Biennial Oscillation and Madden–Julian Oscillation activity. Weather and Climate Dynamics, 1(1), 247-259.

· Mariotti, A., Baggett, C., Barnes, E. A., Becker, E., Butler, A., Collins, D. C., ... & Kirtman, B. P. (2020). Windows of Opportunity for Skillful Forecasts Subseasonal to Seasonal and Beyond. Bulletin of the American Meteorological Society, (2020).

· Dirmeyer, P. A., & Ford, T. W. (2020). A technique for seamless forecast construction and validation from weather to monthly time scales. Monthly Weather Review, 148(9), 3589-3603.

· Alvarez, M. S., Coelho, C. A. S., Osman, M., Firpo, M. Â. F., & Vera, C. S. (2020). Assessment of ECMWF Subseasonal Temperature Predictions for an Anomalously Cold Week Followed by an Anomalously Warm Week in Central and Southeastern South America during July 2017. Weather and Forecasting, 35(5), 1871-1889.

· Phakula, S., Landman, W. A., Engelbrecht, C. J., & Makgoale, T. (2020) Forecast Skill of Minimum and Maximum Temperatures on Subseasonal‐to‐Seasonal Timescales Over South Africa. Earth and Space Science, 7(2), e2019EA000697.

· Manrique-Suñén, A., Gonzalez-Reviriego, N., Torralba, V., Cortesi, N., & Doblas-Reyes, F. J. (2020). Choices in the verification of S2S forecasts and their implications for climate services. Monthly Weather Review, 1-41.

· Wang, Y., Ren, H. L., Zhou, F., Fu, J. X., Chen, Q. L., Wu, J., ... & Zhang, P. Q. (2020). Multi-Model Ensemble Sub-Seasonal Forecasting of Precipitation over the Maritime Continent in Boreal Summer. Atmosphere, 11(5), 515.

· Specq, D., Batté, L., Déqué, M., & Ardilouze, C. (2020). Multimodel forecasting of precipitation at subseasonal timescales over the southwest tropical Pacific. Earth and Space Science, (Accepted)

· Robertson, A. W., Vigaud, N., Yuan, J., & Tippett, M. K. (2020). Towards identifying subseasonal forecasts of opportunity using North American weather regimes. Monthly Weather Review.

· He, H., Yao, S., Huang, A., & Gong, K. (2020). Evaluation and Error Correction of the ECMWF Subseasonal Precipitation Forecast over Eastern China during Summer. Advances in Meteorology, 2020.

· Taguchi, M. (2020). Verification of Subseasonal-to-Seasonal Forecasts for Major Stratospheric Sudden Warmings in Northern Winter from 1998/99 to 2012/13. Advances in Atmospheric Sciences, 37(3), 250-258.

· Phakula, S., Landman, W. A., Engelbrecht, C. J., & Makgoale, T. (2020). Forecast Skill of Minimum and Maximum Temperatures on Subseasonal‐to‐Seasonal Timescales Over South Africa. Earth and Space Science, 7(2), e2019EA000697.

· Son, S. W., Kim, H., Song, K., Kim, S. W., Martineau, P., Hyun, Y. K., & Kim, Y. (2020). Extratropical Prediction Skill of the Subseasonal‐to‐Seasonal (S2S) Prediction Models. Journal of Geophysical Research: Atmospheres, 125(4), e2019JD031273.

· Lin, H. (2020). Subseasonal Forecast Skill over the Northern Polar Region in Boreal Winter. Journal of Climate, 33(5), 1935-1951.

· Wang, S. (2020). A precipitation-based index for tropical intraseasonal oscillations. Journal of Climate, 33(3), 805-823.

· Vigaud, N., Tippett, M. K., Yuan, J., Robertson, A. W., & Acharya, N. (2020). Spatial Correction of Multimodel Ensemble Subseasonal Precipitation Forecasts over North America Using Local Laplacian Eigenfunctions. Monthly Weather Review, 148(2), 523-539.

· Materia, S., Muñoz, Á. G., Álvarez-Castro, M. C., Mason, S. J., Vitart, F., & Gualdi, S. (2020). Multimodel Subseasonal Forecasts of Spring Cold Spells: Potential Value for the Hazelnut Agribusiness. Weather and Forecasting, 35(1), 237-254.

2019

· Muñoz, Á.G. et al. (2019) 'NextGen: A Next-Generation System for Calibrating, Ensembling and Verifying Regional Seasonal and Subseasonal Forecasts', AGUFM, 2019, pp. A23U-3024. Available at: https://ui.adsabs.harvard.edu/abs/2019AGUFM.A23U3024M/abstract

· Muñoz, Á. G. et al. (2019) 'Can We Predict "Climate Migrations"? The 2018 Guatemalan Case', in American Geophysical Union, Fall Meeting 2019, abstract #GC13E-1213. Available at: https://ui.adsabs.harvard.edu/abs/2019AGUFMGC13E1213M/abstract

· Vigaud, N., Tippett, M. K., & Robertson, A. W. (2019). Deterministic Skill of Subseasonal Precipitation Forecasts for the East Africa‐West Asia Sector from September to May. Journal of Geophysical Research: Atmospheres, 124(22), 11887-11896.

· Vigaud, N., Tippett, M. K., Yuan, J., Robertson, A. W., & Acharya, N. (2019). Probabilistic skill of subseasonal surface temperature forecasts over North America. Weather and Forecasting, 34(6), 1789-1806.

· Lin, H. (2019). Subseasonal Forecast Skill over the Northern Polar Region in Boreal Winter. Journal of Climate, (Accepted)

· Pan, B., K. Hsu, A. AghaKouchak, S. Sorooshian, and W. Higgins (2019). Precipitation Prediction Skill for the West Coast United States: From Short to Extended Range. J. Climate, 32, 161–182, https://doi.org/10.1175/JCLI-D-18-0355.1

· Albers, J. R., & Newman, M. (2019). A Priori Identification of Skillful Extratropical Subseasonal Forecasts. Geophysical Research Letters.

· Li, W., J. Chen, L. Li, H. Chen, B. Liu, C. Xu, and X. Li, (2019): Evaluation and Bias Correction of S2S Precipitation for Hydrological Extremes. J. Hydrometeor., 20, 1887–1906, https://doi.org/10.1175/JHM-D-19-0042.1

· Richardson, D., H. Fowler, C. Kilsby, R. neal and R. Dankers, (2019). Improving sub-seasonal forecast skill of meteorological drought: a weather pattern approach. Nat. Hazards Earth Syst. Sci., https://doi.org/10.5194/nhess-2019-221

· Lee, Seungsoo;Kim, Gayoung;Yoon, Soonjo;An, Hyunuk, (2019). Improvement of precipitation forecasting skill of ECMWF data using multi-layer perceptron technique. Journal of Korea Water Resources Association, Volume 52, Issue 7, pp 475-482. https://doi.org/10.3741/JKWRA.2019.52.7.475

· ZHU, Hanchen and Haishan CHEN and Yang ZHOU and Xuan DONG, (2019). Evaluation of the subseasonal forecast skill of surface soil moisture in the S2S database. Atmospheric and Oceanic Science Letters, 12, 6, 467-474, https://doi.org/10.1080/16742834.2019.1663123

· Kim, H., Janiga, M. A., & Pegion, K. ( 2019). MJO propagation processes and mean biases in the SubX and S2S reforecasts. Journal of Geophysical Research: Atmospheres, 124, 9314– 9331. https://doi.org/10.1029/2019JD031139

· Rao, J., Ren, R., Chen, H., Liu, X., Yu, Y., Hu, J., & Zhou, Y. ( 2019). Predictability of stratospheric sudden warmings in the Beijing Climate Center Forecast System with statistical error corrections. Journal of Geophysical Research: Atmospheres, 124, 8385– 8400. https://doi.org/10.1029/2019JD030900

· Olaniyan, E., Adefisan, E. A., Balogun, A. A., & Lawal, K. A. (2019). The influence of global climate drivers on monsoon onset variability in Nigeria using S2S models. Modeling Earth Systems and Environment, 1-24.