Time for timeliness

Time for timeliness

ProescholdtTimo -
回帖数：6

Dear WIGOS community,

the team is happy to announce that a preview of the timeliness monitoring category is available for your feedback in the development system.

As you can see, there are 4 categories, corresponding to the 15, 30 and 120 minute thresholds. There is also a symbol indicating that a station has reported "negative timeliness" (meaning the observation time is after the reception time).
The thresholds are based on the thresholds for global land high-res NWP, and we think that they meaningfully partition the data. In the requirements database, the 30 and 120 minute thresholds are the same for pressure, temperature, humidity and wind, and the 15 minutes seem to be the best upper-threshold for the possible goal values of 6,10 and 15 minutes. (lower values not providing a good partitioning of the data)

The timeliness definition ET-ODQMS agreed for the timeliness template is "reception time - observation time". For WIS2, reception time is equivalent to the time the observation was published on the first WIS2 Global Broker. The data in the system is currently based on ECMWF provided data.

Aggregations are available for 6h, daily and monthly periods. The values is the mean over all non-negative timeliness values from the station in the period. Thus, negative timeliness and empty values do not contribute to the timeliness value. However, if a station has reported one or more negative timeliness values in the period, it is displayed with an orange border.

Timeliness could be released as WDQMS v1.6 once another center operationally exchanges timeliness, completing availability and quality, thus providing full coverage of the vectors in which requirements are expressed by the community.

with kind regards
the WDQMS development team
回复ProescholdtTimo

Re: Time for timeliness

MonnikKarl -
Dear Timo and team

Thank you for this enhancement. I have had only a cursory scan of the data.

It provides valuable insight to the performance of our GTS/WIS system - in Australia I can see distinct state based differences, which will be good to investigate.

I do see this station in the red ring indicating negative times, but the time series graph doesn't indicate it. Could you check?
e.g. WIGOS-ID: 0-20000-0-94668 (daily, temperature, 2025-05-15)

The light blue colour for Between 15 and 30 minutes is very similar as the background ocean colour - could you try another colour? Look at the offshore AWS off Australia/Queensland.

Thank you again for another useful enhancement.

Kind regards
Karl
回复ProescholdtTimo

Re: Time for timeliness

ProescholdtTimo -

Dear Karl, others who provided feedback by email and community,

the release candidate of SYNOP timeliness is now available for test in https://wdqms-test.wmo.int . The proposed release contains the a new monitoring category "timeliness" for the six-hour, daily and monthly periods for the SYNOP NWP category. Timeliness is the difference between the reception time(*) and the observation time. Detailed timeseries allow to see actual and average values over time by station. The release candidate includes your feedback shared earlier. As you can see, the contrast of the light-blue has been increased. The single center 6-hour timeseries (accessible via the popup) contains a timeseries of actual as well as 6h aggregate timeliness values. (combined mode and daily/monthly aggregations only showing aggregate values in the series)
We also improved the timeseries charts, particularly to be able to show a wide range of values (log-scale) and remove outliers (exceeding the interval of [-60,720] minutes).

Timeliness is so far based on data reported by DWD and ECMWF. In combined mode the center with the smallest timeliness determines the color and whether a red marker ring (indicating "negative timeliness") is shown. While the timeliness data from DWD and ECMWF is now stable and has been quality controlled, there are still rare instances of extreme timeliness values which are related to internal processing. WDQMS does not include these into the calculation of the mean, but displays them in the time-series. Centers are monitoring these and will fix issues as they are detected.

The release also contains major non-functional improvements in the software architecture. We would appreciate if you could also test the system in general in addition to the timeliness functions.

If no major bugs are discovered during testing, the release of WDQMS v1.6 is expected for next Wednesday 18/6/2025. An update of the user manual will be available by that time, too.

The release of timeliness is a major step for WDQMS in that now all of availability, quality and timeliness are in the system. Making timeliness available will allow to identify and hopefully resolve timing related issues in the GTS/WIS. It will also allow to use timeliness in the context of GBON.

(*) The manual will document the exact definition of reception time, by center.


with kind regards,
WDQMS development team

回复ProescholdtTimo

Re: Time for timeliness

ProescholdtTimo -
Dear WIGOS community,

thank you for your testing of the timeliness release.

We have realized that additional time is needed for quality control and decided to postpone the release planned for tomorrow.

We are planning to release timeliness next week, once full QC has been performed on metric reported by centers as well as the front-end.

with kind regards,
WDQMS development team
回复ProescholdtTimo

Re: Time for timeliness

MonnikKarl -
Dear Timo and WDQMS development team

Thanks for your ongoing work on this. The colour of dots over oceans are much better.

1) Could you change the colour of the negative timeliness ring to something like green or yellow? A red ring on a red dot is difficult to distinguish.
2) If the Legend 'reporting negative timeliness' item is clicked, then positive timeliness stations disappear. This is the opposite action to the other items in the legend which disappear when the legend item is clicked. In a way it is what is needed, hiding the positive timeliness stations. Maybe it can be worded differently.
3) at the monthly timescale, there is quite discrepancy between ECMWF and DWD data. Looking at AUS, May 2025, Surface pressure, most station report negative timeliness for DWD and positive for ECMWF (only 8 negative). But the DWD values show as positive in the timeseries view.
4) When I download a file for timeliness for DWD & ECMWF surface pressure and daily and 2025-06-10, and sort the file, no negative timeliness values can be found.

Regards
Karl
回复MonnikKarl

Re: Time for timeliness

LINNERTSSamantha -
Dear Timo and WDQMS development team

Thank you for the work on the timeliness.

I presented the Timeliness to the WDQMS focal point at SAWS and the RWC-SA staff.
1. They want to know if it would be possible to use the same colours (availability and quality) for consistency in terms of timeliness.

2. Is there a way to have the stations with "Reporting negative timeliness" display on the map when the "positive timeliness" is switched off in the legend?

3. Is there a possibility of having an additional column in the data file for the number of negative timeliness?
name wigosid country code in OSCAR longitude latitude timeliness (sec) negative_timeliness variable date center



Thank you
Regards
Sam
回复LINNERTSSamantha

Re: Time for timeliness

ProescholdtTimo -
Dear Karl, Sam and all,

we deployed a new version in the test server last week addressing some of the issues raised here.
- The "negative timeliness" filter in the legend is now grey (disabled) by default. Clicking on it activates the filter and limits the visible stations to those having negative timeliness. We hope this is now expected behaviour.
- The actual timeliness values of a single station can be downloaded from the timeseries chart in the popup for 6h individual center. The general download via the map provides the same aggregate values as are the basis of the dots on the map.
- We added a new column "nr_negative_timeliness" to the regular download which represents the number of negative timeliness values from the station in the period in question.
- Timeseries and map are now aligned with respect to treatment of number of negative.

As to the color scheme. We think it would be better to use different colors for timeliness. WDQMS results are mainly communicated visually through the map. We think it is important that timeliness results are not confused with availability.

There is still some discrepancy between centers as regards the number of negative, as you rightly indicate. It is probably best to wait until full QC has been done on the data provided by centers before releasing in production, but this can be discussed further in the team.

with kind regards,
WDQMS dev team