How frequent is the update cycle?

[guidocioni]

Today I was goofing around with same stations and saw that some data is not up to date. For example this station https://meteostat.net/en/station/71765 only has data up to 23 June but on the official website https://climate.weather.gc.ca/climate_data/daily_data_e.html?hlyRange=2006-05-23%7C2021-06-27&dlyRange=2006-05-01%7C2021-06-27&mlyRange=2006-05-01%7C2006-12-01&StationID=44927&Prov=BC&urlExtension=_e.html&searchType=stnName&optLimit=yearRange&StartYear=1840&EndYear=2021&selRowPerPage=25&Line=7&searchMethod=contains&Month=6&Day=27&txtStationName=Lytton&timeframe=2&Year=2021 it goes to 27 June and on some other websites https://www.infoclimat.fr/observations-meteo/archives/28/juin/2021/lytton-rcs/71765.html up to yesterday 28 June.

What are the sources for these differences?

[clampr]

I actually didn't implement Environment Canada as a data source, yet. But it's one of the many things on my backlog 😆 If you like, feel free to add an import routine for Canada. 🇨🇦

[guidocioni]

Is the Canada source here https://dev.meteostat.net/sources.html not the same?

[clampr]

It is the same, but it's only referring to climate normals which I imported manually a few years ago ;)

[guidocioni]

Sorry to resuscitate this but I've been seeing this "issue" more and more so I just want to make sure that I understand how everything works.

How frequently data are ingested into the database?

I know that it depends on the source but, for example, station 60735, which is a standard SYNOP station, has been reporting up to today (https://www.infoclimat.fr/observations-meteo/temps-reel/kairouan/60735.html) but on meteostat it is stuck on 6 Aug (https://meteostat.net/en/station/60735?t=2021-08-02/2021-08-12) for daily data. If I switch to hourly data it seems to be updated but many hours are missing every day. Is this normal?

What is the source for WMO SYNOP stations?

Moreover if I download the daily and hourly data in CSV directly from the web interface they contain data up to 16 Aug, (because I guess it takes model=True as default) but also old data, which is available in the web interface and seems the observed one, seems to be interpolated as it does not resemble the observed values. I know that the model is enabled by default but in occasion like this I do really think it is confusing to interpret the output values.

[clampr]

I understand it's confusing to have real observations and model data mixed together by default. I see these questions popping up all the time 😆 I just struggle disabling model data by default because I fear people will bounce off if many stations contain gaps in the time series.

Surely, we could fix that by investing more effort in the integration of additional data sources. The synop data, for example, is currently coming from DWD, but as far as I can tell, it's pretty incomplete. Also, at some point I will need help with the integration, as I just have too much stuff on my plate 😅 Maybe I need to invest more time in documenting and explaining how people can integrate data sources themselves..