nickdickinson/dstuncrusher
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
A simple approach for meter readings in time series with the time recorded with Daylight Savings Time (DST) and naively compress a week of (cumulative) values into the first day of winter time. For this case, this script takes a dataset with DST applied and removes it be subtracting an hour from records in summer time. It will leave the gap created in the old transition from winter to summer time as this gap is addressed through imputation to fill the missing values with the jump in the cumulative value in the first step of winter time. Useful for some applications that take meter readings from dashboards that incorrectly store timestamps/signatures.
2 Commits 1 Branch 0 Tags 28 KiB
Python 100%
f48d0dd756
Go to file
Nicolas Dickinson f48d0dd756 Script committed
2024-09-12 12:39:39 +02:00
.gitignore Initial commit 2024-09-12 12:20:34 +02:00
LICENSE Initial commit 2024-09-12 12:20:34 +02:00
README.md Script committed 2024-09-12 12:39:39 +02:00
summer_time_correction.py Script committed 2024-09-12 12:39:39 +02:00

README.md

dstcrusher - Summer Time (DST) Uncrushing Tool for Cumulative Meter Readings

A simple approach for meter readings in time series:

  • Are the times recorded with Daylight Savings Time (DST) and naively compress a week of (cumulative) values into the first day of winter time?
  • There will be a jump in the cumulative value in the first step of winter time in this case.

For this case, it removes the timezone signature (if any) and subtracts an hour from records in summer time to convert back to standard time.

It will leave the gap created in the old transition from winter to summer time as this gap should be addressed through imputation afterwards to fill the missing values using the jump in cumulative values.

Useful for some applications that take meter readings from dashboards that incorrectly store timestamps/signatures through application of summer time and crushing the last week into the first value of winter time.

Functions

1. create_test_data()

  • Creates a test dataset in UTC with continuous 15-minute intervals.
  • Converts the dataset to Europe/Amsterdam timezone.
  • Returns a prepared dataframe for testing.

2. get_exact_dst_transitions(year, timezone)

  • Calculates the exact DST transitions for a given year and timezone.
  • Returns the start and end times of DST (summer time).

3. correct_summer_time_with_timezone_check(df, datetime_col, timezone)

  • Corrects the dataset by:
    • Checking if the dataset has timezone-aware data, logging and removing the timezone signature if needed.
    • Dynamically calculating the DST transitions for each year and applying a 1-hour subtraction to return times to standard time.

Example Usage

Run the script with the following steps:

  1. Create test data using create_test_data().
  2. Correct summer time and remove timezone using correct_summer_time_with_timezone_check().
  3. Print the results for the DST transition periods (March and October).

disclaimer

This was completely thought up by Nicolas Dickinson but written with the help of chatgpt 4o. FYI, it strugged greatly with timezone aware and naive datasets and I had to help substantially to get to the right solution and right tests.