dstuncrusher/README.md

# dstuncrusher - Summer Time (DST) Uncrushing Tool for Cumulative Meter Readings
A simple approach for meter readings in time series:
- Are the times recorded with Daylight Savings Time (DST) and naively compress a week of (cumulative) values into the first day of winter time? 
- There will be a jump in the cumulative value in the first step of winter time in this case.

For this case, it removes the timezone signature (if any) and subtracts an hour from records in summer time to convert back to standard time.

It will leave the gap created in the old transition from winter to summer time as this gap should be addressed through imputation afterwards to fill the missing values using the jump in cumulative values. 

Useful for some applications that take meter readings from dashboards that incorrectly store timestamps/signatures through application of summer time and crushing the last week into the first value of winter time.

## Functions

### 1. `create_test_data()`
- Creates a test dataset in UTC with continuous 15-minute intervals.
- Converts the dataset to Europe/Amsterdam timezone.
- Returns a prepared dataframe for testing.

### 2. `get_exact_dst_transitions(year, timezone)`
- Calculates the exact DST transitions for a given year and timezone.
- Returns the start and end times of DST (summer time).

### 3. `correct_summer_time_with_timezone_check(df, datetime_col, timezone)`
- Corrects the dataset by:
  - Checking if the dataset has timezone-aware data, logging and removing the timezone signature if needed.
  - Dynamically calculating the DST transitions for each year and applying a 1-hour subtraction to return times to standard time.

## Example Usage

Run the script with the following steps:

1. Create test data using `create_test_data()`.
2. Correct summer time and remove timezone using `correct_summer_time_with_timezone_check()`.
3. Print the results for the DST transition periods (March and October).

# disclaimer

This was completely thought up by Nicolas Dickinson but written with the help of chatgpt 4o. FYI, it strugged greatly with timezone aware and naive datasets and I had to help substantially to get to the right solution and right tests.