Deploy from GitHub

.streamlit/config.toml

@@ -0,0 +1,9 @@
+[theme]
+base="light"
+primaryColor="#FF4B4B"
+backgroundColor="#f8f9fa"
+textColor="#004080"
+secondaryBackgroundColor="#edf1f7"
+font="sans serif"
+
+

README.md

@@ -0,0 +1,296 @@
+# Energy Forecasting with Transformer and LightGBM
+
+This project focuses on forecasting urban energy consumption based solely on historical usage and temperature data from Chicago (2011–2018). Two model architectures are compared: a LightGBM ensemble model and a Transformer-based neural network (based on the Moments Time Series Transformer). The goal is to predict hourly electricity demand and analyze model performance, interpretability, and generalizability.
+
+The project also simulates a real-time setting, where hourly predictions are made sequentially to mirror operational deployment. The modular design allows for adaptation to other urban contexts, assuming a compatible data structure.
+
+---
+
+## Overview
+
+* **Goal**: Predict hourly energy consumption using timestamp, temperature, and historical consumption features.
+* **Models**: LightGBM and Time Series Transformer Model (moements).
+* **Results**: Both models perform well; LightGBM achieves the best overall performance.
+* **Dashboard**: Live forecast simulation via Streamlit interface.
+* **Usage Context**: Developed as a prototype for real-time hourly forecasting, with a modular structure that supports adaptation to similar operational settings.
+
+---
+
+## Results
+
+### Evaluation Metrics
+
+| Model       | RMSE    | R²    | MAPE   |
+| ----------- | ------- | ----- | ------ |
+| Transformer | 3933.57 | 0.972 | 2.32 % |
+| LightGBM    | 1383.68 | 0.996 | 0.84 % |
+
+> **Note:** All values are in megawatts (MW). Hourly consumption typically ranges from 100,000 to 200,000 MW.
+
+* LightGBM achieves the best trade-off between performance and resource efficiency.  
+* The Transformer model generalizes well to temporal patterns and may scale better in more complex or multi-network scenarios.  
+* Both models show no signs of overfitting, supported by learning curves, consistent evaluation metrics, and additional diagnostics such as residual distribution analysis and noise-feature validation.
+
+---
+
+### Forecast Plots
+
+| LightGBM Prediction Plot | Transformer Prediction Plot |
+| :----------------------: | :--------------------------: |
+| ![LightGBM Prediction](assets/lightgbm_prediction_with_timestamp.png) | ![Transformer Prediction](assets/comparison_plot_1month.png) |
+
+> **Note:** Example forecast windows are shown (LightGBM: 3 months, Transformer: 1 month).  
+> LightGBM maintains highly consistent performance over time, while the Transformer shows occasional over- or underestimation on special peak days.
+
+---
+
+### Learning Curves
+
+These plots visualize training dynamics and help detect overfitting.
+
+| LightGBM Learning Curve | Transformer Learning Curve |
+| :----------------------: | :------------------------: |
+| ![LightGBM LC](assets/lightgbm_learning_curve.png) | ![Transformer LC](assets/training_plot.png) |
+
+* The LightGBM curve shows a stable gap between training and validation RMSE, indicating low overfitting.  
+* The Transformer learning curve also converges smoothly without divergence, supporting generalizability.  
+* In addition to visual inspection, further checks like residual analysis and a noise feature test confirmed robustness.
+
+> **Note:** The LightGBM curve shows boosting rounds with validation RMSE,  
+> while the Transformer plot tracks training loss and test metrics per epoch.
+
+More plots are available in the respective `/results` directories.
+
+---
+
+## Streamlit Simulation Dashboard
+
+* Live hourly forecast simulation
+* Uses the trained models
+* Repeats predictions sequentially for each hour to simulate real-time data flow
+* Hosted on Hugging Face (CPU only, slower prediction speed)
+
+You can try the model predictions interactively in the Streamlit dashboard:
+
+**Try it here:**
+**[Launch Streamlit App](https://huggingface.co/spaces/dlaj/energy-forecasting-demo)**
+
+**Preview:**
+
+![Streamlit Dashboard Preview](assets/streamlit_preview.gif)
+
+---
+
+## Data
+
+* **Source**:
+
+  * [COMED Hourly Consumption Data](https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption)
+  * [NOAA Temperature Data](https://www.ncei.noaa.gov/)
+* **Time range**: January 2011 – August 2018
+* **Merged file**: `data/processed/energy_consumption_aggregated_cleaned.csv`
+
+---
+
+## Feature Engineering
+
+The models rely on timestamp and temperature data, enriched with derived time-based and lag-based features:
+
+* hour\_sin, hour\_cos
+* weekday\_sin, weekday\_cos
+* month\_sin, month\_cos
+* rolling\_mean\_6h
+* temperature\_c
+* consumption\_last\_hour
+* consumption\_yesterday
+* consumption\_last\_week
+
+Feature selection was guided by LightGBM feature importance analysis. Weak features with nearly no impact like "is_weekend" were deleted.
+
+### Final LightGBM Feature Importance
+ 
+<img src="assets/lightgbm_feature_importance.png" alt="Feature Importance" style="width: 80%;"/>
+
+---
+
+## Model Development
+
+### LightGBM
+
+* Custom grid search with over 50 parameter combinations
+* Parameters tested:
+
+  * num\_leaves, max\_depth, learning\_rate, lambda\_l1, lambda\_l2, min\_split\_gain
+* Final Parameters:
+
+  * learning\_rate: 0.05
+  * num\_leaves: 15
+  * max\_depth: 5
+  * lambda\_l1: 1.0
+  * lambda\_l2: 0.0
+  * min\_split\_gain: 0.0
+  * n\_estimators: 1000
+  * objective: regression
+
+Overfitting was monitored using a noise feature and RMSE gaps. See grid search results:
+`notebooks/lightgbm/lightgbm_gridsearch_results.csv`
+
+### Transformer (Moments Time Series Transformer)
+
+* Based on pretrained Moments model
+* Fine-tuned only the forecasting head for regular training
+* Also tested variants with unfrozen encoder layers and dropout
+* Final config:
+
+  * task\_name: forecasting
+  * forecast\_horizon: 24
+  * head\_dropout: 0.1
+  * weight\_decay: 0
+  * freeze\_encoder: True
+  * freeze\_embedder: True
+  * freeze\_head: False
+
+---
+
+## Project Structure
+
+```
+energy-forecasting-transformer-lightgbm/
+├── data/                   # Raw, external, processed datasets
+├── notebooks/              # EDA, lightgbm and transformer prototypes, including hyperparameter tuning and model selection
+├── scripts/                # Data preprocessing scripts
+├── lightgbm_model/         # LightGBM model, scripts, results
+├── transformer_model/      # Transformer model, scripts, results
+├── streamlit_simulation/   # Streamlit dashboard
+├── requirements.txt        # Main environment
+├── requirements_lgbm.txt   # Optional for LightGBM
+├── setup.py
+└── README.md
+```
+
+---
+
+## Reproducibility
+
+You can reuse this pipeline with any dataset, as long as it contains the following key columns:
+
+```csv
+timestamp,       # hourly timestamp (e.g., "2018-01-01 14:00")
+consumption,     # energy usage (aggregated; for individual users, consider adding an ID column)
+temperature      # hourly
+```
+
+### Notes:
+
+* Transformer model training is **very slow on CPU**, also with AMD GPU
+* Recommended: use **CUDA or Google Colab + CUDA GPU runtime** for transformer training
+* All scripts are modular and can be executed separately
+
+---
+
+## CI/CD & DevOps Setup
+
+This project includes a lightweight CI pipeline using GitHub Actions:
+
+* **CI**:  
+  - Runs `pytest` on every push  
+  - Builds and validates the Docker image
+
+* **Code quality checks**:  
+  - Uses `pre-commit` hooks with `black`, `isort`, and `ruff`  
+  - Ensures consistent formatting and linting before commits
+
+To enable pre-commit locally:
+
+```bash
+pre-commit install
+```
+
+---
+
+## Run Locally
+
+### Prerequisites
+
+* Python 3.9–3.11 (required for Moments Transformer)
+
+### Installation
+
+```bash
+git clone https://github.com/dlajic/energy-forecasting-transformer-lightgbm.git
+cd energy-forecasting-transformer-lightgbm
+pip install -r requirements.txt
+```
+
+### Preprocess Data
+
+```bash
+python -m scripts.data_preprocessing.merge_temperature_data  # merges raw temperature and energy data (only needed with raw inputs)
+python -m scripts.data_preprocessing.preprocess_data         # launches full preprocessing pipeline; use if data already matches expected format
+```
+
+### Train Models
+
+```bash
+python -m lightgbm_model.scripts.train.train_lightgbm
+python -m transformer_model.scripts.training.train
+```
+
+### Evaluate Models
+
+```bash
+python -m lightgbm_model.scripts.eval.eval_lightgbm
+python -m transformer_model.scripts.evaluation.evaluate
+python -m transformer_model.scripts.evaluation.plot_learning_curves
+```
+
+### Run Streamlit Dashboard (local)
+
+```bash
+streamlit run streamlit_simulation/app.py
+```
+
+For editable install:
+
+```bash
+pip install -e .
+```
+
+## Run App with Docker
+
+This project also supports containerized execution using Docker:
+
+
+```bash
+# Start app with Docker Compose (Linux)
+./start.sh
+
+# Or on Windows (PowerShell)
+./start.ps1
+```
+
+Make sure Docker (Docker-Desktop) is running before executing the script.
+
+This will:
+
+1. Build the Docker image
+2. Start the Streamlit app on localhost:8501
+3. Open it automatically in your browser
+
+---
+
+## Author
+
+Dean Lajic
+GitHub: [dlajic](https://github.com/dlajic)
+
+---
+
+## References
+
+- Moments Time Series Transformer  
+  https://github.com/moment-timeseries-foundation-model/moment
+- COMED Consumption Dataset  
+  https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption
+- NOAA Weather Data  
+  https://www.ncei.noaa.gov

lightgbm_model/scripts/__init__.py

@@ -0,0 +1 @@
+# __init__.py

lightgbm_model/scripts/config_lightgbm.py

@@ -0,0 +1,41 @@
+# config.py
+import os
+
+# === Paths ===
+BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+DATA_PATH = os.path.join(
+    BASE_DIR, "..", "data", "processed", "energy_consumption_aggregated_cleaned.csv"
+)
+RESULTS_DIR = os.path.join(BASE_DIR, "results")
+MODEL_DIR = os.path.join(BASE_DIR, "model")
+
+# === Feature-Definition ===
+FEATURES = [
+    "hour_sin",
+    "hour_cos",
+    "weekday_sin",
+    "weekday_cos",
+    "rolling_mean_6h",
+    "month_sin",
+    "month_cos",
+    "temperature_c",
+    "consumption_last_week",
+    "consumption_yesterday",
+    "consumption_last_hour",
+]
+TARGET = "consumption_MW"
+
+# === Hyperparameters fpr LightGBM ===
+LIGHTGBM_PARAMS = {
+    "learning_rate": 0.05,
+    "num_leaves": 15,
+    "max_depth": 5,
+    "lambda_l1": 1.0,
+    "lambda_l2": 0.0,
+    "min_split_gain": 0.0,
+    "n_estimators": 1000,
+    "objective": "regression",
+}
+
+# === Early Stopping ===
+EARLY_STOPPING_ROUNDS = 50

lightgbm_model/scripts/eval/eval_lightgbm.py

@@ -0,0 +1,156 @@
+# eval_model.py
+
+import json
+import os
+import pickle
+
+import matplotlib.pyplot as plt
+import numpy as np
+import pandas as pd
+from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
+
+from lightgbm_model.scripts.config_lightgbm import DATA_PATH, RESULTS_DIR
+from lightgbm_model.scripts.utils import load_lightgbm_model
+
+# === Ergebnisse-Ordner vorbereiten ===
+os.makedirs(RESULTS_DIR, exist_ok=True)
+
+# === Modell und eval_result laden ===
+# Modell laden
+model = load_lightgbm_model()
+
+# Eval laden
+with open(os.path.join(RESULTS_DIR, "lightgbm_eval_result.pkl"), "rb") as f:
+    eval_result = pickle.load(f)
+X_train = pd.read_csv(os.path.join(RESULTS_DIR, "X_train.csv"))
+X_test = pd.read_csv(os.path.join(RESULTS_DIR, "X_test.csv"))
+y_test = pd.read_csv(os.path.join(RESULTS_DIR, "y_test.csv"))
+
+# === Lernkurve ===
+train_rmse = eval_result["training"]["rmse"]
+valid_rmse = eval_result["valid_1"]["rmse"]
+
+plt.figure(figsize=(10, 5))
+plt.plot(train_rmse, label="Train RMSE")
+plt.plot(valid_rmse, label="Valid RMSE")
+plt.axvline(model.best_iteration_, color="gray", linestyle="--", label="Best Iteration")
+plt.xlabel("Boosting Round")
+plt.ylabel("RMSE")
+plt.title("LightGBM Learning Curve")
+plt.legend()
+plt.tight_layout()
+plt.savefig(os.path.join(RESULTS_DIR, "lightgbm_learning_curve.png"))
+# plt.show()
+
+# === Metriken berechnen ===
+y_pred = model.predict(X_test)
+mae = mean_absolute_error(y_test, y_pred)
+rmse = np.sqrt(mean_squared_error(y_test, y_pred))
+mape = (
+    np.mean(
+        np.abs(
+            (y_test.values.flatten() - y_pred)
+            / np.where(y_test.values.flatten() == 0, 1e-10, y_test.values.flatten())
+        )
+    )
+    * 100
+)
+r2 = r2_score(y_test, y_pred)
+
+print(f"Test MAPE: {mape:.5f} %")
+print(f"Test MAE: {mae:.5f}")
+print(f"Test RMSE: {rmse:.5f}")
+print(f"Test R2: {r2:.5f}")
+
+metrics = {
+    "model": "LightGBM",
+    "MAE": round(mae, 2),
+    "RMSE": round(rmse, 2),
+    "MAPE (%)": round(mape, 2),
+    "R2": round(r2, 4),
+    "unit": "MW",
+}
+
+# Pfad setzen
+output_path = os.path.join(RESULTS_DIR, "evaluation_metrics_lightgbm.json")
+# Speichern
+with open(output_path, "w") as f:
+    json.dump(metrics, f, indent=4)
+
+print(f"Metriken gespeichert unter {output_path}")
+
+# === Feature Importance ===
+feature_importance = pd.DataFrame(
+    {"Feature": X_train.columns, "Importance": model.feature_importances_}
+).sort_values(by="Importance", ascending=False)
+
+plt.figure(figsize=(10, 6))
+plt.barh(feature_importance["Feature"], feature_importance["Importance"])
+plt.xlabel("Feature Importance")
+plt.title("LightGBM Feature Importance")
+plt.gca().invert_yaxis()
+plt.tight_layout()
+plt.savefig(os.path.join(RESULTS_DIR, "lightgbm_feature_importance.png"))
+# plt.show()
+
+# === Vergleichsplots ===
+results_df = pd.DataFrame(
+    {
+        "True Consumption (MW)": y_test.values.flatten(),
+        "Predicted Consumption (MW)": y_pred,
+    }
+)
+
+# Timestamps anhängen
+full_df = pd.read_csv(DATA_PATH)
+test_dates = full_df.iloc[int(len(full_df) * 0.8) :]["date"].reset_index(drop=True)
+results_df["Timestamp"] = pd.to_datetime(test_dates)
+
+# Voller Plot
+plt.figure(figsize=(15, 6))
+plt.plot(
+    results_df["Timestamp"],
+    results_df["True Consumption (MW)"],
+    label="True",
+    color="darkblue",
+)
+plt.plot(
+    results_df["Timestamp"],
+    results_df["Predicted Consumption (MW)"],
+    label="Predicted",
+    color="red",
+    linestyle="--",
+)
+plt.title("Predicted vs True Consumption")
+plt.xlabel("Timestamp")
+plt.ylabel("Consumption (MW)")
+plt.legend()
+plt.tight_layout()
+plt.savefig(os.path.join(RESULTS_DIR, "lightgbm_comparison_plot.png"))
+# plt.show()
+
+# Subset Plot
+subset = results_df.iloc[: len(results_df) // 10]
+plt.figure(figsize=(15, 6))
+plt.plot(
+    subset["Timestamp"], subset["True Consumption (MW)"], label="True", color="darkblue"
+)
+plt.plot(
+    subset["Timestamp"],
+    subset["Predicted Consumption (MW)"],
+    label="Predicted",
+    color="red",
+    linestyle="--",
+)
+plt.title("Predicted vs True (First decile)")
+plt.xlabel("Timestamp")
+plt.ylabel("Consumption (MW)")
+plt.legend()
+plt.tight_layout()
+plt.savefig(os.path.join(RESULTS_DIR, "lightgbm_prediction_with_timestamp.png"))
+# plt.show()
+
+
+# === Ens message ===
+print("\nEvaluation completed.")
+print(f"All Plots stored in:\n→ {RESULTS_DIR}")

lightgbm_model/scripts/model_loader_wrapper.py

@@ -0,0 +1,11 @@
+from lightgbm_model.scripts.utils import load_lightgbm_model as real_model
+from streamlit_simulation.utils.env import use_dummy
+
+
+def load_lightgbm_model():
+    if use_dummy():
+        from streamlit_simulation.utils.dummy import DummyLightGBMModel
+
+        return DummyLightGBMModel()
+    else:
+        return real_model()

lightgbm_model/scripts/train/train_lightgbm.py

@@ -0,0 +1,66 @@
+# train_lightgbm.py
+
+import os
+import pickle
+
+import pandas as pd
+from lightgbm import LGBMRegressor, early_stopping, record_evaluation
+
+from lightgbm_model.scripts.config_lightgbm import (DATA_PATH,
+                                                    EARLY_STOPPING_ROUNDS,
+                                                    FEATURES, LIGHTGBM_PARAMS,
+                                                    MODEL_DIR, RESULTS_DIR,
+                                                    TARGET)
+
+# === Load Data ===
+df = pd.read_csv(DATA_PATH)
+
+# Drop date (used later for plots only)
+df = df.drop(columns=["date"], errors="ignore")
+
+# === Time-based Split (70% train, 10% valid, 20% test) ===
+train_size = int(len(df) * 0.7)
+valid_size = int(len(df) * 0.1)
+df_train = df.iloc[:train_size]
+df_valid = df.iloc[train_size : train_size + valid_size]
+df_test = df.iloc[train_size + valid_size :]
+
+X_train, y_train = df_train[FEATURES], df_train[TARGET]
+X_valid, y_valid = df_valid[FEATURES], df_valid[TARGET]
+X_test, y_test = df_test[FEATURES], df_test[TARGET]
+
+
+# === Init LightGBM model ===
+eval_result = {}
+
+model = LGBMRegressor(**LIGHTGBM_PARAMS, verbosity=-1)
+
+model.fit(
+    X_train,
+    y_train,
+    eval_set=[(X_train, y_train), (X_valid, y_valid)],
+    eval_metric="rmse",
+    callbacks=[early_stopping(EARLY_STOPPING_ROUNDS), record_evaluation(eval_result)],
+)
+
+# === Save model ===
+os.makedirs(MODEL_DIR, exist_ok=True)
+model_path = os.path.join(MODEL_DIR, "lightgbm_final_model.pkl")
+
+with open(model_path, "wb") as f:
+    pickle.dump(model, f)
+
+# === Save evaluation results ===
+os.makedirs(RESULTS_DIR, exist_ok=True)
+eval_result_path = os.path.join(RESULTS_DIR, "lightgbm_eval_result.pkl")
+
+with open(eval_result_path, "wb") as f:
+    pickle.dump(eval_result, f)
+
+print(f"Model saved to: {model_path}")
+print(f"Eval results saved to: {eval_result_path}")
+
+# === Save data for evaluation ===
+X_train.to_csv(os.path.join(RESULTS_DIR, "X_train.csv"), index=False)
+X_test.to_csv(os.path.join(RESULTS_DIR, "X_test.csv"), index=False)
+y_test.to_csv(os.path.join(RESULTS_DIR, "y_test.csv"), index=False)

lightgbm_model/scripts/utils.py

@@ -0,0 +1,9 @@
+import os
+import pickle
+
+MODEL_PATH = os.path.join("lightgbm_model", "model", "lightgbm_final_model.pkl")
+
+
+def load_lightgbm_model():
+    with open(MODEL_PATH, "rb") as f:
+        return pickle.load(f)

requirements.txt

@@ -0,0 +1,38 @@
+# =============================
+# Requirements for Energy Prediction Project
+# =============================
+
+# Python 3.11 environment recommended since moments dont work with later versions
+
+# Moment Foundation Model (forecasting backbone)
+momentfm @ git+https://github.com/moment-timeseries-foundation-model/moment.git@37a8bde4eb3dd340bebc9b54a3b893bcba62cd4f
+
+# === Core Python stack ===
+numpy==1.25.2                 # Numerical operations
+pandas==2.2.2                 # Data manipulation and analysis
+matplotlib==3.10.0           # Plotting and visualizations
+
+
+# === Machine Learning ===
+scikit-learn==1.6.1          # Evaluation metrics and preprocessing utilities
+torch==2.6.0           # PyTorch with CUDA 12.4 (GPU support)
+#torchvision==0.21.0    # Optional (can support visual tasks, not critical here)
+#torchaudio==2.6.0      # Optional (comes with torch install, can stay)
+
+# === Utilities ===
+tqdm==4.67.1                 # Progress bars
+ipywidgets>=8.0              # Enables tqdm progress bars in Jupyter/Colab
+pprintpp==0.4.0              # Prettier print formatting for nested dicts (used for model output check)
+
+# === lightgbm ===
+lightgbm==4.3.0     # Boosted Trees for tabular modeling (used for baseline and feature selection)
+
+# === Streamlit App ===
+streamlit>=1.30.0
+plotly>=5.0.0
+
+# === for pytest/env dummy/pre-commit/huggingface ====
+pytest
+python-dotenv
+pre-commit
+huggingface_hub

setup.py

@@ -0,0 +1,7 @@
+from setuptools import find_packages, setup
+
+setup(
+    name="energy_prediction",
+    version="0.1",
+    packages=find_packages(),
+)

streamlit_simulation/__init__.py

@@ -0,0 +1 @@
+# __init__.py

streamlit_simulation/app.py

@@ -0,0 +1,556 @@
+import time
+import warnings
+
+import matplotlib.dates as mdates
+import matplotlib.pyplot as plt
+import numpy as np
+import pandas as pd
+import streamlit as st
+import torch
+from config_streamlit import DATA_PATH, PLOT_COLOR, TRAIN_RATIO
+
+from lightgbm_model.scripts.config_lightgbm import FEATURES
+from lightgbm_model.scripts.model_loader_wrapper import load_lightgbm_model
+from streamlit_simulation.utils_streamlit import load_data as load_data_raw
+from transformer_model.scripts.config_transformer import (FORECAST_HORIZON,
+                                                          SEQ_LEN)
+from transformer_model.scripts.utils.informer_dataset_class import \
+    InformerDataset
+from transformer_model.scripts.utils.model_loader_wrapper import \
+    load_model_and_dataset
+
+# ============================== Layout ==============================
+
+# Streamlit & warnings config
+warnings.filterwarnings("ignore", category=FutureWarning)
+st.set_page_config(page_title="Electricity Consumption Forecast", layout="wide")
+
+# CSS part
+st.markdown(
+    f"""
+    <style>
+        .stButton > button {{
+            background-color: {PLOT_COLOR};
+        }}
+
+        /* Entfernt auch den leeren Platz über der App */
+        header[data-testid="stHeader"] {{
+            display: none !important;
+            height: 0px !important;
+            visibility: hidden !important;
+        }}
+
+        .block-container {{
+            padding-top: 0.5rem !important; 
+        }}
+
+    </style>
+""",
+    unsafe_allow_html=True,
+)
+
+
+st.title("Electricity Consumption Forecast: Hourly Simulation")
+st.write("Welcome to the simulation interface!")
+st.info(
+    "**Simulation Overview:**\n\n"
+    "This dashboard provides an hourly electricity consumption forecast using two different models: "
+    "**LightGBM** and a **Transformer (moment-based)**. Both models generate a fresh prediction at every time step "
+    "(i.e., every simulated hour).\n\n"
+    "Note: Since this app runs on a limited CPU on Hugging Face Spaces, the Transformer model may respond slower "
+    "compared to local execution. On a standard local CPU, performance is significantly better."
+)
+
+
+# ============================== Session State Init ===============================
+def init_session_state():
+    defaults = {
+        "is_running": False,
+        "start_index": 0,
+        "true_vals": [],
+        "pred_vals": [],
+        "true_timestamps": [],
+        "pred_timestamps": [],
+        "last_fig": None,
+        "valid_pos": 0,
+        "first_plot_shown": False,
+    }
+    for key, value in defaults.items():
+        if key not in st.session_state:
+            st.session_state[key] = value
+
+
+init_session_state()
+
+
+# ============================== Loaders Cache ==============================
+@st.cache_data
+def load_cached_lightgbm_model():
+    return load_lightgbm_model()
+
+
+@st.cache_resource
+def load_transformer_model_and_dataset():
+    return load_model_and_dataset()
+
+
+@st.cache_data
+def load_data():
+    return load_data_raw()
+
+
+# ============================== Utility Functions ==============================
+
+
+def predict_transformer_step(model, dataset, idx, device):
+    """Performs a single prediction step with the transformer model."""
+    timeseries, _, input_mask = dataset[idx]
+    timeseries = torch.tensor(timeseries, dtype=torch.float32).unsqueeze(0).to(device)
+    input_mask = torch.tensor(input_mask, dtype=torch.bool).unsqueeze(0).to(device)
+
+    with torch.no_grad():
+        output = model(x_enc=timeseries, input_mask=input_mask)
+
+    pred = output.forecast[:, 0, :].cpu().numpy().flatten()
+
+    # Rückskalieren
+    dummy = np.zeros((len(pred), dataset.n_channels))
+    dummy[:, 0] = pred
+    pred_original = dataset.scaler.inverse_transform(dummy)[:, 0]
+
+    return float(pred_original[0])
+
+
+def init_simulation_layout():
+    """Creates layout containers for plot and info sections."""
+    col1, spacer, col2 = st.columns([3, 0.2, 1])
+    plot_title = col1.empty()
+    plot_container = col1.empty()
+    x_axis_label = col1.empty()
+    info_container = col2.empty()
+    return plot_title, plot_container, x_axis_label, info_container
+
+
+def create_prediction_plot(
+    pred_timestamps,
+    pred_vals,
+    true_timestamps,
+    true_vals,
+    window_hours,
+    y_min=None,
+    y_max=None,
+):
+    """Generates the matplotlib figure for plotting prediction vs. actual."""
+    fig, ax = plt.subplots(
+        figsize=(8, 5), constrained_layout=True, facecolor=PLOT_COLOR
+    )
+    ax.set_facecolor(PLOT_COLOR)
+
+    ax.plot(
+        pred_timestamps[-window_hours:],
+        pred_vals[-window_hours:],
+        label="Prediction",
+        color="#EF233C",
+        linestyle="--",
+    )
+    if true_vals:
+        ax.plot(
+            true_timestamps[-window_hours:],
+            true_vals[-window_hours:],
+            label="Actual",
+            color="#0077B6",
+        )
+
+    ax.set_ylabel("Consumption (MW)", fontsize=8)
+    ax.legend(
+        fontsize=8,
+        loc="upper left",
+        bbox_to_anchor=(0, 0.95),
+        # facecolor= INPUT_BG,    # INPUT_BG
+        # edgecolor= ACCENT_COLOR,    # ACCENT_COLOR
+        # labelcolor= TEXT_COLOR    # TEXT_COLOR
+    )
+    ax.yaxis.grid(True, linestyle=":", linewidth=0.5, alpha=0.7)
+    ax.set_ylim(y_min, y_max)
+    ax.xaxis.set_major_locator(mdates.DayLocator(interval=1))
+    ax.xaxis.set_major_formatter(mdates.DateFormatter("%m-%d"))
+    ax.tick_params(axis="x", labelrotation=0, labelsize=5)
+    ax.tick_params(axis="y", labelsize=5)
+    # fig.patch.set_facecolor('#e6ecf0')  # outer area
+
+    for spine in ax.spines.values():
+        spine.set_visible(False)
+
+    st.session_state.last_fig = fig
+    return fig
+
+
+def render_simulation_view(timestamp, prediction, actual, progress, fig, paused=False):
+    """Displays the simulation plot and metrics in the UI."""
+    title = "Actual vs. Prediction (Paused)" if paused else "Actual vs. Prediction"
+    plot_title.markdown(
+        f"<div style='text-align: center; font-size: 20pt; font-weight: bold; margin-bottom: -0.7rem; margin-top: 0rem;'>"
+        f"{title}</div>",
+        unsafe_allow_html=True,
+    )
+    plot_container.pyplot(fig)
+
+    # st.markdown("<div style='margin-bottom: 0.5rem;'></div>", unsafe_allow_html=True)
+    # x_axis_label.markdown(f"<div style='text-align: center; font-size: 13pt; color: {TEXT_COLOR}; margin-top: -0.5rem;'>"f"Time</div>",unsafe_allow_html=True)
+
+    with info_container.container():
+        st.markdown(
+            f"<span style='font-size: 24px; font-weight: 600;'>Time: {timestamp}</span>",
+            unsafe_allow_html=True,
+        )
+        st.metric(
+            "Prediction", f"{prediction:,.0f} MW" if prediction is not None else "–"
+        )
+        st.metric("Actual", f"{actual:,.0f} MW" if actual is not None else "–")
+        st.caption("Simulation Progress")
+        st.progress(progress)
+
+        if len(st.session_state.true_vals) > 1:
+            true_arr = np.array(st.session_state.true_vals)
+            pred_arr = np.array(st.session_state.pred_vals[:-1])
+            min_len = min(len(true_arr), len(pred_arr))
+            if min_len >= 1:
+                errors = np.abs(true_arr[:min_len] - pred_arr[:min_len])
+                mape = (
+                    np.mean(
+                        errors
+                        / np.where(true_arr[:min_len] == 0, 1e-10, true_arr[:min_len])
+                    )
+                    * 100
+                )
+                mae = np.mean(errors)
+                max_error = np.max(errors)
+
+                st.divider()
+                st.markdown(
+                    "<span style='font-size: 24px; font-weight: 600; '>Interim Metrics</span>",
+                    unsafe_allow_html=True,
+                )
+                st.metric("MAPE (so far)", f"{mape:.2f} %")
+                st.metric("MAE (so far)", f"{mae:,.0f} MW")
+                st.metric("Max Error", f"{max_error:,.0f} MW")
+
+
+# ============================== Data Preparation ==============================
+
+df_full = load_data()
+
+# Split Train/Test
+train_size = int(len(df_full) * TRAIN_RATIO)
+test_df_raw = df_full.iloc[train_size:].reset_index(drop=True)
+
+# Start at first full hour (00:00)
+first_full_day_index = test_df_raw[
+    test_df_raw["date"].dt.time == pd.Timestamp("00:00:00").time()
+].index[0]
+test_df_full = test_df_raw.iloc[first_full_day_index:].reset_index(drop=True)
+
+# Select simulation window via date picker
+min_date = test_df_full["date"].min().date()
+max_date = test_df_full["date"].max().date()
+
+# ============================== UI Controls ==============================
+
+with st.sidebar:
+    st.header("⚙️ Simulation Settings")
+
+    st.subheader("General Settings")
+    model_choice = st.selectbox(
+        "Choose prediction model", ["LightGBM", "Transformer Model (moments)"]
+    )
+    if model_choice == "Transformer Model (moments)":
+        st.caption(
+            "⚠️ Note: Transformer model runs slower without GPU. (Use Speed = 10)"
+        )
+    window_days = st.selectbox("Display window (days)", options=[3, 5, 7], index=0)
+    window_hours = window_days * 24
+    speed = st.slider("Speed", 1, 10, 5)
+
+    st.subheader("Date Range")
+    start_date = st.date_input(
+        "Start Date", value=min_date, min_value=min_date, max_value=max_date
+    )
+    end_date = st.date_input(
+        "End Date", value=max_date, min_value=min_date, max_value=max_date
+    )
+
+# ============================== Data Preparation (filtered) ==============================
+
+# final filtered date window
+test_df_filtered = test_df_full[
+    (test_df_full["date"].dt.date >= start_date)
+    & (test_df_full["date"].dt.date <= end_date)
+].reset_index(drop=True)
+
+# For progression bar
+total_steps_ui = len(test_df_filtered)
+
+# ============================== Buttons ==============================
+
+st.markdown("### Start Simulation")
+col1, col2, col3 = st.columns([1, 1, 4])
+with col1:
+    play_pause_text = "▶️ Start" if not st.session_state.is_running else "⏸️ Pause"
+    if st.button(play_pause_text, use_container_width=True):
+        st.session_state.is_running = not st.session_state.is_running
+        st.rerun()
+with col2:
+    reset_button = st.button("🔄 Reset", use_container_width=True)
+
+# Reset logic
+if reset_button:
+    st.session_state.start_index = 0
+    st.session_state.pred_vals = []
+    st.session_state.true_vals = []
+    st.session_state.pred_timestamps = []
+    st.session_state.true_timestamps = []
+    st.session_state.last_fig = None
+    st.session_state.is_running = False
+    st.session_state.valid_pos = 0
+    st.session_state.first_plot_shown = False
+    st.rerun()
+
+# Auto-reset on critical parameter change while running
+if st.session_state.is_running and (
+    start_date != st.session_state.get("last_start_date")
+    or end_date != st.session_state.get("last_end_date")
+    or model_choice != st.session_state.get("last_model_choice")
+):
+    st.session_state.start_index = 0
+    st.session_state.pred_vals = []
+    st.session_state.true_vals = []
+    st.session_state.pred_timestamps = []
+    st.session_state.true_timestamps = []
+    st.session_state.last_fig = None
+    st.session_state.valid_pos = 0
+    st.session_state.first_plot_shown = False
+    st.rerun()
+
+# Track current selections for change detection
+st.session_state.last_start_date = start_date
+st.session_state.last_end_date = end_date
+st.session_state.last_model_choice = model_choice
+
+
+# ============================== Paused Mode ==============================
+
+if not st.session_state.is_running and st.session_state.last_fig is not None:
+    st.write("Simulation paused...")
+    plot_title, plot_container, x_axis_label, info_container = init_simulation_layout()
+
+    timestamp = (
+        st.session_state.pred_timestamps[-1]
+        if st.session_state.pred_timestamps
+        else "–"
+    )
+    prediction = st.session_state.pred_vals[-1] if st.session_state.pred_vals else None
+    actual = st.session_state.true_vals[-1] if st.session_state.true_vals else None
+    progress = st.session_state.start_index / total_steps_ui
+
+    render_simulation_view(
+        timestamp, prediction, actual, progress, st.session_state.last_fig, paused=True
+    )
+
+
+# ============================== initialize values ==============================
+
+# if lightGbm use testdata from above
+if model_choice == "LightGBM":
+    test_df = test_df_filtered.copy()
+
+# Shared state references for storing predictions and ground truths
+
+true_vals = st.session_state.true_vals
+pred_vals = st.session_state.pred_vals
+true_timestamps = st.session_state.true_timestamps
+pred_timestamps = st.session_state.pred_timestamps
+
+# ============================== LightGBM Simulation ==============================
+
+if model_choice == "LightGBM" and st.session_state.is_running:
+    model = load_cached_lightgbm_model()
+    st.write("Simulation started...")
+    st.markdown('<div id="simulation"></div>', unsafe_allow_html=True)
+
+    plot_title, plot_container, x_axis_label, info_container = init_simulation_layout()
+
+    for i in range(st.session_state.start_index, len(test_df)):
+        if not st.session_state.is_running:
+            break
+
+        current = test_df.iloc[i]
+        timestamp = current["date"]
+        features = current[FEATURES].values.reshape(1, -1)
+        prediction = model.predict(features)[0]
+
+        pred_vals.append(prediction)
+        pred_timestamps.append(timestamp)
+
+        if i >= 1:
+            prev_actual = test_df.iloc[i - 1]["consumption_MW"]
+            prev_time = test_df.iloc[i - 1]["date"]
+            true_vals.append(prev_actual)
+            true_timestamps.append(prev_time)
+
+        fig = create_prediction_plot(
+            pred_timestamps,
+            pred_vals,
+            true_timestamps,
+            true_vals,
+            window_hours,
+            y_min=test_df_filtered["consumption_MW"].min() - 2000,
+            y_max=test_df_filtered["consumption_MW"].max() + 2000,
+        )
+
+        render_simulation_view(
+            timestamp,
+            prediction,
+            prev_actual if i >= 1 else None,
+            i / len(test_df),
+            fig,
+        )
+
+        plt.close(fig)  # Speicher freigeben
+
+        st.session_state.start_index = i + 1
+        time.sleep(1 / (speed + 1e-9))
+
+    st.success("Simulation completed!")
+
+
+# ============================== Transformer Simulation ==============================
+
+spinner_placeholder = st.empty()
+
+if model_choice == "Transformer Model (moments)":
+    if st.session_state.is_running:
+        st.write("Simulation started (Transformer)...")
+        st.markdown('<div id="simulation"></div>', unsafe_allow_html=True)
+
+        if not st.session_state.first_plot_shown:
+            spinner_placeholder.markdown("Running first prediction – please wait...")
+
+        plot_title, plot_container, x_axis_label, info_container = (
+            init_simulation_layout()
+        )
+
+        # Zugriff auf Modell, Dataset, Device
+        model, test_dataset, device = load_transformer_model_and_dataset()
+        data = test_dataset.data  # bereits skaliert
+        scaler = test_dataset.scaler
+        n_channels = test_dataset.n_channels
+
+        test_start_idx = (
+            len(InformerDataset(data_split="train", forecast_horizon=FORECAST_HORIZON))
+            + SEQ_LEN
+        )
+        base_timestamp = pd.read_csv(DATA_PATH, parse_dates=["date"])["date"].iloc[
+            test_start_idx
+        ]  # get original timestamp for later, cause not in dataset anymore
+
+        # Schritt 1: Finde Index, ab dem Stunde = 00:00 ist
+        offset = 0
+        while (base_timestamp + pd.Timedelta(hours=offset)).time() != pd.Timestamp(
+            "00:00:00"
+        ).time():
+            offset += 1
+
+        # Neuer Startindex in der Simulation
+        start_index = offset
+
+        # Session-State bei Bedarf initial setzen
+        if "start_index" not in st.session_state or st.session_state.start_index == 0:
+            st.session_state.start_index = start_index
+
+        # Vorbereiten: Liste der gültigen i-Werte im gewünschten Zeitraum
+        valid_indices = []
+        for i in range(start_index, len(test_dataset)):
+            timestamp = base_timestamp + pd.Timedelta(hours=i)
+            if start_date <= timestamp.date() <= end_date:
+                valid_indices.append(i)
+
+        # Fortschrittsanzeige
+        total_steps = len(valid_indices)
+
+        # Aktueller Fortschritt in der Liste (nicht: globaler Dataset-Index!)
+        if "valid_pos" not in st.session_state:
+            st.session_state.valid_pos = 0
+
+        # Hauptschleife: Nur noch über gültige Indizes iterieren
+        for relative_idx, i in enumerate(valid_indices[st.session_state.valid_pos :]):
+
+            # for i in range(st.session_state.start_index, len(test_dataset)):
+            if not st.session_state.is_running:
+                break
+
+            current_pred = predict_transformer_step(model, test_dataset, i, device)
+            current_time = base_timestamp + pd.Timedelta(hours=i)
+
+            pred_vals.append(current_pred)
+            pred_timestamps.append(current_time)
+
+            if i >= 1:
+                prev_actual = test_dataset[i - 1][1][
+                    0, 0
+                ]  # erster Forecast-Wert der letzten Zeile
+                # Rückskalieren
+                dummy_actual = np.zeros((1, n_channels))
+                dummy_actual[:, 0] = prev_actual
+                actual_val = scaler.inverse_transform(dummy_actual)[0, 0]
+
+                true_time = current_time - pd.Timedelta(hours=1)
+
+                if true_time >= pd.to_datetime(start_date):
+                    true_vals.append(actual_val)
+                    true_timestamps.append(true_time)
+
+            # Plot erzeugen
+            fig = create_prediction_plot(
+                pred_timestamps,
+                pred_vals,
+                true_timestamps,
+                true_vals,
+                window_hours,
+                y_min=test_df_filtered["consumption_MW"].min() - 2000,
+                y_max=test_df_filtered["consumption_MW"].max() + 2000,
+            )
+            if len(pred_vals) >= 2 and len(true_vals) >= 1:
+                render_simulation_view(
+                    current_time,
+                    current_pred,
+                    actual_val if i >= 1 else None,
+                    st.session_state.valid_pos / total_steps,
+                    fig,
+                )
+                if not st.session_state.first_plot_shown:
+                    spinner_placeholder.empty()
+                    st.session_state.first_plot_shown = True
+
+            plt.close(fig)  # Speicher freigeben
+
+            st.session_state.valid_pos += 1
+            time.sleep(1 / (speed + 1e-9))
+
+        st.success("Simulation completed!")
+
+
+# ============================== Scroll Sync ==============================
+
+st.markdown(
+    """
+    <script>
+    window.addEventListener("message", (event) => {
+        if (event.data.type === "save_scroll") {
+            const pyScroll = event.data.scrollY;
+            window.parent.postMessage({type: "streamlit:setComponentValue", value: pyScroll}, "*");
+        }
+    });
+    </script>
+""",
+    unsafe_allow_html=True,
+)

streamlit_simulation/config_streamlit.py

@@ -0,0 +1,24 @@
+# config_streamlit
+import os
+
+# Base directory → points to the project root
+BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+
+# Model paths
+MODEL_PATH_LIGHTGBM = os.path.join(
+    BASE_DIR, "lightgbm_model", "model", "lightgbm_final_model.pkl"
+)
+MODEL_PATH_TRANSFORMER = os.path.join(
+    BASE_DIR, "transformer_model", "model", "checkpoints", "model_final.pth"
+)
+
+# Data path
+DATA_PATH = os.path.join(
+    BASE_DIR, "data", "processed", "energy_consumption_aggregated_cleaned.csv"
+)
+
+# Color palette for Streamlit layout
+PLOT_COLOR = "#edf1f7"  # Plot background color
+
+# Constants
+TRAIN_RATIO = 0.7  # Train/test split ratio used by both models

streamlit_simulation/utils/dummy.py

@@ -0,0 +1,43 @@
+# streamlit_simulation/dummy.py
+import numpy as np
+import torch
+
+
+class DummyDataset:
+    def __init__(self, length=100):
+        self.data = np.zeros((length, 10))  # Dummydaten
+        self.scaler = DummyScaler()
+        self.n_channels = 1
+        self.length = length
+
+    def __len__(self):
+        return self.length
+
+    def __getitem__(self, idx):
+        timeseries = np.zeros((48, 1))  # (SEQ_LEN, Channels)
+        target = np.zeros((1, 1))  # Forecast target
+        mask = np.ones((48,))  # Dummy-Maske
+        return timeseries, target, mask
+
+
+class DummyScaler:
+    def inverse_transform(self, x):
+        return x  # keine Skalierung nötig
+
+
+class DummyOutput:
+    def __init__(self, forecast_shape):
+        # gib einen echten Tensor zurück, wie vom echten Modell erwartet
+        self.forecast = torch.tensor(np.full(forecast_shape, 42.0), dtype=torch.float32)
+
+
+class DummyTransformerModel:
+    def __call__(self, x_enc, input_mask):
+        batch_size, seq_len, channels = x_enc.shape
+        forecast_shape = (batch_size, 1, channels)
+        return DummyOutput(forecast_shape)
+
+
+class DummyLightGBMModel:
+    def predict(self, X):
+        return np.zeros(len(X))  # ← gibt jetzt np.ndarray zurück

streamlit_simulation/utils/env.py

@@ -0,0 +1,9 @@
+import os
+
+from dotenv import load_dotenv
+
+load_dotenv()  # einmalig beim Import
+
+
+def use_dummy() -> bool:
+    return os.getenv("USE_DUMMY_MODEL", "false").lower() == "true"

streamlit_simulation/utils_streamlit.py

@@ -0,0 +1,29 @@
+import os
+
+import pandas as pd
+from huggingface_hub import hf_hub_download
+
+from streamlit_simulation.config_streamlit import DATA_PATH
+
+HF_REPO = "dlaj/energy-forecasting-files"
+HF_FILENAME = "data/processed/energy_consumption_aggregated_cleaned.csv"
+
+
+def load_data():
+    # Prüfe, ob lokale Datei existiert
+    if not os.path.exists(DATA_PATH):
+        print(f"Lokale Datei nicht gefunden: {DATA_PATH}")
+        print("Lade von Hugging Face...")
+
+        # Lade von HF Hub
+        downloaded_path = hf_hub_download(
+            repo_id=HF_REPO,
+            filename=HF_FILENAME,
+            repo_type="dataset",
+            cache_dir="hf_cache",  # Optional: lokaler Cache-Ordner
+        )
+
+        return pd.read_csv(downloaded_path, parse_dates=["date"])
+
+    print(f"Lade lokale Datei: {DATA_PATH}")
+    return pd.read_csv(DATA_PATH, parse_dates=["date"])

transformer_model/scripts/__init__.py

@@ -0,0 +1 @@
+# __init__.py

transformer_model/scripts/config_transformer.py

@@ -0,0 +1,33 @@
+# config.py
+import os
+
+# Base Directory
+BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+
+# Data paths
+DATA_PATH = os.path.join(
+    BASE_DIR, "..", "data", "processed", "energy_consumption_aggregated_cleaned.csv"
+)
+
+# Other  paths
+CHECKPOINT_DIR = os.path.join(BASE_DIR, "model", "checkpoints")
+RESULTS_DIR = os.path.join(BASE_DIR, "results")
+
+
+# ========== Model Settings ==========
+SEQ_LEN = 512  # Input sequence length (number of time steps the model sees)
+FORECAST_HORIZON = 1  # Number of future steps the model should predict
+HEAD_DROPOUT = 0.1  # Dropout in the head to prevent overfitting
+WEIGHT_DECAY = 0.0  # L2 regularization (0 means off)
+
+# ========== Training Settings ==========
+MAX_EPOCHS = 9  # Optimal number of epochs based on performance curve
+BATCH_SIZE = 32  # Batch size for training and evaluation
+LEARNING_RATE = 1e-4  # Base learning rate
+MAX_LR = 1e-4  # Max LR for OneCycleLR scheduler
+GRAD_CLIP = 5.0  # Gradient clipping threshold
+
+# ========== Freezing Strategy ==========
+FREEZE_ENCODER = True
+FREEZE_EMBEDDER = True
+FREEZE_HEAD = False  # just unfreeze the last forecasting head for finetuning

transformer_model/scripts/evaluation/__init__.py

@@ -0,0 +1 @@
+# __init__

transformer_model/scripts/evaluation/evaluate.py

@@ -0,0 +1,144 @@
+# evaluate.py
+
+import json
+import logging
+import os
+
+import numpy as np
+import pandas as pd
+import torch
+from momentfm.utils.utils import control_randomness
+from sklearn.metrics import mean_squared_error, r2_score
+from tqdm import tqdm
+
+from transformer_model.scripts.config_transformer import (DATA_PATH,
+                                                          FORECAST_HORIZON,
+                                                          RESULTS_DIR, SEQ_LEN)
+from transformer_model.scripts.utils.check_device import check_device
+from transformer_model.scripts.utils.informer_dataset_class import \
+    InformerDataset
+from transformer_model.scripts.utils.load_final_model import \
+    load_final_transformer_model
+
+# Setup logging
+logging.basicConfig(
+    level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
+)
+
+
+def evaluate():
+    control_randomness(seed=13)
+    # Set device
+    device, backend, scaler = check_device()
+    logging.info(f"Evaluation is running on: {backend} ({device})")
+
+    # Load final model
+    model, _ = load_final_transformer_model(device)
+
+    # Recreate training dataset to get the fitted scaler
+    train_dataset = InformerDataset(
+        data_split="train", random_seed=13, forecast_horizon=FORECAST_HORIZON
+    )
+
+    # Use its scaler in the test dataset
+    test_dataset = InformerDataset(
+        data_split="test", random_seed=13, forecast_horizon=FORECAST_HORIZON
+    )
+
+    test_dataset.scaler = train_dataset.scaler
+
+    test_loader = torch.utils.data.DataLoader(
+        test_dataset, batch_size=32, shuffle=False
+    )
+
+    trues, preds = [], []
+
+    with torch.no_grad():
+        for timeseries, forecast, input_mask in tqdm(
+            test_loader, desc="Evaluating on test set"
+        ):
+            timeseries = timeseries.float().to(device)
+            forecast = forecast.float().to(device)
+            input_mask = input_mask.to(device)  # <- wichtig!
+
+            output = model(x_enc=timeseries, input_mask=input_mask)
+
+            trues.append(forecast.cpu().numpy())
+            preds.append(output.forecast.cpu().numpy())
+
+    trues = np.concatenate(trues, axis=0)
+    preds = np.concatenate(preds, axis=0)
+
+    # Extract only first feature (consumption)
+    true_values = trues[:, 0, :]
+    pred_values = preds[:, 0, :]
+
+    # Inverse normalization
+    n_features = test_dataset.n_channels
+    true_reshaped = np.column_stack(
+        [true_values.flatten()]
+        + [np.zeros_like(true_values.flatten())] * (n_features - 1)
+    )
+    pred_reshaped = np.column_stack(
+        [pred_values.flatten()]
+        + [np.zeros_like(pred_values.flatten())] * (n_features - 1)
+    )
+
+    true_original = test_dataset.scaler.inverse_transform(true_reshaped)[:, 0]
+    pred_original = test_dataset.scaler.inverse_transform(pred_reshaped)[:, 0]
+
+    # Build timestamp index, since date got cutted out in informerdataset we need original dataset and use the index of the beginning of testdata to get the date
+    csv_path = os.path.join(DATA_PATH)
+    df = pd.read_csv(csv_path, parse_dates=["date"])
+
+    train_len = len(train_dataset)
+    test_start_idx = train_len + SEQ_LEN
+    start_timestamp = df["date"].iloc[test_start_idx]
+    logging.info(f"[DEBUG] timestamp: {start_timestamp}")
+
+    timestamps = [
+        start_timestamp + pd.Timedelta(hours=i) for i in range(len(true_original))
+    ]
+
+    df = pd.DataFrame(
+        {
+            "Timestamp": timestamps,
+            "True Consumption (MW)": true_original,
+            "Predicted Consumption (MW)": pred_original,
+        }
+    )
+
+    # Save results to CSV
+    os.makedirs(RESULTS_DIR, exist_ok=True)
+    results_path = os.path.join(RESULTS_DIR, "test_results.csv")
+    df.to_csv(results_path, index=False)
+    logging.info(f"Saved prediction results to: {results_path}")
+
+    # Evaluation metrics
+    mse = mean_squared_error(
+        df["True Consumption (MW)"], df["Predicted Consumption (MW)"]
+    )
+    rmse = np.sqrt(mse)
+    mape = (
+        np.mean(
+            np.abs(
+                (df["True Consumption (MW)"] - df["Predicted Consumption (MW)"])
+                / df["True Consumption (MW)"]
+            )
+        )
+        * 100
+    )
+    r2 = r2_score(df["True Consumption (MW)"], df["Predicted Consumption (MW)"])
+
+    # Save metrics to JSON
+    metrics = {"RMSE": float(rmse), "MAPE": float(mape), "R2": float(r2)}
+    metrics_path = os.path.join(RESULTS_DIR, "evaluation_metrics.json")
+    with open(metrics_path, "w") as f:
+        json.dump(metrics, f)
+
+    logging.info(f"Saved evaluation metrics to: {metrics_path}")
+    logging.info(f"RMSE: {rmse:.3f} | MAPE: {mape:.2f}% | R²: {r2:.3f}")
+
+
+if __name__ == "__main__":
+    evaluate()

transformer_model/scripts/evaluation/plot_metrics.py

@@ -0,0 +1,106 @@
+# plot_metrics.py
+
+import json
+import os
+
+import matplotlib.pyplot as plt
+import pandas as pd
+
+from transformer_model.scripts.config_transformer import RESULTS_DIR
+
+# === Plot 1: Training Metrics ===
+
+# Load training metrics
+training_metrics_path = os.path.join(RESULTS_DIR, "training_metrics.json")
+with open(training_metrics_path, "r") as f:
+    metrics = json.load(f)
+
+train_losses = metrics["train_losses"]
+test_mses = metrics["test_mses"]
+test_maes = metrics["test_maes"]
+
+plt.figure(figsize=(10, 6))
+plt.plot(
+    range(1, len(train_losses) + 1), train_losses, label="Train Loss", color="blue"
+)
+plt.plot(range(1, len(test_mses) + 1), test_mses, label="Test MSE", color="red")
+plt.plot(range(1, len(test_maes) + 1), test_maes, label="Test MAE", color="green")
+plt.xlabel("Epoch")
+plt.ylabel("Loss / Metric")
+plt.title("Training Loss vs Test Metrics")
+plt.legend()
+plt.grid(True)
+
+plot_path = os.path.join(RESULTS_DIR, "training_plot.png")
+plt.savefig(plot_path)
+print(f"[Saved] Training metrics plot: {plot_path}")
+plt.show()
+
+
+# === Plot 2: Predictions vs Ground Truth (Full Range) ===
+
+# Load comparison results
+comparison_path = os.path.join(RESULTS_DIR, "test_results.csv")
+df_comparison = pd.read_csv(comparison_path, parse_dates=["Timestamp"])
+
+plt.figure(figsize=(15, 6))
+plt.plot(
+    df_comparison["Timestamp"],
+    df_comparison["True Consumption (MW)"],
+    label="True",
+    color="darkblue",
+)
+plt.plot(
+    df_comparison["Timestamp"],
+    df_comparison["Predicted Consumption (MW)"],
+    label="Predicted",
+    color="red",
+    linestyle="--",
+)
+plt.title("Energy Consumption: Predictions vs Ground Truth")
+plt.xlabel("Time")
+plt.ylabel("Consumption (MW)")
+plt.legend()
+plt.grid(True)
+plt.tight_layout()
+
+plot_path = os.path.join(RESULTS_DIR, "comparison_plot_full.png")
+plt.savefig(plot_path)
+print(f"[Saved] Full range comparison plot: {plot_path}")
+plt.show()
+
+
+# === Plot 3: Predictions vs Ground Truth (First Month) ===
+
+first_month_start = df_comparison["Timestamp"].min()
+first_month_end = first_month_start + pd.Timedelta(days=25)
+df_first_month = df_comparison[
+    (df_comparison["Timestamp"] >= first_month_start)
+    & (df_comparison["Timestamp"] <= first_month_end)
+]
+
+plt.figure(figsize=(15, 6))
+plt.plot(
+    df_first_month["Timestamp"],
+    df_first_month["True Consumption (MW)"],
+    label="True",
+    color="darkblue",
+)
+plt.plot(
+    df_first_month["Timestamp"],
+    df_first_month["Predicted Consumption (MW)"],
+    label="Predicted",
+    color="red",
+    linestyle="--",
+)
+plt.title("Energy Consumption (First Month): Predictions vs Ground Truth")
+plt.xlabel("Time")
+plt.ylabel("Consumption (MW)")
+plt.legend()
+plt.grid(True)
+plt.tight_layout()
+
+plot_path = os.path.join(RESULTS_DIR, "comparison_plot_1month.png")
+plt.savefig(plot_path)
+print(f"[Saved] 1-Month comparison plot: {plot_path}")
+plt.show()

transformer_model/scripts/training/__init__.py

@@ -0,0 +1 @@
+# __init__

transformer_model/scripts/training/load_basis_model.py

@@ -0,0 +1,69 @@
+# load_basis_model.py
+# Load and initialize the base MOMENT model before finetuning
+
+import logging
+
+import torch
+from momentfm import MOMENTPipeline
+
+from transformer_model.scripts.config_transformer import (FORECAST_HORIZON,
+                                                          FREEZE_EMBEDDER,
+                                                          FREEZE_ENCODER,
+                                                          FREEZE_HEAD,
+                                                          HEAD_DROPOUT,
+                                                          SEQ_LEN,
+                                                          WEIGHT_DECAY)
+
+# Setup logging
+logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
+
+
+def load_moment_model():
+    """
+    Loads and configures the MOMENT model for forecasting.
+    """
+    logging.info("Loading MOMENT model...")
+    model = MOMENTPipeline.from_pretrained(
+        "AutonLab/MOMENT-1-large",
+        model_kwargs={
+            "task_name": "forecasting",
+            "forecast_horizon": FORECAST_HORIZON,  # default = 1
+            "head_dropout": HEAD_DROPOUT,  # default = 0.1
+            "weight_decay": WEIGHT_DECAY,  # default = 0.0
+            "freeze_encoder": FREEZE_ENCODER,  # default = True
+            "freeze_embedder": FREEZE_EMBEDDER,  # default = True
+            "freeze_head": FREEZE_HEAD,  # default = False
+        },
+    )
+
+    model.init()
+    logging.info("Model initialized successfully.")
+    return model
+
+
+def print_trainable_params(model):
+    """
+    Logs all trainable (unfrozen) parameters of the model.
+    """
+    logging.info("Unfrozen parameters:")
+    for name, param in model.named_parameters():
+        if param.requires_grad:
+            logging.info(f"  {name}")
+
+
+def test_dummy_forward(model):
+    """
+    Performs a dummy forward pass to verify the model runs without error.
+    """
+    logging.info(
+        "Running dummy forward pass with random tensors to see if model is running."
+    )
+    dummy_x = torch.randn(16, 1, SEQ_LEN)
+    output = model(x_enc=dummy_x)
+    logging.info(f"Dummy forward pass successful.Output shape: {output.shape}")
+
+
+if __name__ == "__main__":
+    model = load_moment_model()
+    print_trainable_params(model)
+    test_dummy_forward(model)

transformer_model/scripts/training/train.py

@@ -0,0 +1,199 @@
+# train.py
+
+import json
+import logging
+import os
+import time
+
+import numpy as np
+import torch
+from momentfm.utils.utils import control_randomness
+from sklearn.metrics import mean_absolute_error, mean_squared_error
+from tqdm import tqdm
+
+from transformer_model.scripts.config_transformer import (CHECKPOINT_DIR,
+                                                          GRAD_CLIP,
+                                                          LEARNING_RATE,
+                                                          MAX_EPOCHS, MAX_LR,
+                                                          RESULTS_DIR)
+from transformer_model.scripts.training.load_basis_model import \
+    load_moment_model
+from transformer_model.scripts.utils.check_device import check_device
+from transformer_model.scripts.utils.create_dataloaders import \
+    create_dataloaders
+
+# === Setup logging ===
+logging.basicConfig(
+    level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
+)
+
+
+def train():
+    # Start timing
+    start_time = time.time()
+
+    # Setup device (CUDA / DirectML / CPU) and AMP scaler
+    device, backend, scaler = check_device()
+
+    # Load base model
+    model = load_moment_model().to(device)
+
+    # Set random seeds for reproducibility
+    control_randomness(seed=13)
+
+    # Setup loss function and optimizer
+    criterion = torch.nn.MSELoss().to(device)
+    optimizer = torch.optim.Adam(model.parameters(), lr=LEARNING_RATE)
+
+    # Load data
+    train_loader, test_loader = create_dataloaders()
+
+    # Setup learning rate scheduler (OneCycle policy)
+    total_steps = len(train_loader) * MAX_EPOCHS
+    scheduler = torch.optim.lr_scheduler.OneCycleLR(
+        optimizer, max_lr=MAX_LR, total_steps=total_steps, pct_start=0.3
+    )
+
+    # Ensure output folders exist
+    os.makedirs(CHECKPOINT_DIR, exist_ok=True)
+    os.makedirs(RESULTS_DIR, exist_ok=True)
+
+    # Store metrics
+    train_losses, test_mses, test_maes = [], [], []
+
+    best_mae = float("inf")
+    best_epoch = None
+    no_improve_epochs = 0
+    patience = 5
+
+    for epoch in range(MAX_EPOCHS):
+        model.train()
+        epoch_losses = []
+
+        for timeseries, forecast, input_mask in tqdm(
+            train_loader, desc=f"Epoch {epoch}"
+        ):
+            timeseries = timeseries.float().to(device)
+            input_mask = input_mask.to(device)
+            forecast = forecast.float().to(device)
+
+            # Zero gradients
+            optimizer.zero_grad(set_to_none=True)
+
+            # Forward pass (with AMP if enabled)
+            if scaler:
+                with torch.amp.autocast(device_type="cuda"):
+                    output = model(x_enc=timeseries, input_mask=input_mask)
+                    loss = criterion(output.forecast, forecast)
+            else:
+                output = model(x_enc=timeseries, input_mask=input_mask)
+                loss = criterion(output.forecast, forecast)
+
+            # Backward pass + optimization
+            if scaler:
+                scaler.scale(loss).backward()
+                scaler.unscale_(optimizer)
+                torch.nn.utils.clip_grad_norm_(model.parameters(), GRAD_CLIP)
+                scaler.step(optimizer)
+                scaler.update()
+            else:
+                loss.backward()
+                torch.nn.utils.clip_grad_norm_(model.parameters(), GRAD_CLIP)
+                optimizer.step()
+
+            epoch_losses.append(loss.item())
+
+        average_train_loss = np.mean(epoch_losses)
+        train_losses.append(average_train_loss)
+        logging.info(f"Epoch {epoch}: Train Loss = {average_train_loss:.4f}")
+
+        # === Evaluation ===
+        model.eval()
+        trues, preds = [], []
+
+        with torch.no_grad():
+            for timeseries, forecast, input_mask in test_loader:
+                timeseries = timeseries.float().to(device)
+                input_mask = input_mask.to(device)
+                forecast = forecast.float().to(device)
+
+                if scaler:
+                    with torch.amp.autocast(device_type="cuda"):
+                        output = model(x_enc=timeseries, input_mask=input_mask)
+                else:
+                    output = model(x_enc=timeseries, input_mask=input_mask)
+
+                trues.append(forecast.detach().cpu().numpy())
+                preds.append(output.forecast.detach().cpu().numpy())
+
+        trues = np.concatenate(trues, axis=0)
+        preds = np.concatenate(preds, axis=0)
+
+        # Reshape for sklearn metrics
+        trues_2d = trues.reshape(trues.shape[0], -1)
+        preds_2d = preds.reshape(preds.shape[0], -1)
+
+        mse = mean_squared_error(trues_2d, preds_2d)
+        mae = mean_absolute_error(trues_2d, preds_2d)
+
+        test_mses.append(mse)
+        test_maes.append(mae)
+        logging.info(f"Epoch {epoch}: Test MSE = {mse:.4f}, MAE = {mae:.4f}")
+
+        # === Early Stopping Check ===
+        if mae < best_mae:
+            best_mae = mae
+            best_epoch = epoch
+            no_improve_epochs = 0
+
+            # Save best model
+            best_model_path = os.path.join(CHECKPOINT_DIR, "best_model.pth")
+            torch.save(model.state_dict(), best_model_path)
+            logging.info(
+                f"New best model saved to: {best_model_path} (MAE: {best_mae:.4f})"
+            )
+        else:
+            no_improve_epochs += 1
+            logging.info(f"No improvement in MAE for {no_improve_epochs} epoch(s).")
+
+            if no_improve_epochs >= patience:
+                logging.info("Early stopping triggered.")
+                break
+
+        # Save checkpoint
+        checkpoint_path = os.path.join(CHECKPOINT_DIR, f"model_epoch_{epoch}.pth")
+        torch.save(model.state_dict(), checkpoint_path)
+
+        scheduler.step()
+
+    logging.info(f"Best model was at epoch {best_epoch} with MAE: {best_mae:.4f}")
+
+    # Save final model
+    final_model_path = os.path.join(CHECKPOINT_DIR, "model_final.pth")
+    torch.save(model.state_dict(), final_model_path)
+    logging.info(f"Final model saved to: {final_model_path}")
+    logging.info(f"Final Test MSE: {test_mses[-1]:.4f}, MAE: {test_maes[-1]:.4f}")
+
+    # Save training metrics
+    metrics = {
+        "train_losses": [float(x) for x in train_losses],
+        "test_mses": [float(x) for x in test_mses],
+        "test_maes": [float(x) for x in test_maes],
+    }
+
+    metrics_path = os.path.join(RESULTS_DIR, "training_metrics.json")
+    with open(metrics_path, "w") as f:
+        json.dump(metrics, f)
+    logging.info(f"Training metrics saved to: {metrics_path}")
+
+    # Done
+    elapsed = time.time() - start_time
+    logging.info(f"Training complete in {elapsed / 60:.2f} minutes.")
+
+
+# === Entry Point ===
+if __name__ == "__main__":
+    try:
+        train()
+    except Exception as e:
+        logging.error(f"Training failed: {e}")

transformer_model/scripts/utils/__init__.py

@@ -0,0 +1 @@
+# __init__

transformer_model/scripts/utils/check_device.py

@@ -0,0 +1,55 @@
+import importlib
+import subprocess
+import sys
+
+import torch
+
+
+def install_package(package_name):
+    subprocess.check_call([sys.executable, "-m", "pip", "install", package_name])
+
+
+def check_device():
+    # **Check for NVIDIA GPU (CUDA)**
+    if torch.cuda.is_available():
+        device = torch.device("cuda")  # Use NVIDIA GPU
+        backend = "CUDA (NVIDIA)"
+        mixed_precision = True  # Use Automatic Mixed Precision (AMP)
+
+    # **If no NVIDIA GPU, check for AMD GPU (DirectML) only in Windows**
+    else:
+        try:
+            # Only try DirectML if the environment is Windows and DirectML is installed
+            if "win32" in sys.platform:
+                torch_directml = importlib.import_module("torch_directml")
+                if torch_directml.device_count() > 0:
+                    device = torch_directml.device()  # Use AMD GPU with DirectML
+                    backend = "DirectML (AMD)"
+                    mixed_precision = False  # No AMP for AMD GPU
+                else:
+                    raise ImportError  # AMD GPU not found
+            else:
+                device = torch.device("cpu")
+                backend = "CPU"
+                mixed_precision = False  # No AMP for CPU
+
+        except ImportError:
+            # If DirectML is not installed or AMD GPU not found
+            device = torch.device("cpu")
+            backend = "CPU"
+            mixed_precision = False  # No AMP for CPU
+
+    # Print the chosen device info
+    print(f"Training is running on: {backend} ({device})")
+
+    # **Initialize scaler (only for NVIDIA)**
+    if mixed_precision:
+        scaler = torch.amp.GradScaler()
+    else:
+        scaler = None  # No scaler needed for AMD/CPU
+
+    return device, backend, scaler
+
+
+if __name__ == "__main__":
+    device, backend, scaler = check_device()

transformer_model/scripts/utils/create_dataloaders.py

@@ -0,0 +1,46 @@
+# create_dataloaders.py
+
+import logging
+
+from momentfm.utils.utils import control_randomness
+from torch.utils.data import DataLoader
+
+from transformer_model.scripts.config_transformer import (BATCH_SIZE,
+                                                          FORECAST_HORIZON)
+from transformer_model.scripts.utils.informer_dataset_class import \
+    InformerDataset
+
+
+def create_dataloaders():
+    logging.info("Setting random seeds...")
+    control_randomness(seed=13)
+
+    logging.info("Loading training dataset...")
+    train_dataset = InformerDataset(
+        data_split="train", random_seed=13, forecast_horizon=FORECAST_HORIZON
+    )
+    logging.info(
+        "Train set loaded — Samples: %d | Features: %d",
+        len(train_dataset),
+        train_dataset.n_channels,
+    )
+
+    logging.info("Loading test dataset...")
+    test_dataset = InformerDataset(
+        data_split="test", random_seed=13, forecast_horizon=FORECAST_HORIZON
+    )
+    logging.info(
+        "Test set loaded — Samples: %d | Features: %d",
+        len(test_dataset),
+        test_dataset.n_channels,
+    )
+
+    train_loader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True)
+    test_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=True)
+
+    logging.info("Dataloaders created successfully.")
+    return train_loader, test_loader
+
+
+if __name__ == "__main__":
+    create_dataloaders()

transformer_model/scripts/utils/informer_dataset_class.py

@@ -0,0 +1,123 @@
+# informer_dataset.py
+
+import logging
+from typing import Optional
+
+import numpy as np
+import pandas as pd
+from sklearn.preprocessing import StandardScaler
+
+from transformer_model.scripts.config_transformer import DATA_PATH, SEQ_LEN
+
+logging.basicConfig(level=logging.INFO)
+
+
+class InformerDataset:
+    def __init__(
+        self,
+        forecast_horizon: Optional[int],
+        data_split: str = "train",
+        data_stride_len: int = 1,
+        task_name: str = "forecasting",
+        random_seed: int = 42,
+    ):
+        """
+        Parameters
+        ----------
+        forecast_horizon : int
+            Length of the prediction sequence.
+        data_split : str
+            'train' or 'test'.
+        data_stride_len : int
+            Stride length between time windows.
+        task_name : str
+            'forecasting' or 'imputation'.
+        random_seed : int
+            For reproducibility.
+        """
+
+        self.seq_len = SEQ_LEN
+        self.forecast_horizon = forecast_horizon
+        self.full_file_path_and_name = DATA_PATH
+        self.data_split = data_split
+        self.data_stride_len = data_stride_len
+        self.task_name = task_name
+        self.random_seed = random_seed
+
+        self._read_data()
+
+    def _get_borders(self):
+        train_ratio = 0.7
+        n_train = int(self.length_timeseries_original * train_ratio)
+        n_test = self.length_timeseries_original - n_train
+
+        train_end = n_train
+        test_start = train_end - self.seq_len
+        test_end = test_start + n_test + self.seq_len
+
+        # logging.info(f"Train range: 0 to {train_end}")
+        # logging.info(f"Test range: {test_start} to {test_end}")
+
+        return slice(0, train_end), slice(test_start, test_end)
+
+    def _read_data(self):
+        self.scaler = StandardScaler()
+
+        df = pd.read_csv(self.full_file_path_and_name)
+        self.length_timeseries_original = df.shape[0]
+        self.n_channels = df.shape[1] - 1  # exclude timestamp column
+
+        df.drop(columns=["date"], inplace=True)
+        df = df.infer_objects(copy=False).interpolate(method="cubic")
+
+        data_splits = self._get_borders()
+        train_data = df[data_splits[0]]
+
+        self.scaler.fit(train_data.values)
+        df = self.scaler.transform(df.values)
+
+        if self.data_split == "train":
+            self.data = df[data_splits[0], :]
+        elif self.data_split == "test":
+            self.data = df[data_splits[1], :]
+
+        self.length_timeseries = self.data.shape[0]
+
+        # logging.info(f"{self.data_split.capitalize()} set loaded.")
+        # logging.info(f"Time series length: {self.length_timeseries}")
+        # logging.info(f"Number of features: {self.n_channels}")
+
+    def __getitem__(self, index):
+        seq_start = self.data_stride_len * index
+        seq_end = seq_start + self.seq_len
+        input_mask = np.ones(self.seq_len)
+
+        if self.task_name == "forecasting":
+            pred_end = seq_end + self.forecast_horizon
+
+            if pred_end > self.length_timeseries:
+                pred_end = self.length_timeseries
+                seq_end = seq_end - self.forecast_horizon
+                seq_start = seq_end - self.seq_len
+
+            timeseries = self.data[seq_start:seq_end, :].T
+            forecast = self.data[seq_end:pred_end, :].T
+
+            return timeseries, forecast, input_mask
+
+        elif self.task_name == "imputation":
+            if seq_end > self.length_timeseries:
+                seq_end = self.length_timeseries
+                seq_end = seq_end - self.seq_len
+
+            timeseries = self.data[seq_start:seq_end, :].T
+
+            return timeseries, input_mask
+
+    def __len__(self):
+        if self.task_name == "imputation":
+            return (self.length_timeseries - self.seq_len) // self.data_stride_len + 1
+        elif self.task_name == "forecasting":
+            return (
+                self.length_timeseries - self.seq_len - self.forecast_horizon
+            ) // self.data_stride_len + 1

transformer_model/scripts/utils/load_final_model.py

@@ -0,0 +1,39 @@
+import logging
+import os
+
+import torch
+from huggingface_hub import hf_hub_download
+
+from transformer_model.scripts.config_transformer import CHECKPOINT_DIR
+from transformer_model.scripts.training.load_basis_model import \
+    load_moment_model
+
+logging.basicConfig(level=logging.INFO)
+
+
+# load model from checkpoint if available, else download it from hugging face
+def load_real_transformer_model(device=None):  # ⬅️ Name geändert
+    if device is None:
+        device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+    model = load_moment_model()
+    filename = "model_final.pth"
+    local_path = os.path.join(CHECKPOINT_DIR, filename)
+
+    if os.path.exists(local_path):
+        checkpoint_path = local_path
+        print("Loading model from local path...")
+    else:
+        print("Downloading model from Hugging Face Hub...")
+        checkpoint_path = hf_hub_download(
+            repo_id="dlaj/energy-forecasting-files",  # passe ggf. an
+            filename=f"transformer_model/{filename}",
+            repo_type="dataset",
+        )
+
+    model.load_state_dict(torch.load(checkpoint_path, map_location=device))
+    model.to(device)
+    model.eval()
+    logging.info(f"Model loaded from: {checkpoint_path}")
+
+    return model, device

transformer_model/scripts/utils/model_loader_wrapper.py

@@ -0,0 +1,42 @@
+from streamlit_simulation.utils.env import use_dummy
+from transformer_model.scripts.config_transformer import FORECAST_HORIZON
+from transformer_model.scripts.utils.informer_dataset_class import \
+    InformerDataset
+from transformer_model.scripts.utils.load_final_model import \
+    load_real_transformer_model
+
+try:
+    from streamlit_simulation.utils.dummy import (DummyDataset,
+                                                  DummyTransformerModel)
+except ImportError:
+    DummyTransformerModel = None
+    DummyDataset = None
+
+
+def load_final_transformer_model():
+    if use_dummy():
+        if DummyTransformerModel is None:
+            raise ImportError("DummyTransformerModel not available")
+        return DummyTransformerModel(), "cpu"
+    else:
+        return load_real_transformer_model()
+
+
+def load_model_and_dataset():
+    model, device = load_final_transformer_model()
+
+    if use_dummy():
+        if DummyDataset is None:
+            raise ImportError("DummyDataset not available")
+        dataset = DummyDataset(length=200)
+    else:
+        train_dataset = InformerDataset(
+            data_split="train", random_seed=13, forecast_horizon=FORECAST_HORIZON
+        )
+        test_dataset = InformerDataset(
+            data_split="test", random_seed=13, forecast_horizon=FORECAST_HORIZON
+        )
+        test_dataset.scaler = train_dataset.scaler
+        dataset = test_dataset
+
+    return model, dataset, device