Update README for transmission datasets and mining data changes

- Add Transmission Capacity and Transmission Cost to overview
- Update mining description to reflect EUR pricing and power metrics
- Update script descriptions to include transmission data
- Add transmission parameters to configuration section
- Update data specifications table with actual values and 2 new datasets

README.md

@@ -12,7 +12,9 @@ This project generates and processes realistic test data for energy trading stra
 - **Conventional Generation**: Gas, coal, and nuclear plant outputs
 - **Load Profiles**: Regional electricity demand with weather correlations
 - **Data Centers**: Power demand profiles including mining client
-- **Mining**: Hashrate, price, and profitability data (from mempool.space)
+- **Mining**: Hashrate, price (EUR), power efficiency, demand, revenue, and profit per MWh
+- **Transmission Capacity**: Region-to-region interconnector limits and efficiency
+- **Transmission Cost**: Transmission costs including losses, congestion charges, and fees
 
 ## Project Structure
 
@@ -64,9 +66,9 @@ python scripts/04_validate.py
 
 ### Individual scripts
 
-**01_generate_synthetic.py**: Creates synthetic data for battery systems, renewable generation, conventional generation, and data centers.
+**01_generate_synthetic.py**: Creates synthetic data for battery systems, renewable generation, conventional generation, data centers, and transmission capacity/cost.
 
-**02_fetch_historical.py**: Fetches electricity prices, mining data, and load profiles from public APIs (or generates realistic synthetic data when APIs are unavailable).
+**02_fetch_historical.py**: Fetches electricity prices, mining data (with EUR pricing and power metrics), and load profiles from public APIs (or generates realistic synthetic data when APIs are unavailable).
 
 **03_process_merge.py**: Merges datasets, optimizes memory usage, and saves to compressed Parquet format.
 
@@ -82,20 +84,23 @@ Edit `config/data_config.yaml` to customize:
 - **Generation parameters**: Noise levels, outlier rates, missing value rates
 - **Battery parameters**: Capacity ranges, efficiency, degradation
 - **Generation parameters**: Plant capacities, marginal costs
-- **Mining parameters**: Hashrate ranges, mining efficiency
+- **Mining parameters**: Hashrate ranges, power efficiency
+- **Transmission parameters**: Capacity ranges, efficiency, congestion surcharges, fees
 
 ## Data Specifications
 
-| Dataset | Time Range | Rows (10d × 1min) | Est. Size |
+| Dataset | Rows | Actual Size |
-|---------|-----------|-------------------|-----------|
+|---------|------|-------------|
-| electricity_prices | 10 days | 72,000 | ~40MB |
+| electricity_prices | 72,005 | ~2.0 MB |
-| battery_capacity | 10 days | 144,000 | ~20MB |
+| battery_capacity | 144,010 | ~4.0 MB |
-| renewable_generation | 10 days | 216,000 | ~35MB |
+| renewable_generation | 216,015 | ~5.4 MB |
-| conventional_generation | 10 days | 144,000 | ~25MB |
+| conventional_generation | 144,010 | ~3.0 MB |
-| load_profiles | 10 days | 72,000 | ~30MB |
+| load_profiles | 72,005 | ~1.7 MB |
-| data_centers | 10 days | 72,000 | ~15MB |
+| data_centers | 72,005 | ~1.0 MB |
-| mining | 10 days | 14,400 | ~20MB |
+| mining | 14,401 | ~0.5 MB |
-| **Total** | | | **~185MB** |
+| transmission_capacity | 20 | ~0.01 MB |
+| transmission_cost | 20 | ~0.01 MB |
+| **Total** | **734,491** | **~17.9 MB** |
 
 ## Output Format