demo/energy-test-data
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add mining.

Browse Source
This commit is contained in:
kbt-devops
2026-02-10 23:42:37 +07:00
parent 30719abc27
commit 3739a7b4ec
1 changed files with 5 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@@ -12,7 +12,7 @@ This project generates and processes realistic test data for energy trading stra
- **Conventional Generation**: Gas, coal, and nuclear plant outputs - **Conventional Generation**: Gas, coal, and nuclear plant outputs
- **Load Profiles**: Regional electricity demand with weather correlations - **Load Profiles**: Regional electricity demand with weather correlations
- **Data Centers**: Power demand profiles including Bitcoin mining client - **Data Centers**: Power demand profiles including Bitcoin mining client
- **Bitcoin Mining**: Hashrate, price, and profitability data (from mempool.space) - **Mining**: Hashrate, price, and profitability data (from mempool.space)
## Project Structure ## Project Structure
@@ -66,7 +66,7 @@ python scripts/04_validate.py
**01_generate_synthetic.py**: Creates synthetic data for battery systems, renewable generation, conventional generation, and data centers. **01_generate_synthetic.py**: Creates synthetic data for battery systems, renewable generation, conventional generation, and data centers.
**02_fetch_historical.py**: Fetches electricity prices, Bitcoin mining data, and load profiles from public APIs (or generates realistic synthetic data when APIs are unavailable). **02_fetch_historical.py**: Fetches electricity prices, mining data, and load profiles from public APIs (or generates realistic synthetic data when APIs are unavailable).
**03_process_merge.py**: Merges datasets, optimizes memory usage, and saves to compressed Parquet format. **03_process_merge.py**: Merges datasets, optimizes memory usage, and saves to compressed Parquet format.
@@ -82,7 +82,7 @@ Edit `config/data_config.yaml` to customize:
- **Generation parameters**: Noise levels, outlier rates, missing value rates - **Generation parameters**: Noise levels, outlier rates, missing value rates
- **Battery parameters**: Capacity ranges, efficiency, degradation - **Battery parameters**: Capacity ranges, efficiency, degradation
- **Generation parameters**: Plant capacities, marginal costs - **Generation parameters**: Plant capacities, marginal costs
- **Bitcoin parameters**: Hashrate ranges, mining efficiency - **Mining parameters**: Hashrate ranges, mining efficiency
## Data Specifications ## Data Specifications
@@ -94,7 +94,7 @@ Edit `config/data_config.yaml` to customize:
| conventional_generation | 10 days | 144,000 | ~25MB | | conventional_generation | 10 days | 144,000 | ~25MB |
| load_profiles | 10 days | 72,000 | ~30MB | | load_profiles | 10 days | 72,000 | ~30MB |
| data_centers | 10 days | 72,000 | ~15MB | | data_centers | 10 days | 72,000 | ~15MB |
| bitcoin_mining | 10 days | 14,400 | ~20MB | | mining | 10 days | 14,400 | ~20MB |
| **Total** | | | **~185MB** | | **Total** | | | **~185MB** |
## Output Format ## Output Format
@@ -113,7 +113,7 @@ print(df.head())
## Data Sources ## Data Sources
- **Electricity Prices**: Hybrid (synthetic patterns based on EPEX Spot market characteristics) - **Electricity Prices**: Hybrid (synthetic patterns based on EPEX Spot market characteristics)
- **Bitcoin Mining**: Hybrid (mempool.space API + synthetic patterns) - **Mining**: Hybrid (mempool.space API + synthetic patterns)
- **Load Profiles**: Hybrid (ENTSO-E transparency platform patterns + synthetic) - **Load Profiles**: Hybrid (ENTSO-E transparency platform patterns + synthetic)
## Validation Reports ## Validation Reports