demo/energy-test-data
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6 Commits 1 Branch 0 Tags
main
Commit Graph

6 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
kbt-devops
bdda04705f Update README for transmission datasets and mining data changes 
- Add Transmission Capacity and Transmission Cost to overview
- Update mining description to reflect EUR pricing and power metrics
- Update script descriptions to include transmission data
- Add transmission parameters to configuration section
- Update data specifications table with actual values and 2 new datasets
 2026-02-11 01:21:39 +07:00
kbt-devops
faaadc1297 Add transmission datasets and update mining data 
Add two new static datasets for cross-region arbitrage calculations:
- transmission_capacity: region-to-region capacity limits (20 rows)
- transmission_cost: transmission costs per path (20 rows)

Update mining dataset with EUR pricing and power metrics:
- Change btc_price_usd to btc_price_eur
- Add power_efficiency_th_per_mw, power_demand_mw
- Add revenue_eur_per_mwh, profit_eur_per_mwh
- Remove mining_profitability column

Changes include:
- scripts/02_fetch_historical.py: rewrite fetch_bitcoin_mining_data()
- scripts/01_generate_synthetic.py: add transmission data generators
- config/data_config.yaml: add transmission config, update bitcoin config
- config/schema.yaml: add 2 new schemas, update bitcoin_mining schema
- scripts/03_process_merge.py: add 2 new datasets
- scripts/04_validate.py: add 2 new datasets
- test/test_data.py: update for new datasets and bitcoin price reference

Total datasets: 9 (734,491 rows, 17.89 MB)
 2026-02-11 01:09:33 +07:00
kbt-devops
d981f7c56c Add mining. 2026-02-11 00:06:23 +07:00
kbt-devops
3739a7b4ec Add mining. 2026-02-10 23:42:37 +07:00
kbt-devops
30719abc27 Add test_data.py for data verification 
Add utility script to quickly verify and explore generated test data.
Provides data loading, sample previews, time range checks, and key
statistics for all datasets.
 2026-02-10 23:28:27 +07:00
kbt-devops
a643767359 Initial commit: Energy test data generation pipeline 
Add complete test data preparation system for energy trading strategy
demo. Includes configuration, data generation scripts, and validation
tools for 7 datasets covering electricity prices, battery capacity,
renewable/conventional generation, load profiles, data centers, and
mining data.

Excluded from git: Actual parquet data files (data/raw/, data/processed/)
can be regenerated using the provided scripts.

Datasets:
- electricity_prices: Day-ahead and real-time prices (5 regions)
- battery_capacity: Storage system charge/discharge cycles
- renewable_generation: Solar, wind, hydro with forecast errors
- conventional_generation: Gas, coal, nuclear plant outputs
- load_profiles: Regional demand with weather correlations
- data_centers: Power demand profiles including mining operations
- mining_data: Hashrate, price, profitability (mempool.space API)
 2026-02-10 23:28:23 +07:00