Skip to content

Data Ingestion

Our data source: ENTSO-E

The European Network of Transmission System Operators represents 40 electricity transmission system operators across 36 european countries. They are one of the main data sources cited by the Swiss Energy Dashboard, and checking their website shows that they make available – amongst other things – the actual load (hourly in MW) for Switzerland.

Image title
Hourly actual and forecasted load [MW] for Switzerland,
as seen on the ENTSO-E website.

We’ll start building our solution on this data source. To do this, we need a way to access their entire hourly load history.

As it turns out, they have a RESTful API, greatly faciliting the task of ingesting their data – i.e. plugging into our system so that we can use it. Before diving into learning their API, we can check if a Python client exists – and it does: the entsoe-py library, open-sourced on GitHub.1

Don’t reinvent the wheel2

Check if someone has already build what you need.

If appropriate, use it.

To ensure we can access the data, let’s load – using the entsoe-py library – the actual load and forecasted load for the first day of 2024:

from entsoe import EntsoePandasClient
import pandas as pd

entsoe_client = EntsoePandasClient(api_key="<ENTSOE_API_KEY_HERE>")

df = entsoe_client.query_load_and_forecast(
    country_code="CH", 
    start=pd.Timestamp('2024-01-01 00:00', tz="Europe/Zurich"), 
    end=pd.Timestamp('2024-01-02 00:00', tz="Europe/Zurich"),
)
df.head(5)

And voilà!, the data is available to us3:

Image title
First 5 rows of the actual and forecasted load [MW] for Switzerland for the 1st day of 2024, as fetched by the entsoe-py library.

Conclusion

Now that we have access to our data, let’s try to better understand it.

  1. Thank you EnergieID for your work. ↩

  2. It is quite unlikely you are the first person needing that wheel. The previous people having needed that wheel likely stumbled upon – and solved – problems you cannot yet envision. Building off their work, even if not exactly what you had in mind, will likely win you time. ↩

  3. Naturally, we checked if the data retrieved through this 3rd-party client matches the data found on the ENTSO-E official website. ↩