Adam Pantkowski has been involved in the open-source electricity map project as a junior researcher. Among other things, he contributed by putting his own country of origin, Poland, on the map. Read on about his experiences of being part of the electricity map project.
The fall semester of 2016 was my first semester ever to become a junior researcher involved in any kind of project. I had no idea what to suspect, but was very curious and wanted to contribute. At the beginning however, the whole concept of research was very unclear, as I did not have any chance to do something similar before. The first two weeks were spent on simply trying to figure out what to do and how to do it. Many projects were listed, almost all of them seemed very interesting up until there has been an email sent around with one particular project that caught my full attention from the beginning – the project codenamed ElectricityMap.
ElectricityMap, or EM as I tend to refer to it, is a visual representation of CO2 emissions to atmosphere by each European country, shown in real-time. The visualization can be seen through a website: https://electricitymap.tmrow.co which represents the map of Europe, with countries having different colours.
The different colours represent the density of carbon emission to atmosphere of a country, which can be further investigated via the map legend in the bottom-right corner. The stripes however represent the speed of wind. The important fact is, everything visualized is shown in real-time, where the quickest update regarding the current status of a country happens every five minutes (UK).
Having previously seen complex visualizations during the Python Study Group, as well as having a strong love towards programming, I was hooked right in. I’ve decided to contribute to this project and set it as my semester project under ETHOS Lab. It is important to note that this project, run by Tomorrow is an open-source project, which means that everybody can join in and contribute. At the point of my entrance to the project, we were eight people, and there were only very little countries included on the map. By inclusion I mean, that they were representing data about themselves, with a corresponding colour. On the other hand, the countries that are grey indicate that there has not yet been found any data about them, which means they cannot be clicked and analysed. The countries already existing on the map at the time when I’ve joined were France, Spain, UK, Germany, Ireland, Denmark, Norway and Sweden. For this reason, my intentions were to contribute to the project by building scripts using Python programming language, that would fetch the necessary data from a given source, parse it and organize it to be ready for implementation. I was particularly interested in starting with including Poland on the map, as Poland is my home country. This however was my personal intention and this is how I wanted to start contributing. Later on I’ve done additional things, about which I will elaborate later on.
After around 3-4 weeks after joining in my script was ready for implementation. It was 17th of October, very close to midnight when after extensive code reviews (practice during software development during which another person critically assesses your code) my pull request (a request of approval for changes made to the core code) was accepted by the project owner and merged with the master branch (the release branch with the source code to the current build on server). It was then when for the first time in my life, a piece of my code went live for everybody else to see it. That was one of the most important moments in my life and a huge milestone for me, for which I was so proud I could not sleep the entire night. Most importantly however, the script worked and Poland was shown on the map. I had spent around 10h weekly, if not more, learning python and building my script, as well as researching from which source the information could be obtained from. In addition, I had to learn how to test the script by running it in Docker – the container tool used to compile the build as preview it on the local host server, as well as learn Git – the version control system, in order for me to be able to actually include my modifications to the code and be able to contribute to the project with my script. It was a quite steep learning curve at the beginning, considering the fact that the fall semester of 2016 was also the beginning on my M.Sc. in Software Development.
The source ended up to be ENTSOE (https://transparency.entsoe.eu/) – the platform that has the freshest data about electricity production per country in Europe. It is also very important to mention that including Poland on EM was a test-case for the whole team. What I mean by that is, that after adding Poland and seeing how it operates on ENTSOE data, all of already existing countries on EM had been converted to use the data from ENTSOE, as well as all of the other countries to come next would use ENTSOE as their source. Therefore adding Poland was a small milestone for EM as well, as it changed the approach to developing it.
The script was written in a simple but effective way:
- Build the link to access to
- Access it via a network request
- Fetch the previously specified information
- Aggregate it into a map of values to be accessed later on
- Return it as an object that can be visualized on the map
After adding Poland to the map, my work did not stop however. Until the end of the semester, I have accomplished two more things: I had built another script that included Bulgaria on the map and together with another junior researcher – Minna, created a UX survey aiming at obtaining the information about the user experience with it, in order for the whole team to analyse it later and optimize the map in a way that it would become more understandable to people from outside the industry. Doing so I had also gotten a bit of experience with HTML and CSS, as the link to the survey has been included in the welcome box on the map. At the time of writing this report, there has been 63 responses with people providing brilliant feedback regarding the map. Many of the issues has already been fixed, as the development of the map is constant. The survey was created using surveymonkey and put on google docs to aggregate the output.
Therefore, I had spent my semester building scripts and contributing to the project by adding two countries: Poland and Bulgaria, as well as creating a survey that provides the insight into the user experience of the EM users. My biggest struggle was throughout the first month, where I was just introduced to many different tools (Git, Docker) I knew nothing about, as well as building something in a language I was not really familiar with. However, after spending a decent amount of time doing it, I was finally able to achieve my goals which I’ve set at the beginning.
Although the semester is almost over, I plan on continuing with the project and contributing to it. I feel honoured that I am able to be a part of it and even more, that I’ve actually given my own brick that builds the whole thing. In the middle of November, I have had a meeting with Olivier – project owner, and talked about the future of EM. So far we are working on filling up the map with all the countries (which might not be possible after all, as some of them, like Russia, are almost impossible to get access to), however soon we will be extending the map to other continents. The plans are to first extend the EM to Americas and then assess the outcome. The team has also grown extensively since I’ve joined at the beginning, with currently 24 people being a part of it. Along with this extension, there is an ongoing development of solar filter, showing the sunlight over Europe. Therefore, for any future contributors, there is a lot of diverse areas open for contribution:
- Create scripts visualizing the missing countries
- Develop relevant overlay filters
- Develop mobile app of EM (it is in very early stage at this point)
- Extend the map to include other continents
By writing this, I would like to encourage anybody who reads this report to get involved and contribute. This project has taught me a ton of new things and gave me valuable experiences with developing for real products. In addition, the team that stands behind it is highly helpful, understanding and smart. For this reason I intend to continue contributing to the project of EM.