By Emilie Mørch Groth, Junior Researcher
The recap
I want to briefly take you back to the beginning of this project. Driven by an interest in public use of data and AI, and with an aim to build a better understanding of publicly used AI solutions and the citizens they affect, I set off on a mapping project of the 40 Danish Signature Projects under the National Uptake Fund, which can be found at Digitaliseringstyrelsen’s (Digst) webpage. The keywords in my exploration were accountability and transparency.
I began plotting information on the projects into a spreadsheet to determine their geographical placement and involved partners, and I tried to pinpoint the technology used. The idea was to use this information to build a visual map, where the user could explore who is involved, where, and how. But alas. Working through my spreadsheet, I ran into the same issue over and over: I could not find the information I was looking for, which led to a halt in the mapping exercise. More specifically, I could not uncover the partners involved in the projects. Many were mentioned as ‘private partners’ on the official project descriptions, but I could find no traces of them elsewhere.
At my previous blogpost, I was at this pivotal point, unable to map the projects, as originally intended, and I left it with questions on how to fill the blank holes in the data, while contemplating questions on accountable AI and potential motives.
The prototypes
Working with the incomplete data, I began constructing map prototypes based on the preliminary dataset. I wanted to explore how different maps could convey different messages, and how this could be turned into something useful.
The heatmap edition
The first iteration was a vector graphic based heatmap of Denmark, with all municipalities outlined and all involved hospitals and universities marked. The idea was also to map other types of partners, such as cluster and collaborations, and of course all the private partners, once that information was available.
Heatmap prototype:
colour strength = no. of projects, H = hospital, U = university, red X = cancelled project
This prototype quickly presented an array of questions on mapping method relating to what story the map is meant to convey. How do you map and colour a project related to a hospital? On paper it is a regional project, but is it fair to mark an entire region based on a single project taking place in a single hospital corridor? And what about projects related to larger, national collaborations? It may be centred in a single municipality, but how do you map national collaboration efforts?
The interactive edition
This prototype iteration explored the opportunity to let the audience create their own narrative, to zoom in and see details on chosen areas. The projects would be divided by geographical area, municipality or region, and the summarized information would be available.
Interactive prototype
Despite the interactive element, this prototype failed to convey the intertwined nature of the projects. They came across as isolated and static, and to be honest less interesting. To continue with this map, effort should be put into showing the interconnectedness that hides behind some of the projects.
The network edition
The final prototype was meant to showcase the connections across projects by making them and all partners be nodes in a connected network.
Network prototype: No = project id, Size = million DKK, X = cancelled, Colour = sector
(blue: health, green: energy, pink: social, purple: employment, orange: admin, grey: stakeholder)
The prototype succeeded best in showcasing the interconnectedness, but it quickly became chaotic and hard to interpret the map. It would need more work and effort put into distinguishing between partners and projects, and tinkering with other aspects of the projects before it would become really helpful.
All in all, the three prototypes carry the same issue, they are built on the incomplete dataset, making them non representative at best and faulty at worst. I wanted to overcome this issue, so I started reaching out.
The reaching out
I went back to the source, and I sent mails directly to the employees listed at the Signature Project page at Digitaliseringsstyrelsen.
I stated my purpose, described my project, and put forth my problem. As my focus was accountability, I kept to the project partners as my main investigation and inquiry. I asked them if Digst could help me with uncovering the ‘private partners’ mentioned in the official project descriptions on their page. Relatively shortly after, I received an answer, though not the answer I had hoped for. I was told that they could not help me with the names of the private companies involved, because they do not know. The project contracts are handled by the projects themselves, and Digst does not have a full overview.
I’ll let that hang there a moment.
[…]
Maybe it’s just me being too caught up in my interests, but I found this response surprising to say the least.
AI has been a hot topic for a while, and AI ethics, transparency and accountability is both discussed and preached left and right. This underlines, why I find it so surprising that Digst do not have an overview of who is involved in experimental Signature Projects funded by the state and involving many citizens. I am not trying to make an argument for a completely centralized process, and I do not pretend to know much about how the public institutions work and usually deal with partnerships – but I honestly expected more transparency.
The pivot
This insight – or lack of same – combined with tenaciously searching my way through the internet on search terms and keywords without success, resulted in me giving up on the ambition of having a complete dataset to map from, which let me to abandon the mapping altogether.
I circled back to my original intention; to enable a better and broader understanding of the use of AI in the public sectors in Denmark, and my frustration that information was so obscured and the landscape so murky led to the project pivoting from mapping exercise to dataset building. The focal point had become the partners of the Signature Projects, as they are an essential factor when discussing accountability and transparency, and the project thus came to be centred on creating a dataset around these.
The dataset
Left with a spreadsheet instead of an interactive and exploratory map, I realised that this dataset encompasses what turned out to be the most interesting factor of working with the Signature Projects: the lack of transparency. In the first phases of the data collection, the columns and rows related to project partners were full of holes. As I worked along, I was able to fill most of them – but it took investigative and persistent effort. In the end, it is not about the few data gaps still existing, it is about the inherent difficulty in understanding who is involved in the data work in these projects.
To me, it seems very strange that money is granted to AI and data-driven projects without knowledge of who is involved in the data work. In my opinion we cannot achieve accountability, unless we have transparency relating to all partners involved, especially when we are dealing with the data of citizens.
So this becomes the story I want to tell and the lesson I have learned; there is more work to do if we want full transparency and accountability in the use of data and AI in public projects. The Signature Projects represent our state’s attitude towards AI technology and solutions, and they should set a prime example on how to work with citizens data.
The end
My project has come to an official finish, and it ended up far different from when it set sail nearly a year ago. It became a project on critical investigation more than creative mapping, and the process has only enhanced my interest in AI accountability and how we work with it in the Danish state.
Though the project is at an end, I will do my best to keep the work from stranding in a junk folder at my desktop. My ambition is to find a suitable format and make the dataset available to interested peers. My insights into what could – and could not – be uncovered on the Signature Projects and their partnerships might prove interesting and relevant to others.