NU.nl published wednesday march 13th an article calling on reader to help with our research. Twenty readers helped with entering structuring information about drones. How did this collaboration go?
The Government Gazette regularly publishes papers on partial deposits of airspace. These also include deposits for drones: unmanned aircraft vehicles equipped with cameras.
It is unclear what these planes do in our airspace. What information is recorded? What is done with that information? Is that information stored somewhere? These are all questions which NU.nl wanta to answer.
Application
There are 165 PDF documents in total. In here we find the dates and locations of airspace deposits. To be able to say something meaningful about the necessity of these flights, we firstly wanted to get an idea of the scope of this information.
Manually entering all those document would mean several weeks of work for our editors. That’s why we came up with the following plan: we ask the help of our readers. This question was placed online and we waited to see what the reaction would be.
At the end of the day, we found 67 enthusiastic participants in the mailbox. Later in the week we were at almost a hundred people. Far above our expectations.
Scraping
From some readers, we got the advice to scrape the documents. That means you write a program that recognizes the structure of a document, picking out useful information and placing that in a spreadsheet.
We already spoke with a number of experts on this subject. They could not assure us that we could get 100 percent reliable data with this method. For many participants that message sounded like a challenge.
Ultimately, Dirk-Jan Faber made a scraper, which could convert 140 to convert spreadsheets fairly reliable.
We could correct the wrong dates and locations using the manually entered information. we supplemented the missing documents by means of our other readers.
Of the large group of readers who responded initially, a small group carried over to the final group.
Results
The first result is our article about the lack of clarity surrounding drone flights. It appears that since 2009 on 551 different days drones are deployed. Last year, every two days a drone flew around. And the flights are less frequently announced in advance.
What we learned from this project?
- We have a spreadsheet with reliable information on drone flights in the Netherlands.
- A first experience with crowdsourcing. Normally readers only see our final product. Now they can help with the creation of news.
- We have received many useful tips from the experts. Because we placed an open call and actively sought contact with readers, these experts contacted us.
- We currently have circulated a survey among the participants. With the results we want to tackle the next project even better.
Once we have structured the information in terms of locations, we make the spreadsheet available to our readers.
The people who helped us (not everyone wanted to be mentioned by name):
- Marek
- Ralf Groenveld
- Piet Kalkhoven
- Dirk-Jan Faber
- Arnoud Aten
- Dimitri van Hoven
- Leo Duursma
- Antonio
- Herbert Klinkvis
- L. Wanders
- Hugo Leisink
- Wegman
- Ahad Khan