Geostore Artificial Intelligence: How we categorized over 200,000 road images

"Is this a road made of dirt, gravel, asphalt or concrete?"

That was the question. As part of our Geostore project, the World Bank wanted to know the road surface type and quality of the farm to market roads in the Philippines.

We had an image repository covering many of the roads, and lots of glorious road images. Over 200,000 of them, to be precise, taken during the different phases of the construction projects: survey, planning, in-progress and completed.

Luckily, most of the images had latitude and longitude coordinates embedded in the photo. That let us pinpoint where each image was taken, plot it on a map, and work out where the road was.

One problem was that a road could have several photos taken at the same spot on different dates, matching different phases of construction. A photo might show the road during the survey, when it was still dirt, and a later one show it concreted and finished.

Luckily again, each image also carried the timestamp of when it was taken. When there were several photos of the same spot with different surface types, we picked the most recent one, and that told us the latest state of the road.

The next problem was labeling each image with the right surface type. How would we know a dirt road from a gravel road? What if it was partly concrete, or concrete with some asphalt on it? What if the person labeling it made a mistake?

We had to pin down the surface types and put a quality control process around the labeling, and that is how the Geoprocessing Dashboard started.

We wrote training documentation with several examples of each surface type so everyone labeling would do it the same way. We also wrote a protocol for photos with more than one surface type. A single lane of concrete next to a lane of dirt, for example, got labeled as both a dirt road and a concrete road. Yes, there were a LOT of roads like that.

People make mistakes

For quality assurance we built a feature that had several people label the same images, so inconsistent labels showed up. Those got flagged, and someone else would look at the image and settle on the right label. That way the disputed images got resolved quickly.

Labeling from the keyboard

The labelers stared at their screens all day, and we noticed that labeling with the mouse was eating a lot of time.

What if we can label the images and proceed to the next one without having to lift our fingers from the keyboard? To test whether that helped, we built a special tool with shortcut keys that were comfortable and easy to learn, and BOOM, productivity doubled!

We were still barely making a dent in the 200,000 images. Because several people labeled each one, an image took about 2 minutes of combined labeling time. At that rate, even 10 people working non-stop for 8 hours a day would need almost 4 months to finish. We needed a faster way.

Artificial Intelligence

Our team had been experimenting with Artificial Intelligence and Machine Learning that could classify and label images automatically. Could it label photos of roads?

First we needed to give the Machine Learning (ML) model plenty of example images for each category. Fortunately we had those. We had already labeled about 10,000 images, which was the base we needed for a custom ML model.

Screenshot from: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets

After several hours of setup and training, the model started to work. We piped the remaining 190,000 images into it, and in less than 24 hours all 200,000 images were labeled. Oh yeah!

We still had to double check the images the model said it wasn't sure about. Most of those turned out to be selfies and group photos of the inspection teams. :)

So we started with a labeling tool, made it faster, found it still wasn't fast enough, and handed the job to Artificial Intelligence.

Learn more about our work with the Geostore here.

Contact Us

If you need processes in your company's operations automated, talk to us and we will see how we can help.