Sewer defect classification on imbalanced data

Can GAN-generated images fix extreme class imbalance in sewer inspection? An honest negative result, and a labelling problem found on the way.

23defect types in the inspection dataset, from real sewer pipes
230,000+images between the most common and the rarest defect class
4training setups compared: raw, standard augmentation, GAN, both
0.31 → 0.59recall on the rare class with standard augmentation. GAN images made it worse

Four camera images from inside sewer pipes: settled deposits, a pipe connection, a displaced joint, and a rat — What the inspection robot sees: settled deposits, a pipe connection, a displaced joint, and vermin, the rarest kind of finding. Place names in the camera overlay are blurred.

Problem

Sewer pipes are inspected by driving a camera robot through them while an expert watches and notes every defect. It is slow and tiring, so Veolia wanted a classifier. The obstacle is the data: some defects appear in hundreds of thousands of images and others in a few thousand, and a model learns to ignore what it rarely sees.

Bar chart of images per defect class, falling from over 230,000 for the most common class to almost none for the rarest — Images per defect class. The rare classes on the right are the ones the model struggles with. Images can also carry up to 8 labels at once, and 38% are zoomed views of the same pipe.

The question

Can a GAN generate realistic images of a rare defect, and does adding them to the training set make the classifier better at that defect? As far as we knew, nobody had tried this on sewer inspection data.

What I did

Step	What happened
1. Generate	Trained a balancing GAN (BAGAN: an autoencoder, then a class-conditional generator) on a benchmark first, then on the sewer images
2. Hit a wall	On the raw data the GAN collapsed and produced nothing usable for the rare class
3. Look at the data	Went through the rare class image by image: many samples did not show the defect they were labelled with
4. Clean and rebalance	Removed the wrong labels by hand, under-sampled common classes, augmented the rare one. The GAN then trained and produced realistic images
5. Test the idea	Trained four classifiers and compared them on the rare class, judged by recall and F1: missing a real defect costs more than a false alarm

Two rows of sewer images labelled as vermin: the top row mostly shows empty pipes, the bottom row shows actual rats — The rare class before and after cleaning. Top: images labelled "vermin" in the original data, most with no animal in sight. Bottom: what was left after I checked them one by one.

Two grids of small sewer images side by side, real on the left and GAN-generated on the right, hard to tell apart at a glance — Real images on the left, GAN-generated on the right. At a glance they are hard to tell apart.

Result

Bar chart of precision, recall and F1 on the rare class for four training setups. Standard augmentation has the highest recall and F1, the two GAN setups the lowest — Rare class, four setups. Standard augmentation (green) wins on recall and F1. Both GAN setups (orange, yellow) fall far below the untouched baseline (blue).

Trained on	Recall, rare class
Original data	0.31
Standard augmentation	0.59
GAN images	0.11
Standard + GAN	about 0.01

On the common classes all four setups scored the same, about 0.6.

The generated images looked right and still hurt the classifier. My explanation in the thesis: the GAN reproduced the look of a sewer pipe but not the fine detail that defines the defect (the generated “rats” had no clear legs, eyes or heads), so the classifier learned a blurred version of the class.

What I took from it

Look at the data before tuning the model. The most useful finding of the project was upstream: the rare class was full of wrong labels, and no model could have fixed that.
Report the negative result. “Realistic to the eye” is not the same as “useful for training”, and the simple method won.
I still work this way in inspection projects, including semiconductor defect detection.

Master’s thesis, written during a research internship at Veolia Research & Innovation (France), 2022.