Echo state networks for level set image segmentation

The question

The level set method segments an image by evolving a curve under a partial differential equation, one iteration at a time. The dissertation asked whether that iterative evolution can be treated as a spatiotemporal sequence and learned by recurrent neural networks, and in particular whether an echo state network, a reservoir computing model whose recurrent weights are fixed at random and never trained, can learn it as well as recurrent networks trained end to end.

How it was done

Level set evolutions were generated in MATLAB from four image datasets and used as training sequences. Five models were compared on learning the evolution: a convolutional echo state network, and convolutional RNN, LSTM and GRU models and a 3D CNN, all trained with backpropagation. Hyperparameters were searched in parallel runs tracked with TensorBoard, and results were compared on intersection over union and F1 score. A separate set of experiments varied the reservoir itself: leaking rate, spectral radius, reservoir size and sparsity.

Data

• Weizmann Segmentation Database (200 images)
• Berkeley Segmentation Data Set (BSD500)
• CIFAR-10
• CIFAR-100

What was found

• The trained gated models did best. The convolutional GRU outperformed the other models on the Weizmann, BSD and CIFAR-100 experiments, and the convolutional LSTM on CIFAR-10.
• The echo state network did not outperform the trained recurrent models under the conditions tested. On the two small datasets it did no better than random, which the dissertation attributes to reservoir parameters that did not satisfy the echo state property.
• The trained models improved as the datasets grew. The echo state network had its best results on the largest dataset, CIFAR-100, though they were still not the best of the five.
• Raising the leaking rate improved IoU and F1, and raising the spectral radius slightly reduced them, which the dissertation reads as the reservoir treating level set evolution as a short term memory problem.
• Larger reservoirs improved performance, and 80% sparsity made reservoir updates cheap at the cost of capacity.
• Training the echo state network was fast because it needs no backpropagation, but finding reservoir parameters that work required a search the trained models did not.

Limitations

Echo state network performance was sensitive to its hyperparameters, and the dissertation notes that the best reservoir settings are task specific. The CIFAR experiments were capped at 10 million generated examples because of compute limits. The per dataset detail here is taken from the thesis source in the public code repository and agrees with the deposited abstract, which names the convolutional GRU and LSTM as the best architectures and the leaking rate and spectral radius as the main causes of the low performance of the echo state network. The deposited dissertation on WIReDSpace is the authoritative text.

What it might mean

A negative result with a useful shape: for this task a cheap untrained reservoir is not a drop in replacement for a trained gated network, and the leaking rate and spectral radius are where the difference lies.

Publications

• Learning Level Set Method by Echo State Network for Image Segmentation

  TL Mashinini. MSc Thesis, University of the Witwatersrand, 2022, 2022.

Code

• ESNIterativeSegmentation. Models, data generation scripts, notebooks and the thesis source. The code is on the master branch.
• Echo-State-Master. An earlier echo state network repository from 2019.

Lineage

Came before

A BSc in computational and applied mathematics and astronomy, and an honours project on wildfire estimation with kernel density estimators.

Led to

The proposed doctoral research, at proposal stage and not registered, is described in the CV record as following on from this work.