Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Bohan Chen, Kevin Miller, Andrea L. Bertozzi, Jon Schwenk

doi:10.1007/s42967-023-00284-8

Communications on Applied Mathematics and Computation >

2024 , Vol. 6 >Issue 2: 1013 - 1033

DOI: https://doi.org/10.1007/s42967-023-00284-8

REVIEW ARTICLE

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Expand

1. Department of Mathematics, University of California, Los Angeles, 520 Portola Plaza, Los Angeles 90095, CA, USA;
2. Oden Institute for Computational Engineering and Sciences, University of Texas at Austin, 201 E 24th St, Austin 78712, TX, USA;
3. Los Alamos National Laboratory, Los Alamos, NM 87545, USA

Received date: 2022-11-27

Revised date: 2023-04-11

Accepted date: 2023-05-04

Online published: 2023-07-20

Supported by

Bohan Chen is supported by the UC-National Lab In-Residence Graduate Fellowship Grant L21GF3606. Kevin Miller was supported by a DOD National Defense Science and Engineering Graduate (NDSEG) Research Fellowship. Jon Schwenk is supported by the Laboratory Directed Research and Development program of Los Alamos National Laboratory under project numbers 20170668PRD1 and 20210213ER. Andrea Bertozzi is supported by the NGA under Contract No. HM04762110003. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NGA. Approved for public release, NGA-U-2023-00757.

Fold

Abstract

Graph learning, when used as a semi-supervised learning (SSL) method, performs well for classification tasks with a low label rate. We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi- or hyperspectral image segmentation. Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification. This work builds on recent advances in the design of novel active learning acquisition functions (e.g., the Model Change approach in arXiv:2110.07739) while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods. In addition to improvements in the accuracy, our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

Key words： Image segmentation; Graph learning; Batch active learning; Hyperspectral image

Cite this article

Bohan Chen, Kevin Miller, Andrea L. Bertozzi, Jon Schwenk . Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs[J]. Communications on Applied Mathematics and Computation, 2024 , 6(2) : 1013 -1033 . DOI: 10.1007/s42967-023-00284-8

References

[1] Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching in fixed dimensions. J. ACM 45(6), 891-923 (1998). https://doi.org/10.1145/293347.293348
[2] Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399-2434 (2006)
[3] Bertozzi, A.L., Flenner, A.: Diffuse interface models on graphs for classification of high dimensional data. Multiscale Model. Simul. 10(3), 1090-1118 (2012)
[4] Bertozzi, A.L., Hosseini, B., Li, H., Miller, K., Stuart, A.M.: Posterior consistency of semi-supervised regression on graphs. Inverse Problems 37(10), 105011 (2021)
[5] Bertozzi, A.L., Luo, X., Stuart, A.M., Zygalakis, K.C.: Uncertainty quantification in graph-based classification of high dimensional data. SIAM/ASA J. Uncertain. Quantif. 6(2), 568-595 (2018)
[6] Bertozzi, A.L., Merkurjev, E.: Graph-based optimization approaches for machine learning, uncertainty quantification and networks. In: Processing, Analyzing and Learning of Images, Shapes, and Forms. Part 2, 503-531, Handb. Numer. Anal., 20, Elsevier/North-Holland, Amsterdam (2019)
[7] Boyd, Z.M., Bae, E., Tai, X.-C., Bertozzi, A.L.: Simplified energy landscape for modularity using total variation. SIAM J. Appl. Math. 78(5), 2439-2464 (2018)
[8] Boyd, Z.M., Porter, M.A., Bertozzi, A.L.: Stochastic block models are a discrete surface tension. J. Nonlinear Sci. 30(5), 2429-2462 (2020)
[9] Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.-P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vision 28(2), 151-167 (2007)
[10] Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60-65. IEEE (2005)
[11] Cai, W., Zhang, Y., Zhou, J.: Maximizing expected model change for active learning in regression. In: 2013 IEEE 13th International Conference on Data Mining, pp. 51-60. IEEE (2013)
[12] Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266-277 (2001)
[13] Chapman, J., Chen, B., Tan, Z., Calder, J., Miller, K., Bertozzi, A.L.: Novel batch active learning approach and its application on the synthetic aperture radar datasets. In: SPIE Defense and Commercial Sensing: Algorithms for Synthetic Aperture Radar Imagery XXX (2023)
[14] Ciurte, A., Bresson, X., Cuisenaire, O., Houhou, N., Nedevschi, S., Thiran, J.-P., Cuadra, M.B.: Semi-supervised segmentation of ultrasound images based on patch representation and continuous min cut. PLoS ONE 9(7), e100972 (2014)
[15] Dasgupta, S.: Two faces of active learning. Theoret. Comput. Sci. 412(19), 1767-1781 (2011). https://doi.org/10.1016/j.tcs.2010.12.054
[16] Gal, Y., Islam, R., Ghahramani, Z.: Deep Bayesian active learning with image data. In: International Conference on Machine Learning, pp. 1183-1192. PMLR (2017)
[17] Garcia-Cardona, C., Merkurjev, E., Bertozzi, A.L., Flenner, A., Percus, A.G.: Multiclass data segmentation using diffuse interface methods on graphs. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1600-1613 (2014)
[18] Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005-1028 (2009). https://doi.org/10.1137/070698592
[19] Hu, H., Sunu, J., Bertozzi, A.L.: Multi-class graph Mumford-Shah model for plume detection using the MBO scheme. In: Proceedings of the EMMCVPR Conference in Hong Kong. 8932, 209-222. Tai, X.-C. et al. (Eds), Springer Lecture Notes in Computer Science (2015)
[20] Hu, H., Laurent, T., Porter, M.A., Bertozzi, A.L.: A method based on total variation for network modularity optimization using the MBO scheme. SIAM J. Appl. Math. 73(6), 2224-2246 (2013)
[21] Iyer, G., Chanussot, J., Bertozzi, A.L.: A graph-based approach for data fusion and segmentation of multimodal images. IEEE Trans. Geosci. Remote Sensing 59(5), 4419-4429 (2021). https://doi.org/10.1109/TGRS.2020.2971395
[22] Ji, M., Han, J.: A variance minimization criterion to active learning on graphs. In: Artificial Intelligence and Statistics, pp. 556-564. PMLR (2012)
[23] Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comp. Vision 1, 321-331 (2004)
[24] Kushnir, D., Venturi, L.: Diffusion-based deep active learning. arXiv:2003.10339 (2020)
[25] Ma, Y., Garnett, R., Schneider, J.G.: Sigma-optimality for active learning on Gaussian random fields. In: NIPS, pp. 2751-2759 (2013)
[26] Ma, Y., Huang, T.-K., Schneider, J.G.: Active search and bandits on graphs using sigma-optimality. In: UAI, vol. 542, pp. 551 (2015)
[27] Meng, Z., Merkurjev, E., Koniges, A., Bertozzi, A.L.: Hyperspectral image classification using graph clustering methods. IPOL J. Image Process. Online 7, 218-245 (2017). https://doi.org/10.5201/ipol.2017.204
[28] Merkurjev, E., Garcia-Cardona, C., Bertozzi, A.L., Flenner, A., Percus, A.G.: Diffuse interface methods for multiclass segmentation of high-dimensional data. Appl. Math. Lett. 33, 29-34 (2014)
[29] Merkurjev, E., Kostić, T., Bertozzi, A.L.: An MBO scheme on graphs for classification and image processing. SIAM J. Imaging Sci. 6(4), 1903-1930 (2013)
[30] Merkurjev, E., Sunu, J., Bertozzi, A.L.: Graph MBO method for multiclass segmentation of hyperspectral stand-off detection video. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 689-693. IEEE (2014)
[31] Miller, K., Bertozzi, A.L.: Model-change active learning in graph-based semi-supervised learning. https://doi.org/10.48550/arXiv.2110.07739 (2021)
[32] Miller, K., Li, H., Bertozzi, A.L.: Efficient graph-based active learning with probit likelihood via Gaussian approximations. arXiv:2007.11126 (2020).
[33] Miller, K., Mauro, J., Setiadi, J., Baca, X., Shi, Z., Calder, J., Bertozzi, A.L.: Graph-based active learning for semi-supervised classification of SAR data. arXiv:2204.00005 (2022)
[34] Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42(5), 577-685 (1989). https://doi.org/10.1002/cpa.3160420503
[35] O’Shea, K., Nash, R.: An introduction to convolutional neural networks. arXiv:1511.08458 (2015)
[36] Qiao, Y., Shi, C., Wang, C., Li, H., Haberland, M., Luo, X., Stuart, A.M., Bertozzi, A.L.: Uncertainty quantification for semi-supervised multi-class classification in image processing and ego-motion analysis of body-worn videos. Electron. Imaging 31(11), 1-264 (2019)
[37] Qin, J., Lee, H., Chi, J.T., Drumetz, L., Chanussot, J., Lou, Y., Bertozzi, A.L.: Blind hyperspectral unmixing based on graph total variation regularization. IEEE Trans. Geosci. Remote Sensing 59(4), 3338-3351 (2021). https://doi.org/10.1109/TGRS.2020.3020810
[38] Schwenk, J., Rowland, J.: RiverPIXELS: paired Landsat images and expert-labeled sediment and water pixels for a selection of rivers v1.0. United States. https://data.ess-dive.lbl.gov/view/, https://doi.org/10.15485/1865732
[39] Settles, B.: Active Learning vol. 6, pp. 1-114. Morgan & Claypool Publishers LLC, Carnegie Mellon University, USA (2012). https://doi.org/10.2200/s00429ed1v01y201207aim018
[40] Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Carnegie-Mellon University, Pittsburgh, PA (1994)
[41] Thorpe, M., Nguyen, T.M., Xia, H., Strohmer, T., Bertozzi, A., Osher, S., Wang, B.: Grand++: graph neural diffusion with a source term. In: International Conference on Learning Representations (2021)
[42] Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395-416 (2007)
[43] Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: International Conference on Machine Learning, pp. 6861-6871. PMLR (2019)
[44] Zhu, F., Wang, Y., Xiang, S., Fan, B., Pan, C.: Structured sparse method for hyperspectral unmixing. ISPRS-J. Photogramm. Remote Sens. 88, 101-118 (2014)
[45] Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using Gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 912-919 (2003)

Options

Abstract

Outlines

模态框（Modal）标题

Abstract

Cite this article

References