{"id":15680,"date":"2020-07-27T10:00:00","date_gmt":"2020-07-27T14:00:00","guid":{"rendered":"https:\/\/pyimagesearch.com\/?p=15680"},"modified":"2023-02-20T05:15:13","modified_gmt":"2023-02-20T10:15:13","slug":"opencv-grabcut-foreground-segmentation-and-extraction","status":"publish","type":"post","link":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/","title":{"rendered":"OpenCV GrabCut: Foreground Segmentation and Extraction"},"content":{"rendered":"\n<script src=\"https:\/\/fast.wistia.com\/embed\/medias\/ijdj48ow2j.jsonp\" async=\"\"><\/script><script src=\"https:\/\/fast.wistia.com\/assets\/external\/E-v1.js\" async=\"\"><\/script><div class=\"wistia_responsive_padding\" style=\"padding:56.25% 0 0 0;position:relative;\"><div class=\"wistia_responsive_wrapper\" style=\"height:100%;left:0;position:absolute;top:0;width:100%;\"><div class=\"wistia_embed wistia_async_ijdj48ow2j videoFoam=true\" style=\"height:100%;position:relative;width:100%\"><div class=\"wistia_swatch\" style=\"height:100%;left:0;opacity:0;overflow:hidden;position:absolute;top:0;transition:opacity 200ms;width:100%;\"><img decoding=\"async\" src=\"https:\/\/fast.wistia.com\/embed\/medias\/ijdj48ow2j\/swatch\" style=\"filter:blur(5px);height:100%;object-fit:contain;width:100%;\" alt=\"\" aria-hidden=\"true\" onload=\"this.parentNode.style.opacity=1;\"><\/div><\/div><\/div><\/div>\n\n\n\n<p>In this tutorial, you will learn how to use OpenCV and GrabCut to perform foreground segmentation and extraction.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header.png\" alt=\"\" class=\"wp-image-15682\" width=\"500\" height=\"500\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header-150x150.png?lossy=2&amp;strip=1&amp;webp=1 150w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header-300x300.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header.png?size=378x378&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 500px) 100vw, 500px\" \/><\/figure><\/div>\n\n\n<p>Prior to deep learning and instance\/semantic segmentation networks such as Mask R-CNN, U-Net, etc., <strong>GrabCut was <em>the method<\/em> to accurately segment the foreground of an image from the background.<\/strong><\/p>\n\n\n\n<p>The GrabCut algorithm works by:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Accepting an input image with <em>either<\/em> (1) a <strong>bounding box<\/strong> that specified the location of the object in the image we wanted to segment or (2) a <strong>mask<\/strong> that <em>approximated<\/em> the segmentation<\/li><li>Iteratively performing the following steps:<ul><li><strong>Step #1: <\/strong>Estimating the color distribution of the foreground and background via a Gaussian Mixture Model (GMM)<\/li><li><strong>Step #2: <\/strong>Constructing a Markov random field over the pixels labels (i.e., foreground vs. background)<\/li><li><strong>Step #3: <\/strong>Applying a graph cut optimization to arrive at the final segmentation<\/li><\/ul><\/li><\/ul>\n\n\n\n<p>Sounds complicated, doesn\u2019t it?<\/p>\n\n\n<p>Luckily, OpenCV has an implementation of GrabCut via the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.grabCut<\/code> function that makes applying GrabCut a breeze (once you know the parameters to the function and how to tweak them, of course).<\/p>\n\n\n<p>But before you go saying:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p><em>Hey Adrian, isn&#8217;t the GrabCut algorithm old news? <\/em><\/p><p><em>Shouldn&#8217;t we just be applying Mask R-CNN, U-Net, or other image segmentation networks to segment background and foreground instead?<\/em><\/p><\/blockquote>\n\n\n\n<p>The above is the perfect example of how deep learning and traditional computer vision are being blended together.<\/p>\n\n\n\n<p>If you\u2019ve ever used Mask R-CNN or U-Net before, you know these deep neural networks are <em>super powerful<\/em>, but the masks are not always perfect. <strong>In practice, you can actually use GrabCut to clean up these segmentation masks<\/strong> (and I\u2019ll be showing you how to do that in a future post).<\/p>\n\n\n\n<p>But in the meantime, let\u2019s learn about the fundamentals of GrabCut.<\/p>\n\n\n\n<p><strong>To learn how to use OpenCV and GrabCut for foreground segmentation, <em>just keep reading.<\/em><\/strong><\/p>\n\n\n\n<div id=\"pyi-source-code-block\" class=\"source-code-wrap\"><div class=\"gpd-source-code\">\n    <div class=\"gpd-source-code-content\">\n        <img decoding=\"async\" src=\"\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/01\/source-code-icon.png\" alt=\"\">\n        <h4>Looking for the source code to this post?<\/h4>\n                    <a href=\"#download-the-code\" class=\"pyis-cta-modal-open-modal\">Jump Right To The Downloads Section <svg class=\"svg-icon arrow-right\" width=\"12\" height=\"12\" aria-hidden=\"true\" role=\"img\" focusable=\"false\" viewBox=\"0 0 14 14\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M6.8125 0.1875C6.875 0.125 6.96875 0.09375 7.09375 0.09375C7.1875 0.09375 7.28125 0.125 7.34375 0.1875L13.875 6.75C13.9375 6.8125 14 6.90625 14 7C14 7.125 13.9375 7.1875 13.875 7.25L7.34375 13.8125C7.28125 13.875 7.1875 13.9062 7.09375 13.9062C6.96875 13.9062 6.875 13.875 6.8125 13.8125L6.1875 13.1875C6.125 13.125 6.09375 13.0625 6.09375 12.9375C6.09375 12.8438 6.125 12.75 6.1875 12.6562L11.0312 7.8125H0.375C0.25 7.8125 0.15625 7.78125 0.09375 7.71875C0.03125 7.65625 0 7.5625 0 7.4375V6.5625C0 6.46875 0.03125 6.375 0.09375 6.3125C0.15625 6.25 0.25 6.1875 0.375 6.1875H11.0312L6.1875 1.34375C6.125 1.28125 6.09375 1.1875 6.09375 1.0625C6.09375 0.96875 6.125 0.875 6.1875 0.8125L6.8125 0.1875Z\" fill=\"#169FE6\"><\/path><\/svg><\/a>\n            <\/div>\n<\/div>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\">OpenCV GrabCut: Foreground Segmentation and Extraction<\/h2>\n\n\n<p>In the first part of this tutorial, we\u2019ll discuss GrabCut, its implementation in OpenCV via the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.grabCut<\/code> function, and its associated parameters.<\/p>\n\n\n<p>From there, we\u2019ll learn how to implement GrabCut with OpenCV via both:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>GrabCut initialization with bounding boxes<\/li><li>GrabCut initialization with mask approximations<\/li><\/ol>\n\n\n\n<p>Afterward, we\u2019ll apply GrabCut and review our results.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"grabcut-in-opencv\">GrabCut in OpenCV<\/h3>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"203\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_paper.jpg\" alt=\"\" class=\"wp-image-15683\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_paper.jpg?size=126x43&amp;lossy=2&amp;strip=1&amp;webp=1 126w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_paper-300x102.jpg?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_paper.jpg?size=378x128&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_paper.jpg?size=504x171&amp;lossy=2&amp;strip=1&amp;webp=1 504w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_paper.jpg?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption> <strong>Figure 1:<\/strong> A selection of methods for performing foreground segmentation. <em>Column f<\/em> shows GrabCut results; compared to the other methodologies, GrabCut results in a high quality output segmentation. In today\u2019s tutorial, we\u2019ll apply GrabCut with OpenCV for foreground and background segmentation and extraction. (image source:  Figure 2 from <a rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\" href=\"https:\/\/cvg.ethz.ch\/teaching\/cvl\/2012\/grabcut-siggraph04.pdf\" target=\"_blank\">Kolmogorov and Blake, 2004<\/a>) <\/figcaption><\/figure><\/div>\n\n<p>The <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.grabCut<\/code> function has the following signature:<\/p>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"1\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"1\">grabCut(img, mask, rect, bgdModel, fgdModel, iterCount[, mode]) ->\n\tmask, bgdModel, fgdModel<\/pre>\n\n\n\n<p><strong>To obtain a complete understanding of the implementation, let\u2019s review each of these parameters:<\/strong><\/p>\n\n\n<ul>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">img<\/code><\/code>: The input image, which GrabCut assumes to be an 8-bit, 3-channel image (i.e., unsigned 8-bit integer in BGR channel ordering).<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code><\/code>: The input\/output mask. This mask is assumed to be a single-channel image with an unsigned 8-bit integer data type. This mask is initialized automatically if you use bounding box initialization (i.e., <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_INIT_WITH_RECT<\/code>); otherwise, GrabCut assumes you are performing mask initialization (<code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_INIT_WITH_MASK<\/code>).<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">rect<\/code><\/code>: The bounding box rectangle that contains the region that we want to segment. This parameter is only used when you set the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mode<\/code> to <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_INIT_WITH_MASK<\/code>).<\/li>\n<li><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">bgModel<\/code>: Temporary array used by GrabCut internally when modeling the <em>background<\/em>.<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">fgModel<\/code><\/code>: Temporary array used by GrabCut when modeling the <em>foreground.<\/em><\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">iterCount<\/code><\/code>: Number of iterations GrabCut will perform when modeling the foreground versus background. The more iterations, the longer GrabCut will run, and ideally the results will be better.<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mode<\/code><\/code>: Either <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_INIT_WITH_RECT<\/code> or <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_INIT_WITH_MASK<\/code>, depending on whether you are initializing GrabCut with a bounding box or a mask, respectively.<\/li>\n<\/ul>\n<div class=\"cl-preview-section\">\n<p>OpenCV\u2019s GrabCut implementation returns a 3-tuple of:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ul>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code><\/code>: The output mask after applying GrabCut<\/li>\n<li><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">bgModel<\/code>: The temporary array used to model the background (you can ignore this value)<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">fgModel<\/code><\/code>: The temporary array for the foreground (again, you can ignore this value)<\/li>\n<\/ul>\n<\/div>\n<p>Now that we have an understanding of the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.grabCut<\/code> function including its parameters and the values that it returns, let\u2019s move on to applying GrabCut to an example computer vision project.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"configuring-your-development-environment\">Configuring your development environment<\/h3>\n\n\n\n<p>You can set up your system today with a Python virtual environment containing OpenCV by following my <em><a href=\"https:\/\/pyimagesearch.com\/2018\/09\/19\/pip-install-opencv\/\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">pip install opencv<\/a><\/em> tutorial (instructions included for Ubuntu, macOS, and Raspbian).<\/p>\n\n\n\n<p>Please note that <a href=\"https:\/\/pyimagesearch.com\/faqs\/single-faq\/can-you-help-me-do-___-on-windows\/\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">PyImageSearch does not recommend or support Windows for computer vision and deep learning development<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"project-structure\">Project structure<\/h3>\n\n\n<p>Before we move on, use the <strong><em>\u201cDownloads\u201d<\/em><\/strong> section of today&#8217;s tutorial to grab the .zip associated with this blog post. From there, let\u2019s inspect the layout of the files and folders directly in our terminal with the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">tree<\/code> command:<\/p>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"1\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"2\">$ tree --dirsfirst\n.\n\u251c\u2500\u2500 images\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 adrian.jpg\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 lighthouse.png\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 lighthouse_mask.png\n\u251c\u2500\u2500 grabcut_bbox.py\n\u2514\u2500\u2500 grabcut_mask.py\n\n1 directory, 5 files<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Our project today consists of one folder of <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">images\/<\/code> and two Python scripts:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ul>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">images\/<\/code><\/code>: Two input photos and one <em>manually<\/em> created approximation mask image<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">grabcut_bbox.py<\/code><\/code>: A script that accomplishes GrabCut by means of bounding box initialization<\/li>\n<li><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">grabcut_mask.py<\/code>: Performs GrabCut via mask initialization<\/li>\n<\/ul>\n<\/div>\n\n\n<p>Using both of the Python scripts, we are going to learn how to perform GrabCut using two methods (bounding box initialization vs. mask initialization). We\u2019ll begin with the bounding box approach in the next section.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"grabcut-with-opencv-initialization-with-bounding-boxes\">GrabCut with OpenCV: Initialization with bounding boxes<\/h3>\n\n\n\n<p>Let\u2019s get started implementing GrabCut with OpenCV \u2014 we\u2019ll start by reviewing the bounding box implementation method.<\/p>\n\n\n\n<p>Here, we\u2019ll specify the bounding box of the object we want to segment in the image. The bounding box could be generated by:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Manually examining the image and labeling the <em>(x, y)<\/em>-coordinates of the bounding box<\/li><li>Applying a Haar cascade<\/li><li>Using HOG + Linear SVM to detect the object<\/li><li>Utilizing deep learning-based object detectors such as Faster R-CNN, SSDs, YOLO, etc.<\/li><\/ul>\n\n\n\n<p><strong>As long as the algorithm generates a bounding box, you can use it in conjunction with GrabCut.<\/strong><\/p>\n\n\n\n<p>For the purposes of our demo script today, we will <strong><em>manually<\/em><\/strong> define the bounding box <em>(x, y)<\/em>-coordinates (i.e., rather than applying an automated object detector).<\/p>\n\n\n\n<p>Let\u2019s take a look at the bounding box initialization method of GrabCut now.<\/p>\n\n\n<p>Open up a new file, name it <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">grabcut_bbox.py<\/code>, and insert the following code:<\/p>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"1\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"3\"># import the necessary packages\nimport numpy as np\nimport argparse\nimport time\nimport cv2\nimport os\n\n# construct the argument parser and parse the arguments\nap = argparse.ArgumentParser()\nap.add_argument(\"-i\", \"--image\", type=str,\n\tdefault=os.path.sep.join([\"images\", \"adrian.jpg\"]),\n\thelp=\"path to input image that we'll apply GrabCut to\")\nap.add_argument(\"-c\", \"--iter\", type=int, default=10,\n\thelp=\"# of GrabCut iterations (larger value => slower runtime)\")\nargs = vars(ap.parse_args())<\/pre>\n\n\n\n<p>We begin this script with a selection of imports, namely OpenCV and NumPy (the rest are built into Python). Please refer to the <em>\u201cConfiguring your development environment\u201d<\/em> section above to install Python, OpenCV, and associated software on your system.<\/p>\n\n\n\n<p>Our script handles two <a rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\" href=\"https:\/\/pyimagesearch.com\/2018\/03\/12\/python-argparse-command-line-arguments\/\" target=\"_blank\">command line arguments<\/a>:<\/p>\n\n\n<div class=\"cl-preview-section\">\n<ul>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code><\/code>: The path to your input image. By <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">default<\/code>, we\u2019ll use the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">adrian.jpg<\/code> image in the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">images\/<\/code> directory.<\/li>\n<li><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--iter<\/code>: The number of GrabCut iterations to perform, where smaller values lead to faster overall time and larger values result in a slower runtime (but ideally better segmentation results)<\/li>\n<\/ul>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Let\u2019s go ahead and load our input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code> and allocate space for an equivalently sized mask:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"17\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"4\"># load the input image from disk and then allocate memory for the\n# output mask generated by GrabCut -- this mask should hae the same\n# spatial dimensions as the input image\nimage = cv2.imread(args[\"image\"])\nmask = np.zeros(image.shape[:2], dtype=\"uint8\")<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Here, <strong>Line 20<\/strong> loads your input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code> from disk and <strong>Line 21<\/strong> creates a <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> (i.e., empty image) with the same dimensions. The <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> will soon be populated with the results of the GrabCut algorithm.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Next, we will <strong><em>manually<\/em><\/strong> define the coordinates of the face in the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">adrian.jpg<\/code> image:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"23\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"5\"># define the bounding box coordinates that approximately define my\n# face and neck region (i.e., all visible skin)\nrect = (151, 43, 236, 368)<\/pre>\n\n\n\n<p><strong>Line 25<\/strong> defines the bounding box coordinates of the face in the image. These <em>(x, y)<\/em>-coordinates were determined <strong><em>manually<\/em><\/strong> by means of a mouse hovering over pixels in the image and me jotting them down. You can accomplish this with most photo editing software including Photoshop or free alternatives such as GIMP and other apps you find online.<\/p>\n\n\n<div class=\"cl-preview-section\">\n<p>It is important to note here that while these face <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">rect<\/code> coordinates were determined manually, any object detector could do the job. Given that our first example is a face, you could have chosen a Haar, HOG, or <a href=\"https:\/\/pyimagesearch.com\/2018\/02\/26\/face-detection-with-opencv-and-deep-learning\/\" target=\"_blank\" rel=\"noopener noreferrer\">DL-based face detector<\/a> to find the bounding box coordinates of the face (substitute a different object detector for different types of objects).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>In this next code block, we\u2019ll will <strong>execute the GrabCut algorithm with bounding box initialization<\/strong> on our input:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"27\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"6\"># allocate memory for two arrays that the GrabCut algorithm internally\n# uses when segmenting the foreground from the background\nfgModel = np.zeros((1, 65), dtype=\"float\")\nbgModel = np.zeros((1, 65), dtype=\"float\")\n\n# apply GrabCut using the the bounding box segmentation method\nstart = time.time()\n(mask, bgModel, fgModel) = cv2.grabCut(image, mask, rect, bgModel,\n\tfgModel, iterCount=args[\"iter\"], mode=cv2.GC_INIT_WITH_RECT)\nend = time.time()\nprint(\"[INFO] applying GrabCut took {:.2f} seconds\".format(end - start))<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Before we perform the GrabCut computation, we need two empty arrays for GrabCut to use internally when segmenting the foreground from the background (<code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">fgModel<\/code> and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">bgModel<\/code>). <strong>Lines 29 and 30<\/strong> generate both arrays with NumPy\u2019s <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">zeros<\/code> method.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>From there, <strong>Lines 34 and 35<\/strong> apply GrabCut (timestamps are collected before\/after the operation), and the elapsed time is printed via <strong>Line 37<\/strong>.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>GrabCut returns our populated <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> as well as two arrays that we can ignore. If you need a <strong>review of the GrabCut method signature<\/strong> including the input parameters and return values, please refer to the <em>\u201cGrabCut in OpenCV\u201d<\/em> section above.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Let\u2019s go ahead and post-process our <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code>:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"39\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"7\"># the output mask has for possible output values, marking each pixel\n# in the mask as (1) definite background, (2) definite foreground,\n# (3) probable background, and (4) probable foreground\nvalues = (\n\t(\"Definite Background\", cv2.GC_BGD),\n\t(\"Probable Background\", cv2.GC_PR_BGD),\n\t(\"Definite Foreground\", cv2.GC_FGD),\n\t(\"Probable Foreground\", cv2.GC_PR_FGD),\n)\n\n# loop over the possible GrabCut mask values\nfor (name, value) in values:\n\t# construct a mask that for the current value\n\tprint(\"[INFO] showing mask for '{}'\".format(name))\n\tvalueMask = (mask == value).astype(\"uint8\") * 255\n\n\t# display the mask so we can visualize it\n\tcv2.imshow(name, valueMask)\n\tcv2.waitKey(0)<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p><strong>Lines 42-47<\/strong> define possible <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">values<\/code> in the output GrabCut <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> including our definite\/probable backgrounds and foregrounds.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>We then proceed to loop over these <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">values<\/code> so that we can visualize each. Inside the loop (<strong>Lines 50-57<\/strong>), we (1) construct a mask for the current <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">value<\/code> and (2) display it until any key is pressed.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>After each of our definite\/probable backgrounds and foregrounds have been displayed, our code will begin generating an <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code> and an <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">output<\/code> image:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"59\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"8\"># we'll set all definite background and probable background pixels\n# to 0 while definite foreground and probable foreground pixels are\n# set to 1\noutputMask = np.where((mask == cv2.GC_BGD) | (mask == cv2.GC_PR_BGD),\n\t0, 1)\n\n# scale the mask from the range [0, 1] to [0, 255]\noutputMask = (outputMask * 255).astype(\"uint8\")\n\n# apply a bitwise AND to the image using our mask generated by\n# GrabCut to generate our final output image\noutput = cv2.bitwise_and(image, image, mask=outputMask)<\/pre>\n\n\n\n<p>Here we produce two visualizations:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>GrabCut output mask<\/li><li>Output image (with the background masked out)<\/li><\/ol>\n\n\n<div class=\"cl-preview-section\">\n<p>To produce our GrabCut <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code>, <strong>Lines 62 and 63<\/strong> find all pixels that are either definite background or probable background and set them to <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">0<\/code> \u2014 all other pixels should be marked as <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">1<\/code> (i.e., foreground). Notice how we take advantage of NumPy\u2019s <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">where<\/code> function while OR-ing each mask and setting the values to <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">0<\/code> and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">1<\/code> accordingly. Then, <strong>Line 66<\/strong> scales the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code> from the range <em>[0, 1]<\/em> to <em>[0, 255]<\/em>.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>We then generate our <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">output<\/code> image with the background masked out by means of a <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">bitwise_and<\/code> operation and pass the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code> as the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> parameter (<strong>Line 70<\/strong>).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>At this point, we have:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ul>\n<li>Prepared inputs to the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">grabCut<\/code> function including our input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">image<\/code>, <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code>, <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">rect<\/code> coordinates, and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">fgModel<\/code> and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">bgModel<\/code> zero arrays. Note that the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">rect<\/code> coordinates were determined <em>manually<\/em>.<\/li>\n<li>Executed the GrabCut algorithm.<\/li>\n<li>Generated and visualized our definite\/probable background and foreground masks.<\/li>\n<li>Generated our (1) GrabCut output mask (<code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code>) and our (2) output image with the background masked out (<code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">output<\/code>).<\/li>\n<\/ul>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Let\u2019s go ahead and display our final results:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"72\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"9\"># show the input image followed by the mask and output generated by\n# GrabCut and bitwise masking\ncv2.imshow(\"Input\", image)\ncv2.imshow(\"GrabCut Mask\", outputMask)\ncv2.imshow(\"GrabCut Output\", output)\ncv2.waitKey(0)<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>To wrap up, we show each of the following in separate windows:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ul>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">image<\/code><\/code>: Our original input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code><\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code><\/code>: The GrabCut mask<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">output<\/code><\/code>: The results of our hard work \u2014 only the foreground from our original image (i.e., the background has been masked out by means of GrabCut)<\/li>\n<\/ul>\n<\/div>\n\n\n<p>Now that GrabCut with bounding box initialization has been implemented, let\u2019s move on to applying it to our input images.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"bounding-box-grabcut-results\">Bounding box GrabCut results<\/h3>\n\n\n\n<p>Start by using the <strong><em>\u201cDownloads\u201d<\/em><\/strong> section of this blog post to download the source code and example image.<\/p>\n\n\n\n<p>From there, open up a terminal, and execute the following command:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"1\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"10\">$ python grabcut_bbox.py\n[INFO] applying GrabCut took 1.08 seconds\n[INFO] showing mask for 'Definite Background'\n[INFO] showing mask for 'Probable Background'\n[INFO] showing mask for 'Definite Foreground'\n[INFO] showing mask for 'Probable Foreground'<\/pre>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"301\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_face.png\" alt=\"\" class=\"wp-image-15684\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_face.png?size=126x63&amp;lossy=2&amp;strip=1&amp;webp=1 126w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_face-300x151.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_face.png?size=378x190&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_face.png?size=504x253&amp;lossy=2&amp;strip=1&amp;webp=1 504w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_face.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption> <strong>Figure 2:<\/strong> The GrabCut bounding box initialization method requires that the bounding box coordinates be provided as input to the algorithm. Here, I\u2019ve <em>manually<\/em> found the coordinates of the bounding box; however, you could apply any type of object detector to grab the <em>(x, y)<\/em>-coordinates. Either way, you\u2019ll be able to apply GrabCut with OpenCV to perform foreground segmentation and extraction. <\/figcaption><\/figure><\/div>\n\n<p>On the <em>left,<\/em> you can see the original input image, while on the <em>right,<\/em> you can see the same face with a bounding box drawn around the face\/neck region (this bounding box corresponds to the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">rect<\/code> variable in the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">grabcut_bbox.py<\/code> script).<\/p>\n\n\n<p>Our goal here is to automatically segment the face and neck region from the above image using GrabCut and OpenCV.<\/p>\n\n\n\n<p>Next, you can see our output from <strong>Lines 45-60<\/strong> where we visualize the definite and probable background and foreground segmentations:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_montage.png\" alt=\"\" class=\"wp-image-15685\" width=\"500\" height=\"500\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_montage-150x150.png?lossy=2&amp;strip=1&amp;webp=1 150w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_montage-300x300.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_montage.png?size=378x378&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_montage.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 500px) 100vw, 500px\" \/><figcaption> <strong>Figure 3:<\/strong> The various GrabCut masks (bounding box initialization) visualized with OpenCV. <em>Top-left:<\/em> Definite background. <em>Top-right:<\/em> Probable background. <em>Bottom-left:<\/em> Definite foreground. <em>Bottom-right:<\/em> Probable foreground. <\/figcaption><\/figure><\/div>\n\n<div class=\"cl-preview-section\">\n<p>These values map to:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ol>\n<li><strong>Definite background<\/strong> <em>(top-left):<\/em> <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_BGD<\/code><\/li>\n<li><strong>Probable background<\/strong> <em>(top-right)<\/em>: <code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_PR_BGD<\/code><\/code><\/li>\n<li><strong>Definite foreground<\/strong> <em>(bottom-left)<\/em>: <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_FGD<\/code><\/li>\n<li><strong>Probable foreground<\/strong> <em>(bottom-right)<\/em>: <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_PR_FGD<\/code><\/li>\n<\/ol>\n<\/div>\n\n\n<p>Finally, we have the output of GrabCut itself:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_output.png\" alt=\"\" class=\"wp-image-15686\" width=\"500\" height=\"500\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_output-150x150.png?lossy=2&amp;strip=1&amp;webp=1 150w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_output-300x300.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_output.png?size=378x378&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_bbox_output.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 500px) 100vw, 500px\" \/><figcaption> <strong>Figure 4:<\/strong> <em>Left:<\/em> Our original input image of me. <em>Right:<\/em> GrabCut mask via bounding box initialization. <em>Bottom:<\/em> Our output image where the foreground is segmented from the background via GrabCut masking. Each of these images was generated by means of OpenCV and applying GrabCut for foreground segmentation and extraction. <\/figcaption><\/figure><\/div>\n\n\n<p>On the <em>left,<\/em> we have our original input image.<\/p>\n\n\n\n<p>The <em>right<\/em> shows the output mask generated by GrabCut, while the <em>bottom<\/em> shows the output of applying the mask to the input image \u2014 <strong>notice how my face and neck region is cleanly segmented and extracted via GrabCut.<\/strong><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"grabcut-with-opencv-initialization-with-masks\">GrabCut with OpenCV: Initialization with masks<\/h3>\n\n\n\n<p>Previously, we learned how to initialize OpenCV\u2019s GrabCut using bounding boxes \u2014 but there\u2019s actually a <em>second<\/em> method to initialize GrabCut.<\/p>\n\n\n\n<p><strong>Using masks, we can supply the <em>approximate<\/em> segmentation of the object in the image.<\/strong> GrabCut can then iteratively apply graph cuts to improve the segmentation and extract the foreground from the image.<\/p>\n\n\n\n<p>These masks could be generated by:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Manually creating them in photo editing software such as Photoshop, GIMP, etc.<\/li><li>Applying basic image processing operations such as thresholding, edge detection, contour filtering, etc.<\/li><li>Utilizing deep learning-based segmentation networks (ex., Mask R-CNN and U-Net)<\/li><\/ul>\n\n\n\n<p><strong>How the mask is generated is irrelevant to GrabCut.<\/strong> As long as you have a mask that approximates the segmentation of the object in an image, <em>you can use GrabCut to further improve the segmentation.<\/em><\/p>\n\n\n\n<p>Let\u2019s see how GrabCut with mask initialization works.<\/p>\n\n\n<p>Open up the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">grabcut_mask.py<\/code> file in your project directory structure, and insert the following code:<\/p>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"1\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"11\"># import the necessary packages\nimport numpy as np\nimport argparse\nimport time\nimport cv2\nimport os\n\n# construct the argument parser and parse the arguments\nap = argparse.ArgumentParser()\nap.add_argument(\"-i\", \"--image\", type=str,\n\tdefault=os.path.sep.join([\"images\", \"lighthouse.png\"]),\n\thelp=\"path to input image that we'll apply GrabCut to\")\nap.add_argument(\"-mask\", \"--mask\", type=str,\n\tdefault=os.path.sep.join([\"images\", \"lighthouse_mask.png\"]),\n\thelp=\"path to input mask\")\nap.add_argument(\"-c\", \"--iter\", type=int, default=10,\n\thelp=\"# of GrabCut iterations (larger value => slower runtime)\")\nargs = vars(ap.parse_args())<\/pre>\n\n\n\n<p>Again, our most notable imports are OpenCV and NumPy. Please follow the <em>\u201cConfiguring your development environment\u201d<\/em> section instructions if you need to set up your system to perform GrabCut with mask initialization.<\/p>\n\n\n<div class=\"cl-preview-section\">\n<p>Our script handles three <a href=\"https:\/\/pyimagesearch.com\/2018\/03\/12\/python-argparse-command-line-arguments\/\" target=\"_blank\" rel=\"noopener noreferrer\">command line arguments<\/a>:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ul>\n<li><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code>: The path to your input image. This time, by <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">default<\/code>, we\u2019ll use the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">lighthouse.png<\/code> photo available in the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">images\/<\/code> directory.<\/li>\n<li><code><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--mask<\/code><\/code>: The path to input approximation mask associated with the input image. Again, you could create this mask in a number of ways listed at the top of this section, but for the sake of this example, I <em>manually<\/em> created the mask.<\/li>\n<li><code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--iter<\/code>: The number of GrabCut iterations to perform, where smaller values lead to faster overall time and larger values result in a slower runtime (but ideally better segmentation results)<\/li>\n<\/ul>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Now that our imports and command line arguments are taken care of, let\u2019s go ahead and load our input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code> and input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--mask<\/code>:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"20\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"12\"># load the input image and associated mask from disk\nimage = cv2.imread(args[\"image\"])\nmask = cv2.imread(args[\"mask\"], cv2.IMREAD_GRAYSCALE)\n\n# apply a bitwise mask to show what the rough, approximate mask would\n# give us\nroughOutput = cv2.bitwise_and(image, image, mask=mask)\n\n# show the rough, approximated output\ncv2.imshow(\"Rough Output\", roughOutput)\ncv2.waitKey(0)<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Before we get into the weeds of this second GrabCut method, we need to load our input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--image<\/code> and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">--mask<\/code> from disk (<strong>Lines 21 and 22<\/strong>).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Please note that our rough mask was <strong><em>manually<\/em><\/strong> generated for the sake of this example (using Photoshop\/GIMP); however, in a future post we\u2019ll be showing you how to <em>automatically<\/em> generate the mask via a deep learning Mask R-CNN.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p><strong>Line 26<\/strong> applies a bitwise AND to the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">image<\/code> using the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code>, resulting in our rough approximation of our foreground segmentation. Subsequently <strong>Lines 29 and 30<\/strong> display the approximation until any key is pressed.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>From here, we\u2019ll set our probable\/definite foreground values into the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> array:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"32\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"13\"># any mask values greater than zero should be set to probable\n# foreground\nmask[mask > 0] = cv2.GC_PR_FGD\nmask[mask == 0] = cv2.GC_BGD<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Any pixel values in the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">mask<\/code> greater than zero are set to probable foreground (<strong>Line 34<\/strong>); all other pixel values are set to definite background (<strong>Line 35<\/strong>).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>We\u2019re now ready to <strong>apply GrabCut with mask initialization:<\/strong><\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"37\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"14\"># allocate memory for two arrays that the GrabCut algorithm internally\n# uses when segmenting the foreground from the background\nfgModel = np.zeros((1, 65), dtype=\"float\")\nbgModel = np.zeros((1, 65), dtype=\"float\")\n\n# apply GrabCut using the the mask segmentation method\nstart = time.time()\n(mask, bgModel, fgModel) = cv2.grabCut(image, mask, None, bgModel,\n\tfgModel, iterCount=args[\"iter\"], mode=cv2.GC_INIT_WITH_MASK)\nend = time.time()\nprint(\"[INFO] applying GrabCut took {:.2f} seconds\".format(end - start))<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Again, we allocate memory for the foreground and background models of GrabCut (<strong>Lines 39 and 40<\/strong>).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>And then we execute GrabCut on the image using the approximate mask segmentation (<strong>Lines 44 and 45<\/strong>). Note how the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">rect<\/code> parameter is set to <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">None<\/code> (we don\u2019t need it for this method), unlike the first bounding box-based method described in this blog post.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Moving on, we\u2019ll post-process the results:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"49\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"15\"># the output mask has for possible output values, marking each pixel\n# in the mask as (1) definite background, (2) definite foreground,\n# (3) probable background, and (4) probable foreground\nvalues = (\n\t(\"Definite Background\", cv2.GC_BGD),\n\t(\"Probable Background\", cv2.GC_PR_BGD),\n\t(\"Definite Foreground\", cv2.GC_FGD),\n\t(\"Probable Foreground\", cv2.GC_PR_FGD),\n)\n\n# loop over the possible GrabCut mask values\nfor (name, value) in values:\n\t# construct a mask that for the current value\n\tprint(\"[INFO] showing mask for '{}'\".format(name))\n\tvalueMask = (mask == value).astype(\"uint8\") * 255\n\n\t# display the mask so we can visualize it\n\tcv2.imshow(name, valueMask)\n\tcv2.waitKey(0)<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>This block should look especially familiar. In fact, it is identical to a block in our first GrabCut method code walkthrough.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Again, we define definite\/probable foreground and background <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">values<\/code> (<strong>Lines 52-57<\/strong>) and display each of the resulting <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">valueMask<\/code> images (<strong>Lines 60-67<\/strong>).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Next, we\u2019ll prepare our GrabCut mask and output image with the background removed:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"69\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"18\"># set all definite background and probable background pixels to 0\n# while definite foreground and probable foreground pixels are set\n# to 1, then scale teh mask from the range [0, 1] to [0, 255]\noutputMask = np.where((mask == cv2.GC_BGD) | (mask == cv2.GC_PR_BGD),\n\t0, 1)\noutputMask = (outputMask * 255).astype(\"uint8\")\n\n# apply a bitwise AND to the image using our mask generated by\n# GrabCut to generate our final output image\noutput = cv2.bitwise_and(image, image, mask=outputMask)<\/pre>\n\n\n<div class=\"cl-preview-section\">\n<p>Again, this code on <strong>Lines 72-78<\/strong> should be familiar at this point (they are identical to the previous script).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>Here, we find all pixels that are either definite background or probable background and set them to <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">0<\/code>; all other pixels are marked as <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">1<\/code> (i.e., foreground). We then scale the mask to the range <em>[0, 255]<\/em>.<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>We then apply a bitwise AND operation to the input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">image<\/code> using the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code>, resulting in the background being removed (masked out).<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>And finally we display the results on screen:<\/p>\n<\/div>\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"80\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"16\"># show the input image followed by the mask and output generated by\n# GrabCut and bitwise masking\ncv2.imshow(\"Input\", image)\ncv2.imshow(\"GrabCut Mask\", outputMask)\ncv2.imshow(\"GrabCut Output\", output)\ncv2.waitKey(0)<\/pre>\n\n\n<p>Again, to conclude our script, we show the input <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">image<\/code>, GrabCut <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">outputMask<\/code>, and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">output<\/code> of GrabCut after applying the mask.<\/p>\n\n\n<p>With GrabCut mask initialization now implemented, let\u2019s move on to testing it with our own example images.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"mask-grabcut-results\">Mask GrabCut results<\/h3>\n\n\n\n<p>We are now ready to use OpenCV and GrabCut to segment an image via mask initialization.<\/p>\n\n\n\n<p>Start by using the <strong><em>\u201cDownloads\u201d<\/em><\/strong> section of this tutorial to download the source code and example images.<\/p>\n\n\n\n<p>From there, open up a terminal, and execute the following command:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"1\" data-enlighter-title=\"OpenCV GrabCut: Foreground Segmentation and Extraction\" data-enlighter-group=\"17\">$ python grabcut_mask.py\n[INFO] applying GrabCut took 0.56 seconds\n[INFO] showing mask for 'Definite Background'\n[INFO] showing mask for 'Probable Background'\n[INFO] showing mask for 'Definite Foreground'\n[INFO] showing mask for 'Probable Foreground'<\/pre>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"206\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_rough_output.png\" alt=\"\" class=\"wp-image-15687\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_rough_output.png?size=126x43&amp;lossy=2&amp;strip=1&amp;webp=1 126w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_rough_output-300x103.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_rough_output.png?size=378x130&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_rough_output.png?size=504x173&amp;lossy=2&amp;strip=1&amp;webp=1 504w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_rough_output.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption> <strong>Figure 5:<\/strong> <em>Left:<\/em> Our original photo of a lighthouse. <em>Right:<\/em> The output of applying GrabCut via mask initialization.<\/figcaption><\/figure><\/div>\n\n\n<p>On the <em>left,<\/em> you can see our original input image. On the <em>right<\/em> you can see the output of applying GrabCut via mask initialization.<\/p>\n\n\n\n<p>The image on the <em>right<\/em> shows the mask associated with the lighthouse. For the sake of this blog post\/example I manually created this mask in Photoshop; however any algorithm capable of producing a mask could be used here (ex., basic image processing via thresholding, edge detection, contours; deep learning-based segmentation; etc.) <strong>Notice how the mask\/segmentation isn\u2019t very \u201cclean\u201d \u2014 we can easily see the blue sky of the background \u201cleaking\u201d into our mask.<\/strong><\/p>\n\n\n\n<p>From there, we can visualize our definite and probable masks for the background and foreground, respectively:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"412\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_montage.png\" alt=\"\" class=\"wp-image-15688\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_montage.png?size=126x87&amp;lossy=2&amp;strip=1&amp;webp=1 126w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_montage-300x206.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_montage.png?size=378x260&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_montage.png?size=504x346&amp;lossy=2&amp;strip=1&amp;webp=1 504w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_montage.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption> <strong>Figure 6:<\/strong> The various GrabCut masks (mask initialization) visualized with OpenCV. <em>Top-left:<\/em> Definite background. <em>Top-right:<\/em> Probable background. <em>Bottom-left:<\/em> Definite foreground. <em>Bottom-right:<\/em> Probable foreground. <\/figcaption><\/figure><\/div>\n\n<div class=\"cl-preview-section\">\n<p>These values map to:<\/p>\n<\/div>\n<div class=\"cl-preview-section\">\n<ol>\n<li><strong>Definite background<\/strong> <em>(top-left):<\/em> <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_BGD<\/code><\/li>\n<li><strong>Probable background<\/strong> <em>(top-right)<\/em>: <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_PR_BGD<\/code><\/li>\n<li><strong>Definite foreground<\/strong> <em>(bottom-left)<\/em>: <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_FGD<\/code><\/li>\n<li><strong>Probable foreground<\/strong> <em>(bottom-right)<\/em>: <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.GC_PR_FGD<\/code><\/li>\n<\/ol>\n<\/div>\n<div class=\"cl-preview-section\">\n<p>And finally, we have the output of OpenCV\u2019s GrabCut with mask initialization:<\/p>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"412\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_output.png\" alt=\"\" class=\"wp-image-15689\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_output.png?size=126x87&amp;lossy=2&amp;strip=1&amp;webp=1 126w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_output-300x206.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_output.png?size=378x260&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_output.png?size=504x346&amp;lossy=2&amp;strip=1&amp;webp=1 504w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_output.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption> <strong>Figure 7:<\/strong> <em>Left:<\/em> Our original input image of a lighthouse. <em>Right:<\/em> GrabCut mask via mask initialization. <em>Bottom:<\/em> Our output image where the foreground segmented from the background via GrabCut masking. Each of these images was generated by means of OpenCV and applying GrabCut for foreground segmentation and extraction. <\/figcaption><\/figure><\/div>\n\n\n<p>For reference, the <em>left<\/em> displays our input image.<\/p>\n\n\n\n<p>The <em>right<\/em> shows our output mask generated by GrabCut, while the <em>bottom<\/em> displays the output of applying the mask created by GrabCut to the original input image.<\/p>\n\n\n\n<p><strong>Notice that we have cleaned up our segmentation<\/strong> \u2014 the blue background from the sky has been removed, while the lighthouse is left as the foreground.<\/p>\n\n\n\n<p>The only problem is that the area where the actual spotlight sits in the lighthouse has been marked as background:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"412\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_issue.png\" alt=\"\" class=\"wp-image-15690\" srcset=\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_issue.png?size=126x87&amp;lossy=2&amp;strip=1&amp;webp=1 126w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_issue-300x206.png?lossy=2&amp;strip=1&amp;webp=1 300w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_issue.png?size=378x260&amp;lossy=2&amp;strip=1&amp;webp=1 378w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_issue.png?size=504x346&amp;lossy=2&amp;strip=1&amp;webp=1 504w, https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2020\/07\/opencv_grabcut_mask_issue.png?lossy=2&amp;strip=1&amp;webp=1 600w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption> <strong>Figure 8:<\/strong> Click to view this in your browser with the option to zoom in (<code>ctrl + \"+\"<\/code>). As you can observe, the results of GrabCut with mask initialization aren\u2019t perfect. I suggest you use the <em>definite background<\/em> mask <code>value<\/code> result rather than both the <em>definite\/probable foreground<\/em> masks in this specific case. You will need to invert the definite background mask image using your OpenCV\/NumPy knowledge. From there, your GrabCut mask initialization method will produce a better foreground segmentation. <\/figcaption><\/figure><\/div>\n\n\n<p>The problem here is that the area where the light sits in the lighthouse is more-or-less transparent, causing the blue sky background to shine through, thereby causing GrabCut to mark this area as background.<\/p>\n\n\n<p>You could fix this problem by updating your mask to use the definite background (i.e., <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv.GC_BGD<\/code>) when loading your mask from disk. I will leave this as an exercise to you, the reader, to implement.<\/p>\n\n\n<h3 class=\"wp-block-heading\" id=\"why-grabcut-is-good-but-not-perfect\">Why GrabCut is good, but not perfect<\/h3>\n\n\n\n<p>GrabCut is one of my favorite computer vision algorithms ever invented, but it\u2019s not perfect.<\/p>\n\n\n\n<p>Furthermore, deep learning-based segmentation networks such as Faster R-CNN and U-Net can <em>automatically<\/em> generate masks that can segment objects (foreground) from their backgrounds \u2014 <strong>does that mean that GrabCut is irrelevant in the age of deep learning?<\/strong><\/p>\n\n\n\n<p><em>Actually, far from it.<\/em><\/p>\n\n\n\n<p>While Faster R-CNN and U-Net are super powerful methods, they can result in masks that are a bit messy. We can use GrabCut to help clean up these masks. I\u2019ll be showing you how to do exactly that in a future blog post.<\/p>\n\n\n\n<div id=\"pitch\" style=\"padding: 40px; width: 100%; background-color: #F4F6FA;\">\r\n\t<h3>What's next? We recommend <a target=\"_blank\" href=\"https:\/\/pyimagesearch.com\/pyimagesearch-university\/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend\">PyImageSearch University<\/a>.<\/h3>\r\n\r\n\t<script src=\"https:\/\/fast.wistia.com\/embed\/medias\/kno0cmko2z.jsonp\" async><\/script><script src=\"https:\/\/fast.wistia.com\/assets\/external\/E-v1.js\" async><\/script><div class=\"wistia_responsive_padding\" style=\"padding:56.25% 0 0 0;position:relative;\"><div class=\"wistia_responsive_wrapper\" style=\"height:100%;left:0;position:absolute;top:0;width:100%;\"><div class=\"wistia_embed wistia_async_kno0cmko2z videoFoam=true\" style=\"height:100%;position:relative;width:100%\"><div class=\"wistia_swatch\" style=\"height:100%;left:0;opacity:0;overflow:hidden;position:absolute;top:0;transition:opacity 200ms;width:100%;\"><img decoding=\"async\" src=\"https:\/\/fast.wistia.com\/embed\/medias\/kno0cmko2z\/swatch\" style=\"filter:blur(5px);height:100%;object-fit:contain;width:100%;\" alt=\"\" aria-hidden=\"true\" onload=\"this.parentNode.style.opacity=1;\" \/><\/div><\/div><\/div><\/div>\r\n\r\n\t<div style=\"margin-top: 32px; margin-bottom: 32px; \">\r\n\t\t<strong>Course information:<\/strong><br\/>\r\n\t\t86+ total classes \u2022 115+ hours hours of on-demand code walkthrough videos \u2022 Last updated: May 2026<br\/>\r\n\t\t<span style=\"color: #169FE6;\">\u2605\u2605\u2605\u2605\u2605<\/span> 4.84 (128 Ratings) \u2022 16,000+ Students Enrolled\r\n\t<\/div>\r\n\r\n\t<p><strong>I strongly believe that if you had the right teacher you could <em>master<\/em> computer vision and deep learning.<\/strong><\/p>\r\n\r\n\t<p>Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?<\/p>\r\n\r\n\t<p>That\u2019s <em>not<\/em> the case.<\/p>\r\n\r\n\t<p>All you need to master computer vision and deep learning is for someone to explain things to you in <em>simple, intuitive<\/em> terms. <em>And that\u2019s exactly what I do<\/em>. My mission is to change education and how complex Artificial Intelligence topics are taught.<\/p>\r\n\r\n\t<p>If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you\u2019ll learn how to <em>successfully<\/em> and <em>confidently<\/em> apply computer vision to your work, research, and projects. Join me in computer vision mastery.<\/p>\r\n\r\n\t<p><strong>Inside PyImageSearch University you'll find:<\/strong><\/p>\r\n\r\n\t<ul style=\"margin-left: 0px;\">\r\n\t\t<li style=\"list-style: none;\">&check; <strong>86+ courses<\/strong> on essential computer vision, deep learning, and OpenCV topics<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; <strong>86 Certificates<\/strong> of Completion<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; <strong>115+ hours hours<\/strong> of on-demand video<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; <strong>Brand new courses released <em>regularly<\/em><\/strong>, ensuring you can keep up with state-of-the-art techniques<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; <strong>Pre-configured Jupyter Notebooks in Google Colab<\/strong><\/li>\r\n\t\t<li style=\"list-style: none;\">&check; Run all code examples in your web browser \u2014 works on Windows, macOS, and Linux (no dev environment configuration required!)<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; Access to <strong>centralized code repos for <em>all<\/em> 540+ tutorials<\/strong> on PyImageSearch<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; <strong> Easy one-click downloads<\/strong> for code, datasets, pre-trained models, etc.<\/li>\r\n\t\t<li style=\"list-style: none;\">&check; <strong>Access<\/strong> on mobile, laptop, desktop, etc.<\/li>\r\n\t<\/ul>\r\n\r\n\t<p style=\"text-align: center;\">\r\n\t\t<a target=\"_blank\" class=\"button link\" href=\"https:\/\/pyimagesearch.com\/pyimagesearch-university\/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend\" style=\"background-color: #6DC713; border-bottom: none;\">Click here to join PyImageSearch University<\/a>\r\n\t<\/p>\r\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"summary\">Summary<\/h2>\n\n\n\n<p>In this tutorial, you learned how to use OpenCV and the GrabCut algorithm to perform foreground segmentation and extraction.<\/p>\n\n\n<p>The GrabCut algorithm is implemented in OpenCV via the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">cv2.grabCut<\/code> function and can be initialized via either:<\/p>\n\n\n<ol class=\"wp-block-list\"><li>A bounding box that specifies the location of the object you want to segment in the input image<\/li><li>A mask that approximates the pixel-wise location of the object in the image<\/li><\/ol>\n\n\n\n<p>The GrabCut algorithm takes the bounding box\/mask and then iteratively approximates the foreground and background.<\/p>\n\n\n\n<p>While deep learning-based image segmentation networks (ex., Mask R-CNN and U-Net) tend to be more powerful in actually <em>detecting<\/em> and <em>approximating the mask<\/em> of objects in an image, we know that these masks can be less than perfect \u2014 <strong>we can actually use GrabCut to clean up \u201cmessy\u201d masks returned by these segmentation networks!<\/strong><\/p>\n\n\n\n<p>In a future tutorial, I\u2019ll show you how to do exactly that.<\/p>\n\n\n\n<p><strong>To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), <em>simply enter your email address in the form below!<\/em><\/strong><\/p>\n\n\n\n<div id=\"download-the-code\" class=\"post-cta-wrap\">\n<div class=\"gpd-post-cta\">\n\t<div class=\"gpd-post-cta-content\">\n\t\t\n\n\t\t\t<div class=\"gpd-post-cta-top\">\n\t\t\t\t<div class=\"gpd-post-cta-top-image\"><img decoding=\"async\" src=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/01\/cta-source-guide-1.png\" alt=\"\" \/><\/div>\n\t\t\t\t\n\t\t\t\t<div class=\"gpd-post-cta-top-title\"><h4>Download the Source Code and FREE 17-page Resource Guide<\/h4><\/div>\n\t\t\t\t<div class=\"gpd-post-cta-top-desc\"><p>Enter your email address below to get a .zip of the code and a <strong>FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.<\/strong> Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!<\/p><\/div>\n\n\n\t\t\t<\/div>\n\n\t\t\t<div class=\"gpd-post-cta-bottom\">\n\t\t\t\t<form id=\"footer-cta-code\" class=\"footer-cta\" action=\"https:\/\/www.getdrip.com\/forms\/4130035\/submissions\" method=\"post\" target=\"blank\" data-drip-embedded-form=\"4130035\">\n\t\t\t\t\t<input name=\"fields[email]\" type=\"email\" value=\"\" placeholder=\"Your email address\" class=\"form-control\" \/>\n\n\t\t\t\t\t<button type=\"submit\">Download the code!<\/button>\n\n\t\t\t\t\t<div style=\"display: none;\" aria-hidden=\"true\"><label for=\"website\">Website<\/label><br \/><input type=\"text\" id=\"website\" name=\"website\" tabindex=\"-1\" autocomplete=\"false\" value=\"\" \/><\/div>\n\t\t\t\t<\/form>\n\t\t\t<\/div>\n\n\n\t\t\n\t<\/div>\n\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, you will learn how to use OpenCV and GrabCut to perform foreground segmentation and extraction. Prior to deep learning and instance\/semantic segmentation networks such as Mask R-CNN, U-Net, etc., GrabCut was the method to accurately segment the&hellip;<\/p>\n","protected":false},"author":1,"featured_media":15682,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[14,276,27],"tags":[592,55,146],"class_list":{"0":"post-15680","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-image-processing","8":"category-opencv","9":"category-tutorials","10":"tag-grabcut","11":"tag-image-processing-2","12":"tag-segmentation","13":"entry"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v19.6.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>OpenCV GrabCut: Foreground Segmentation and Extraction - PyImageSearch<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"OpenCV GrabCut: Foreground Segmentation and Extraction - PyImageSearch\" \/>\n<meta property=\"og:description\" content=\"In this tutorial, you will learn how to use OpenCV and GrabCut to perform foreground segmentation and extraction. Prior to deep learning and instance\/semantic segmentation networks such as Mask R-CNN, U-Net, etc., GrabCut was the method to accurately segment the&hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/\" \/>\n<meta property=\"og:site_name\" content=\"PyImageSearch\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-27T14:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-20T10:15:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header.png\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"601\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Adrian Rosebrock\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Adrian Rosebrock\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"22 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/\",\"url\":\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/\",\"name\":\"OpenCV GrabCut: Foreground Segmentation and Extraction - PyImageSearch\",\"isPartOf\":{\"@id\":\"https:\/\/pyimagesearch.com\/#website\"},\"datePublished\":\"2020-07-27T14:00:00+00:00\",\"dateModified\":\"2023-02-20T10:15:13+00:00\",\"author\":{\"@id\":\"https:\/\/pyimagesearch.com\/#\/schema\/person\/5901b399e2f20b986362a00636181cca\"},\"breadcrumb\":{\"@id\":\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/pyimagesearch.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"OpenCV GrabCut: Foreground Segmentation and Extraction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/pyimagesearch.com\/#website\",\"url\":\"https:\/\/pyimagesearch.com\/\",\"name\":\"PyImageSearch\",\"description\":\"You can master Computer Vision, Deep Learning, and OpenCV - PyImageSearch\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/pyimagesearch.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/pyimagesearch.com\/#\/schema\/person\/5901b399e2f20b986362a00636181cca\",\"name\":\"Adrian Rosebrock\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/pyimagesearch.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/57e1f3a95feeb9b113f80510f086d7d81b6f62badd9bd69134e51037a8b79925?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/57e1f3a95feeb9b113f80510f086d7d81b6f62badd9bd69134e51037a8b79925?s=96&d=mm&r=g\",\"caption\":\"Adrian Rosebrock\"},\"description\":\"Hi there, I\u2019m Adrian Rosebrock, PhD. All too often I see developers, students, and researchers wasting their time, studying the wrong things, and generally struggling to get started with Computer Vision, Deep Learning, and OpenCV. I created this website to show you what I believe is the best possible way to get your start.\",\"url\":\"https:\/\/pyimagesearch.com\/author\/adrian\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"OpenCV GrabCut: Foreground Segmentation and Extraction - PyImageSearch","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/","og_locale":"en_US","og_type":"article","og_title":"OpenCV GrabCut: Foreground Segmentation and Extraction - PyImageSearch","og_description":"In this tutorial, you will learn how to use OpenCV and GrabCut to perform foreground segmentation and extraction. Prior to deep learning and instance\/semantic segmentation networks such as Mask R-CNN, U-Net, etc., GrabCut was the method to accurately segment the&hellip;","og_url":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/","og_site_name":"PyImageSearch","article_published_time":"2020-07-27T14:00:00+00:00","article_modified_time":"2023-02-20T10:15:13+00:00","og_image":[{"width":600,"height":601,"url":"https:\/\/pyimagesearch.com\/wp-content\/uploads\/2020\/07\/opencv_grabcut_header.png","type":"image\/png"}],"author":"Adrian Rosebrock","twitter_misc":{"Written by":"Adrian Rosebrock","Est. reading time":"22 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/","url":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/","name":"OpenCV GrabCut: Foreground Segmentation and Extraction - PyImageSearch","isPartOf":{"@id":"https:\/\/pyimagesearch.com\/#website"},"datePublished":"2020-07-27T14:00:00+00:00","dateModified":"2023-02-20T10:15:13+00:00","author":{"@id":"https:\/\/pyimagesearch.com\/#\/schema\/person\/5901b399e2f20b986362a00636181cca"},"breadcrumb":{"@id":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/pyimagesearch.com\/2020\/07\/27\/opencv-grabcut-foreground-segmentation-and-extraction\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/pyimagesearch.com\/"},{"@type":"ListItem","position":2,"name":"OpenCV GrabCut: Foreground Segmentation and Extraction"}]},{"@type":"WebSite","@id":"https:\/\/pyimagesearch.com\/#website","url":"https:\/\/pyimagesearch.com\/","name":"PyImageSearch","description":"You can master Computer Vision, Deep Learning, and OpenCV - PyImageSearch","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/pyimagesearch.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/pyimagesearch.com\/#\/schema\/person\/5901b399e2f20b986362a00636181cca","name":"Adrian Rosebrock","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/pyimagesearch.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/57e1f3a95feeb9b113f80510f086d7d81b6f62badd9bd69134e51037a8b79925?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/57e1f3a95feeb9b113f80510f086d7d81b6f62badd9bd69134e51037a8b79925?s=96&d=mm&r=g","caption":"Adrian Rosebrock"},"description":"Hi there, I\u2019m Adrian Rosebrock, PhD. All too often I see developers, students, and researchers wasting their time, studying the wrong things, and generally struggling to get started with Computer Vision, Deep Learning, and OpenCV. I created this website to show you what I believe is the best possible way to get your start.","url":"https:\/\/pyimagesearch.com\/author\/adrian\/"}]}},"_links":{"self":[{"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/posts\/15680","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/comments?post=15680"}],"version-history":[{"count":6,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/posts\/15680\/revisions"}],"predecessor-version":[{"id":37748,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/posts\/15680\/revisions\/37748"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/media\/15682"}],"wp:attachment":[{"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/media?parent=15680"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/categories?post=15680"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pyimagesearch.com\/wp-json\/wp\/v2\/tags?post=15680"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}