Auto-Segmentation

Generate face masks automatically with the BiSeNet AI model, then refine the results with manual editing tools.

Overview

Auto-segmentation uses the BiSeNet (Bilateral Segmentation Network) model to analyze a face image and automatically generate a mask that covers the face region. This is dramatically faster than painting a mask from scratch and provides a solid starting point that you can refine with the manual editing tools.

BiSeNet is specifically designed for face parsing -- it understands the different semantic regions of a face (skin, eyes, nose, mouth, eyebrows, hair) and can distinguish the face area from the background with high accuracy.

BiSeNet Model

The BiSeNet model is an ONNX-format neural network that runs locally on your machine. It requires ONNX Runtime to execute, which is included as an optional dependency.

Model Details

Property	Value
Architecture	BiSeNet (Bilateral Segmentation Network)
Format	ONNX
File Size	~52 MB
Runtime	ONNX Runtime (onnxruntime package)
Download	Automatic on first use
Execution	CPU or GPU (if CUDA provider available)

Automatic Download

The BiSeNet model is downloaded automatically the first time you use the auto-segmentation feature. The download is approximately 52 MB and happens transparently in the background. A progress indicator appears while the model is being fetched.

Once downloaded, the model is cached locally and reused on subsequent launches. You do not need to download it again unless you clear the application cache.

Requires ONNX Runtime

Auto-segmentation requires the onnxruntime Python package to be installed. If it is not available, the auto-segmentation button will be disabled with a tooltip explaining the requirement. Install it with pip install onnxruntime.

One-Click Mask Generation

Generating a mask with auto-segmentation takes a single click. The model analyzes the face image and produces a mask in under a second on most hardware.

Open a face in the Face Editor

Double-click a face thumbnail in the Face Browser to open it in the editor.

Click the Auto-Segment button

In the Tool Panel on the left, click the Auto-Segment button (brain icon). The model runs and generates a mask overlay on the canvas.

Review the generated mask

The mask appears as a semi-transparent overlay on the face. Check that it covers the entire face region without excessive bleed into the background.

Refine if needed

Switch to the Paint or Eraser tool to fix any areas where the auto-generated mask needs adjustment. Common fixes include the hairline, jawline, and ear regions.

Batch Auto-Segmentation

For large face sets, you can run auto-segmentation on multiple faces at once using the batch mask tool in the Face Browser. Select multiple faces, right-click, and choose Auto-Segment Selected. This processes all selected faces sequentially without opening the editor.

How BiSeNet Works

Understanding how the model generates masks can help you predict where it will succeed and where manual refinement is likely needed.

Recognized Face Regions

BiSeNet classifies each pixel of the face image into one of several semantic regions:

Skin -- Forehead, cheeks, chin, and neck skin.
Eyes -- Left and right eye regions including eyebrows.
Nose -- The full nose region.
Mouth -- Upper and lower lips, inner mouth area.
Hair -- Hair regions (excluded from the default mask).
Background -- Everything that is not part of the face (excluded).

By default, the auto-segmentation mask includes skin, eyes, nose, and mouth regions while excluding hair and background. This produces a mask that covers the core face area.

Accuracy Expectations

BiSeNet performs well in most conditions, but certain situations can reduce accuracy:

Works Well

Front-facing or slightly angled faces
Good lighting conditions
Clean backgrounds
Standard facial features
Faces without occlusions

May Need Refinement

Extreme profile angles
Heavy shadows or backlighting
Faces partially covered by hands or objects
Unusual makeup or face paint
Very low resolution face images

Refining Auto-Generated Masks

Even with a good auto-segmentation result, manual refinement is usually needed for professional-quality output. Here are the most common areas that benefit from manual touch-up:

Hairline Boundary

The boundary between the forehead and hair is the most challenging area for any segmentation model. BiSeNet may include some hair in the mask or miss small areas of the forehead. Use the eraser to remove hair from the mask and the paint tool to fill in any gaps on the forehead.

Jawline and Chin

The jawline boundary may be slightly inaccurate, especially with angled faces or strong shadows under the chin. Zoom in and use a small brush to trace the exact jawline contour.

Ear Regions

Depending on the face angle, ears may or may not be included in the auto-generated mask. For most face swaps, excluding the ears produces more natural results because ear shape is distinctive and difficult to blend convincingly.

Re-Running Auto-Segmentation

Running auto-segmentation again on the same face will overwrite any manual mask edits you have made. If you have spent time refining a mask, be careful not to accidentally click the Auto-Segment button. Use Undo immediately if this happens.

Performance

Auto-segmentation performance depends on your hardware and whether a GPU is available:

Hardware	Speed per Face
GPU (CUDA)	~50-100ms
CPU (modern)	~200-500ms
CPU (older)	~500ms-1s

Batch auto-segmentation on a set of 100 faces typically takes 5-10 seconds on a GPU or 20-50 seconds on CPU. Progress is displayed in the status bar during batch operations.

Troubleshooting

Auto-Segment button is disabled

ONNX Runtime is not installed. Install it with pip install onnxruntime and restart Recaster.

Model download fails

Check your internet connection. The model is downloaded from Hugging Face and requires an active connection for the first download. If behind a proxy, configure your system proxy settings.

Mask quality is poor

Very small or heavily occluded faces can confuse the model. Try extracting faces at a larger resolution, or use the manual mask editing tools instead for difficult cases.

Segmentation is slow

If ONNX Runtime is using CPU instead of GPU, performance will be slower. Install onnxruntime-gpu for CUDA acceleration. Verify your GPU is detected with the system info panel.

Was this page helpful?