BOUNTY: Mouth Mask Feature

[KRSHH]

Summary by Sourcery

Implement a mouth mask feature in the face swapping module to enhance realism by separately handling the mouth area. Update the UI to include switches for enabling the mouth mask and visualizing the mouth mask box, giving users more control over the feature.

New Features:

• Introduce a mouth mask feature in the face swapping module, allowing for more realistic face swaps by handling the mouth area separately.

Enhancements:

• Add UI controls for enabling the mouth mask feature and visualizing the mouth mask box, providing users with more control over the face swapping process.

[sourcery-ai]

Reviewer's Guide by Sourcery

This PR implements a mouth mask feature for face swapping that preserves the original mouth area of the target face. The implementation includes mask creation, color correction, and visualization capabilities. The changes primarily affect the face swapper module and add new UI controls.

Class diagram for the updated face swapper module

classDiagram
    class FaceSwapper {
        +get_face_swapper() : Any
        +swap_face(source_face: Face, target_face: Face, temp_frame: Frame) : Frame
        +process_frame(source_face: Face, temp_frame: Frame) : Frame
        +process_frame_v2(temp_frame: Frame, temp_frame_path: str) : Frame
        +process_frames(source_path: str, temp_frame_paths: List[str], progress: Any) : None
        +process_image(source_path: str, target_path: str, output_path: str) : None
        +process_video(source_path: str, temp_frame_paths: List[str]) : None
        +create_lower_mouth_mask(face: Face, frame: Frame) : (np.ndarray, np.ndarray, tuple, np.ndarray)
        +draw_mouth_mask_visualization(frame: Frame, face: Face, mouth_mask_data: tuple) : Frame
        +apply_mouth_area(frame: np.ndarray, mouth_cutout: np.ndarray, mouth_box: tuple, face_mask: np.ndarray, mouth_polygon: np.ndarray) : np.ndarray
        +create_face_mask(face: Face, frame: Frame) : np.ndarray
        +apply_color_transfer(source, target)
    }

    class UI {
        +create_root(start: Callable[[], None], destroy: Callable[[], None]) : ctk.CTk
    }

    class Globals {
        +mouth_mask: bool
        +show_mouth_mask_box: bool
        +mask_feather_ratio: int
        +mask_down_size: float
        +mask_size: int
    }

    FaceSwapper --> UI : uses
    FaceSwapper --> Globals : uses
    UI --> Globals : uses
    note for FaceSwapper "New methods for mouth mask feature added"
    note for UI "New UI controls for mouth mask feature added"
    note for Globals "New global variables for mouth mask feature added"

Loading

File-Level Changes

Change	Details	Files
Added mouth mask functionality to preserve original mouth area during face swapping	Implemented create_lower_mouth_mask function to generate a mask for the lower face area Added color transfer functionality to blend the preserved mouth area naturally Created face mask generation for better blending with the swapped face Integrated mouth mask processing into the main face swapping pipeline	modules/processors/frame/face_swapper.py
Added UI controls for mouth mask features	Added toggle switch for enabling/disabling mouth mask Added toggle switch for showing mouth mask visualization Added mouth mask configuration parameters	modules/ui.py modules/globals.py
Added visualization and debugging features for mouth mask	Implemented draw_mouth_mask_visualization function for debugging Added mask feathering for smooth blending Added configurable parameters for mask size and feathering	modules/processors/frame/face_swapper.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @KRSHH - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟡 General issues: 2 issues found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟡 Complexity: 1 issue found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue (bug_risk): Use more specific exception handling to avoid masking critical errors

Catching all exceptions silently could hide serious issues. Consider catching specific exceptions (e.g., ValueError, cv2.error) and logging the error details for debugging.

[sourcery-ai]

suggestion: Consider making magic numbers configurable parameters

The forehead extension factor (5.0) and face width padding (5%) should be configurable parameters since they may need adjustment for different face shapes and sizes.

FOREHEAD_EXTENSION_FACTOR = 5.0

extended_forehead_height = int(forehead_height * FOREHEAD_EXTENSION_FACTOR)

[sourcery-ai]

issue (complexity): Consider separating the face swapping code into dedicated classes and modules for mask generation, visualization, and core swapping logic

The face swapping code mixes multiple concerns that should be separated. Consider:

1. Extract mouth masking into a dedicated class:

class MouthMaskGenerator:
    def __init__(self, expansion_factor=1.0, feather_ratio=10):
        self.expansion_factor = expansion_factor
        self.feather_ratio = feather_ratio

    def generate_mask(self, face: Face, frame: Frame) -> dict:
        # Current create_lower_mouth_mask logic goes here
        return {
            'mask': mask,
            'cutout': mouth_cutout,
            'bounds': (min_x, min_y, max_x, max_y),
            'polygon': lower_lip_polygon
        }

2. Move visualization to a separate debug module:

# debug_visualizer.py
def visualize_mouth_mask(frame: Frame, mask_data: dict) -> Frame:
    # Current visualization code goes here
    return annotated_frame

3. Simplify swap_face to focus on high-level flow:

def swap_face(source_face: Face, target_face: Face, frame: Frame) -> Frame:
    face_swapper = get_face_swapper()
    result = face_swapper.get(frame, target_face, source_face, paste_back=True)

    if modules.globals.mouth_mask:
        mask_gen = MouthMaskGenerator()
        mask_data = mask_gen.generate_mask(target_face, frame)
        result = apply_mouth_area(result, **mask_data)

        if modules.globals.show_mouth_mask_box:
            result = debug.visualize_mouth_mask(result, mask_data)

    return result

This separates concerns while preserving functionality, making the code more maintainable and easier to test.

[KRSHH]

Directly commited to main last time (now reverted)... my mistake