The Corridor Key: Modernizing Chroma Keying with Neural Networks and Procedural Data

Overview

Traditional chroma keying has remained stagnant for decades, relying on primitive color subtraction math that fails to handle semi-transparency, motion blur, and fine details like hair. Most green screen tools essentially eyeball a color range, leaving artists with a binary choice: a hard, crunchy edge or a messy green fringe.

Corridor Key
solves this by replacing manual color picking with a neural network trained to understand the relationship between a subject and its background. This technique doesn't just isolate pixels; it mathematically unmixes the foreground from the background, effectively calculating what a pixel would look like if it were shot on a truly transparent plate.

Prerequisites

To understand or implement the logic behind this tool, you should be familiar with:

  • Machine Learning Concepts: Specifically supervised learning, training loops, and ground truth data.
  • 3D Production Pipelines: Experience with
    Houdini
     or
    Blender
     for generating synthetic data.
  • VFX Compositing: Familiarity with alpha channels, premultiplied vs. straight color, and EXR file formats.
  • Python Programming: Basic knowledge for handling batch scripts and data loading.

Key Libraries & Tools

  • Houdini
    : A procedural 3D application used to generate thousands of unique training samples with randomized lighting and materials.
  • Blender
    : Utilized for character-focused synthetic data generation, particularly for hair and organic shapes.
  • PyTorch/TensorFlow: The underlying frameworks for the neural network architecture (though accessed via the tool's wrapper).
  • After Effects
     /
    Nuke
    : Professional compositing software used to verify the EXR outputs and integrate them into a final scene.

Code Walkthrough: Synthetic Data Generation

The core of this breakthrough isn't the network itself, but the data fed into it. To train a model to handle every variable, we use procedural generation to create a ground truth dataset that is impossible to film in reality.

# Pseudocode for a Procedural Training Iteration
import random

def generate_training_sample(subject_model, background_color):
    # 1. Randomize Environment
    lighting = random.uniform(0.5, 2.0)
    rotation = random.randint(0, 360)
    
    # 2. Render Green Screen Version (Input)
    input_img = render(subject_model, bg=background_color, light=lighting, rot=rotation)
    
    # 3. Render Ground Truths (Answers)
    target_fg = render(subject_model, bg=None, light=lighting, rot=rotation) # No BG
    target_alpha = extract_alpha(target_fg)
    
    return input_img, target_fg, target_alpha

In

Houdini
, this logic translates to a node network where a "switch" node cycles through hundreds of models. We randomize materials (metal, fabric, skin) and lighting rigs every frame. This forces the model to learn that "green" is the background regardless of whether the foreground is a shiny sword or a frizzy wig.

Training Logic and Loss Functions

A critical hurdle was the "green fringe"—the leftover color on semi-transparent pixels. To solve this, the training script was updated to recomposite the predicted foreground onto a random new background during the loss calculation. This highlights errors that are invisible against a black background.

# Training Loop Logic
for input_img, true_fg, true_alpha in dataset:
    prediction_fg, prediction_alpha = model(input_img)
    
    # Test against different backgrounds to expose fringe
    random_bg = get_random_texture()
    comp_pred = composite(prediction_fg, prediction_alpha, random_bg)
    comp_true = composite(true_fg, true_alpha, random_bg)
    
    # Calculate loss based on the final composite
    loss = calculate_difference(comp_pred, comp_true)
    optimizer.step(loss)

By comparing the final composite rather than just the isolated mask, the model learns that a red gel in front of a green screen must become a semi-transparent red pixel, not a purple one.

Syntax Notes

  • NAN Handling: Neural networks occasionally produce "Not a Number" (NAN) glitches. The tool implements a cleanup pass to identify and interpolate these mathematical errors.
  • EXR Standards: The tool outputs linear, 32-bit float data. This ensures that when you import the footage into
    After Effects
    , the dynamic range is preserved for professional relighting.

Practical Examples

  • Son of a Dungeon
    : Used to process over 500 shots featuring complex chainmail and translucent magical effects.
  • Complex Refractions: Keying subjects through glassware or transparent visors where traditional tools fail.
  • Low-Resolution Sources: The model's pattern recognition allows it to pull usable keys from compressed, 8-bit web footage that would normally require manual rotoscoping.

Tips & Gotchas

  • VRAM Requirements: The current model is computationally heavy. You need roughly 24GB of VRAM (an NVIDIA 3090/4090 class card) to run the full resolution inference.
  • Tracking Markers: The model treats tracking markers as part of the "background" it wants to remove. If your markers are a contrasting color (like blue on green), ensure your training data includes similar contrast to prevent the model from getting confused.
  • Despill: While the tool performs exceptionally well, a secondary despill pass in your compositor may still be necessary to perfectly match the lighting of your new background.
The Corridor Key: Modernizing Chroma Keying with Neural Networks and Procedural Data

Fancy watching it?

Watch the full video and context

5 min read