Creating an Object Detection Dataset for Film Slates

Back in 2022, when I first started tinkering with custom dataset creation for Paperdot, the goal was raw classification: training a model to distinguish between different materials. There weren't a lot of good tools around to speed up the labeling process. You mostly hand-rolled your own scripts. Now, four years later, there's a whole ecosystem of annotation tools built for exactly this (CVAT, Label Studio, and Roboflow being the ones I keep coming back to).

In my recent research project, things moved on again. From static labeling into dynamic, real-time tracking. The goal this time: track film slates - the clapperboards that mark the start (or end) of a take. Well enough to read their state and content on the fly. This post is about the part that actually determines whether the rest of the project works: building the dataset.

1. The Core Philosophy: What Are We Actually Tracking

Before you annotate a single frame, figure out what you actually need - not what's convenient to label. For detecting film slates, the requirements were:

• Realtime speed - this has to run live on set, not as an offline batch job.
• Rotational awareness - slates aren't always held upright. Operators tilt them for the camera, and "end slates" (also called tail slates, used when a slate is filmed at the end of a take instead of the start) are often flipped a full 180°, sometimes held upside-down mid-frame.
• State awareness - the system needs to know whether the slate is open or closed, since that's the moment that actually marks the cut point.

Only the first two are really about geometry. The third is a labeling/taxonomy problem, which I'll get to in the next section. For the geometry, there are three standard ways to frame an object detection problem, and it's worth being deliberate about which one you pick before you invest annotation hours:

• Standard Object Detection (BBox): Outputs an axis-aligned $x, y, w, h$ rectangle. Because slates are almost always held at an angle, an axis-aligned box has to grow to contain the full tilted shape, which means it's mostly background dead-space. Worse, a bbox literally cannot encode rotation - there's no field for it - so downstream steps like straightening the slate for OCR get no help from the detector at all. And you can not easily detect tail slates.
• Instance Segmentation: Gives you pixel-level masks, which would technically solve the rotation problem, but the annotation cost is punishing (tracing a polygon per frame instead of dragging four corners), and a pixel mask is mathematical overkill when all you need is a clean rectangle to crop and de-skew. It's also slower at inference - a real cost when the whole point is realtime tracking.
• Oriented Bounding Boxes (OBB): the winner. A rotated rectangle, typically stored as four corner points (Ultralytics' format: class x1 y1 x2 y2 x3 y3 x4 y4, normalized) rather than a center point plus an angle $\theta$.

2. Defining Your Class Taxonomy

This is the step that's easiest to skip and most expensive to fix later, because it decides what every single annotation in your dataset actually means. "Track a slate" is not a taxonomy. You have to decide, in writing, before you label anything:

• Two full classes, not one class with an attribute. I settled on slate_open and slate_closed as two entirely separate labels, rather than a single slate class carrying an open/closed attribute on the side. A shared class with an attribute is the more "normalized" data-modeling choice on paper, but attributes are a feature of whatever annotation tool and training pipeline you're using today - they don't necessarily survive a change of export format or a different model architecture down the line. Two plain classes are the lowest-common-denominator representation: they carry over cleanly no matter what you point at the dataset next, and if a future model only cares about one state, it can just train on that class and ignore the other.
• The boundary is easier than it sounds, because there mostly isn't one. I expected to need a fuzzy "half-open" middle category. In practice a slate is a hinge with two rest positions, so it behaves like a binary switch: if there's any visible gap at the hinge, it's slate_open - full stop, no matter how small the gap. You're not judging degrees of openness, you're just checking whether it's flush shut or not. That single rule resolves almost every frame without a judgment call, including the brief "just starting to close" frames that used to worry me.
• Decide how to handle occlusion and partial visibility explicitly. A hand gripping the slate, a slate half out of frame, a slate reflected in a mirror or monitor in the background - pick a rule (annotate down to some minimum visible fraction, otherwise skip) and apply it uniformly, rather than making a judgment call per frame.

None of this needs to be a formal document - a few bullet points pinned next to your annotation tool is enough. The goal is just that the rule exists and is the same on frame 1 and frame 10,000.

3. Smart Sampling Strategy for Video Clips

Film slates sit still or move in short, predictable bursts. If you dump raw 25 fps video straight into your annotation tool, you end up annotating thousands of frames that are visually identical to their neighbors - burning hours of labeling time for close to zero additional information.

• The base clip: In most of our recent takes, the slate is on screen for roughly 2 seconds (at 25 fps, about 50 raw frames).
• The sampling interval: Since we don't want to annotate every frame, and dataset quality depends on the images actually varying, we sample every 2nd frame. That nets about 25 annotated frames per clip, which is enough to capture the full progression - open, moving, the moment of impact, settled closed - without redundancy. Go coarser (every 5th frame, say) and you risk skipping straight over the brief "slightly closed, still moving" state entirely, which is exactly the transition your model most needs examples of.
• Don't sample blindly through static holds. The interval above assumes motion. If a take has the slate held rock-steady in frame for a second or two before the clap, sampling every 2nd frame there just produces a run of near-duplicate images. A cheap perceptual-similarity check (or even eyeballing it) to thin out visually identical runs is worth the extra few minutes - those slots are better spent on a different lighting setup or a different take.
• Background & negative frames: Never train a detector exclusively on positive examples, or it will start hallucinating phantom slates everywhere. Aim for 15–20% of your dataset as empty frames - roughly 4–6 per clip, split evenly before the slate enters and after it leaves - annotated with no boxes at all, so the model learns them as pure background.
• Make your negatives count as hard negatives, not just empty rooms. A blank shot of a wall teaches the model almost nothing it doesn't already know. Frames containing other rectangular, high-contrast objects - clipboards, laptops, monitors, whiteboards, notebooks - are far more valuable negatives, because they're the things your model is actually likely to confuse for a slate later.
• Chase diversity on every axis you can, not just frame count. Camera distance and angle, lighting setup (daylight, tungsten, low-light on-set conditions), the slate's physical design (chalk, dry-erase, digital/smart slate), operator hands and skin tones, and the amount of motion blur during the clap all matter more to generalization than raw image count. Twenty takes across five different lighting setups will beat a hundred takes shot in one room, every time.

4. Splitting Your Data Without Leaking the Future

This is the part that's genuinely easy to get wrong with video-sourced data, and it doesn't show up as an obvious bug - it shows up as a validation score that looks great and a model that quietly underperforms once it meets footage it hasn't seen.

The trap: if you sample 25 frames from a clip and then split frames randomly into train/val, you'll end up with, say, frame 14 in training and frame 15 - nearly pixel-identical - in validation. Your validation metric is then measuring how well the model memorized that specific take, not how well it generalizes. It'll look fantastic and mean very little.

• Split by take/clip, never by frame. Every frame from a given clip goes entirely into train, entirely into validation, or entirely into test - never split across two of them.
• A reasonable starting ratio is 70/20/10 or 80/10/10 at the clip level, adjusted once you see how many distinct takes you actually have.
• Stratify the split across your diversity axes, not just randomly by clip. If all your low-light or unusual-slate-design takes happen to land in training, your validation score will look better than reality, and you won't find out until deployment.
• Hold back at least one or two clips with a slate design or setup the model has never seen anywhere else - reserved exclusively for test. That's your actual generalization sanity check, separate from the "did it learn this specific set of takes" question that train/val answers.

5. Dataset Size & the "More Is Better" Trap

How many images do you actually need? Because a clapperboard has high visual contrast and a rigid, well-defined structure, you don't need millions of images to get solid performance.

• Proof of concept: 200–300 images (roughly 8–12 fully annotated takes) is enough to get a basic model running and validate the whole pipeline.
• Production-grade sweet spot: 500–1,000 images (20–40 takes plus backgrounds). This is where you start getting real variance in lighting, focal length, and motion blur - enough to hold up against messy real-world conditions.
• Is more always better? (The 2,000+ image question): Technically, modern YOLO models won't break if you feed them 2,000+ images. But you'll hit diminishing returns fast. Film slates and on-set environments are visually repetitive by nature, so thousands of extra look-alike frames buy you very little additional mAP (mean Average Precision - the standard accuracy metric for detection models, averaging precision across classes and confidence thresholds) for a lot of extra annotation time. A diverse, well-sampled 600–800 image set will typically outperform a larger but repetitive one.
• These numbers are per class, not per dataset. With slate_open and slate_closed as two separate classes, each one needs its own share of solid examples - a total of 600 images split 90/10 between them isn't a 600-image dataset, it's really a 60-image dataset for whichever state is underrepresented. Since a closed slate is on screen for a fraction of the time an open one is per take (Section 3's sampling interval naturally captures more "open" frames than "closed" ones), keep an eye on the per-class count as you go, not just the running total.

6. Annotation Quality: Tight Boxes, Consistent Corners

Volume and sampling strategy get you diverse data; this is what makes each individual annotation actually worth something.

• Fit boxes tight to the physical edges of the slate, not to its motion-blur halo. This matters more for OBB than it ever did for axis-aligned boxes, because the model isn't just learning "object here," it's learning the exact rotation from your corner placement - sloppy corners teach it a sloppy angle.
• Keep corner ordering consistent across frames. Since OBB is stored as four explicit corner points, the order in which those points are recorded implicitly encodes orientation. If that order flips between one frame and the next (which can happen with careless manual correction of exported labels), you're teaching the model two contradictory ideas of which way the box is "facing" for what should be a smooth, continuous rotation. Most annotation tools default to a consistent order automatically - the failure mode is almost always a human hand-editing a label file afterward, not the tool itself.
• Use interpolation for video, but don't trust it blindly. Most modern annotation tools can interpolate a box's position between two manually placed keyframes, which is a huge time-saver for a moving slate. Spot-check the interpolated frames anyway, especially during the fast part of the motion right around impact - that's exactly where linear interpolation tends to drift away from the true position.
• If more than one person is labeling, write your taxonomy rules down (see Section 2) and spot-check across annotators periodically. Inter-annotator drift on a fuzzy boundary case is invisible until you're staring at a confusion matrix wondering why the model can't decide.

References and helpful Videos

Create Dataset with CVAT | Traing YOLO Object Detection Models | CVAT - Accelerate annotation with Hugging Face