Creating an Object Detection Dataset for Film Slates

Back in 2022, when I first started tinkering with custom dataset creation for Paperdot, the goal was raw classification: training a model to distinguish between different materials. There weren't a lot of good tools around to speed up the labeling process. You mostly hand-rolled your own scripts. Now, four years later, there's a whole ecosystem of annotation tools built for exactly this (CVAT, Label Studio, and Roboflow being the ones I keep coming back to).

In my recent research project, things moved on again. From static labeling into dynamic, real-time tracking. The goal this time: track film slates - the clapperboards that mark the start (or end) of a take. Well enough to read their state and content on the fly. This post is about the part that actually determines whether the rest of the project works: building the dataset.

1. The Core Philosophy: What Are We Actually Tracking

Before you annotate a single frame, figure out what you actually need - not what's convenient to label. For detecting film slates, the requirements were:

• Realtime speed - this has to run live on set, not as an offline batch job.
• Rotational awareness - slates aren't always held upright. Operators tilt them for the camera, and "end slates" (also called tail slates, used when a slate is filmed at the end of a take instead of the start) are often flipped a full 180°, sometimes held upside-down mid-frame.
• State awareness - the system needs to know whether the slate is open or closed, since that's the moment that actually marks the cut point.

Only the first two are really about geometry. The third is a labeling/taxonomy problem, which I'll get to in the next section. Before locking in a geometry, it's worth ruling out the simplest option first, then being deliberate about which of the remaining, more complex ones you pick:

• Whole-Image Classification (ruled out first): The simplest option on paper - skip boxes entirely, feed the frame (or a crop of it) into a classifier, and let it output slate_open, slate_closed, or background. It's also the least precise choice for this job, for two compounding reasons. First, a classifier gives you no location - even a correct "slate_open" prediction on a 4K frame doesn't tell you where, which is useless when the actual downstream step is cropping and de-skewing the slate for OCR. Second, classification accuracy degrades badly once the slate stops dominating the frame: in a wide shot where the slate is 5% of the image, that signal gets diluted by everything else the network has to look at, so precision drops exactly in the real-world wide-angle shots you care about most. Detection - in any of the three flavors below - solves both problems at once, because it gives you a location and lets the model attend to just that region instead of the whole frame.
• Horizontal Bounding Boxes (HBB) - the default, and why it falls short here: The standard object-detection output: an axis-aligned $x, y, w, h$ rectangle. Because slates are almost always held at an angle, an axis-aligned box has to grow to contain the full tilted shape, which means a large chunk of it is background dead-space - for a slate tilted 30-45°, you can easily lose 30-40% of the box area to padding. That's not just wasted crop pixels; it actively confuses the model's learned notion of "what a slate looks like," since a chunk of every positive training box is, visually, whatever happens to be behind the operator's hand. Worse, an HBB literally cannot encode rotation - there's no field for it - so downstream steps like straightening the slate for OCR get no help from the detector at all. And you can not easily detect tail slates.
• Instance Segmentation: Gives you pixel-level masks, which would technically solve the rotation problem, but the annotation cost is punishing (tracing a polygon per frame instead of dragging four corners), and a pixel mask is mathematical overkill when all you need is a clean rectangle to crop and de-skew. It's also slower at inference - a real cost when the whole point is realtime tracking.
• Oriented Bounding Boxes (OBB) - the winner: A rotated rectangle, typically stored as four corner points (Ultralytics' format: class x1 y1 x2 y2 x3 y3 x4 y4, normalized) rather than a center point plus an angle $\theta$. Against HBB, this buys you a tight, padding-free crop (better signal for the OCR/read step downstream) and rotation for free - the four corner points are the angle, so there's nothing extra to infer or post-process. Against segmentation, it keeps annotation to four clicks per box instead of a traced polygon, and inference stays fast enough for realtime tracking. For an object that's rigid, always roughly rectangular, and needs to be de-skewed downstream, OBB is the representation that costs the least to annotate and gives the most useful output.

2. Defining Your Class Taxonomy

This is the step that's easiest to skip and most expensive to fix later, because it decides what every single annotation in your dataset actually means. "Track a slate" is not a taxonomy. You have to decide, in writing, before you label anything:

• Two full classes, not one class with an attribute. I settled on slate_open and slate_closed as two entirely separate labels, rather than a single slate class carrying an open/closed attribute on the side. A shared class with an attribute is the more "normalized" data-modeling choice on paper, but attributes are a feature of whatever annotation tool and training pipeline you're using today - they don't necessarily survive a change of export format or a different model architecture down the line. Two plain classes are the lowest-common-denominator representation: they carry over cleanly no matter what you point at the dataset next, and if a future model only cares about one state, it can just train on that class and ignore the other.
• The boundary is easier than it sounds, because there mostly isn't one. I expected to need a fuzzy "half-open" middle category. In practice a slate is a hinge with two rest positions, so it behaves like a binary switch: if there's any visible gap at the hinge, it's slate_open - full stop, no matter how small the gap. You're not judging degrees of openness, you're just checking whether it's flush shut or not. That single rule resolves almost every frame without a judgment call, including the brief "just starting to close" frames that used to worry me.
• Decide how to handle occlusion and partial visibility explicitly. A hand gripping the slate, a slate half out of frame, a slate reflected in a mirror or monitor in the background - pick a rule (annotate down to some minimum visible fraction, otherwise skip) and apply it uniformly, rather than making a judgment call per frame.

Why Not Just One slate Class?

It's worth being honest about the tradeoff here, because it isn't free. If you asked "will one class or two perform better," the honest answer is: one class will usually win, especially on a small-to-medium dataset. A single-class model only has to answer a binary question at the candidate-box level - "is this a slate, or is it background?" A two-class model has to answer a three-way question - "is this slate_open, slate_closed, or background?" - which is a strictly harder discrimination problem for the same amount of training data.

The cost is concrete. Splitting a visually distinct object into sub-categories forces the model to learn the subtle nuances between those states (the exact hinge angle of an open clapper stick versus a shut one) instead of just learning what makes something a clapperboard at all. If your dataset isn't massive, the model will periodically confuse slate_open and slate_closed on borderline frames or awkward viewing angles - and that confusion doesn't just misclassify, it depresses confidence scores across the board, which shows up as missed detections (false negatives), not just wrong labels, since most inference pipelines threshold on confidence before they even get to the class prediction.

So why pay that cost here? Because for this project, knowing the state is the entire point - open vs. closed is the cut-point signal the whole pipeline exists to produce. A single slate class would just push the same problem one step downstream: you'd still need a second model, or a hand-written heuristic, bolted on afterward to answer the one question that mattered from the start. If your use case only needs "there's a slate here, go crop it," and the open/closed state is someone else's problem later, collapsing to one class is the better call - it trains faster, converges with less data, and hands you higher-confidence boxes to feed into whatever second-stage classifier you build. Two classes is a decision you make because the state is the deliverable, not a default you reach for automatically.

None of this needs to be a formal document - a few bullet points pinned next to your annotation tool is enough. The goal is just that the rule exists and is the same on frame 1 and frame 10,000.

3. Smart Sampling Strategy for Video Clips

Film slates sit still or move in short, predictable bursts. If you dump raw 25 fps video straight into your annotation tool, you end up annotating thousands of frames that are visually identical to their neighbors - burning hours of labeling time for close to zero additional information.

• The base clip: In most of our recent takes, the slate is on screen for roughly 2 seconds (at 25 fps, about 50 raw frames).
• The sampling interval: Since we don't want to annotate every frame, and dataset quality depends on the images actually varying, we sample every 3rd frame. That nets about 25 annotated frames per clip, which is enough to capture the full progression - open, moving, the moment of impact, settled closed - without redundancy. Go coarser (every 5th frame, say) and you risk skipping straight over the brief "slightly closed, still moving" state entirely, which is exactly the transition your model most needs examples of.
• Don't sample blindly through static holds. The interval above assumes motion. If a take has the slate held rock-steady in frame for a second or two before the clap, sampling every 2nd frame there just produces a run of near-duplicate images. A cheap perceptual-similarity check (or even eyeballing it) to thin out visually identical runs is worth the extra few minutes - those slots are better spent on a different lighting setup or a different take.
• Background & negative frames: Never train a detector exclusively on positive examples, or it will start hallucinating phantom slates everywhere. Aim for 15–20% of your dataset as empty frames - roughly 4–6 per clip, split evenly before the slate enters and after it leaves - annotated with no boxes at all, so the model learns them as pure background.
• Make your negatives count as hard negatives, not just empty rooms. A blank shot of a wall teaches the model almost nothing it doesn't already know. Frames containing other rectangular, high-contrast objects - clipboards, laptops, monitors, whiteboards, notebooks - are far more valuable negatives, because they're the things your model is actually likely to confuse for a slate later.
• Chase diversity on every axis you can, not just frame count. Camera distance and angle, lighting setup (daylight, tungsten, low-light on-set conditions), the slate's physical design (chalk, dry-erase, digital/smart slate), operator hands and skin tones, and the amount of motion blur during the clap all matter more to generalization than raw image count. Twenty takes across five different lighting setups will beat a hundred takes shot in one room, every time.

4. Splitting Your Data Without Leaking the Future

This is the part that's genuinely easy to get wrong with video-sourced data, and it doesn't show up as an obvious bug - it shows up as a validation score that looks great and a model that quietly underperforms once it meets footage it hasn't seen.

The trap: if you sample 25 frames from a clip and then split frames randomly into train/val, you'll end up with, say, frame 14 in training and frame 15 - nearly pixel-identical - in validation. Your validation metric is then measuring how well the model memorized that specific take, not how well it generalizes. It'll look fantastic and mean very little.

• Split by take/clip, never by frame. Every frame from a given clip goes entirely into train, entirely into validation, or entirely into test - never split across two of them.
• A reasonable starting ratio is 70-80% train / 15-20% validation / 10-15% test, at the clip level, adjusted once you see how many distinct takes you actually have. Lean toward the higher end of the train range only once you have enough clips that validation and test still land on a statistically meaningful number of takes - a 90/5/5 split sounds efficient, but if that 5% test slice is only two clips, one unusual take can swing your test mAP by a huge margin and you won't be able to tell if that's real or just noise.
• Each split should preserve the same labeled-to-negative ratio as the full dataset, not just the same overall size. Aim for roughly 90-95% labeled (positive) frames to 5-10% unlabeled/negative frames in train, validation, and test individually - see Section 5 for exactly how to hit that number instead of eyeballing it. If your negatives all end up bunched in train because that's where you added them last, validation and test won't actually be testing whether the model avoids hallucinating slates - only whether it does so correctly on takes it was trained on.
• Stratify the split across your diversity axes, not just randomly by clip. If all your low-light or unusual-slate-design takes happen to land in training, your validation score will look better than reality, and you won't find out until deployment.
• Hold back at least one or two clips with a slate design or setup the model has never seen anywhere else - reserved exclusively for test. That's your actual generalization sanity check, separate from the "did it learn this specific set of takes" question that train/val answers.

5. Dataset Size & the "More Is Better" Trap

How many images do you actually need? Because a clapperboard has high visual contrast and a rigid, well-defined structure, you don't need millions of images to get solid performance.

• Proof of concept: 200–300 images (roughly 8–12 fully annotated takes) is enough to get a basic model running and validate the whole pipeline.
• Production-grade sweet spot: 500–1,000 images (20–40 takes plus backgrounds). This is where you start getting real variance in lighting, focal length, and motion blur - enough to hold up against messy real-world conditions.
• Is more always better? (The 2,000+ image question): Technically, modern YOLO models won't break if you feed them 2,000+ images. But you'll hit diminishing returns fast. Film slates and on-set environments are visually repetitive by nature, so thousands of extra look-alike frames buy you very little additional mAP (mean Average Precision - the standard accuracy metric for detection models, averaging precision across classes and confidence thresholds) for a lot of extra annotation time. A diverse, well-sampled 600–800 image set will typically outperform a larger but repetitive one.
• These numbers are per class, not per dataset. With slate_open and slate_closed as two separate classes, each one needs its own share of solid examples - a total of 600 images split 90/10 between them isn't a 600-image dataset, it's really a 60-image dataset for whichever state is underrepresented. Since a closed slate is on screen for a fraction of the time an open one is per take (Section 3's sampling interval naturally captures more "open" frames than "closed" ones), keep an eye on the per-class count as you go, not just the running total.

A Worked Example: From Class Targets to Split Counts

The numbers above are useful as targets, but turning "600-800 images, per class" into an actual folder structure across train/val/test takes a couple of arithmetic steps. Here's the formula, then a worked example using the slate_open/slate_closed split from this project.

Step 1 - Total labeled images. Sum your per-class targets from above (adjusted, per the note above, for the fact that slate_closed is naturally underrepresented per take):

Total labeled = sum of per-class targets
Example: slate_open 1,500 + slate_closed 1,000 = 2,500 labeled images

Step 2 - Back into the negative frames. Negatives are a percentage of the whole dataset (Section 3), not of the labeled portion, so solve for the total:

Total dataset = Total labeled / (1 - negative fraction)
Example at 8% negatives: 2,500 / 0.92 ~= 2,717 -> round to 2,720 total, ~220 negative

Step 3 - Apply the split ratio at the clip level (Section 4). Using 75/15/10 as a midpoint of the ranges above:

• Train (75%): 2,040 images
• Validation (15%): 408 images
• Test (10%): 272 images

Step 4 - Distribute class balance and negatives proportionally, inside each split - not just across the dataset as a whole:

Getting to this table isn't just arithmetic after the fact - it's why Section 4's "split by clip, not frame" and "stratify across diversity axes" rules matter in practice. Hitting these numbers requires tracking, per clip, how many frames of each class and how many negatives it contributes as you annotate - not just eyeballing a train folder once labeling is done and hoping the ratios worked out.

6. Annotation Quality: Tight Boxes, Consistent Corners

Volume and sampling strategy get you diverse data; this is what makes each individual annotation actually worth something.

• Fit boxes tight to the physical edges of the slate, not to its motion-blur halo. This matters more for OBB than it ever did for axis-aligned boxes, because the model isn't just learning "object here," it's learning the exact rotation from your corner placement - sloppy corners teach it a sloppy angle.
• Keep corner ordering consistent across frames. Since OBB is stored as four explicit corner points, the order in which those points are recorded implicitly encodes orientation. If that order flips between one frame and the next (which can happen with careless manual correction of exported labels), you're teaching the model two contradictory ideas of which way the box is "facing" for what should be a smooth, continuous rotation. Most annotation tools default to a consistent order automatically - the failure mode is almost always a human hand-editing a label file afterward, not the tool itself.
• Use interpolation for video, but don't trust it blindly. Most modern annotation tools can interpolate a box's position between two manually placed keyframes, which is a huge time-saver for a moving slate. Spot-check the interpolated frames anyway, especially during the fast part of the motion right around impact - that's exactly where linear interpolation tends to drift away from the true position.
• If more than one person is labeling, write your taxonomy rules down (see Section 2) and spot-check across annotators periodically. Inter-annotator drift on a fuzzy boundary case is invisible until you're staring at a confusion matrix wondering why the model can't decide.

7. The Active Learning Loop: Let the Model Help You Label

Hand-labeling every frame from scratch doesn't scale, and past a certain point you don't need to. Once you have even a rough model, it's a faster annotator than you are - your job shifts from drawing boxes to correcting them. This is the loop that actually got this dataset built:

1. Cut clips. Shutter Encoder to trim raw takes down to the ~2-second slate windows described in Section 3, so you're not importing hours of dead footage into the annotation tool.
2. Import and hand-label a seed set. Load the clips into CVAT and manually annotate the first batch - roughly 100 labeled frames is enough to get started. Use CVAT's rectangle tracking (interpolation between manually placed keyframes, per Section 6) so you're placing boxes on the first and last frame of a motion and letting the tool fill in the rest, not annotating every single frame by hand.
3. Export and train. Pull the labeled set out with my own tool, Dataset Studio, and train a first-pass model. At ~100 images this model will be rough, but it doesn't need to be good yet - it just needs to be better than a blank annotation.
4. Auto-annotate the next batch. Point that first model at the next batch of unlabeled clips instead of hand-drawing from scratch, and trigger auto-annotation. CVAT supports model-assisted annotation directly, or you can serve your own model through Nuclio and wire it in as a proper serverless annotation function inside CVAT rather than a one-off script - either way, the model proposes boxes and you're now correcting instead of creating from nothing, which is a large speedup once the model's confidence is reasonably high on the easy majority of frames.
5. Review, correct, retrain, repeat. Fix what the model got wrong - this is also your best signal for where the taxonomy or sampling strategy from Sections 2-3 is weak; if the model keeps failing on tilted end-slates, go find more tilted end-slates. Fold the corrected batch into the training set and retrain. Each cycle both grows the dataset and improves the model doing the pre-labeling, so the loop compounds - later batches take a fraction of the manual effort the first 100 images did.

The main thing to watch for: don't let the model's own mistakes quietly become training data. A wrong auto-annotated box that goes uncorrected teaches the next model the same mistake with more confidence. Spot-check auto-annotated batches at least as carefully as you would a new annotator's first week of work - probably more, since a model doesn't improve simply from being told "right" or "wrong" after the fact, it needs someone to actually notice the correction was needed in the first place.

References and helpful Videos

Create Dataset with CVAT | Traing YOLO Object Detection Models | CVAT - Accelerate annotation with Hugging Face