Five ways to turn a packet capture into training rows

Project: Activity Detection from Network Traffic

Before any model gets trained, a packet capture and a list of labelled time intervals have to become a table: fixed-width feature vectors, one label each. That sounds like plumbing. It isn't: it's a modeling decision as consequential as which classifier to use, and it's the one that's easiest to skip past without noticing you made a choice at all.

The setup: a small home network (two smart bulbs, a camera, a motion sensor, a plug, an Alexa device) captured to .pcap while labelled activities happened (device turned on, motion detected, stream started). The features are all metadata: frame length, inter-arrival time, TCP flags, TTL, TLS record type, which fixed device IP a packet was to or from. Never payload: the premise of the whole project is that this has to work against fully encrypted traffic, the same visibility a network-level observer would actually have.

The five strategies

The gap between "activity happened from t=12.3s to t=15.1s" and "a row in a training table" has to be filled in somehow, and there isn't one obviously correct way to do it:

Sliding: fixed 1-second windows, stepped every 0.5s across the whole capture, labelled by whichever activity covers at least half the window (else idle). Produces the most rows by far, and the least clean labels: a window straddling the start of an event gets a label that's only partially true, and the 50% threshold is an arbitrary line with real cases sitting right on either side of it.
Event-centered: a fixed 2-second window centered on each activity's midpoint. Labels are exact, but a short event gets padded with quiet time on either side, and anything happening outside that ±1s gets thrown away entirely.
Event-centered with idle: the same, plus a matching 2s window centered on the midpoint of each gap between activities, so idle is represented by windows drawn the same way as everything else, instead of being whatever's left over.
Full-event: one window per activity, sized to its actual start and end. No padding, no truncation, no arbitrary duration, but now duration itself varies row to row, and several features (raw packet counts especially) scale with window length. A fixed-feature classifier has no way to know that unless duration is fed in explicitly.
Full-event with idle: full-event windows, plus the entire gap between each pair of activities kept as one long idle window, rather than a fixed-size sample of it.

All five are implemented, behind a flag, and all five produce a CSV shaped identically enough to run through the same downstream feature extraction and modeling code. That wasn't an accident: it was the only way to actually compare them instead of picking one on instinct and hoping.

Why this mattered more than the model

Swapping logistic regression for XGBoost changes performance at the margins. Swapping the windowing strategy changes what the model is being asked to learn in the first place. A sliding-window dataset has boundary-blurred labels but a lot of them; a full-event dataset has exact labels but leaks window duration into the feature space unless you're careful; an event-centered dataset throws away real data to keep windows a consistent size. None of these are bugs: they're the actual cost of turning continuous, variable-length activity into the fixed-shape rows a classifier needs, but each one biases the resulting model in a different, specific direction, and that bias doesn't show up as an error message. It shows up as a model that looks fine on paper and generalizes badly, for a reason that has nothing to do with which algorithm was chosen.

The other constant across all five strategies is class imbalance: in every one of them, idle dominates, because most of a home network's life is quiet. That's why every model here is scored and refit on balanced accuracy during grid search rather than plain accuracy, and why SMOTE oversampling and class-weighted variants are part of the training pipeline rather than an afterthought bolted on at the end. A model that predicts idle unconditionally would post a misleadingly good accuracy number on any of these five datasets; balanced accuracy is what stops that from looking like success.