Pavan Varahad V

Home / Blog / WiFi CSI

Seeing without a camera

Building a real-time activity detector from WiFi signals: an ESP32, a signal-processing pipeline, a Random Forest, and a live dashboard with a 3D pose view.

  • WiFi CSI
  • ESP32
  • Signal processing
  • Random Forest
  • FastAPI
  • Three.js

When a person moves through a room, they change how WiFi signals bounce around it. No camera, wearable or microphone is involved. The router is already there; the person is the thing disturbing its signal.

This project turns that disturbance into something readable in real time: is anyone here, how much are they moving, are they breathing, did someone fall, and are they sitting or standing. This post covers how the system is built, from the serial port to the browser, and the signal-processing choices that made it behave.

The demo, from my LinkedIn post. Open on LinkedIn

What CSI is, in one paragraph

Channel State Information describes how a WiFi signal arrived at the receiver, subcarrier by subcarrier. A still room gives a steady pattern. A moving person reshapes the reflections, so the pattern changes. An ESP32 running CSI firmware streams these values over USB serial, and everything else is software that decides what the changes mean.

The data flow

Data flow

HardwareESP32-WROOM, CSI firmwaretext lines over USB serial
↓ serial
Thread · sourceSerial source
  • baud autodetect (921600, 115200, …)
  • line parser: CSV, JSON or array
  • alternatives with the same interface: replay and simulated sources
↓ frames (timestamp, 52 subcarrier amplitudes, RSSI) via a bounded queue
Thread · consumerEngine consumer
  • feeds a 60 s ring buffer
  • writes labelled CSV while recording
↓ shared, lock-guarded buffer
Presencedetector
Motion leveldetector
Breathing + falldetectors
ActivityRandom Forest inference
↓
ServerFastAPI, REST + WebSocket /ws
↓
ClientWeb dashboardvanilla JS, no build step

One interface, three sources

The pipeline never knows where frames come from. One source reads the real board, one plays back a recorded session, and one generates synthetic frames. That means I can develop the dashboard with no hardware attached, replay a recorded session to debug a detector, and run the same code path in all three cases.

The recorder writes labelled CSV, and that is also the replay format. One file format serves as training data, test fixture and demo reel.

Threads and concurrency

Threads

ThreadRole
Source threadreads serial, file or simulator, parses, and pushes frames to a bounded queue. Drops the oldest frame when full.
Consumer threaddrains the queue into the pipeline ring buffer and the recorder.
uvicorn event loopREST handlers and a WebSocket tick of about 15 Hz. Heavy metric math runs in the default executor so the loop never blocks.

Two decisions matter most. First, the queue is bounded and drops the oldest frame when it fills: for a live display, fresh data beats complete data, and a slow consumer should not create an ever-growing lag. Second, switching sources stops the old thread and rebuilds the pipeline, detectors and classifier from scratch, so state from a recording can never leak into a live session.

Signal processing choices

This is where most of the real work is. Each choice is a small decision with a reason.

  • Amplitude only. ESP32 CSI phase is dominated by sampling-time offset and packet-boundary jitter. Without multi-antenna phase sanitisation it adds noise rather than signal, so I use amplitude and ignore phase.
  • EMA at ingest (α = 0.3). An exponential moving average keeps the per-frame cost proportional to the number of subcarriers.
  • Hampel filter per window, not per frame. It is more expensive, so it runs once per analysis window instead of on every frame. Cheap work runs often; costly work runs rarely.
  • Resample irregular timing. CSI arrives per received packet, so the samples are not evenly spaced. Before any FFT or Welch analysis, I resample onto a uniform grid by interpolation. Skipping this step quietly corrupts every frequency estimate.
  • Motion energy. The mean over subcarriers of the temporal standard deviation over a 2 s window. It is the standard, robust statistic for "is the channel changing", and most detectors build on it.

Every tunable lives in one place, so changing a threshold never means hunting through the code.

From signal to meaning

The detectors read the same shared buffer in parallel:

  • Presence uses calibration of the empty room and hysteresis, so it does not flicker around its threshold.
  • Motion level reports how much the channel is changing.
  • Breathing estimates breaths per minute from the power spectral density, which only works when the person is still.
  • Fall is a heuristic on the motion pattern.
  • Activity is a Random Forest over features from sliding windows, trained on labelled recordings, with the model hot-reloaded when a new one is saved so I can retrain without restarting the server.

Heuristics handle what has a clear physical signature (presence, breathing). A learned model handles what does not (sitting versus standing). Choosing the simple tool where it works kept the classifier small.

The dashboard

The web UI has no build step and is served as plain static files. A single WebSocket message carries three things:

  • Heatmap rows: only the new columns since the last tick.
  • Metrics: detector, activity and status values, sent about three times a second.
  • Motion series: a 20 s motion-energy sparkline.

The waterfall heatmap scrolls by shifting the existing canvas left and painting only the new columns, so the cost per tick stays constant however long the session runs. Sending deltas rather than the full history is what keeps a 15 Hz display light on both the server and the browser.

The 3D pose view

The most fun part is the 3D pose view: a Three.js scene with a procedural skeleton avatar, bundled locally with no CDN. The dashboard passes every metrics message to the 3D scene, which maps it onto the avatar.

What drives the avatar

SignalAvatar response
activity labelpose target (standing, sitting, walking, waving, lying). Joints ease toward it each frame; walking adds a gait cycle scaled by motion level.
breathing_bpmchest scale oscillation, visible when the person is still.
fallsnaps to lying and shows a pulsing red floor ring.
motion levelagitation, and the colour of the router-to-ESP32 link line.
label = emptyavatar ghosts to 10 % opacity.

A bug that taught me something

With a trained model loaded, the avatar follows the classifier label directly and fades out only when the label is empty. It is deliberately not gated on the presence detector. The presence detector can briefly drop while someone sits very still, and gating on it would freeze the avatar a label behind. Presence only controls opacity when no model has been trained yet. When two signals disagree, decide which one owns each decision.

On the dashboard, activity colours live in one place, applied to the Activity tile and to each per-class probability bar. The avatar itself keeps fixed material colours, so colour means one thing in each place.

The honest limit

The avatar is a visualisation of the classifier's pose class. It is not measured joint positions. A single-antenna ESP32 cannot provide those, so I would be misleading anyone who read the skeleton as a pose estimate. Showing the label as an animated figure is a good way to make the output understandable, as long as the post and the UI say what it really is.

What I would do next

These are ideas, not shipped work:

  • More than one room and one person. The model learns the room it was trained in, so a quick recalibration flow would make it usable elsewhere.
  • More antennas, which is what makes phase usable and opens the door to real location and pose information.
  • More activity classes, now that recording and retraining with hot-reload is cheap.

Takeaways

  1. Make the data source swappable. Real, recorded and simulated sources behind one interface made everything easier to test.
  2. Prefer fresh data over complete data in anything live: bounded queues that drop the oldest frame.
  3. Do cheap work often and expensive work rarely: EMA per frame, Hampel per window.
  4. Fix timing before frequency analysis. Uneven samples break FFTs silently.
  5. Be clear about what a visualisation is. A moving avatar says "classifier label", not "measured skeleton".

If you work on WiFi sensing, I would like to hear how you handle phase on single-antenna hardware, and what you do about moving between rooms.

← All posts