dsh/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

施工现场安全行为智能识别系统 v2.0.0 - Construction site safety behavior intelligent recognition system.

A dual-camera coordinated capture system that:

• Uses panoramic camera for real-time monitoring with YOLO11 detection (person/hard hat/reflective vest)
• Controls PTZ dome camera to zoom and track detected targets
• Performs OCR number recognition via llama-server API
• Detects safety violations (missing hard hat/reflective vest)
• Pushes events to business platform and announces via TTS

Running Commands

OCR Mode (Number Recognition)

cd dual_camera_system
python main.py --panorama-ip 192.168.1.100 --ptz-ip 192.168.1.101
python main.py --interactive      # Interactive mode
python main.py --skip-calibration # Skip auto-calibration

Safety Mode (Safety Detection)

cd dual_camera_system
python safety_main.py --panorama-ip 192.168.1.100 --ptz-ip 192.168.1.101

Common Parameters

--model-size {n,s,m,l,x}    # YOLO11 model size
--no-gpu                    # Disable GPU
--ocr-host localhost --ocr-port 8111  # OCR API endpoint

Dependencies

pip install opencv-python opencv-contrib-python ultralytics
# OCR requires llama-server running on localhost:8111

Architecture

dual_camera_system/
├── main.py              # OCR mode entry point
├── safety_main.py       # Safety mode entry point
├── config.py            # Config aggregation (imports from config/)
├── config/              # Modular configuration
│   ├── system.py        # Feature toggles and working mode
│   ├── camera.py        # Camera IPs, credentials, SDK paths
│   ├── detection.py     # YOLO detection config
│   └── ...              # Other module configs
├── dahua_sdk.py         # Dahua SDK ctypes wrapper
├── panorama_camera.py   # Panoramic camera + YOLO detection
├── ptz_camera.py        # PTZ dome camera control
├── calibration.py       # Visual calibration (motion detection + feature matching)
├── coordinator.py       # Camera coordination logic (AsyncCoordinator, SequentialCoordinator)
├── safety_detector.py   # Safety detection (hard hat/reflective vest)
├── safety_coordinator.py # Safety mode coordinator
├── ocr_recognizer.py    # OCR number recognition
├── event_pusher.py      # Event push to platform
├── voice_announcer.py   # TTS voice announcement
└── llm_service.py       # LLM service wrapper (Qwen2.5-VL)

Critical Technical Details

SDK Initialization Order (CRITICAL)

In main.py, YOLO/PyTorch MUST load before Dahua SDK. The SDK's CLIENT_Init modifies process memory mapping; if loaded before PyTorch, it causes segfault. The current code handles this correctly - do not change the order.

SDK Path Auto-Selection

config/camera.py auto-selects SDK path by CPU architecture:

• aarch64 → /home/admin/dsh/dh/arm/Bin (Orange Pi)
• x86_64 → /home/wen/dsh/dh/Bin (x86 Linux server)
• Other → ../dh/Bin (development reference)

Camera Ports

• SDK login: port 37777
• RTSP stream: port 554 (rtsp_port in config)

Calibration

• Interval: 24 hours (not 5 minutes as noted in some docs)
• Daily calibration time: configurable via CALIBRATION_CONFIG.daily_calibration_time
• Methods: motion detection + SIFT/ORB feature matching
• The system can fallback to angle estimation if visual calibration fails

Working Modes

Controlled by config/system.py:

• mode: 'safety' - Safety detection mode (safety_main.py)
• mode: 'ocr' - Number recognition mode (main.py)

Feature toggles control enabled modules:

• enable_detection, enable_safety_detection
• enable_calibration, enable_ptz_tracking
• enable_ocr, enable_llm
• enable_event_push, enable_voice_announce

RKNN Test Environment

ssh admin@192.168.20.84
conda activate rknn
cd /home/admin/dsh/dual_camera_system

Model Paths

• Safety detection model: /home/wen/dsh/yolo/yolo11m_safety.pt
• Class mapping: 0=helmet, 3=person, 4=reflective vest
• YOLO11 weights auto-download on first run

RTSP URL Pattern

rtsp://username:password@ip:554/cam/realmonitor?channel=0&subtype=0

Note: subtype=0 for main stream, subtype=1 for sub stream.