AI Powered GAME Controller
An AI-powered Python game controller applying Machine Learning, real-time Computer Vision, and PyAutoGUI for hands-free interactive tracking. It captures real-time video feed via a webcam, tracking specific hand gestures or body movements, and translating them into precise in-game actions for an immersive experience without traditional physical hardware.
The Challenge
Traditional game controllers are great, but I wanted to explore whether a computer could interpret natural human movement as game input — no hardware required. The core challenge was achieving low-latency gesture recognition from a standard webcam feed while keeping the system responsive enough for real-time gameplay. Existing solutions were either too expensive or too slow.
Technical Approach
The controller uses a webcam feed as the input stream, OpenCV for frame processing, MediaPipe-style landmark tracking for gesture interpretation, and PyAutoGUI to convert recognized movement into keyboard or media-control actions. The important design constraint was keeping the frame-processing loop simple enough to remain responsive. Gesture mapping was kept explicit so each recognized pose could be tested independently before being connected to a real game or media workflow.
Outcome
This project became the foundation for my interest in applied computer vision because it connected perception directly to user interaction. It showed that AI projects become much more useful when model output is translated into a clear action layer. The work also exposed practical tradeoffs around lighting, camera angle, gesture ambiguity, and frame latency that do not appear in static ML demos.
What I Built & Learned
This was my introduction to Computer Vision and it completely changed how I think about human-computer interaction. I learned to implement real-time pose estimation and hand tracking using MediaPipe and OpenCV, map landmark coordinates to keypress events via PyAutoGUI, and optimize frame processing pipelines to minimize input lag. It sparked a deep interest in applied AI and perception systems that continues to drive my projects today.
