GestureOS is a computer vision-powered system that lets you control your mouse and keyboard using hand gestures — no hardware required, just a webcam!
Built using Python, OpenCV, MediaPipe, and PyAutoGUI, this project enables seamless gesture-based interactions like cursor movement, clicks, scrolling, and even system commands — all in real time.
- 🖱️ Move the mouse with finger tracking
- 👆 Pinch gesture for left-click
- 🤜 Fist gesture to drag
- ✌️ Two-finger scroll
- 🤚 Palm to pause interaction
- 🔄 Swipe gestures for window switching
- 🔧 Fully touchless control system
- Python 3.8+
- OpenCV
- MediaPipe (Google)
- PyAutoGUI
- Tkinter (optional – GUI feedback)
GestureOS/ │ ├── main.py # Core logic ├── gestures.py # Gesture definitions and logic ├── controller.py # Mouse + keyboard controller ├── utils.py # Helper functions ├── requirements.txt # Dependencies ├── demo.gif # Demo recording (add yours) └── README.md # This file
git clone https://github.com/fahadnasir13/GestureOS.git
cd GestureOS
pip install -r requirements.txt
python main.py
Or install manually:
bash
pip install opencv-python mediapipe pyautogui
👋 How It Works
Gesture Action
🟢 Index finger only Move cursor
🟡 Index + thumb (pinch) Left click
🔴 Fist Drag mode
🟠 Two fingers down Scroll
🔵 Palm open Pause all actions
🟣 Swipe left/right Switch apps (Alt+Tab)
The system uses MediaPipe Hand Tracking to detect hand landmarks and interprets gestures using custom logic in gestures.py.
Mouse and keyboard control is handled via PyAutoGUI.
🌱 Future Improvements
Multi-hand support (dual gestures)
Voice + gesture combo control
Custom gesture training module
GUI dashboard with settings
OS-level tray application
📄 License
MIT License. Feel free to use, improve, and share!
🙌 Credits
MediaPipe
OpenCV
PyAutoGUI
Built with ❤️ by Fahad Nasir