OmniParser SOP

Appearance

Test ​

Status: NEEDS-ACCOUNT (model weights required) Install method: GitHub clone + HuggingFace model download Cost/access: Free (open-source), but requires GPU for inference

Classification (2026-06-30) ​

OmniParser (microsoft/OmniParser on HuggingFace and GitHub) is a vision model for parsing UI screenshots. No pip package exists under the name omniparser or any common variant.

What was actually run ​

bash
pip install omniparser            # -> No matching distribution found
pip install omniparser-utils      # -> No matching distribution found
pip install omniparser-microsoft  # -> No matching distribution found

No installable pip package found.

What is required ​

  1. Clone the GitHub repo: git clone https://github.com/microsoft/OmniParser
  2. Download model weights from HuggingFace (IconDetect model + caption model - several GB)
  3. GPU with sufficient VRAM for vision inference (CUDA recommended)
  4. Run inference scripts from the cloned repo

Smoke test after setup (requires GPU environment) ​

bash
python weights/convert_florence2.py
python gradio_demo.py

The Gradio demo launches a local web UI where you can upload a screenshot and see parsed elements.