Vision Detector

Vision Detector is an AI image analysis tool that supports CoreML machine learning models, multi-modal Large Language Models, and on-device Apple Intelligence. Choose the mode that best fits your use case. You can choose one of the following image input sources: - Live video by the built-in camera - Still image by the built-in camera - Photo Library - Files (iCloud, Google Drive, Dropbox, etc.) For live video inputs, continuous inference is performed on the camera feed. However, the frame rate and other parameters depend on the device. # Notification Center Vision Detector can call Notification Center for predefined conditions. Broadcasting notifications sends notifications to all Apple devices signed in with the same Apple Account. ## CoreML Mode Vision Detector locally executes CoreML models using Apple’s Vision framework. Before using this mode, prepare a machine learning model in the CoreML format (.mlmodel or .mlpackage) using Create ML. Import ML models via iCloud or AirDrop. Vision Detector supports the following types of CoreML models: - Image classification - Object detection - Object Segmentation - Style transfer Built-in Vision framework is also available for human mask and image classification. ## Vision LLM Mode Vision Detector can use multi-modal large language model (LLMs). Define what to do with the prompt as follows: - If the image is a screenshot, answer the name of the software - Call a notification when you see a cat - Count the number of persons in the image, and notify when it changes ## Apple Intelligence mode The images are completely processed on your device locally with your prompt.