(Ironically, a prototype itself...) 😅
Status: Work In Progress
- Make it easier to prototype basic Machine Learning apps with SwiftUI
- Provide an easy interface for commonly built views to assist with prototyping and idea validation
- Effectively a wrapper around the more complex APIs, providing a simpler interface (perhaps not all the same functionality, but enough to get you started and inspired!)
- iOS 14.0+ / macOS 13.0+
- Swift 5.9+
- Xcode 15+
- Sound recognition (
recognizeSounds) requires iOS 15.0+ and is unavailable on macOS
PrototypeKit is distributed as a Swift Package.
- In Xcode, choose File → Add Package Dependencies…
- Paste the package URL into the search field:
https://github.com/FridayTechnologies/PrototypeKit - For Dependency Rule, select Up to Next Major Version starting from
0.1.0for reproducible builds. (To live on the latest unreleased changes instead, select Branch and entermaster.) - Click Add Package, then add the PrototypeKit library to your app target.
If you maintain your own Swift package, add PrototypeKit to your dependencies:
dependencies: [
.package(url: "https://github.com/FridayTechnologies/PrototypeKit", from: "0.1.0")
]…then add it to your target's dependencies:
.target(
name: "YourTarget",
dependencies: ["PrototypeKit"]
)Note: PrototypeKit follows Semantic Versioning. It is still pre-1.0, so minor version bumps (
0.x) may include breaking changes; pin with.upToNextMinor(from: "0.1.0")if you need stricter guarantees. See the CHANGELOG for what's in each release.
- API reference — DocC documentation is published to GitHub Pages: https://fridaytechnologies.github.io/PrototypeKit/documentation/prototypekit
- Sample code — the
Examples/directory contains a runnable SwiftUI gallery covering every feature; copyPrototypeKitExamples.swiftinto an app to try them.
Here are a few basic examples you can use today.
- Ensure you have created your Xcode project
- Ensure you have added the PrototypeKit package to your project (see Installation above)
- Select your project file within the project navigator.
- Ensure that your target is selected
- Select the info tab.
- Right-click within the "Custom iOS Target Properties" table, and select "Add Row"
- Use
Privacy - Camera Usage Descriptionfor the key. Type the reason your app will use the camera as the value.
Utilise PKCameraView
PKCameraView()Full Example
import SwiftUI
import PrototypeKit
struct ContentView: View {
var body: some View {
VStack {
PKCameraView()
}
.padding()
}
}- Required Step: Drag in your Create ML / Core ML model into Xcode.
- Change
FruitClassifierbelow to the name of your Model. - You can use latestPrediction as you would any other state variable (i.e refer to other views such as Slider)
Utilise ImageClassifierView
ImageClassifierView(modelURL: FruitClassifier.urlOfModelInThisBundle,
latestPrediction: $latestPrediction)Full Example
import SwiftUI
import PrototypeKit
struct ImageClassifierViewSample: View {
@State var latestPrediction: String = ""
var body: some View {
VStack {
ImageClassifierView(modelURL: FruitClassifier.urlOfModelInThisBundle,
latestPrediction: $latestPrediction)
Text(latestPrediction)
}
}
}Detect and locate objects in the live camera feed using a Create ML / Core ML Object Detector model.
- Required Step: Drag in your Create ML / Core ML object detector model into Xcode.
- Change
MyObjectDetectorbelow to the name of your Model. detectedObjectsholds the labels of the objects found in the latest frame; use it as you would any other state variable.
Utilise ObjectDetectorView
ObjectDetectorView(modelURL: MyObjectDetector.urlOfModelInThisBundle,
detectedObjects: $detectedObjects)Full Example
import SwiftUI
import PrototypeKit
struct ObjectDetectorViewSample: View {
@State var detectedObjects: [String] = []
var body: some View {
VStack {
ObjectDetectorView(modelURL: MyObjectDetector.urlOfModelInThisBundle,
detectedObjects: $detectedObjects)
ScrollView {
ForEach(Array(detectedObjects.enumerated()), id: \.offset) { index, object in
Text(object)
}
}
}
}
}Need to know where each object is (for example, to draw bounding boxes)? Bind an array of
DetectedObject instead of [String]. Each DetectedObject carries the label, a confidence
(0–1), and a normalized boundingBox (CGRect, origin bottom-left as Vision reports it):
ObjectDetectorView(modelURL: MyObjectDetector.urlOfModelInThisBundle,
detectedObjects: $detectedObjects) // $detectedObjects is [DetectedObject]Full Example (with bounding boxes)
import SwiftUI
import PrototypeKit
struct ObjectDetectorBoxesSample: View {
@State var detectedObjects: [DetectedObject] = []
var body: some View {
ZStack {
ObjectDetectorView(modelURL: MyObjectDetector.urlOfModelInThisBundle,
detectedObjects: $detectedObjects)
GeometryReader { geometry in
ForEach(Array(detectedObjects.enumerated()), id: \.offset) { _, object in
let box = object.boundingBox
Rectangle()
.stroke(.red, lineWidth: 2)
// Vision's origin is bottom-left; SwiftUI's is top-left, so flip Y.
.frame(width: box.width * geometry.size.width,
height: box.height * geometry.size.height)
.position(x: box.midX * geometry.size.width,
y: (1 - box.midY) * geometry.size.height)
.overlay(Text(object.label))
}
}
}
}
}Classify hand poses in real-time using a Create ML / Core ML hand action classifier.
- Required Step: Drag in your Create ML / Core ML hand pose model into Xcode.
- Change
HandPoseClassifierbelow to the name of your Model. - You can use
latestPredictionas you would any other state variable.
Utilise HandPoseClassifierView
HandPoseClassifierView(modelURL: HandPoseClassifier.urlOfModelInThisBundle,
latestPrediction: $latestPrediction)Full Example
import SwiftUI
import PrototypeKit
struct HandPoseClassifierViewSample: View {
@State var latestPrediction: String = ""
var body: some View {
VStack {
HandPoseClassifierView(modelURL: HandPoseClassifier.urlOfModelInThisBundle,
latestPrediction: $latestPrediction)
Text(latestPrediction)
}
}
}Classify a person's action from their body movement in real-time using a Create ML / Core ML Action Classifier model.
- Required Step: Drag in your Create ML / Core ML action classifier model into Xcode.
- Change
ActionClassifierbelow to the name of your Model. - You can use
latestPredictionas you would any other state variable.
An action unfolds over time, so ActionClassifierView detects body-pose keypoints with Vision and
feeds a sliding window of frames (two seconds by default) into your model, updating latestPrediction
as the window advances.
Utilise ActionClassifierView
ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle,
latestPrediction: $latestPrediction)If your model uses different feature names or a different prediction window, supply an
ActionClassifierConfiguration:
ActionClassifierView(
modelURL: ActionClassifier.urlOfModelInThisBundle,
configuration: ActionClassifierConfiguration(predictionWindowSize: 90),
latestPrediction: $latestPrediction
)Full Example
import SwiftUI
import PrototypeKit
struct ActionClassifierViewSample: View {
@State var latestPrediction: String = ""
var body: some View {
VStack {
ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle,
latestPrediction: $latestPrediction)
Text(latestPrediction)
}
}
}Utilise LiveTextRecognizerView
LiveTextRecognizerView(detectedText: $detectedText)Full Example
import SwiftUI
import PrototypeKit
struct TextRecognizerView: View {
@State var detectedText: [String] = []
var body: some View {
VStack {
LiveTextRecognizerView(detectedText: $detectedText)
ScrollView {
ForEach(Array(detectedText.enumerated()), id: \.offset) { line, text in
Text(text)
}
}
}
}
}Utilise LiveBarcodeRecognizerView
LiveBarcodeRecognizerView(detectedBarcodes: $detectedBarcodes)Full Example
import SwiftUI
import PrototypeKit
struct BarcodeRecognizerView: View {
@State var detectedBarcodes: [String] = []
var body: some View {
VStack {
LiveBarcodeRecognizerView(detectedBarcodes: $detectedBarcodes)
ScrollView {
ForEach(Array(detectedBarcodes.enumerated()), id: \.offset) { index, barcode in
Text(barcode)
}
}
}
}
}Recognize cats and dogs in real-time using the built-in Vision animal recognizer (no Core ML model required).
Utilise LiveAnimalRecognizerView
LiveAnimalRecognizerView(detectedAnimals: $detectedAnimals)Full Example
import SwiftUI
import PrototypeKit
struct AnimalRecognizerView: View {
@State var detectedAnimals: [String] = []
var body: some View {
VStack {
LiveAnimalRecognizerView(detectedAnimals: $detectedAnimals)
ScrollView {
ForEach(Array(detectedAnimals.enumerated()), id: \.offset) { index, animal in
Text(animal)
}
}
}
}
}Detect faces in real-time using the built-in Vision face detector (no Core ML model required).
Utilise LiveFaceDetectorView
LiveFaceDetectorView(faceCount: $faceCount)Full Example
import SwiftUI
import PrototypeKit
struct FaceDetectorView: View {
@State var faceCount: Int = 0
var body: some View {
VStack {
LiveFaceDetectorView(faceCount: $faceCount)
Text("Faces: \(faceCount)")
}
}
}Detect human body poses in real-time using the built-in Vision human body pose request (no Core ML model required).
Utilise LiveBodyPoseDetectorView
LiveBodyPoseDetectorView(bodyCount: $bodyCount)Full Example
import SwiftUI
import PrototypeKit
struct BodyPoseDetectorView: View {
@State var bodyCount: Int = 0
var body: some View {
VStack {
LiveBodyPoseDetectorView(bodyCount: $bodyCount)
Text("Bodies: \(bodyCount)")
}
}
}Detect rectangular shapes (documents, cards, signs) in real-time using the built-in Vision rectangle detector (no Core ML model required).
Utilise LiveRectangleDetectorView
LiveRectangleDetectorView(rectangleCount: $rectangleCount)Full Example
import SwiftUI
import PrototypeKit
struct RectangleDetectorView: View {
@State var rectangleCount: Int = 0
var body: some View {
VStack {
LiveRectangleDetectorView(rectangleCount: $rectangleCount)
Text("Rectangles: \(rectangleCount)")
}
}
}Required Step: Sound recognition uses the microphone, so you must add the
Privacy - Microphone Usage Description(NSMicrophoneUsageDescription) key to your target's Info properties — follow the same steps as the camera setup above, using the microphone key instead. Without it, classification fails silently.
Utilise recognizeSounds modifier to detect sounds in real-time. This feature supports both the system sound classifier and custom Core ML models.
.recognizeSounds(recognizedSound: $recognizedSound)For custom configuration, you can use the SoundAnalysisConfiguration:
.recognizeSounds(
recognizedSound: $recognizedSound,
configuration: SoundAnalysisConfiguration(
inferenceWindowSize: 1.5, // Window size in seconds
overlapFactor: 0.9, // Overlap between consecutive windows
mlModel: yourCustomModel // Optional custom Core ML model
)
)Full Example
import SwiftUI
import PrototypeKit
struct SoundRecognizerView: View {
@State var recognizedSound: String?
var body: some View {
VStack {
Text("Recognized Sound: \(recognizedSound ?? "None")")
}
// Attach the modifier to a view to start listening; updates `recognizedSound` live.
.recognizeSounds(recognizedSound: $recognizedSound)
}
}Classify the device's physical activity (for example walking, running, or standing still) in real-time from the accelerometer and gyroscope, using a Create ML / Core ML Activity Classifier.
- Required Step: Drag in your Create ML / Core ML activity classifier model into Xcode.
- Change
ActivityClassifierbelow to the name of your Model. - You can use
latestActivityas you would any other state variable.
Utilise the classifyActivity modifier to detect activity in real-time.
.classifyActivity(modelURL: ActivityClassifier.urlOfModelInThisBundle,
latestActivity: $latestActivity)If your model's input/output feature names or sample rate differ from the Create ML defaults,
supply an ActivityClassifierConfiguration:
.classifyActivity(
modelURL: ActivityClassifier.urlOfModelInThisBundle,
configuration: ActivityClassifierConfiguration(
sensorUpdateInterval: 1.0 / 50.0, // Sensor sample rate in seconds (50 Hz)
predictionWindowSize: 50 // Samples per prediction
),
latestActivity: $latestActivity
)Note: Activity classification relies on
CoreMotionand is available on iOS only. The modifier produces no visible content of its own — attach it to a view to drive classification and readlatestActivity.
Full Example
import SwiftUI
import PrototypeKit
struct ActivityClassifierViewSample: View {
@State var latestActivity: String?
var body: some View {
VStack {
Text("Activity: \(latestActivity ?? "Detecting…")")
}
// Attach the modifier to a view to start classifying; updates `latestActivity` live.
.classifyActivity(modelURL: ActivityClassifier.urlOfModelInThisBundle,
latestActivity: $latestActivity)
}
}Analyse text on-device with Apple's Natural Language framework. Unlike the camera and sound features, these need no camera, microphone, permissions, or Core ML model — everything ships with the OS, and they work on both iOS and macOS. That makes them the gentlest way to get started with on-device ML.
Each is a View modifier that re-runs whenever the text you pass in changes, updating a binding with
the result.
Score how positive or negative a piece of text is, from -1 (very negative) to 1 (very positive).
.analyzeSentiment(text: text, score: $score)Full Example
import SwiftUI
import PrototypeKit
struct SentimentView: View {
@State var text: String = "I love this!"
@State var score: Double = 0
var body: some View {
VStack {
TextField("Type something", text: $text)
Text("Sentiment: \(score, specifier: "%.2f")")
}
.analyzeSentiment(text: text, score: $score)
}
}Detect the dominant language of a piece of text as a BCP-47 code (for example en, fr).
.identifyLanguage(text: text, language: $language)Full Example
import SwiftUI
import PrototypeKit
struct LanguageView: View {
@State var text: String = "Bonjour tout le monde"
@State var language: String?
var body: some View {
VStack {
TextField("Type something", text: $text)
Text("Language: \(language ?? "Detecting…")")
}
.identifyLanguage(text: text, language: $language)
}
}Extract the people, places, and organizations mentioned in a piece of text.
.tagEntities(text: text, entities: $entities)Full Example
import SwiftUI
import PrototypeKit
struct EntitiesView: View {
@State var text: String = "Tim Cook announced the news in London."
@State var entities: [String] = []
var body: some View {
VStack {
TextField("Type something", text: $text)
ScrollView {
ForEach(Array(entities.enumerated()), id: \.offset) { index, entity in
Text(entity)
}
}
}
.tagEntities(text: text, entities: $entities)
}
}PrototypeKit is designed to fail gracefully — it will not crash your app on bad input:
- If a Core ML model can't be loaded (wrong URL, incompatible model), the affected view still shows the camera feed but produces no predictions. The failure is logged rather than fatal.
- Per-frame Vision and sound-classification errors are logged and skipped.
- If your app is missing the
NSCameraUsageDescriptionkey, the camera view shows an on-screen message explaining what to add instead of a black preview.
Diagnostics use Apple's unified logging system (os.Logger) under the subsystem
com.prototypekit.PrototypeKit, so nothing is printed to your console in Release builds. To watch
PrototypeKit's logs while developing:
log stream --predicate 'subsystem == "com.prototypekit.PrototypeKit"'PrototypeKit only logs developer-facing diagnostics — never the contents of camera frames, audio, or recognized text.
Is this production ready?
Not yet — PrototypeKit is intended for prototyping and idea validation, and it does not yet publish versioned releases. That said, it no longer crashes the host app on bad input (missing/invalid models, denied permissions, audio interruptions): those paths now degrade gracefully and log through
os.Logger. See the CHANGELOG for details.