How to Use Apple’s Foundation Models in a Web App with a macOS Companion

<p>Not every AI feature needs a cloud model, with its per-token bills, network round-trips, and private data leaving your machine. If you're on a modern Mac, a capable language model is already on your disk.</p> <p><strong>Foundation Models</strong> is Apple's Swift framework for working with large language models. It's the on-device model behind Apple Intelligence, Apple's Private Cloud Compute, or another provider's server model.</p> <p>This tutorial targets the on-device model: you send it a prompt and it runs entirely on the Mac's own hardware locally, free-per-call, and offline-friendly.</p> <p>Paired with Apple Vision for reading images on device, that's enough to build real AI features like summaries, classification, and structured extraction without the data ever leaving your machine.</p> <h2 id="heading-table-of-contents">Table Of Contents</h2> <ul> <li><p><a href="#heading-what-you-will-build">What You Will Build</a></p> </li> <li><p><a href="#heading-prerequisites">Prerequisites</a></p> </li> <li><p><a href="#heading-why-a-macos-companion-app">Why a macOS Companion App?</a></p> </li> <li><p><a href="#heading-foundation-models-cant-read-images-directly">Foundation Models Can't Read Images Directly</a></p> </li> <li><p><a href="#heading-project-structure">Project Structure</a></p> </li> <li><p><a href="#heading-build-the-react-app">Build the React App</a></p> <ul> <li><p><a href="#heading-check-companion-health">Check Companion Health</a></p> </li> <li><p><a href="#heading-convert-the-image-to-base64">Convert the Image to Base64</a></p> </li> <li><p><a href="#heading-analyze-immediately-after-upload">Analyze Immediately After Upload</a></p> </li> <li><p><a href="#heading-send-the-image-to-the-companion">Send the Image to the Companion</a></p> </li> <li><p><a href="#heading-render-the-json-output">Render the JSON Output</a></p> </li> </ul> </li> <li><p><a href="#heading-build-the-macos-companion-app">Build the macOS Companion App</a></p> </li> <li><p><a href="#heading-check-foundation-models-availability">Check Foundation Models Availability</a></p> </li> <li><p><a href="#heading-extract-text-with-apple-vision">Extract Text with Apple Vision</a></p> </li> <li><p><a href="#heading-ask-foundation-models-to-explain-the-vision-output">Ask Foundation Models to Explain the Vision Output</a></p> </li> <li><p><a href="#heading-return-json-to-the-browser">Return JSON to the Browser</a></p> </li> <li><p><a href="#heading-run-the-app">Run the App</a></p> </li> <li><p><a href="#heading-conclusion">Conclusion</a></p> </li> <li><p><a href="#heading-resources">Resources</a></p> </li> </ul> <h2 id="heading-what-you-will-build">What You Will Build</h2> <p>You'll build <strong>Vision Bridge</strong>, a web app that sends an image to a local macOS companion. The companion reads the image with Apple Vision, reasons about it with Foundation Models, and returns structured JSON to the browser: private, on-device AI behind a plain web interface.</p> <p>You can find the complete source code in this GitHub repository: <a href="http://github.com/03balogun/vision-bridge">github.com/03balogun/vision-bridge</a>.</p> <p>The goal isn't to build a giant product but rather to understand the architecture behind how this works.</p> <img src="https://cdn.hashnode.com/uploads/covers/5db93b3da2342e8354088115/6d18db01-e921-4291-bb2e-26be2c02b304.png" alt="Screenshot of the Vision Bridge app, with image upload on the left and JSON output on the right" style="display:block;margin:0 auto" width="3024" height="1714" loading="lazy"> <p>Vision Bridge has two parts:</p> <ul> <li><p>A React app with a split-screen interface.</p> </li> <li><p>A macOS companion app that exposes a local API.</p> </li> </ul> <p>The React app has:</p> <ul> <li><p>An image upload area</p> </li> <li><p>An image preview</p> </li> <li><p>Automatic analysis after upload</p> </li> <li><p>A JSON output viewer</p> </li> <li><p>A companion health status indicator</p> </li> </ul> <p>The macOS companion app has:</p> <ul> <li><p><code>GET /v1/health</code></p> </li> <li><p><code>POST /v1/analyze-image</code></p> </li> <li><p>Apple Vision OCR</p> </li> <li><p>Foundation Models availability checks</p> </li> <li><p>Foundation Models reasoning over Vision output</p> </li> </ul> <p>The final response looks like this:</p> <pre><code class="language-json">{ "support": { "visionAvailable": true, "foundationModelAvailable": true, "foundationModelStatus": "available" }, "image": { "filename": "screenshot.png", "contentType": "image/png", "byteCount": 1048576, "width": 1440, "height": 900 }, "vision": { "detectedText": [ { "text": "Build failed", "confidence": 0.96, "boundingBox": { "x": 0.12, "y": 0.31, "width": 0.45, "height": 0.08 } } ] }, "model": { "summary": "The image appears to show a software build failure.", "description": "A developer tool window is showing an error state with diagnostic text.", "suggestedTags": ["screenshot", "developer-tool", "error"], "possibleUses": [ "Generate alt text", "Summarize screenshots", "Extract document data" ] } } </code></pre> <h2 id="heading-prerequisites">Prerequisites</h2> <p>To follow along, you need:</p> <ul> <li><p>macOS 26 or newer</p> </li> <li><p>Xcode with the macOS 26 SDK</p> </li> <li><p>Node.js 20 or newer</p> </li> <li><p>Basic React knowledge</p> </li> <li><p>Basic Swift knowledge</p> </li> <li><p>A Mac that supports Apple Intelligence</p> </li> </ul> <p>Foundation Models availability depends on the Mac, the OS version, and Apple Intelligence settings. The companion checks this at runtime, which we'll cover below.</p> <h2 id="heading-why-a-macos-companion-app">Why a macOS Companion App?</h2> <p>You can't write this in a regular React app:</p> <pre><code class="language-ts">import FoundationModels from "apple-frameworks"; </code></pre> <p>That API doesn't exist in the browser. A native macOS app, however, can use any Apple framework, so the companion acts as a local bridge. The same pattern works for any native capability the web platform doesn't expose.</p> <h2 id="heading-foundation-models-cant-read-images-directly">Foundation Models Can't Read Images Directly</h2> <p>The public Foundation Models framework is a language model interface. It doesn't currently expose direct image input the way a multimodal cloud model might, so this tutorial never sends the image to the model. Instead, the companion feeds the Vision OCR observations and image metadata into the prompt. The model reasons over structured text, never the original pixels.</p> <p>That split plays to each framework's strength: Vision is excellent at pulling machine-readable information out of images, and Foundation Models turns that information into summaries, labels, explanations, and structured output.</p> <img src="https://cdn.hashnode.com/uploads/covers/5db93b3da2342e8354088115/a5c11ad4-dcac-4690-bc6d-08b27fd6fed8.png" alt="Vision Bridge architecture: the browser sends the image over localhost to the Swift companion, which runs Apple Vision OCR, feeds the observations to Foundation Models, and returns structured JSON" style="display:block;margin:0 auto" width="2492" height="1572" loading="lazy"> <p>The above diagram shows the round trip that the rest of this tutorial builds. The browser sends the uploaded image as base64 JSON over localhost to the Swift companion. Inside the companion, Apple Vision runs OCR on the image and produces text observations: the recognized strings, their confidence scores, and their bounding boxes.</p> <p>Those observations, not the image itself, are formatted into a prompt for Foundation Models, which generates a summary, description, and tags. The companion then bundles the Vision output and the model output into one JSON response and returns it to the browser.</p> <h2 id="heading-project-structure">Project Structure</h2> <p>Create a project with this structure:</p> <pre><code class="language-text">vision-bridge/ apps/ web/ src/ main.tsx styles.css package.json vite.config.ts macos-companion/ Package.swift Sources/ VisionBridgeCompanion/ main.swift package.json README.md </code></pre> <p>The root <code>package.json</code> gives us a few convenient commands:</p> <pre><code class="language-json">{ "scripts": { "dev": "npm --workspace apps/web run dev", "build": "npm --workspace apps/web run build", "companion": "swift run --package-path apps/macos-companion VisionBridgeCompanion" }, "workspaces": ["apps/web"] } </code></pre> <h2 id="heading-build-the-react-app">Build the React App</h2> <p>The web app is intentionally simple. It has one job: let the user pick an image and show the JSON returned by the companion.</p> <p>The web app uses Vite, React, Lucide icons, and a JSON viewer:</p> <pre><code class="language-json">{ "dependencies": { "@vitejs/plugin-react": "^6.0.3", "lucide-react": "^0.468.0", "react": "^18.3.1", "react-dom": "^18.3.1", "react-json-view-lite": "^2.5.0", "vite": "^8.1.3" } } </code></pre> <p>After defining the dependencies, install them:</p> <pre><code class="language-plaintext">npm install </code></pre> <p>The API base URL points to the local companion:</p> <pre><code class="language-ts">const API_BASE_URL = "http://127.0.0.1:43119"; </code></pre> <h3 id="heading-check-companion-health">Check Companion Health</h3> <p>The web app pings the companion so the UI can show whether the native bridge is online:</p> <pre><code class="language-ts">async function checkHealth() { setHealthError(null); try { const response = await fetch(`${API_BASE_URL}/v1/health`); if (!response.ok) { throw new Error(`Health check failed with ${response.status}`); } const payload = await response.json(); setHealth(payload); } catch (error) { setHealth(null); setHealthError(error instanceof Error ? error.message : "Companion unavailable"); } } </code></pre> <img src="https://cdn.hashnode.com/uploads/covers/5db93b3da2342e8354088115/dc3c37eb-1d9c-4b44-82db-f38adada4f19.png" alt="Screenshot of the companion online status pill" style="display:block;margin:0 auto" width="732" height="212" loading="lazy"> <h3 id="heading-convert-the-image-to-base64">Convert the Image to Base64</h3> <p>When the user selects a file, the app converts it to base64 so it can be sent as JSON:</p> <pre><code class="language-ts">function readFileAsBase64(file: File) { return new Promise&lt;string&gt;((resolve, reject) =&gt; { const reader = new FileReader(); reader.onload = () =&gt; { const result = String(reader.result); resolve(result.includes(",") ? result.split(",")[1] : result); }; reader.onerror = () =&gt; reject(reader.error); reader.readAsDataURL(file); }); } </code></pre> <p>This isn't the only way to upload files. You could also use <code>multipart/form-data</code>, but JSON keeps the demo easy to inspect.</p> <h3 id="heading-analyze-immediately-after-upload">Analyze Immediately After Upload</h3> <p>The app starts analysis as soon as an image is uploaded:</p> <pre><code class="language-ts">async function handleFile(file: File) { if (!file.type.startsWith("image/")) { setError("Choose a PNG, JPEG, HEIC, or another browser-readable image."); return; } const base64 = await readFileAsBase64(file); const nextImage = { file, previewUrl: URL.createObjectURL(file), base64, }; setSelectedImage(nextImage); setAnalysis(null); setError(null); setCopied(false); analyzeImage(nextImage); } </code></pre> <p><code>handleFile</code> does the preparation work for every new image. It rejects anything that isn't a browser-readable image, converts the file to base64, and builds a single object holding everything the rest of the flow needs: the original <code>File</code> (for its name and MIME type), an object URL for the preview, and the base64 payload for the API call.</p> <p>It then clears out the previous run the old analysis, any error message, and the "copied" indicator so the UI never shows results from the last image next to a new one. Finally, it kicks off <code>analyzeImage(nextImage)</code> immediately.</p> <p>Note that it passes the fresh object directly instead of relying on the <code>selectedImage</code> state: React state updates don't apply until the next render, so reading the state here would still give you the <em>previous</em> image.</p> <p>The <code>Analyze</code> button still exists in the UI, but it works as a manual rerun button.</p> <h3 id="heading-send-the-image-to-the-companion">Send the Image to the Companion</h3> <p>Here's the core request:</p> <pre><code class="language-ts">const analysisRequestId = useRef(0); async function analyzeImage(image = selectedImage) { if (!image) { setError("Choose an image first."); return; } const requestId = analysisRequestId.current + 1; analysisRequestId.current = requestId; setRequestState("loading"); setError(null); setCopied(false); try { const response = await fetch(`${API_BASE_URL}/v1/analyze-image`, { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ filename: image.file.name, mimeType: image.file.type || "application/octet-stream", base64: image.base64, }), }); const payload = await response.json(); if (requestId !== analysisRequestId.current) { return; } if (!response.ok) { throw new Error(payload.error?.message ?? `Analysis failed with ${response.status}`); } setAnalysis(payload); setRequestState("success"); } catch (error) { if (requestId !== analysisRequestId.current) { return; } setRequestState("error"); setError(error instanceof Error ? error.message : "Could not analyze image"); } } </code></pre> <p>This function is the entire client side of the bridge. It flips <code>requestState</code> to <code>loading</code> (which drives the spinner and disables the button), then sends a <code>POST</code> to <code>/v1/analyze-image</code> with a JSON body containing three fields: the filename, the MIME type, and the base64 image data. That body maps one-to-one onto the <code>AnalyzeImageRequest</code> struct the Swift companion decodes later.</p> <p>Notice that the response is parsed as JSON <em>before</em> checking <code>response.ok</code>. That's deliberate: when the companion rejects a request (bad base64, oversized image), it still returns a JSON body with an <code>error.message</code> field, so the UI can show the companion's own explanation instead of a generic status code. On success, the payload goes straight into state, and the JSON viewer re-renders with the result.</p> <p>The <code>requestId</code> bookkeeping guards against stale responses. If a user uploads a second image while the first is still analyzing, whichever request finishes <em>last</em> would win, and OCR plus model generation takes long enough that responses can genuinely arrive out of order. So every call increments a counter stored in a ref and remembers its own ID.</p> <p>After the <code>await</code>, it checks whether it's still the newest request; if a newer upload started in the meantime, the older response is silently discarded instead of overwriting the latest image's result. The same check runs in the <code>catch</code> block, so an old failure can't clobber a newer success either. If you also want to cancel the in-flight HTTP request rather than just ignore its result, an <code>AbortController</code> is the natural next step.</p> <h3 id="heading-render-the-json-output">Render the JSON Output</h3> <p>The output pane uses <code>react-json-view-lite</code>:</p> <pre><code class="language-tsx">&lt;JsonView data={jsonData} shouldExpandNode={allExpanded} style={jsonViewTheme} /&gt; </code></pre> <h2 id="heading-build-the-macos-companion-app">Build the macOS Companion App</h2> <p>The companion is a Swift command-line app. It exposes a small local HTTP API.</p> <p>If you come from the web side, the mapping is simple: Swift Package Manager is Swift's npm, <code>Package.swift</code> is its <code>package.json</code>, and <code>swift run</code> is its <code>npm start</code>. It ships with Xcode, so there's nothing extra to install.</p> <p>The <code>Package.swift</code> file looks like this:</p> <pre><code class="language-swift">// swift-tools-version: 6.0 import PackageDescription let package = Package( name: "VisionBridgeCompanion", platforms: [ .macOS("26.0") ], products: [ .executable( name: "VisionBridgeCompanion", targets: ["VisionBridgeCompanion"] ) ], targets: [ .executableTarget( name: "VisionBridgeCompanion" ) ] ) </code></pre> <p>The companion imports the Apple frameworks it needs:</p> <pre><code class="language-swift">import Foundation import FoundationModels import ImageIO import Network import Vision </code></pre> <p>It listens on <code>127.0.0.1:43119</code>:</p> <pre><code class="language-swift">private let defaultPort: UInt16 = 43119 </code></pre> <p>The app exposes two routes:</p> <pre><code class="language-swift">switch (request.method, request.path) { case ("GET", "/v1/health"): let health = HealthResponse(support: ModelSupport.current) return try json(health) case ("POST", "/v1/analyze-image"): let payload = try JSONDecoder().decode(AnalyzeImageRequest.self, from: request.body) let response = try await service.analyze(payload) return try json(response) default: return try json( ErrorResponse(error: APIErrorPayload(message: "Route not found")), status: .notFound ) } </code></pre> <p>This <code>switch</code> is the companion's entire routing layer — no web framework, just pattern matching on the method and path.</p> <p>The two routes split the work cleanly:</p> <ul> <li><p><code>GET /v1/health</code> is the cheap, read-only route. It runs no analysis, it just reports whether Vision and Foundation Models are usable on this Mac via <code>ModelSupport.current</code> (covered in the next section). The React app calls it on load to render the online/offline status pill, so the user knows the bridge is up before they upload anything.</p> </li> <li><p><code>POST /v1/analyze-image</code> is where the real work happens. It decodes the request body into an <code>AnalyzeImageRequest</code> (with the same <code>filename</code>, <code>mimeType</code>, and <code>base64</code> fields the browser sent) and hands it to the analysis service. This validates the image, runs Vision OCR, prompts Foundation Models, and returns the combined result. The <code>try await</code> matters here: analysis is asynchronous, and the route simply waits for it before serializing the response.</p> </li> </ul> <p>Anything else falls through to a JSON 404, so even unknown routes respond in the same format the browser already knows how to parse.</p> <p>Errors work the same way: thrown errors are caught in one place and converted into JSON error responses with an appropriate status code, which is exactly what the web app's <code>payload.error?.message</code> check reads.</p> <p>One practical detail: because the browser calls the companion from a different origin (the Vite dev server), every response also carries CORS headers, and the router answers preflight <code>OPTIONS</code> requests with an empty <code>204</code>. Without that, the browser would block the <code>fetch</code> before it ever reached these routes.</p> <h2 id="heading-check-foundation-models-availability">Check Foundation Models Availability</h2> <p>The companion shouldn't assume that the model is available. Check it first:</p> <pre><code class="language-swift">private struct ModelSupport: Encodable { let visionAvailable: Bool let foundationModelAvailable: Bool let foundationModelStatus: String static var current: ModelSupport { let model = SystemLanguageModel.default switch model.availability { case .available: return ModelSupport( visionAvailable: true, foundationModelAvailable: true, foundationModelStatus: "available" ) case .unavailable(let reason): return ModelSupport( visionAvailable: true, foundationModelAvailable: false, foundationModelStatus: "unavailable.\(reason.description)" ) @unknown default: return ModelSupport( visionAvailable: true, foundationModelAvailable: false, foundationModelStatus: "unavailable.unknown" ) } } } </code></pre> <p>A user might have an unsupported Mac, Apple Intelligence might be disabled, or the model might not be ready yet. The response tells the browser which case it's dealing with.</p> <h2 id="heading-extract-text-with-apple-vision">Extract Text with Apple Vision</h2> <p>The companion decodes the base64 image, checks its metadata, then runs Vision OCR.</p> <p>Here's the text recognition flow:</p> <pre><code class="language-swift">private func recognizeText(in imageData: Data) async throws -&gt; [DetectedText] { var request = RecognizeTextRequest() request.recognitionLevel = .accurate request.automaticallyDetectsLanguage = true request.usesLanguageCorrection = true let observations = try await request.perform(on: imageData) var detectedText: [DetectedText] = [] for observation in observations { guard let candidate = observation.topCandidates(1).first else { continue } let bounds = NormalizedBox.from(points: [ observation.topLeft, observation.topRight, observation.bottomRight, observation.bottomLeft ]) detectedText.append(DetectedText( text: candidate.string, confidence: Double(candidate.confidence), boundingBox: bounds )) } return detectedText } </code></pre> <p>Vision gives us structured observations:</p> <ul> <li><p>recognized text</p> </li> <li><p>confidence scores</p> </li> <li><p>normalized bounding boxes</p> </li> </ul> <p>Those observations become the model’s context.</p> <h2 id="heading-ask-foundation-models-to-explain-the-vision-output">Ask Foundation Models to Explain the Vision Output</h2> <p>Now the companion creates a prompt from the image metadata and OCR results.</p> <p>Notice the instruction:</p> <pre><code class="language-text">You cannot see the original image. Use only the metadata and OCR observations below. </code></pre> <p>That keeps the model honest. It shouldn't pretend to see pixels it never received.</p> <p>Here's the prompt shape:</p> <pre><code class="language-swift">let textPreview = detectedText .prefix(30) .map { "- \($0.text) (confidence: \(String(format: "%.2f", $0.confidence)))" } .joined(separator: "\n") let prompt = """ You are summarizing Apple Vision OCR output for a developer tool named Vision Bridge. You cannot see the original image. Use only the metadata and OCR observations below. Image: - filename: \(image.filename) - content type: \(image.contentType) - size: \(image.width ?? 0)x\(image.height ?? 0) OCR observations: \(textPreview.isEmpty ? "- No text detected." : textPreview) Return a compact JSON object with these exact keys: summary: one sentence description: one short paragraph suggestedTags: 3 to 6 short tags possibleUses: 3 to 5 practical use cases for this kind of image analysis """ </code></pre> <p>Then call the model:</p> <pre><code class="language-swift">let session = LanguageModelSession( model: .default, instructions: "Return valid JSON only. Do not include Markdown fences." ) let response = try await session.respond(to: prompt) let raw = response.content.trimmingCharacters(in: .whitespacesAndNewlines) </code></pre> <p>Even when you ask for JSON, always validate the output. Models can still return Markdown fences or malformed text. The sample app strips simple Markdown code fences and falls back to a raw response if parsing fails.</p> <h2 id="heading-return-json-to-the-browser">Return JSON to the Browser</h2> <p>The companion combines the support state, image metadata, Vision results, and model output:</p> <pre><code class="language-swift">return AnalyzeImageResponse( support: support, image: metadata, vision: VisionPayload(detectedText: detectedText), model: modelInsight ) </code></pre> <p>The browser doesn't need to know how Vision or Foundation Models work. It just receives JSON. The native app owns the native capabilities, while the web app owns the interface.</p> <p>It's worth pausing on what each of the four blocks actually gives you, because they're not all the same kind of data:</p> <ul> <li><p><code>support</code> tells you what was possible on this Mac. If <code>foundationModelAvailable</code> is <code>false</code>, the <code>model</code> block still exists but contains a fallback message rather than real analysis, and the <code>foundationModelStatus</code> string (for example, <code>unavailable.appleIntelligenceNotEnabled</code>) tells the UI <em>why</em>, so it can explain rather than silently degrade.</p> </li> <li><p><code>image</code> echoes back the file's metadata plus the measured pixel dimensions. It's useful as a sanity check, and you need the width and height to do anything spatial with the Vision results.</p> </li> <li><p><code>vision</code> is the ground truth. Each entry in <code>detectedText</code> is a string Vision actually found, with a confidence score between 0 and 1 and a normalized bounding box: coordinates expressed as fractions of the image size, so <code>x: 0.12, width: 0.45</code> means "starts 12% from the left and spans 45% of the width." Because the boxes are normalized, you can draw highlight overlays on the preview at any display size by multiplying by the rendered dimensions. Low-confidence entries are worth filtering or flagging before you trust them.</p> </li> <li><p><code>model</code> is interpretation, not observation. The <code>summary</code>, <code>description</code>, <code>suggestedTags</code>, and <code>possibleUses</code> fields are generated by the language model from the OCR text. This is useful as alt text, captions, or tag suggestions, but they inherit whatever the OCR missed and should be treated as a draft, not a fact. When the model's output can't be parsed as JSON, <code>rawResponse</code> carries the unparsed text so nothing is lost.</p> </li> </ul> <p>For a screenshot of a failed build, the model block might come back like this:</p> <pre><code class="language-json">{ "model": { "summary": "The image appears to show a software build failure.", "description": "A developer tool window is showing an error state with diagnostic text.", "suggestedTags": ["screenshot", "developer-tool", "error"], "possibleUses": [ "Generate alt text", "Summarize screenshots", "Extract document data" ] } } </code></pre> <p>That combination (exact text with positions from Vision, plus a human-readable interpretation from the model) is enough to build real features on top of a searchable screenshot library indexed by <code>detectedText</code> and <code>suggestedTags</code>, automatic alt text for uploaded images, or click-to-highlight overlays powered by the bounding boxes.</p> <p>And because the prompt lives in the companion, changing what comes back (say, extracting line items from receipts instead of tagging screenshots) is a prompt edit, not an architecture change.</p> <h2 id="heading-run-the-app">Run the App</h2> <p>Start the companion:</p> <pre><code class="language-sh">npm run companion </code></pre> <p>In another terminal, start the web app:</p> <pre><code class="language-sh">npm run dev </code></pre> <p>Open the Vite URL:</p> <pre><code class="language-text">http://127.0.0.1:5173 </code></pre> <p>If that port is busy, Vite will choose another one.</p> <p>The companion should be available at:</p> <pre><code class="language-text">http://127.0.0.1:43119 </code></pre> <p>You can test it directly:</p> <pre><code class="language-sh">curl http://127.0.0.1:43119/v1/health </code></pre> <p>Expected response:</p> <pre><code class="language-json">{ "app": "Vision Bridge Companion", "ok": true, "support": { "foundationModelAvailable": true, "foundationModelStatus": "available", "visionAvailable": true }, "version": "0.1.0" } </code></pre> <img src="https://cdn.hashnode.com/uploads/covers/5db93b3da2342e8354088115/76a47c9a-934c-4133-ba7c-e2a9c6b6dad4.png" alt="Screenshot of terminal running companion" style="display:block;margin:0 auto" width="1448" height="556" loading="lazy"> <h2 id="heading-conclusion">Conclusion</h2> <p>You now have a React interface that uploads an image, a Swift companion that analyzes it with Apple-native frameworks, and structured JSON flowing between them.</p> <p>Vision Bridge is intentionally small, but the bridge itself is reusable. Once you have a trusted native companion, a web app can do more than send prompts to a remote model: it can ask the Mac to work with local context, use any Apple framework, and return structured data the browser can render, store, or sync.</p> <h2 id="heading-resources">Resources</h2> <ul> <li><p><a href="https://developer.apple.com/documentation/foundationmodels">Apple Foundation Models documentation</a></p> </li> <li><p><a href="https://developer.apple.com/documentation/vision">Apple Vision documentation</a></p> </li> </ul>