v-modal/vmodal_sdk_swift_iphoneduo

VModal for Apple platforms

Give your iOS and macOS apps a multimodal memory.

Upload video. Find moments by meaning, speech, text, or imagery.
Keep the experience native, typed, and 100% Swift.

Turn every video library into an experience your users can explore.

VModal brings multimodal video search and mobile-friendly uploads to Swift with a small, strongly typed API. Your app owns the interface; the SDK handles the VModal gateway, request models, responses, streaming uploads, progress, and cancellation.

Copy this prompt into your coding agent:

1. Clone https://github.com/v-modal/vmodal_sdk_swift_iphoneduo.git and enter
   the vmodal_sdk_swift_iphoneduo directory.
2. Inspect the repository instructions and example/StarterIOS/README.md before
   making changes.
3. Use the repository's selected Xcode and Swift toolchain; do not install or
   switch Xcode. Run:
     bash install.sh check
     bash build.sh analyze
     bash test.sh test
4. Start or select an available iPhone simulator. List devices with:
     bash install.sh device_list
   Get the first available iPhone simulator identifier with:
     bash install.sh device_id
5. Run example/StarterIOS on the selected simulator with:
     bash run.sh example --device DEVICE_ID

Keep working until the app builds, installs, and opens. Fix any repository
setup issue you can safely resolve. Do not hard-code credentials or persist an
API key. When the app opens, explain how to enter a runtime VModal API key and
complete the authentication, collection, upload, index, and search flow. If a
required Xcode installation, simulator, or API key is unavailable, stop at that
boundary and report the exact blocker and the next command I should run.

Request an API key from VModal.

SDK reference: Sources/VModalSDK/VModalSDK.docc

Add the package in Xcode with File → Add Package Dependencies…, using:

https://github.com/v-modal/vmodal_sdk_swift_iphoneduo

Attach the VModalSDK product to your application target. Then create one project from the API key already loaded by your authenticated app and retain immutable scopes wherever your app performs content operations:

import VModalSDK

let keys = try MutableAPIKeyProvider("runtime-api-key")
let project = try VModal.configure(
    projectID: "food_app",
    apiKeyProvider: keys
)
let favorites = try project.scope(
    collectionName: "user_123",
    streamName: "favorites"
)

projectID, collectionName, and streamName accept only letters, digits, and underscore. Each is trimmed and limited to 80 characters. Project and collection names cannot contain the reserved __ separator, and their encoded backend value is also limited to 80 characters. The SDK performs that encoding internally.

The SDK never owns your login screen or persists your API key. Authentication identity is separate from project, collection, and stream organization.

let collections = try await project.listCollections(mode: "vid_file")
guard collections.contains("user_123") else {
    throw ValidationError("No video collection exists for this API key")
}

let results = try await favorites.search(
    "the cyclist crossing the bridge at sunset",
    options: ScopedSearchOptions(
        searchSources: ["image"],
        limit: 20
    )
)

print("\(results.cntActual) moments found")
for moment in results.data {
    print(moment)
}

Collection access is key-scoped. A logical name copied from another account or environment can return HTTP 404 even when the search route is healthy. Use ScopedSearchOptions(versionLancedb: version) when your application tracks a specific index version.

The response stays typed where the contract is stable and preserves raw JSON so new server fields remain available immediately.

The SDK reads an app-accessible file URL as a stream. It does not load the entire video into memory.

let source = try UploadSource(fileURL: movieURL)
let upload = favorites.upload(source)

let progress = Task {
    for await value in upload.progress {
        print("Uploading \(value.percent)%")
    }
}

// Connect this to your SwiftUI cancel button when needed:
// upload.cancel()

let uploaded = try await upload.result
progress.cancel()
print("Ready: \(uploaded.filename)")

For CCTV footage, provide the public filename and offset-aware recording origin in VideoUploadOptions. The backend—not the SDK—normalizes the datetime and returns canonical UTC epoch milliseconds. Metadata tags are repeated independently on the wire.

let task = favorites.upload(
    try UploadSource(fileURL: cameraClipURL),
    options: ScopedUploadOptions(
        uploadOptions: VideoUploadOptions(
            videoFilename: "entrance-camera.mp4",
            metadataText: "north entrance delivery lane",
            metadataTags: ["entrance", "delivery", "camera-3"],
            startDatetimeUser: "2026-07-30T09:15:00+09:00"
        )
    )
)
let uploaded = try await task.result
print(uploaded.startTsUnixUserMs) // canonical backend value

Search the same footage using a metadata string and an absolute range. Start is inclusive and end is exclusive. Datetime values must include Z or an explicit UTC offset; the SDK preserves the caller text without timezone conversion.

let moments = try await favorites.search(
    "vehicle",
    options: ScopedSearchOptions(
        queryMetadataText: "delivery",
        startDate: "2026-07-30T09:15:00.000+09:00",
        endDate: "2026-07-30T09:16:00.000+09:00",
        searchSources: ["image"]
    )
)

Signed single upload is the production default for every supported file size. Multipart upload is experimental and must be enabled explicitly with VideoUploadOptions(multipart: true); it fails with FeatureDisabledError when the complete backend route family is unavailable.

Uploads use exact file ranges, awaited socket streaming, coalesced progress, and one network-concurrency budget per bulk task. See the API reference for upload limits, timeout behavior, and the performance benchmark command.

• Rotate credentials without rebuilding the client: try await keys.rotate(newAPIKey).
• Cancel search or upload work when a view, scene, or session closes.
• Observe upload progress with AsyncStream and update SwiftUI state on the main actor.
• Keep photo picking, Keychain storage, background scheduling, and lifecycle UI in the parent app.
• Close network resources deterministically with await project.close().

For logout or account switching, cancel active work, clear upload persistence, call await keys.clear(), close the project, and create a new project and scopes for the next identity. Key rotation alone is not an identity, project, collection, or stream switch.

global index       project=video_search  collection=global           stream=uploads
per-user index     project=food_app      collection=user_123         stream=personal_videos
multiple streams   project=food_app      collection=user_123         stream=camera/favorites
catalog            project=shopping_app  collection=product_catalog  stream=merchant_uploads

Create a separate VModalProject for each developer project. On account switch, create fresh project/client state; an already running task retains the immutable scope with which it started.

The included Apple examples are practical product and data-flow references. Choose the example that matches the feature you are building, then apply the same SDK contract in your SwiftUI state and lifecycle ownership.

When translating these flows:

• Keep one immutable state model for loading, progress, empty, success, error, and cleanup states.
• Convert a selected photo or movie into an app-readable local URL, then create UploadSource(fileURL:).
• Replace callback-style networking with UploadTask.progress, UploadTask.result, and upload.cancel().
• Preserve collection and stream coupling across upload, index creation, search, and bulk image lookup. Never display a job or result after the user has switched scope or identity.
• Load presigned result images without adding the VModal bearer credential. Refresh expired URLs by repeating the image lookup.
• Treat background transfer and processing as application-owned and platform-specific; the SDK does not schedule background work for your app.

VModalClient remains supported for auth, usage, image lookup, and advanced wire-level integration. To combine it with scopes, construct the client first and transfer lifecycle ownership to the project:

let client = try await VModalClient.fromEnvironment(
    ProcessInfo.processInfo.environment
)
let project = try VModal.fromClient(
    projectID: "food_app",
    client: client
)

let profile = try await client.auth.me()
let scope = try project.scope(
    collectionName: "user_123",
    streamName: "favorites"
)

await project.close() // closes the transferred client

Gateway mode is the default and sends caller identity only as a bearer credential. VModalClient.unsafeDirect is reserved for trusted private networks.

Minimum toolchain: Swift 6.0, Xcode 26.6, iOS 16.0, and macOS 13.0.

• VModal home
• VModal for developers
• VModal AI
• Read the Swift getting-started guide
• Review complete operation parity
• Run the StarterIOS app
• Review the iPhone Duo and adaptive-layout guide
• Review the API compatibility specification
• Open an issue

git clone https://github.com/v-modal/vmodal_sdk_swift_iphoneduo.git
cd vmodal_sdk_swift_iphoneduo
bash install.sh check
bash build.sh analyze
bash test.sh test

The offline gate resolves the package, analyzes Swift concurrency, runs SDK tests, checks route synchronization, and validates the StarterIOS simulator build. Live tests require the repository's existing test credentials and are intentionally separate:

bash test.sh live
bash test.sh cctv_live

VModalSDK is an open-source Swift package that abstracts multimodal video infrastructure into focused, developer-friendly operations. It lets Apple app teams build semantic media experiences without managing vector databases or ML pipelines locally. The SDK sends typed requests to VModal services and returns structured, raw-JSON-preserving responses suitable for production UI.

• Multimodal search: Query media with natural-language text, metadata, and image-aware sources.
• Video analytics: Index uploaded footage and return contextual timestamps for matching moments.
• Image recognition: Search visual objects, text, and scene relationships through the VModal backend.
• Scoped organization: Keep project, collection, and stream identity explicit across every operation.
• Runtime authorization: Supply and rotate bearer credentials in your app without SDK-owned login UI.
• Swift concurrency: Use async/await, AsyncStream, cancellation tokens, and typed failures.

• Swift native: Built with Foundation and Swift concurrency for iOS and macOS applications.
• Small footprint: Keeps ML and vector-index work in VModal services rather than bundling local models.
• Streaming transport: Streams file bytes and progress instead of loading complete video files in memory.
• Reliable operations: Provides typed failures, bounded retries, cancellation, and idempotent resource closing.

• Camera and field apps: Upload and find specific moments from captured footage.
• Security surveillance: Search recorded video for events using natural-language descriptions.
• Digital asset management: Organize and retrieve large collections of business media.
• Editorial tools: Find scenes or actions inside long-form footage and B-roll.

Explore the full platform and developer resources:

• VModal — the official home of VModal multimodal video and image search.
• VModal for Developers — API docs, SDKs, and integration guides for building on VModal.
• VModal AI — learn how VModal AI powers semantic search across video, speech, text, and imagery.

VModalSDK is available under the MIT License.