Coreml

Core ML Swift Integration

Load, configure, and run Core ML models in iOS apps. This skill covers the Swift side: model loading, prediction, MLTensor, profiling, and deployment.

Scope boundary: Python-side model conversion, optimization (quantization, palettization, pruning), and framework selection live in the apple-on-device-ai skill. This skill owns Swift integration only.

See references/coreml-swift-integration.md for complete code patterns including actor-based caching, batch inference, image preprocessing, and testing.

Contents

• Loading Models
• Model Configuration
• Making Predictions
• MLTensor (iOS 18+)
• Working with MLMultiArray
• Image Preprocessing
• Multi-Model Pipelines
• Vision Integration
• Performance Profiling
• Model Deployment
• Memory Management
• Common Mistakes
• Review Checklist
• References

Loading Models

Auto-Generated Classes

When you add a .mlmodel or .mlpackage to an app target, Xcode generates a Swift class with typed input/output. Use this whenever possible.

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .all

let model = try MyImageClassifier(configuration: config)

Manual Loading

Load from a URL when the model is downloaded at runtime or stored outside the bundle.

let modelURL = Bundle.main.url(
    forResource: "MyModel", withExtension: "mlmodelc"
)!
let model = try MLModel(contentsOf: modelURL, configuration: config)

Async Loading (iOS 15+)

Load models without blocking the main thread. Prefer this for large models.

let model = try await MLModel.load(
    contentsOf: modelURL,
    configuration: config
)

Compile at Runtime (iOS 16+)

Compile a .mlpackage or .mlmodel to .mlmodelc on device. Useful for models downloaded from a server. Do this once per model version, not on every launch.

let compiledURL = try await MLModel.compileModel(at: packageURL)
let model = try await MLModel.load(contentsOf: compiledURL, configuration: config)

Cache the compiled URL -- recompiling on every launch is a bug. Copy compiledURL to a persistent location (e.g., Application Support). When reviewing runtime-loaded models, call out both facts together: async MLModel.compileModel(at:) is iOS 16+, and compiled models must be cached so the app does not recompile on every launch.

Model Configuration

MLModelConfiguration controls compute units, GPU access, and model parameters.

Compute Units Decision Table

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine

// Optional fallback for constrained work after profiling and policy review
config.computeUnits = .cpuOnly

Configuration Properties

let config = MLModelConfiguration()
config.computeUnits = .all
config.allowLowPrecisionAccumulationOnGPU = true // faster, slight precision loss

Making Predictions

With Auto-Generated Classes

The generated class provides typed input/output structs.

let model = try MyImageClassifier(configuration: config)
let input = MyImageClassifierInput(image: pixelBuffer)
let output = try model.prediction(input: input)
print(output.classLabel)        // "golden_retriever"
print(output.classLabelProbs)   // ["golden_retriever": 0.95, ...]

With MLDictionaryFeatureProvider

Use when inputs are dynamic or not known at compile time.

let inputFeatures = try MLDictionaryFeatureProvider(dictionary: [
    "image": MLFeatureValue(pixelBuffer: pixelBuffer),
    "confidence_threshold": MLFeatureValue(double: 0.5),
])
let output = try model.prediction(from: inputFeatures)
let label = output.featureValue(for: "classLabel")?.stringValue

Prediction Inside Async Workflows

MLModel.prediction(...) is synchronous. In async pipelines, keep model loading async, then run prediction from an actor or non-main task without adding await to the prediction call.

let output = try model.prediction(from: inputFeatures)

Batch Prediction

Process multiple inputs in one call for better throughput.

let batchInputs = try MLArrayBatchProvider(array: inputs.map { input in
    try MLDictionaryFeatureProvider(dictionary: ["image": MLFeatureValue(pixelBuffer: input)])
})
let batchOutput = try model.predictions(fromBatch: batchInputs)
for i in 0..<batchOutput.count {
    let result = batchOutput.features(at: i)
    print(result.featureValue(for: "classLabel")?.stringValue ?? "unknown")
}

Use predictions(fromBatch:) when batching without explicit MLPredictionOptions. Use predictions(from:options:) only when passing both an MLBatchProvider and MLPredictionOptions; predictions(from:) by itself is not the no-options batch API.

Validate a representative single input before batching. Then verify batch output count/order, feature types, domain invariants, and agreement with the single-input result. On failure, fix the deterministic input, shape, model, or configuration issue before rerunning fixtures and physical-device profiling.

Stateful Prediction (iOS 18+)

Use MLState for models that maintain state across predictions (sequence models, LLMs, audio accumulators). Create state once and pass it to each prediction call.

let state = model.makeState()

// Each synchronous prediction carries forward the internal model state
for frame in audioFrames {
    let input = try MLDictionaryFeatureProvider(dictionary: [
        "audio_features": MLFeatureValue(multiArray: frame)
    ])
    let output = try model.prediction(from: input, using: state)
    let classification = output.featureValue(for: "label")?.stringValue
}

MLState is Sendable, but Sendable does not make one state safe for concurrent inference. Predictions using the same state must be serialized; do not read or write state buffers while a prediction is in flight. Call model.makeState() for each independent concurrent stream. If you need MLPredictionOptions, iOS 18+ also provides the async prediction(from:using:options:) overload; the same one-in-flight-per-state rule still applies.

MLTensor (iOS 18+)

MLTensor is a Swift-native multidimensional array for pre/post-processing. Operations run lazily -- call await tensor.shapedArray(of:) to materialize results.

import CoreML

// Creation
let tensor = MLTensor([1.0, 2.0, 3.0, 4.0])
let zeros = MLTensor(zeros: [3, 224, 224], scalarType: Float.self)

// Reshaping
let reshaped = tensor.reshaped(to: [2, 2])

// Math operations
let softmaxed = tensor.softmax(alongAxis: -1)
let centered = tensor - tensor.mean()

// Interop with MLShapedArray / MLMultiArray
let shaped = await tensor.shapedArray(of: Float.self)
let multiArray = try MLMultiArray(shaped)
let shapedAgain = MLShapedArray<Float>(multiArray)

Do not invent MLTensor APIs for statistics or bridging. Avoid examples such as MLTensor(multiArray), tensor.std(), tensor.standardDeviation(), direct lazy-buffer access, or synchronous extraction; perform unsupported DSP/statistics outside the tensor pipeline or with source-confirmed tensor operations.