Speech Recognition

Speech Recognition

Transcribe live and pre-recorded audio to text using Apple's Speech framework. Covers SpeechAnalyzer / SpeechTranscriber (iOS 26+) and SFSpeechRecognizer (iOS 10+) fallback guidance.

Scope boundary: Use this skill for speech-to-text recognition, speech authorization, microphone capture plumbing, and result handling. Hand off text analysis, language identification after transcription, sentiment, embeddings, and translation to natural-language; hand off audio playback UI to avkit; hand off summarization or generation over transcripts to apple-on-device-ai.

Contents

• SpeechAnalyzer Strategy (iOS 26+)
• SFSpeechRecognizer Setup
• Authorization
• Live Microphone Transcription
• Pre-Recorded Audio File Recognition
• On-Device vs Server Recognition
• Handling Results
• Common Mistakes
• Review Checklist
• References

SpeechAnalyzer Strategy (iOS 26+)

Use SpeechAnalyzer for modern iOS 26+ speech analysis, especially long-form recordings, live transcription, time-indexed transcripts, and fully on-device flows. Keep SFSpeechRecognizer for iOS 10+ deployment targets, server-backed locale coverage, or existing callback/delegate implementations.

Read SpeechAnalyzer patterns when implementing an iOS 26+ transcription pipeline, model asset handling, volatile results, or file/buffer examples.

SpeechAnalyzer setup checklist

1. Choose the module:

• SpeechTranscriber for the newer general-purpose on-device model.
• DictationTranscriber when SpeechTranscriber is unavailable for the current device or locale and dictation-compatible support is acceptable.
• SpeechDetector only in conjunction with a transcriber when voice activity detection is worth the accuracy/power tradeoff.

1. Check support before creating the session:

• SpeechTranscriber.isAvailable
• SpeechTranscriber.supportedLocale(equivalentTo:)
• SpeechTranscriber.installedLocales / supportedLocales when showing language choices.

1. Pick a documented preset:

• .transcription for basic accurate transcription.
• .progressiveTranscription for live UI updates.
• .timeIndexedProgressiveTranscription when playback highlighting needs audioTimeRange.

1. Install required assets with AssetInventory.assetInstallationRequest.
2. Convert live audio buffers to SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith:) before yielding AnalyzerInput.
3. Consume module results from their AsyncSequence in a separate task.
4. Finish explicitly with finalizeAndFinish(through:), finalizeAndFinishThroughEndOfInput(), or cancelAndFinishNow().

Do not use an offlineTranscription preset; Apple does not document one. Finishing an AsyncStream input sequence does not finish the analyzer session.

SFSpeechRecognizer Setup

Creating a recognizer with locale

import Speech

// Default locale (user's current language)
let recognizer = SFSpeechRecognizer()

// Specific locale
let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))

// Check if recognition is available for this locale
guard let recognizer, recognizer.isAvailable else {
    print("Speech recognition not available")
    return
}

Monitoring availability changes

final class SpeechManager: NSObject, SFSpeechRecognizerDelegate {
    private let recognizer = SFSpeechRecognizer()!

    override init() {
        super.init()
        recognizer.delegate = self
    }

    func speechRecognizer(
        _ speechRecognizer: SFSpeechRecognizer,
        availabilityDidChange available: Bool
    ) {
        // Update UI — disable record button when unavailable
    }
}

Authorization

Request both speech recognition and microphone permissions before starting live transcription. Add these keys to Info.plist:

• NSSpeechRecognitionUsageDescription
• NSMicrophoneUsageDescription

import Speech
import AVFoundation

func requestPermissions() async -> Bool {
    let speechStatus = await withCheckedContinuation { continuation in
        SFSpeechRecognizer.requestAuthorization { status in
            continuation.resume(returning: status)
        }
    }
    guard speechStatus == .authorized else { return false }

    let micStatus: Bool
    if #available(iOS 17, *) {
        micStatus = await AVAudioApplication.requestRecordPermission()
    } else {
        micStatus = await withCheckedContinuation { continuation in
            AVAudioSession.sharedInstance().requestRecordPermission { granted in
                continuation.resume(returning: granted)
            }
        }
    }
    return micStatus
}

Live Microphone Transcription

The standard pattern: AVAudioEngine captures microphone audio → buffers are appended to SFSpeechAudioBufferRecognitionRequest → results stream in.

import Speech
import AVFoundation

final class LiveTranscriber {
    private let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))!
    private let audioEngine = AVAudioEngine()
    private var recognitionRequest: SFSpeechAudioBufferRecognitionRequest?
    private var recognitionTask: SFSpeechRecognitionTask?

    func startTranscribing() throws {
        // Cancel any in-progress task
        recognitionTask?.cancel()
        recognitionTask = nil

        // Configure audio session
        let audioSession = AVAudioSession.sharedInstance()
        try audioSession.setCategory(.record, mode: .measurement, options: .duckOthers)
        try audioSession.setActive(true, options: .notifyOthersOnDeactivation)

        // Create request
        let request = SFSpeechAudioBufferRecognitionRequest()
        request.shouldReportPartialResults = true
        self.recognitionRequest = request

        // Start recognition task
        recognitionTask = recognizer.recognitionTask(with: request) { result, error in
            if let result {
                let text = result.bestTranscription.formattedString
                print("Transcription: \(text)")

                if result.isFinal {
                    self.stopTranscribing()
                }
            }
            if let error {
                print("Recognition error: \(error)")
                self.stopTranscribing()
            }
        }

        // Install audio tap
        let inputNode = audioEngine.inputNode
        let recordingFormat = inputNode.outputFormat(forBus: 0)
        inputNode.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) {
            buffer, _ in
            request.append(buffer)
        }

        audioEngine.prepare()
        try audioEngine.start()
    }

    func stopTranscribing() {
        audioEngine.stop()
        audioEngine.inputNode.removeTap(onBus: 0)
        recognitionRequest?.endAudio()
        recognitionRequest = nil
        recognitionTask?.cancel()
        recognitionTask = nil
    }
}

Pre-Recorded Audio File Recognition

Use SFSpeechURLRecognitionRequest for audio files on disk:

func transcribeFile(at url: URL) async throws -> String {
    guard let recognizer = SFSpeechRecognizer(), recognizer.isAvailable else {
        throw SpeechError.unavailable
    }
    let request = SFSpeechURLRecognitionRequest(url: url)
    request.shouldReportPartialResults = false

    return try await withCheckedThrowingContinuation { continuation in
        var didResume = false
        recognizer.recognitionTask(with: request) { result, error in
            guard !didResume else { return }
            if let error {
                didResume = true
                continuation.resume(throwing: error)