AnyDecisionModel and AnySpeechRecognizer split a Swift app's listening from its choices

A support recording contains two different problems. First, an application must recover what the caller said. Then it must decide whether the request belongs with billing, technical support or another team. Treating both as one open-ended conversation makes the app's responsibilities harder to see.

Developer Mattt introduced AnySpeechRecognizer and AnyDecisionModel on October 29, 2026. These are third-party Swift packages for Apple-platform developers: one offers a choice of speech recognition engines, while the other returns typed choices and probability distributions. They build on familiar Swift patterns without being Apple product announcements. Mattt's introduction explains the two jobs through the same support-call example.

The interesting development is the ability to choose and evaluate those jobs separately. This guide explains what the packages contribute, how local and hosted backends change an app's design, and why a numerical confidence value still needs a product policy around it.

Mattt’s Emoji Magnet example app uses AnyDecisionModel to sort emoji. Author-supplied documentation screenshot, Apache 2.0; resized with margins. Image source · License and image notes.

A transcript and a routing decision need separate acceptance tests

Imagine a caller saying, “The invoice is wrong, and I cannot sign in to download it.” A transcript can reproduce that sentence correctly while a routing policy still fails. Does the company send it to billing because of the invoice, to technical support because of the login failure, or to a general queue because both are present?

The answer depends on the service's policy. A model cannot supply a missing ownership rule simply by returning a strongly preferred label. Before evaluating any backend, write down what each allowed option means and what should happen when several options fit. That gives the app a target more useful than “the output looked sensible.”

Separating stages also makes errors easier to diagnose. If the recognizer changes a product name, preserve that transcript and check whether the decision would have changed with the correct words. If the transcript is accurate but the selected team is wrong, inspect the choice definitions and decision model. These are different repair tasks, and combining them hides the source of failure.

Our take is that the most valuable abstraction here is a boundary between hearing and choosing. Each stage should produce an artifact the next stage can inspect. A product team can then decide which stages deserve automatic execution and which should remain suggestions for a person.

Original workflow diagram: recognition supplies evidence; a bounded decision supplies a distribution; the application decides what action is permitted.

Engine choice matters when the app's audio changes

AnySpeechRecognizer's README, checked November 2, describes an API modeled on Apple's modern Speech framework, with Apple, Parakeet and Qwen3-ASR options. Engine capabilities differ: the published comparison lists live provisional results and word timing for some paths, while Qwen3-ASR supplies segment results without word timing. The package expects Swift 6.2 or later and macOS or iOS 26 or later, with additional hardware requirements for particular models.

That difference is visible in product design. A live caption display benefits from text arriving before the speaker finishes, but it must show that provisional words can change. A file-import workflow can prioritize final results and let the user wait for the recording to finish processing. The same transcript screen should not pretend both modes have identical timing behavior.

For an illustrative interview app, word timing may be essential because tapping a sentence should seek to the relevant part of the recording. For an illustrative voice-inbox app, a segment-level transcript may be sufficient because the main task is reviewing a short message. Choose the engine after defining that interaction, then test with the language, accents and recording conditions the audience actually uses.

Language support deserves the same treatment. A list of supported languages is an eligibility check, while a noisy customer recording is an acceptance test. Keep original audio available to authorized reviewers and provide a way to correct important names or numbers. Those corrections are useful evidence when deciding whether the recognizer needs a different configuration or whether the product needs a better recovery interaction.

The package also leaves audio capture with the application. That is a useful ownership line: the app remains responsible for recording permission, source selection and the lifecycle of capture. A recognition library should not be mistaken for an entire recording feature, especially when the UI must handle interruptions, cancellation and a user who stops early.

Typed decisions reduce parsing work, not responsibility

AnyDecisionModel's README, checked November 2, documents binary probabilities, choices from allowed options and scores on ordered scales. It offers local MLX, Core AI and llama.cpp paths, plus Jev and Ollama backends. The API returns structured results instead of requiring the application to recover a label from an explanatory paragraph.

That can simplify an app considerably. When the allowed result is a department, the application's code can handle each department explicitly. When the question is whether a message needs review, the result can feed a policy with a review state. The app still needs to define what “review” means and who owns the queue.

An illustrative maintenance-report app makes the distinction clear. A worker records a description, the recognizer produces text, and the decision layer suggests one of several equipment categories. The application can show the suggestion alongside the transcript before saving. It should not silently transform a category suggestion into a safety determination, a repair order or permission to operate equipment.

Ordered scores need similarly concrete definitions. If a team asks for urgency on a three-level scale, each level should describe an action or service expectation. “Low,” “medium” and “high” are easy labels to display but difficult labels to evaluate when reviewers interpret them differently. A short written rubric makes disagreements visible before they become model-training or threshold problems.

The comparison is about interface design, not a claim that one technique always wins. Some screens need both a bounded choice and a concise explanation. Keep the explanation distinguishable from evidence and keep the decision editable when the workflow permits it.

Confidence describes a distribution, not an accuracy guarantee

The AnyDecisionModel README makes an unusually useful distinction: local confidence measures how concentrated the answer distribution is, not the probability that the selected answer is correct. It also warns against comparing confidence values from different backends as though they shared one formula. Calibration should be fitted separately from evaluation data. Project documentation, checked November 2, 2026.

Suppose an illustrative test returns one department with 0.90 probability and divides the rest between other departments. That says the model favored that department under the question and options it received. It does not establish a 90% historical success rate for that sort of customer message. The category definitions may be poor, the input may omit crucial context, or the model may systematically prefer one option.

A useful product policy therefore needs an explicit “hold for review” path. Start with all decisions visible to reviewers and collect corrections. Then evaluate whether any subset is reliable enough for a narrowly defined automatic action. The threshold is a consequence of observed errors and the cost of those errors, rather than a number chosen because it looks confident.

The evaluation set should contain ambiguous messages, unfamiliar names and cases where none of the available departments is a good fit. A system can appear strong when every example fits one clean category, then fail on the mixed requests that dominate real support work. Have reviewers record why they changed a choice so the team can distinguish a bad model answer from a missing policy.

Also examine the cost of each error direction. Sending a routine request for review consumes time; silently routing a sensitive request to the wrong audience can have a different consequence. A single average score hides that difference. Report the cases that matter to the product's owners, then choose an action policy they can explain to users and maintainers.

This is also why changing backends is a product change. The same Swift call can hide different behavior, numerical semantics and operational dependencies. Re-run the evaluation before carrying an old threshold into a new configuration. Keep the model identifier, configuration and option definitions with the result so a later review can explain how the decision was produced.

Original diagram: a strong preference does not remove the need to evaluate correctness or define an action policy. Confidence and accuracy answer different questions.

Local operation changes the work the team owns

Running a model on the device can be appealing for a voice inbox or offline reference tool. The team can design around locally retained inputs and a predictable network boundary. But model downloads, supported hardware, memory use and first-run preparation become visible parts of the user experience. A privacy promise should describe the whole path, including any hosted stage later added to the application.

A hosted decision backend shifts some operational work to a service, while introducing network behavior, credentials and service availability into the flow. The user needs a comprehensible state when a request fails or remains pending. Retrying should preserve the original transcript and should not accidentally repeat a downstream action that has already completed.

The current project documentation also distinguishes tested compatibility from live service verification. AnyDecisionModel says Jev and Ollama wire compatibility tests use fixtures. That is valuable test coverage, but a shipping application should still verify its selected service configuration end to end. AnyDecisionModel README, November 2 check.

Avoid promising identical performance across a developer Mac and the smallest supported phone. A deployment target states where code is intended to run; it does not supply latency, battery or memory evidence for every device. An app can begin with file processing on one qualified machine, then expand its supported surface after the intended interaction passes on additional hardware.

Begin with a reviewable file workflow

A first pilot should make errors easy to inspect. A small file-based application is often a clearer starting point than a live assistant because the same authorized audio can be replayed and compared. The goal is to prove the stages and their handoff before adding real-time interaction.

1. Define one task with a short list of allowed decisions and a written rule for ambiguous cases. Include a review option in the application's policy.
2. Check the current package requirements and add the relevant Swift package dependency. Select one recognition engine and one decision backend rather than qualifying every combination at once.
3. Import a small set of authorized recordings. Keep the original files, expected transcripts and reviewer decisions in separate records.
4. Install or stage the required model assets explicitly, following the package documentation. Show preparation and processing as distinct UI states.
5. Run recognition, retain final text, and ask the bounded questions about that text. Display the result alongside the source rather than immediately performing an external action.
6. Review errors by stage, then evaluate a separate set of recordings after making changes. Record the configuration with every result.
7. Add cancellation, unavailable-model and network-failure behavior before expanding to live capture or automatic routing.

The success criterion should be understandable to a product owner. For example, a voice-inbox pilot might require that a reviewer can find the source recording, correct the transcript and change the suggested queue without losing any original material. That is a useful workflow result even before the team decides to automate anything.

Small decisions still deserve a complete product design

These packages make an attractive architectural proposition: use one component to hear, another to choose, and application code to enforce the allowed action. That creates places to measure quality and places to recover from failure. It also makes the limits of a numerical answer easier to discuss with the people who must trust it.

The expectation worth testing is maintainability. Can a team replace a recognizer without rewriting the whole app? Can it evaluate a new decision backend without pretending old thresholds still apply? Can users correct the result at the point where the error actually occurred?

A small model becomes useful when its job is clear and its output has a responsible owner. The Swift abstraction helps connect the parts; the product's decision policy gives those parts a purpose.

Sources

• Mattt: Introducing AnySpeechRecognizer and AnyDecisionModel, October 29, 2026.
• AnySpeechRecognizer: requirements, engine comparison and examples, checked November 2, 2026.
• AnyDecisionModel: backends, calibration and limitations, checked November 2, 2026. Package code licenses and individual model licenses require separate consideration.
• Mattt: Emoji Magnet documentation and screenshot, Apache-2.0 documentation; checked November 2, 2026.

Keep an implementation decision connected to its source

Telli.sh is one option for retaining authorized source material and the notes behind an implementation decision. Create a note workspace.