Whisper

Whisper is completely free to download and run locally under the MIT open-source license. There are no usage fees, no rate limits, and no commercial restrictions when self-hosting. For users who want a managed service without infrastructure overhead, OpenAI offers Whisper as an API at $0.006 per minute of audio — approximately $0.36 per hour of audio — which is among the most affordable transcription API pricing available. The model weights, code, and documentation are all freely available on GitHub.

Running Whisper locally requires Python and pip. Install it with 'pip install openai-whisper', then run transcription from the command line with 'whisper audio.mp3 --model large-v3'. The first run will download the selected model weights automatically. For the large-v3 model, a GPU with at least 10GB of VRAM is recommended for fast inference, though smaller models like 'medium' and 'small' run acceptably on CPUs and less powerful GPUs. The Python API is also available for integration into custom applications.

Model selection depends on your accuracy requirements and hardware. The 'tiny' and 'base' models are fastest and suitable for English with clean audio on any hardware. The 'small' and 'medium' models offer a good balance of accuracy and speed, working well on modern CPUs. The 'large-v3' model delivers the highest accuracy across all languages and conditions, but requires a capable GPU for reasonable inference speed. For most production use cases requiring high accuracy, large-v3 is recommended, and this is what the OpenAI API uses.

Whisper large-v3 is competitive with or exceeds the accuracy of many commercial transcription services on diverse audio benchmarks, particularly for non-English languages, accented speech, and noisy audio. It achieves word error rates below 5% on many standard English benchmarks. For specialized domains with very specific vocabulary, fine-tuned models may outperform Whisper, and for certain languages, purpose-built models (such as Vito for Korean) may deliver better accuracy. However, for general-purpose multilingual transcription, Whisper is widely regarded as the best freely available option.

Yes, Whisper supports direct audio-to-English translation as a built-in task. You can pass audio in any of the 99 supported languages and receive an English text output without a separate translation step. This is accomplished by specifying '--task translate' in the CLI or setting the task parameter in the API. Note that Whisper's translation is designed for English as the target language only — for translation into other target languages, you would transcribe first and then use a separate translation model.