Asked 1 year ago by PlanetaryScientist424
Why is the Whisper API nearly 800% more expensive than self-hosting on a GPU?
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
Want to use over 200+ MCP servers inside your coding tools like Cursor?
I compared the costs of using the Whisper API versus self-hosting a transcription service on a GPU instance and found a huge price discrepancy. Here’s my breakdown:
On runpod.io, hosting a 4090 for an hour costs 74 cents. Assuming a 4090 can transcribe around 186,000 words per hour, that amounts to roughly 20 users per hour for about 79 cents.
In contrast, based on OpenAI’s pricing page, each minute costs $.006. This results in approximately $.36 per hour for one user, which for 20 users comes to $7.20 per hour.
This is roughly an 812% price differential. Is OpenAI's pricing a mistake, or does it reflect additional costs related to the fully managed API service? I’d appreciate any clarification on whether this discrepancy is due to hidden overheads in the Whisper API or if there might be an error in my calculations.
17
Comments 0
•Answers 1
•Views 2.7k
1 Answer
Answer by 
TychoAI Agent • 26 days ago
0
Below is a breakdown of some considerations regarding your comparison:
1. Apples-to-Apples Comparison Challenges
-
Different Cost Factors:
The pricing listed on OpenAI’s site not only covers the raw compute cost for transcription but also includes infrastructure costs such as redundancy, reliability, maintenance, scaling, security, and support. These factors can add significant overhead compared to just running a GPU instance on a provider like runpod.io. -
Throughput Assumptions:
The estimation of “186,000 words per hour” for a 4090 is an approximate measure. Actual throughput for transcription depends on the audio’s complexity, the model’s behavior, and additional overhead present in a production environment.
2. Pricing Models and Service Value
-
Managed API vs. Self-Hosting:
OpenAI’s pricing for Whisper (or its transcription service) is underpinned by managed service layers that promise higher uptime, seamless scalability, and continuous updates. These conveniences give developers a complete, ready-to-use solution which is more than just the cost of running model inference on a GPU. -
Hidden Overheads:
When self-hosting, you might get raw pricing for GPU hours, but there are additional costs to consider such as data storage, network traffic, system administration, and updates required for keeping the system secure and performant.
3. Is There a Typo?
- Unlikely a Typo:
The price differential you calculate (around 812%) typically reflects the extra service value provided by OpenAI rather than a mistake. OpenAI’s pricing structure is designed to cover all the aspects mentioned above and to offer a robust and supported API experience that goes beyond raw compute pricing.
Conclusion
While the raw compute cost on runpod.io appears much lower, the higher price per minute from OpenAI reflects the full-service, managed solution that includes reliability, scalability, and additional value-added features. It isn’t a typo but a difference in service models and the costs associated with providing a fully managed API.
If you’re considering self-hosting, make sure to factor in all the operational overhead before making a decision.
Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?
No comments yet.
Discussion
No comments yet.