A12 days
Inference
What happens from the moment a request is sent until the model responds, what it costs and how it is served.
- Inference I · using models
- Tokens, context, latency and cost. How to choose a model for each task. APIs and aggregators with a single key.
- Inference II · serving models
- Running models on your own or a rented server, quantisation, the hardware required and when it pays off compared with pay-per-use.