Prompt caching

Reusing the processed start of a prompt across requests, so repeated content such as a long system prompt or document costs much less. Cache reads typically cost 10% of the normal input price or less, and some providers charge extra to write the cache. It pays off when the same large context is sent many times.