start_load_kv could be executed asynchronously and in parallel with multiple requests—why was synchronous serial execution selected instead? Same question here, the get_num_new_matched_tokens is fixed ...

Are there any best practices for deploying Cog in a production-grade setup with autoscaling, load balancing, or using async workers behind a gateway? I'd appreciate any suggestions, workarounds, or ...