software-engineering
Posts filed under software-engineering. All posts.
-
Batching Requests to a Model Server
How dynamic batching trades a little latency for much higher throughput, with a small asyncio implementation.
-
Structured Logging for Training Jobs
How to emit one JSON object per event from a training loop so that runs can be searched and compared later.