ParrotServe
“Parrot: Efficient Serving of LLM-based Applications with Semantic Variable” Parrot is a distributed serving system for LLM-based Applications. The Parrot API w/ Semantic Variable is served by a centralized cluster manager called ServeCore, which manages…