frugal
1.0.0
An open-source Cost & Resource Optimization Platform for LLMs. Be Frugal! ?
Numexa is an AI-driven cost and resource optimization tool designed to enhance operational efficiency. It achieves this by leveraging contextual insights derived from usage metrics. Numexa employs cutting-edge techniques such as intelligent caching and data retrieval, harnessing the power of vector databases to streamline operations. Explore how Numexa can revolutionize your resource management and cost-saving endeavors.
Model agnostic functionality records unlimited requests from various providers like OpenAI, Cohere, Anthropic and more.
? Model management
? Alerting & Notification with predefined policies, like error rate, threshold, cost, etc.
? Caching, Custom Rate Limits, and Retries,
Track costs and latencies by users, applications, and endpoints
(Coming soon) Intellegient caching and data retrieval
(Coming soon) Cost and resource optimization
Before you begin, ensure you have the following installed on your system:
Clone the Repository:
git clone <repository_url>
cd <repository_directory>
Build and Start the Services: Run the following commands to build and start the project services
make all
docker compose -f docker-compose.dev.yaml up -dVerify Services: After running the above commands, your project services should be up and running. You can verify this by checking the logs
Join our #Discord or drop email at [email protected]
