Spaces:

sethmcknight
/

msse-ai-engineering

Sleeping

# Production server configuration (gunicorn.conf.py)
workers = 1                    # Single worker optimized for memory
threads = 2                    # Minimal threading for I/O
max_requests = 50              # Prevent memory leaks with worker restart
timeout = 30                   # Balance for LLM response times
preload_app = false           # Avoid memory duplication

Embedding Model Optimization

Memory-Efficient AI Models:

Production Model: paraphrase-MiniLM-L3-v2
- Dimensions: 384
- Memory Usage: ~132MB
- Quality: Maintains semantic search accuracy
Alternative Model: all-MiniLM-L6-v2 (not used in production)
- Memory Usage: ~550-1000MB (exceeds platform limits)

Database Strategy

Pre-built Vector Database:

Approach: Vector database built locally and committed to repository
Benefit: Zero embedding generation on deployment (avoids memory spikes)
Size: ~25MB for 98 document chunks with metadata
Persistence: ChromaDB with SQLite backend for reliability

📊 Performance Metrics

Response Time Performance

Production Response Times:

Health Checks: <100ms
Document Search: <500ms
RAG Chat Responses: 2-3 seconds (including LLM generation)
System Initialization: <2 seconds (lazy loading)

Memory Monitoring

Real-time Memory Tracking:

{
  "memory_usage_mb": 187,
  "memory_available_mb": 325,
  "memory_utilization": 0.36,
  "gc_collections": 247,
  "embedding_model": "paraphrase-MiniLM-L3-v2",
  "vector_db_size_mb": 25
}

Capacity & Scaling

Current Capacity:

Concurrent Users: 20-30 simultaneous requests
Document Corpus: 98 chunks from 22 policy documents
Daily Queries: Supports 1000+ queries/day within free tier limits
Storage: 100MB total (including application code and database)

🔧 Production Features

Memory Management System

Automated Memory Optimization:

# Memory monitoring and cleanup utilities
class MemoryManager:
    def track_usage(self):      # Real-time memory monitoring
    def optimize_memory(self):  # Garbage collection and cleanup
    def get_stats(self):       # Detailed memory statistics

Error Handling & Recovery

Memory-Aware Error Handling:

Out of Memory: Automatic garbage collection and request retry
Memory Pressure: Request throttling and service degradation
Memory Leaks: Automatic worker restart (max_requests=50)

Health Monitoring

Production Health Checks:

# System health endpoint
GET /health

# Response includes:
{
  "status": "healthy",
  "components": {
    "vector_store": "operational",
    "llm_service": "operational",
    "embedding_service": "operational",
    "memory_manager": "operational"
  },
  "performance": {
    "memory_usage_mb": 187,
    "response_time_avg_ms": 2140,
    "uptime_hours": 168
  }
}

🚀 Deployment Pipeline

Automated CI/CD

GitHub Actions Integration:

Pull Request Validation:
- Full test suite (138 tests)
- Memory usage validation
- Performance benchmarking
Deployment Triggers:
- Automatic deployment on merge to main
- Manual deployment via GitHub Actions
- Rollback capability for failed deployments
Post-Deployment Validation:
- Health check verification
- Memory usage monitoring
- Performance regression testing

Environment Configuration

Required Environment Variables:

# Production deployment configuration
OPENROUTER_API_KEY=sk-or-v1-***     # LLM service authentication
FLASK_ENV=production                 # Production optimizations
PORT=10000                          # Render platform default

# Optional optimizations
MAX_TOKENS=500                      # Response length limit
GUARDRAILS_LEVEL=standard           # Safety validation level
VECTOR_STORE_PATH=/app/data/chroma_db # Database location

📈 Production Improvements

Memory Optimizations Implemented

Before Optimization:

Startup Memory: ~400MB (exceeded platform limits)
Model Memory: ~550-1000MB (all-MiniLM-L6-v2)
Architecture: Monolithic with all services loaded at startup

After Optimization:

Startup Memory: ~50MB (87% reduction)
Model Memory: ~60MB (paraphrase-MiniLM-L3-v2)
Architecture: App Factory with lazy loading

Performance Improvements

Response Time Optimizations:

Lazy Loading: Services initialize only when needed
Caching: ML services cached after first request
Database: Pre-built vector database for instant availability
Gunicorn: Optimized worker/thread configuration for I/O

Reliability Improvements

Error Handling & Recovery:

Memory Monitoring: Real-time tracking with automatic cleanup
Graceful Degradation: Fallback responses for service failures
Circuit Breaker: Automatic service isolation for stability
Worker Restart: Prevent memory leaks with automatic recycling

🔄 Monitoring & Maintenance

Production Monitoring

Key Metrics Tracked:

Memory Usage: Real-time monitoring with alerts
Response Times: P95 latency tracking
Error Rates: Service failure monitoring
User Engagement: Query patterns and usage statistics

Maintenance Schedule

Automated Maintenance:

Daily: Health check validation and performance reporting
Weekly: Memory usage analysis and optimization review
Monthly: Dependency updates and security patching
Quarterly: Performance benchmarking and capacity planning

This production deployment demonstrates successful implementation of comprehensive memory management for cloud-constrained environments while maintaining full RAG functionality and enterprise-grade reliability.