fine_tuning/README.md

Fine-tuning Module 🎯

Train custom models on your healthcare conversation data.

Overview

This module automatically collects conversation data and enables fine-tuning of specialized healthcare agents using OpenAI's fine-tuning API.

Features

• ✅ Automatic Data Collection - Logs all agent conversations
• ✅ Quality Filtering - Filter by user ratings
• ✅ OpenAI Format Export - Ready for fine-tuning
• ✅ Multi-Agent Support - Train each agent separately
• ✅ Job Management - Track fine-tuning progress

How It Works

1. Data Collection (Automatic)

The system automatically logs conversations when enabled:

# In coordinator (already integrated)
coordinator = AgentCoordinator(enable_data_collection=True)

Data is stored in fine_tuning/data/ organized by agent:

fine_tuning/data/
├── nutrition/
│   ├── conversations_20241025.jsonl
│   └── multi_turn_20241025.jsonl
├── exercise/
├── symptom/
├── mental_health/
└── general_health/

2. Export Training Data

Export conversations in OpenAI fine-tuning format:

from fine_tuning import get_data_collector

collector = get_data_collector()

# Export all conversations
training_file = collector.export_for_openai_finetuning(
    agent_name='nutrition_agent',
    output_file='nutrition_training.jsonl'
)

# Export only high-quality conversations (rating >= 4.0)
training_file = collector.export_for_openai_finetuning(
    agent_name='nutrition_agent',
    min_quality_rating=4.0
)

3. Fine-tune Agent

Option A: Using Script (Recommended)

# Fine-tune nutrition agent
python scripts/fine_tune_agent.py --agent nutrition

# With quality filtering
python scripts/fine_tune_agent.py --agent nutrition --min-rating 4.0

# Start job without waiting
python scripts/fine_tune_agent.py --agent exercise --no-wait

Option B: Using Python API

from fine_tuning import fine_tune_agent

# Fine-tune and wait for completion
model_id = fine_tune_agent(
    agent_name='nutrition',
    training_file='nutrition_training.jsonl',
    model='gpt-4o-mini-2024-07-18',
    wait_for_completion=True
)

print(f"Fine-tuned model: {model_id}")

4. Use Fine-tuned Model

Update agent configuration to use the fine-tuned model:

# config/settings.py or agent file
MODEL = 'ft:gpt-4o-mini-2024-07-18:your-org:nutrition:abc123'

Data Format

Conversation Entry

{
  "timestamp": "2024-10-25T10:00:00",
  "agent": "nutrition_agent",
  "user_message": "Tôi muốn giảm cân",
  "agent_response": "Để giảm cân hiệu quả...",
  "user_data": {
    "age": 25,
    "gender": "male",
    "weight": 70,
    "height": 175
  },
  "metadata": {
    "rating": 5.0,
    "feedback": "Very helpful"
  }
}

OpenAI Fine-tuning Format

{
  "messages": [
    {"role": "system", "content": "You are a nutrition specialist."},
    {"role": "user", "content": "Tôi muốn giảm cân"},
    {"role": "assistant", "content": "Để giảm cân hiệu quả..."}
  ]
}

API Reference

ConversationDataCollector

from fine_tuning import get_data_collector

collector = get_data_collector()

# Log single conversation
collector.log_conversation(
    agent_name='nutrition_agent',
    user_message='User question',
    agent_response='Agent answer',
    user_data={'age': 25},
    metadata={'rating': 5.0}
)

# Log multi-turn conversation
collector.log_multi_turn_conversation(
    agent_name='nutrition_agent',
    conversation_history=[
        ('User msg 1', 'Agent response 1'),
        ('User msg 2', 'Agent response 2')
    ],
    user_data={'age': 25}
)

# Get conversation counts
counts = collector.get_conversation_count()
# {'nutrition': 150, 'exercise': 89, ...}

# Export for fine-tuning
training_file = collector.export_for_openai_finetuning(
    agent_name='nutrition_agent',
    min_quality_rating=4.0
)

FineTuningTrainer

from fine_tuning import FineTuningTrainer

trainer = FineTuningTrainer()

# Upload training file
file_id = trainer.upload_training_file('training.jsonl')

# Create fine-tuning job
job_id = trainer.create_fine_tuning_job(
    training_file_id=file_id,
    model='gpt-4o-mini-2024-07-18',
    suffix='nutrition-v1'
)

# Check job status
status = trainer.check_job_status(job_id)

# Wait for completion
result = trainer.wait_for_completion(job_id)

# List all fine-tuned models
models = trainer.list_fine_tuned_models()

Best Practices

1. Data Quality

• ✅ Collect at least 100-200 conversations per agent
• ✅ Include diverse user queries and scenarios
• ✅ Filter by quality rating (>= 4.0 recommended)
• ✅ Review and clean data before fine-tuning

2. Training

• ✅ Start with gpt-4o-mini (faster, cheaper)
• ✅ Use descriptive suffixes (e.g., nutrition-v1)
• ✅ Monitor training progress
• ✅ Test fine-tuned model before deployment

3. Evaluation

• ✅ Compare fine-tuned vs base model responses
• ✅ Test on held-out conversations
• ✅ Collect user feedback on fine-tuned model
• ✅ Iterate based on results

Cost Estimation

Fine-tuning costs (OpenAI pricing):

• Training: ~$0.008 per 1K tokens
• Usage: Same as base model

Example:

• 200 conversations × 500 tokens avg = 100K tokens
• Training cost: ~$0.80
• Usage: Same as gpt-4o-mini

Troubleshooting

No conversations collected

# Check if data collection is enabled
# In coordinator initialization:
coordinator = AgentCoordinator(enable_data_collection=True)

Training file format error

# Validate JSONL format
python -c "import json; [json.loads(line) for line in open('file.jsonl')]"

Fine-tuning job failed

# Check job status
trainer = FineTuningTrainer()
status = trainer.check_job_status('job-id')
print(status)

Examples

Complete Workflow

from fine_tuning import get_data_collector, fine_tune_agent

# 1. Check collected data
collector = get_data_collector()
counts = collector.get_conversation_count()
print(f"Nutrition conversations: {counts.get('nutrition', 0)}")

# 2. Export training data
training_file = collector.export_for_openai_finetuning(
    agent_name='nutrition_agent',
    min_quality_rating=4.0
)

# 3. Fine-tune
model_id = fine_tune_agent(
    agent_name='nutrition',
    training_file=training_file,
    suffix='v1',
    wait_for_completion=True
)

# 4. Update configuration
print(f"Update MODEL to: {model_id}")

Future Enhancements

• Automatic quality scoring
• Data augmentation
• Multi-model comparison
• A/B testing framework
• Continuous learning pipeline