# NotebookLM Timeout Issue Analysis & Fix Plan

## Issue Summary
On https://jadwaai.com/study/notebook/35, users see "انتهت مهلة الإنشاء. اضغط إعادة المحاولة..." (Creation timeout. Click retry...) but the assets were actually created successfully.

## Root Cause Analysis

### Current Situation
- **7 assets successfully created**: infographic, slide_deck, mind_map, report, flashcards, quiz, audio
- **1 asset failed**: video (30 min timeout)
- **Specific Issue**: Audio asset (ID: 27) shows "ready" status with error message "انتهت مهلة الإنشاء. اضغط إعادة المحاولة." but file exists (27.97 MB)

### Database vs File System Analysis
```
Asset ID 27 (audio):
- Status: ready ✓
- File Path: uploads/notebooks/35/audio.mp3 ✓
- File Format: mp3 ✓
- Error Message: "انتهت مهلة الإنشاء. اضغط إعادة المحاولة." ❌
- Created: 2026-05-03 19:38:39
- Completed: 2026-05-03 20:02:10 (23 minutes!)

Asset ID 28 (video):
- Status: failed ❌
- File Path: None ❌
- Error Message: "Task d3a5a760-3e49-4a1e-b080-5dfef69bfb08 timed out after 1800s" ❌
```

## Problem Identification

### 1. Timeout vs Success Detection
The NotebookLM asset generation process times out after 30 minutes, but the asset may be successfully created during that time. The current code doesn't distinguish between:
- **Timeout with success**: Asset created but process timed out
- **Timeout with failure**: Asset not created due to actual error

### 2. Error Message Handling
Error messages are set even when files are successfully created, causing confusion in the UI.

### 3. File Verification Missing
No post-creation verification to check if files exist before marking as failed.

## Fix Plan

### Phase 1: Database Repair (Immediate)

**1.1 Create Asset Repair Script**
- Script to scan for assets with error messages but valid files
- Clear error messages for assets with valid files
- Verify file existence vs database records

**1.2 Database Cleanup**
```sql
-- Clear error messages for ready assets with valid files
UPDATE notebook_assets 
SET error_message = NULL 
WHERE status = 'ready' 
  AND file_path IS NOT NULL 
  AND error_message IS NOT NULL;
```

### Phase 2: Code Improvements (Short-term)

**2.1 Enhanced File Verification**
```python
# In app/routes/study.py - api_notebook_generate function
def verify_asset_file(asset, report_id):
    """Verify if asset file exists and is valid."""
    if not asset.file_path:
        return False
    
    full_path = os.path.join(current_app.static_folder, asset.file_path)
    if not os.path.exists(full_path):
        return False
    
    # Check file size (should be > 0 for most assets)
    file_size = os.path.getsize(full_path)
    if file_size == 0 and asset.asset_type not in ['mind_map', 'quiz', 'flashcards']:
        return False
    
    return True
```

**2.2 Improved Timeout Handling**
```python
# Modify exception handling in asset generation
except Exception as e:
    current_app.logger.error(f"NotebookLM generate {asset_type} error: {e}")
    
    # CRITICAL: Check if file was actually created despite timeout
    if verify_asset_file(asset, report.id):
        # File exists - clear error and mark as ready
        asset.status = "ready"
        asset.error_message = None
        asset.completed_at = datetime.utcnow()
        current_app.logger.info(f"Asset {asset_type} created despite timeout/error")
    else:
        # Real failure
        err_str = str(e).lower()
        if "rate limit" in err_str or "quota" in err_str:
            asset.status = "failed"
            asset.error_message = "تم تجاوز حد الاستخدام في NotebookLM. حاول مرة أخرى بعد عدة دقائق."
        elif "timeout" in err_str or "timed out" in err_str:
            asset.status = "failed"
            asset.error_message = "انتهت مهلة الإنشاء. المحتوى كبير جداً أو تعذر إنشاؤه."
        else:
            asset.status = "failed"
            asset.error_message = str(e)
    
    db.session.commit()
```

**2.3 Auto-Reset Stale Assets**
```python
# Enhanced auto-reset logic for assets stuck > 10 minutes
def auto_reset_stale_assets(report_id):
    """Reset stale generating assets and verify files."""
    assets = NotebookAsset.query.filter_by(report_id=report_id).all()
    
    for asset in assets:
        if asset.status == "generating" and asset.created_at:
            age_minutes = (datetime.utcnow() - asset.created_at).total_seconds() / 60
            
            if age_minutes > 10:
                # Check if file exists despite timeout
                if verify_asset_file(asset, report_id):
                    # File was created - mark as ready
                    asset.status = "ready"
                    asset.error_message = None
                    asset.completed_at = datetime.utcnow()
                else:
                    # No file - mark as failed
                    asset.status = "failed"
                    asset.error_message = "انتهت مهلة الإنشاء. اضغط إعادة المحاولة."
```

### Phase 3: UI Improvements (User Experience)

**3.1 Enhanced Asset Display**
```javascript
// In notebook.html template
function showAssetWithWarning(assetType) {
    // Show asset as ready but with timeout warning
    const warningBadge = `
        <span class="px-2 py-1 rounded-full bg-amber-500/10 text-amber-400 text-[10px] font-bold">
            جاهز ⚠️ (انتهت المهلة)
        </span>
    `;
    // Show download/view buttons normally
}
```

**3.2 Better Status Messages**
- Separate "Timeout with success" from "Timeout with failure"
- Show warnings instead of errors for successful timeouts
- Add "Verify & Repair" button for administrators

### Phase 4: Monitoring & Prevention

**4.1 Asset Generation Monitoring**
- Log asset generation start/end times
- Track which asset types commonly timeout
- Alert on assets > 15 minutes generation time

**4.2 Timeout Configuration**
```python
# Asset-specific timeouts
ASSET_TIMEOUTS = {
    "audio": 1800,      # 30 minutes
    "video": 3600,      # 60 minutes  
    "mind_map": 300,    # 5 minutes
    "quiz": 300,        # 5 minutes
    "flashcards": 300,  # 5 minutes
    "report": 600,      # 10 minutes
    "infographic": 900, # 15 minutes
    "slide_deck": 1200, # 20 minutes
}
```

## Implementation Priority

1. **HIGH PRIORITY** (Do immediately):
   - Run database repair script to fix existing assets
   - Add file verification to timeout handling
   - Clear error messages for ready assets

2. **MEDIUM PRIORITY** (Do this week):
   - Implement enhanced timeout handling
   - Add auto-reset for stale assets with file verification
   - Improve error messages and UI display

3. **LOW PRIORITY** (Do next sprint):
   - Add monitoring and alerting
   - Implement asset-specific timeouts
   - Create admin tools for bulk asset repair

## Success Criteria

✓ Assets with valid files never show error messages
✓ Timeout detection distinguishes success vs failure
✓ Users can access all successfully created assets
✓ Clear error messages only for actual failures
✓ No false "failed" statuses for successful creations

## Files to Modify

1. `app/routes/study.py` - Asset generation and timeout handling
2. `app/templates/study/notebook.html` - UI status display
3. `app/ai/notebooklm_service.py` - File verification utilities
4. Create new: `scripts/repair_notebook_assets.py` - Database repair script

## Testing Plan

1. Test with report 35 to verify fix works
2. Create new test report and generate assets
3. Simulate timeout scenarios
4. Verify file existence checking
5. Test UI display with various asset states