TASK_CONTEXT_API_ROUTING_ERROR.md

Task Context: HuggingFace API Routing Error

Current Work: Debugging Agent Display Failure on HuggingFace Deployment

Critical Error Summary

The SAAP platform deployed on HuggingFace is experiencing complete failure of agent display functionality. The frontend loads successfully, but agents cannot be retrieved from the backend.

Error Pattern:

GET /api/v1/agents HTTP/1.1" 500 Internal Server Error
RuntimeError: Not found (from spa_static_files.py line 27)

Success Indicators:

• Platform initializes successfully ✅
• 7 agents loaded into database ✅
• colossus connection successful ✅
• OpenRouter client initialized ✅
• Frontend static assets serve correctly ✅
• API endpoint fails catastrophically ❌

Key Technical Concepts

Architecture:

• Backend: Python FastAPI with hybrid agent support (colossus + OpenRouter)
• Frontend: Vue.js SPA with Vite build system
• Deployment: Docker containerized on HuggingFace Spaces
• Database: SQLite with SQLAlchemy ORM
• Agents: 7 base templates (Jane, John, Lara, Theo, Justus, Leon, Luna Alesi)

Critical Components:

1. backend/main.py - FastAPI application initialization and middleware ordering
2. backend/spa_static_files.py - Custom static file handler (line 27 = error source)
3. backend/api/agents.py - Agent API endpoints (should handle /api/v1/agents)
4. backend/services/agent_manager_hybrid.py - Hybrid agent manager service

Pydantic Warning (Secondary Issue):

UserWarning: Valid config keys have changed in V2:
* 'schema_extra' has been renamed to 'json_schema_extra'

Relevant Files and Code

Error Source Location:

• File: backend/spa_static_files.py
• Line: 27
• Method: get_response
• Error: raise RuntimeError("Not found")

Stack Trace Analysis:

Starlette Middleware Chain:
1. proxy_headers.py
2. CORS middleware  
3. Exception middleware
4. Routing (fails here - goes to StaticFiles instead of API route)
5. StaticFiles handler (spa_static_files.py line 105)
6. get_response method (spa_static_files.py line 27) → RuntimeError

Request Flow (ACTUAL):

Frontend → GET /api/v1/agents 
  → Starlette routing 
  → StaticFiles middleware catches route (WRONG!)
  → spa_static_files.py:get_response() 
  → RuntimeError("Not found")

Request Flow (EXPECTED):

Frontend → GET /api/v1/agents 
  → Starlette routing 
  → FastAPI agent router 
  → agent_api.py:get_agents()
  → Return agent list JSON

Problem Solved Thus Far

Initialization Success:

• Database connection established
• 7 agents registered from templates
• colossus server connection verified (14.06s test successful)
• OpenRouter client initialized with free model fallback
• Hybrid agent manager operational

Configuration Verified:

• Environment: OPENROUTER_API_KEY warning present (expected in free mode)
• Database: sqlite:///./saap_production.db created successfully
• Providers: Both colossus (✅) and OpenRouter (✅) initialized

Pending Tasks and Next Steps

Immediate Investigation Required:

1. Examine spa_static_files.py implementation

   # Need to review line 27 in get_response method
   # Understand why it's intercepting API routes
   

2. Check main.py middleware ordering

   # Verify API router mounted BEFORE static file handler
   # Pattern should be:
   app.include_router(api_router)  # FIRST
   app.mount("/", SPAStaticFiles())  # LAST (catch-all)
   

3. Verify route registration

   # Confirm /api/v1/agents endpoint properly registered
   # Check FastAPI route debugging output
   

4. Fix Pydantic V2 deprecation warning

   # Replace 'schema_extra' with 'json_schema_extra' in Pydantic models
   

Direct Quote from Error Logs:

"ich bekomme folgende Fehler in huggingface und die Aagenten werden aus diesem Grund nicht angezeigt:

INFO: 10.20.38.235:38000 - "GET /api/v1/agents HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/app/spa_static_files.py", line 27, in get_response raise RuntimeError("Not found") RuntimeError: Not found"

Where Development Left Off:

The user encountered this error in production HuggingFace deployment. The platform starts successfully, all backend services initialize correctly, but the critical /api/v1/agents endpoint fails with a routing error. The static file handler is incorrectly intercepting API requests that should be handled by FastAPI routers.

Root Cause Hypothesis:

The middleware/route mounting order in backend/main.py likely has the SPA static file handler registered before (or with higher priority than) the API routes, causing all requests to be evaluated by the static file handler first. When /api/v1/agents doesn't match a static file, the handler raises RuntimeError("Not found") instead of allowing the request to continue to FastAPI routers.

Required Fix Pattern:

# backend/main.py - CORRECT PATTERN

app = FastAPI()

# 1. Register API routes FIRST (specific routes)
app.include_router(agent_router, prefix="/api/v1")
app.include_router(hybrid_router, prefix="/api/v1")

# 2. Mount static files LAST (catch-all fallback)
app.mount("/", SPAStaticFiles(directory="frontend/dist"))

Test Verification Needed:

• Confirm /api/v1/agents returns JSON agent list (not 500)
• Verify static assets still load (/, /assets/*)
• Check agent display in frontend UI
• Validate both API and SPA work without conflicts

Technical Context - File Structure:

backend/
├── main.py (← FIX NEEDED: middleware ordering)
├── spa_static_files.py (← line 27 raising error)
├── api/
│   ├── agents.py (← should handle /api/v1/agents)
│   └── hybrid_endpoints.py
└── services/
    └── agent_manager_hybrid.py

Environment Details:

• Python: 3.11
• FastAPI: Latest (from requirements.txt)
• Uvicorn: Running on 0.0.0.0:7860
• Platform: HuggingFace Spaces
• Deployment: Docker container

Error Frequency: Continuous - every frontend refresh attempts to fetch agents and fails. The error repeats every few seconds as the frontend retries the request.

Debugging Methodology to Apply

Per Rules/debugging-workflows.md - Four-Phase Framework:

Phase 1: Root Cause Investigation (IN PROGRESS)

• ✅ Read error messages - RuntimeError from spa_static_files.py line 27
• ✅ Stack trace analyzed - Starlette routing → StaticFiles → error
• ⚠️ Need to read spa_static_files.py and main.py to confirm hypothesis
• ⚠️ Need to trace middleware ordering

Phase 2: Pattern Analysis (PENDING)

• Find working FastAPI + SPA patterns
• Compare against current implementation
• Identify middleware ordering differences

Phase 3: Hypothesis Testing (PENDING)

• Hypothesis: Middleware ordering causes route interception
• Test: Reorder middleware in main.py
• Verify: API routes accessible, SPA still works

Phase 4: Implementation (PENDING)

• Create failing test for /api/v1/agents endpoint
• Fix middleware ordering
• Verify all tests pass
• Deploy and validate on HuggingFace

Integration Requirements

Security Considerations:

• API endpoints must not expose sensitive data
• CORS properly configured for HuggingFace domain
• Error messages sanitized (no stack traces to frontend)

Code Quality Standards:

• Fix Pydantic V2 deprecation warnings
• Follow FastAPI best practices for middleware ordering
• Ensure clear separation: API routes vs static file serving

Testing Requirements:

• Integration test: GET /api/v1/agents returns 200 + agent list
• Integration test: GET / returns 200 + SPA HTML
• Integration test: GET /assets/* returns 200 + static assets
• Coverage target: ≥75% (MCP/tool-heavy architecture acceptable)

Critical Files to Examine

1. backend/spa_static_files.py - Line 27 error source
2. backend/main.py - Middleware mounting order
3. backend/api/agents.py - Agent endpoint implementation
4. backend/services/agent_manager_hybrid.py - Agent retrieval logic

Success Criteria

• /api/v1/agents returns 200 status with valid JSON
• Agents display in frontend UI
• Static assets continue loading correctly
• No RuntimeError exceptions
• Pydantic deprecation warning resolved
• Route precedence clearly documented