src/api/README.md

🚀 Cidadão.AI API Layer

📋 Overview

The API Layer is the primary interface for the Cidadão.AI platform, providing RESTful endpoints for transparency analysis, multi-agent orchestration, and real-time monitoring. Built with FastAPI and async/await patterns for high-performance concurrent processing.

🏗️ Architecture

src/api/
├── app.py              # FastAPI application entry point
├── auth.py             # OAuth2 authentication
├── oauth.py            # OAuth provider integration  
├── websocket.py        # Real-time WebSocket communication
├── middleware/         # Security & logging middleware
│   ├── authentication.py    # JWT authentication middleware
│   ├── logging_middleware.py # Structured request logging
│   ├── rate_limiting.py     # Rate limiting with Redis
│   └── security.py          # Security headers & CORS
└── routes/             # API endpoints organized by domain
    ├── investigations.py    # Anomaly detection endpoints
    ├── analysis.py         # Pattern analysis endpoints  
    ├── reports.py          # Report generation endpoints
    ├── health.py           # Health checks & monitoring
    ├── auth.py             # Authentication endpoints
    ├── oauth.py            # OAuth2 flow endpoints
    ├── audit.py            # Audit logging endpoints
    └── websocket.py        # WebSocket event handlers

🔌 API Endpoints

Core Endpoints

Endpoint	Method	Description	Authentication
/	GET	API information	Public
/docs	GET	Swagger UI documentation	Public
/health	GET	Basic health check	Public
/health/detailed	GET	Comprehensive system status	Public
/health/live	GET	Kubernetes liveness probe	Public
/health/ready	GET	Kubernetes readiness probe	Public

Authentication

Endpoint	Method	Description
/auth/login	POST	User authentication
/auth/refresh	POST	Token refresh
/auth/logout	POST	Session termination
/auth/oauth/google	GET	Google OAuth2 flow
/auth/oauth/github	GET	GitHub OAuth2 flow

Investigations 🔍

Endpoint	Method	Description	Agent
/api/v1/investigations/start	POST	Start anomaly investigation	InvestigatorAgent
/api/v1/investigations/{id}	GET	Get investigation results	-
/api/v1/investigations/{id}/status	GET	Check investigation progress	-
/api/v1/investigations/stream	GET	Stream real-time results	InvestigatorAgent

Anomaly Types Supported:

• price - Price anomalies using statistical methods
• vendor - Vendor concentration analysis
• temporal - Suspicious timing patterns
• payment - Payment irregularities
• duplicate - Duplicate contract detection
• pattern - Custom pattern matching

Analysis 📊

Endpoint	Method	Description	Agent
/api/v1/analysis/trends	POST	Spending trend analysis	AnalystAgent
/api/v1/analysis/patterns	POST	Pattern correlation analysis	AnalystAgent
/api/v1/analysis/efficiency	POST	Efficiency metrics calculation	AnalystAgent
/api/v1/analysis/{id}	GET	Get analysis results	-

Analysis Types:

• spending_trends - Linear regression trend analysis
• vendor_patterns - Vendor behavior analysis
• organizational_behavior - Cross-org pattern comparison
• seasonal_analysis - Seasonal pattern detection
• efficiency_metrics - Performance indicators
• correlation_analysis - Multi-dimensional correlations

Reports 📝

Endpoint	Method	Description	Agent
/api/v1/reports/generate	POST	Generate investigation report	ReporterAgent
/api/v1/reports/{id}	GET	Retrieve generated report	-
/api/v1/reports/{id}/download	GET	Download report (PDF/HTML)	-

Report Formats:

• json - Structured data format
• markdown - Human-readable markdown
• html - Web-formatted report
• pdf - Professional PDF document (planned)

Audit & Security 🛡️

Endpoint	Method	Description
/audit/events	GET	Audit event history
/audit/security	GET	Security event analysis
/audit/compliance	GET	Compliance status

🔐 Security Features

Authentication & Authorization

# JWT-based authentication with refresh tokens
Authentication: Bearer <jwt_token>

# API Key authentication for service-to-service
X-API-Key: <api_key>

# OAuth2 providers supported
- Google OAuth2
- GitHub OAuth2

Security Middleware Stack

# Middleware execution order (LIFO)
1. SecurityMiddleware      # Security headers, CORS
2. LoggingMiddleware       # Request/response logging  
3. RateLimitMiddleware     # Rate limiting per IP/user
4. AuthenticationMiddleware # JWT validation
5. TrustedHostMiddleware   # Host validation (production)

Rate Limiting

# Default limits per authenticated user
- 60 requests/minute
- 1000 requests/hour  
- 10000 requests/day

# Configurable per endpoint
investigations: 10/minute  # CPU-intensive operations
analysis: 20/minute       # Medium complexity
reports: 5/minute         # Resource-intensive generation

📊 Request/Response Models

Investigation Request

{
  "query": "Analyze contracts from Ministry of Health 2024",
  "data_source": "contracts",
  "filters": {
    "year": "2024",
    "orgao": "20000",
    "valor_min": 100000
  },
  "anomaly_types": ["price", "vendor", "temporal"],
  "include_explanations": true,
  "stream_results": false
}

Investigation Response

{
  "investigation_id": "uuid4-string",
  "status": "completed",
  "query": "Analyze contracts...",
  "data_source": "contracts",
  "started_at": "2025-01-24T10:00:00Z",
  "completed_at": "2025-01-24T10:05:30Z",
  "anomalies_found": 23,
  "total_records_analyzed": 15420,
  "results": [
    {
      "anomaly_id": "uuid4-string",
      "type": "price",
      "severity": "high",
      "confidence": 0.92,
      "description": "Price 340% above expected range",
      "explanation": "Statistical analysis shows...",
      "affected_records": [...],
      "suggested_actions": [...]
    }
  ],
  "summary": "Found 23 anomalies across 15,420 records...",
  "confidence_score": 0.87,
  "processing_time": 330.5
}

🔄 Async Processing Patterns

Background Tasks

# Long-running investigations use background tasks
@router.post("/investigations/start")
async def start_investigation(
    request: InvestigationRequest,
    background_tasks: BackgroundTasks
):
    investigation_id = str(uuid4())
    
    # Start investigation in background
    background_tasks.add_task(
        run_investigation,
        investigation_id,
        request
    )
    
    return {"investigation_id": investigation_id, "status": "started"}

Real-time Streaming

# Stream results as they're discovered
@router.get("/investigations/stream")
async def stream_investigation(investigation_id: str):
    async def generate():
        async for result in investigate_with_streaming(investigation_id):
            yield f"data: {json.dumps(result)}\n\n"
    
    return StreamingResponse(generate(), media_type="text/plain")

🚦 Error Handling

Custom Exception Hierarchy

CidadaoAIError (base)
├── ValidationError (400)
├── DataNotFoundError (404)  
├── AuthenticationError (401)
├── UnauthorizedError (403)
├── RateLimitError (429)
├── LLMError (503)
├── TransparencyAPIError (502)
└── AgentExecutionError (500)

Error Response Format

{
  "status": "error",
  "status_code": 400,
  "error": {
    "error": "ValidationError",
    "message": "Invalid data source provided",
    "details": {
      "field": "data_source",
      "allowed_values": ["contracts", "expenses", "agreements"]
    }
  },
  "request_id": "uuid4-string",
  "timestamp": "2025-01-24T10:00:00Z"
}

📈 Monitoring & Observability

Health Checks

# Basic health check
GET /health
{
  "status": "healthy",
  "timestamp": "2025-01-24T10:00:00Z",
  "version": "1.0.0",
  "uptime": 86400.5,
  "services": {
    "transparency_api": {"status": "healthy", "response_time": 0.145},
    "database": {"status": "healthy", "response_time": 0.003},
    "redis": {"status": "healthy", "response_time": 0.001}
  }
}

Audit Logging

# All API requests are automatically audited
Audit Event Types:
- AUTHENTICATION_SUCCESS/FAILURE
- API_ACCESS
- INVESTIGATION_STARTED/COMPLETED
- REPORT_GENERATED
- SECURITY_VIOLATION
- DATA_ACCESS

Structured Logging

# All logs use structured format
{
  "timestamp": "2025-01-24T10:00:00Z",
  "level": "INFO",
  "logger": "api.routes.investigations",
  "message": "investigation_started",
  "investigation_id": "uuid4-string",
  "user_id": "user123",
  "data_source": "contracts",
  "anomaly_types": ["price", "vendor"],
  "processing_time": 0.045
}

🧪 Testing Strategy

Test Categories

# Unit tests - individual endpoint logic
pytest tests/unit/api/

# Integration tests - full request/response cycles  
pytest tests/integration/api/

# E2E tests - complete workflows
pytest tests/e2e/api/

# Load tests - performance validation
pytest tests/performance/api/

Test Configuration

# Test database isolation
@pytest.fixture
async def test_client():
    # Use TestContainers for real databases
    with TestClient(app) as client:
        yield client

# Authentication test helpers
@pytest.fixture
def authenticated_headers():
    token = create_test_jwt()
    return {"Authorization": f"Bearer {token}"}

🔧 Configuration

Environment Variables

# Server configuration
HOST=0.0.0.0
PORT=8000
WORKERS=4

# Database
DATABASE_URL=postgresql+asyncpg://user:pass@localhost/cidadao_ai

# Redis  
REDIS_URL=redis://localhost:6379/0

# API Keys
TRANSPARENCY_API_KEY=your_api_key
GROQ_API_KEY=your_groq_key

# Security
SECRET_KEY=your-super-secret-key
JWT_SECRET_KEY=your-jwt-secret

# CORS
CORS_ORIGINS=["http://localhost:3000", "https://cidadao.ai"]

Feature Flags

# Progressive feature rollout
ENABLE_FINE_TUNING=false
ENABLE_AUTONOMOUS_CRAWLING=false  
ENABLE_ADVANCED_VISUALIZATIONS=false
ENABLE_ETHICS_GUARD=true

🚀 Development

Local Development

# Install dependencies
pip install -r requirements/base.txt

# Run with hot reload
uvicorn src.api.app:app --reload --host 0.0.0.0 --port 8000

# Or use Makefile
make dev

Docker Development

# Build development image
docker build -f Dockerfile.api -t cidadao-api:dev .

# Run with Docker Compose
docker-compose -f docker-compose.dev.yml up api

Code Quality

# Code formatting
black src/api/
ruff check src/api/

# Type checking
mypy src/api/

# Security scanning  
bandit -r src/api/

# All quality checks
make lint

📚 API Documentation

Interactive Documentation

• Swagger UI: /docs - Interactive API explorer
• OpenAPI Schema: /openapi.json - Machine-readable spec

Authentication for Documentation

# Test authentication in Swagger UI
1. Click "Authorize" button
2. Enter: Bearer <your_jwt_token>
3. Test endpoints with authentication

---

🤝 Integration Patterns

Frontend Integration

// TypeScript client example
const response = await fetch('/api/v1/investigations/start', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    query: 'Analyze suspicious contracts',
    data_source: 'contracts',
    anomaly_types: ['price', 'vendor']
  })
});

Webhook Integration

# Receive investigation results via webhook
@app.post("/webhook/investigation-complete")
async def handle_investigation_complete(payload: dict):
    investigation_id = payload["investigation_id"]
    results = payload["results"]
    # Process results...

This API layer provides a robust, secure, and scalable interface for the Cidadão.AI platform, enabling efficient access to transparency analysis capabilities through well-designed RESTful endpoints.