DAG Visualization for QA_LLM_Module

Overview

This enhancement adds DAG (Directed Acyclic Graph) visualization capability to the QA_LLM_Module, allowing users to see visual representations of robot task dependencies generated by the LLM.

Features

Automatic DAG Generation: Visualizations are generated automatically when the LLM outputs valid task JSON
IEEE-Style Formatting: Professional, publication-ready graphs
Hierarchical Layout: Tasks are organized by dependency levels
Color-Coded Nodes:
- 🔴 Red: Start tasks (no dependencies)
- 🟣 Purple: End tasks (no dependents)
- 🟠 Orange: Intermediate tasks
Detailed Information: Each task shows function name, assigned robots, and object keywords
Two Visualization Modes: Detailed and simplified versions
🔒 Safety Confirmation Workflow:
- Task plans require operator approval before deployment
- "Validate & Deploy Task Plan" button for safety compliance
- Visual status updates (Pending → Approved & Deployed)
- Prevents accidental execution of unverified plans

Installation

The required dependencies are already added to requirements.txt:

pip install matplotlib==3.9.4
pip install networkx==3.4

Usage

Automatic Integration with Safety Workflow

When you run the main application and input a query that generates task JSON:

DAG Generation: The visualization appears automatically with "Pending Approval" status
Safety Review: Operator reviews the task dependency graph
Confirmation: Click "🔒 Validate & Deploy Task Plan" to send to construction site
Deployment: Graph updates to "APPROVED & DEPLOYED" status

python3 main.py

Enhanced Safety Workflow Steps

🤖 LLM generates task plan → DAG shows "Pending Approval"
👨‍💼 Operator reviews visualization → Confirms task safety and correctness
📝 [Optional] Edit Task Plan → Manual JSON editing if modifications needed
🔄 [Optional] Update DAG Visualization → Regenerate graph with edits
🔒 Validate & Deploy Task Plan → Final approval and deployment to robots
✅ Confirmation displayed → DAG shows "APPROVED & DEPLOYED"

Three-Button IEEE-Style Workflow

📝 Edit Task Plan: Opens JSON editor for manual modifications
🔄 Update DAG Visualization: Regenerates graph from edited JSON
🔒 Validate & Deploy Task Plan: Final safety approval and deployment

Manual Testing

You can test the DAG visualization independently:

python3 test_dag_visualization.py
python3 dag_demo.py

API Reference

DAGVisualizer Class

from dag_visualizer import DAGVisualizer

visualizer = DAGVisualizer()

Methods

`create_dag_visualization(task_data, title="Robot Task Dependency Graph")`

Creates a detailed DAG visualization with full task information.

Parameters:

task_data (dict): Task data in the expected JSON format
title (str): Title for the graph

Returns:

str: Path to the generated PNG image, or None if failed

`create_simplified_dag_visualization(task_data, title="Robot Task Graph")`

Creates a simplified DAG visualization suitable for smaller displays.

Parameters:

task_data (dict): Task data in the expected JSON format
title (str): Title for the graph

Returns:

str: Path to the generated PNG image, or None if failed

Expected JSON Format

The visualizer expects task data in the following format:

{
  "tasks": [
    {
      "task": "task_name",
      "instruction_function": {
        "name": "function_name",
        "robot_ids": ["robot_id_1", "robot_id_2"],
        "dependencies": ["dependency_task_name"],
        "object_keywords": ["object1", "object2"]
      }
    }
  ]
}

Examples

Single Task

{
  "tasks": [
    {
      "task": "target_area_for_specific_robots_1",
      "instruction_function": {
        "name": "target_area_for_specific_robots",
        "robot_ids": ["robot_dump_truck_01"],
        "dependencies": [],
        "object_keywords": ["puddle1"]
      }
    }
  ]
}

Multiple Tasks with Dependencies

{
  "tasks": [
    {
      "task": "excavate_soil",
      "instruction_function": {
        "name": "excavate_soil_from_pile",
        "robot_ids": ["robot_excavator_01"],
        "dependencies": [],
        "object_keywords": ["soil_pile"]
      }
    },
    {
      "task": "transport_soil",
      "instruction_function": {
        "name": "transport_soil_to_pit",
        "robot_ids": ["robot_dump_truck_01"],
        "dependencies": ["excavate_soil"],
        "object_keywords": ["pit"]
      }
    }
  ]
}

File Structure

QA_LLM_Module/
├── dag_visualizer.py          # Main visualization module
├── test_dag_visualization.py  # Test suite
├── dag_demo.py               # Demo script
├── gradio_llm_interface.py   # Updated with DAG integration
├── main.py                   # Updated Gradio interface
└── requirements.txt          # Updated dependencies

Integration Details

Modified Files

gradio_llm_interface.py:
- Added DAGVisualizer import
- Modified predict() method to generate visualizations
- Returns additional DAG image output
main.py:
- Added gr.Image component for DAG display
- Updated input/output mappings for DAG visualization
requirements.txt:
- Added matplotlib==3.9.4
- Added networkx==3.4

Troubleshooting

Common Issues

"No module named 'matplotlib'"
- Solution: pip install matplotlib networkx
"No tasks found or invalid graph structure"
- Solution: Ensure your JSON follows the expected format
Images not displaying in Gradio
- Solution: Check that the image paths are accessible and files exist

Debug Mode

Enable detailed logging by setting the log level:

from loguru import logger
logger.add("dag_debug.log", level="DEBUG")

Performance Notes

Visualizations are generated in temporary directories
Images are automatically cleaned up by the system
Large graphs (>20 nodes) may take longer to render
Memory usage scales with graph complexity

Future Enhancements

Potential improvements for future versions:

Interactive Graphs: Clickable nodes with detailed information
Real-time Updates: Live visualization updates as tasks execute
Export Options: Support for PDF, SVG, and other formats
Custom Styling: User-configurable colors and layouts
Animation: Animated execution flow visualization