bluefly/ai_agent_huggingface

HuggingFace AI agent with advanced model discovery and pipeline management

Maintainers

Details

gitlab.bluefly.io/llm/drupal/ai_agent_huggingface

Installs: 79

Dependents: 0

Suggesters: 1

Security: 0

Type:drupal-module

pkg:composer/bluefly/ai_agent_huggingface

v0.1.3 2025-10-12 01:57 UTC

Requires

  • php: >=8.1
  • bluefly/llm: ^0.1
  • drupal/ai: ^1.0
  • drupal/ai_agents: ^1.1
  • drupal/ai_provider_huggingface: ^1.0
  • drupal/ai_search: ^1.1
  • drupal/ai_vdb_provider_milvus: ^1.1
  • drupal/core: ^10.3 || ^11
  • drupal/search_api: ^1.0
  • drupal/search_api_solr: ^4.3
  • drupal/search_api_solr_dense_vector: ^1.0

Requires (Dev)

Suggests

  • drupal/ai_provider_huggingface: HuggingFace provider for AI module (^1.0)
  • drupal/key: Key management for secure API key storage
  • drupal/token: Token replacement system for dynamic values

GPL-2.0-or-later a314036fe1f587dc981f6f5b5686220ef83c2888

This package is auto-updated.

Last update: 2025-11-18 23:05:39 UTC

README

Drupal module for HuggingFace model management and pipeline execution, integrating with the Drupal AI ecosystem and ECA workflow engine.

Description

AI Agent HuggingFace extends the Drupal AI module ecosystem with comprehensive HuggingFace integration, providing model discovery, pipeline execution, fine-tuning capabilities, and workflow automation through ECA (Event-Condition-Action).

This module bridges HuggingFace's extensive ML model hub with Drupal's content management and workflow capabilities, enabling:

  • Automated model discovery and synchronization
  • Pipeline-based inference execution
  • Fine-tuning and model version management
  • RAG (Retrieval-Augmented Generation) workflows
  • Vector database integration for embeddings
  • ECA-driven AI workflows

Features

Core Capabilities

  • Model Management: Discovery, import, and lifecycle management of HuggingFace models
  • Pipeline Execution: Support for multiple pipeline types (text generation, classification, translation, summarization, question answering, image classification, object detection)
  • Fine-Tuning: Model training and fine-tuning orchestration
  • RAG Integration: Retrieval-Augmented Generation with vector database support
  • ECA Workflow Integration: Native ECA actions, conditions, and events for AI automation
  • API Key Management: Secure token storage via Drupal Key module
  • Performance Monitoring: Built-in analytics and quota management

Pipeline Types Supported

  • Text Generation
  • Text Classification
  • Translation
  • Summarization
  • Question Answering
  • Image Classification
  • Object Detection

ECA Integration

  • Actions: Fine-tuning, model execution, orchestration
  • Conditions: GPU availability checks, model status
  • Events: Model loaded, training completion, inference results

Requirements

Drupal Requirements

  • Drupal core: ^10.3 || ^11.0
  • PHP: >= 8.1

Required Modules

Optional Modules

  • llm - LLM platform integration
  • ai_search - RAG capabilities (https://www.drupal.org/project/ai_search)
  • ai_vdb_provider_qdrant - Qdrant vector database integration
  • search_api - Search API framework
  • search_api_solr - Solr backend
  • search_api_solr_dense_vector - Dense vector support
  • advancedqueue - Advanced queue processing
  • eck - Entity Construction Kit
  • key - Key management system
  • token - Token replacement system

Recommended Modules

  • eca_content - Content operations in ECA
  • eca_endpoint - API triggers for ECA
  • eca_tamper - Data transformation in ECA
  • ai_logging - Advanced AI operation logging
  • entity_browser - Enhanced entity selection
  • monitoring - System monitoring and alerts
  • admin_toolbar - Enhanced admin interface

Installation

Via Composer (Recommended)

composer require bluefly/ai_agent_huggingface
drush en ai_agent_huggingface -y

Manual Installation

  1. Download and extract the module to /modules/contrib/ai_agent_huggingface or /modules/custom/ai_agent_huggingface
  2. Install required dependencies via Composer:
    composer require drupal/ai drupal/ai_provider_huggingface drupal/eca drupal/ai_eca
  3. Enable the module:
    drush en ai_agent_huggingface -y

Post-Installation Configuration

  1. Configure HuggingFace API Token:

    • Navigate to Configuration > AI > HuggingFace Settings (/admin/config/ai/huggingface)
    • Add your HuggingFace API token (get one from https://huggingface.co/settings/tokens)
    • Recommended: Use the Key module for secure storage

  2. Discover Models:

    • Visit Configuration > AI > HuggingFace Models (/admin/config/ai/huggingface/models)
    • Use the model discovery interface to browse available models
    • Import models you wish to use

  3. Configure ECA Workflows (Optional):

    • Navigate to Configuration > Workflow > ECA (/admin/config/workflow/eca)
    • Create workflows using HuggingFace actions and conditions

  4. Set Up Vector Database (Optional, for RAG):

    • Install and configure ai_vdb_provider_qdrant or similar
    • Configure embedding models for vectorization

Configuration

Module Settings

Access module settings at: /admin/config/ai/huggingface/settings

  • API Configuration: HuggingFace API endpoints and authentication
  • Model Defaults: Default pipeline configurations
  • Performance Settings: Caching, timeouts, batch processing
  • Quota Management: API usage limits and monitoring

Drush Commands

# Model management
drush huggingface:discover              # Discover available models
drush huggingface:sync                  # Sync models from HuggingFace Hub
drush huggingface:list                  # List imported models

# Pipeline operations
drush huggingface:execute <model> <input>   # Execute model inference

# Fine-tuning
drush huggingface:finetune <model> <dataset>  # Start fine-tuning job

# Orchestration
drush ai:orchestrate <workflow>         # Run AI orchestration workflow

# Testing
drush huggingface:test                  # Run module tests

Usage Examples

Basic Model Inference

// Load the HuggingFace service
$huggingface = \Drupal::service('ai_agent_huggingface.service');

// Execute text generation
$result = $huggingface->execute([
  'model' => 'gpt2',
  'task' => 'text-generation',
  'input' => 'Once upon a time',
]);

ECA Workflow Integration

Create an ECA model that:

  1. Triggers on content creation
  2. Uses HuggingFace to generate summaries
  3. Saves summary to a field

RAG Implementation

// Configure RAG pipeline
$rag_service = \Drupal::service('ai_agent_huggingface.rag');
$rag_service->setEmbeddingModel('sentence-transformers/all-MiniLM-L6-v2');
$rag_service->setGenerationModel('google/flan-t5-base');

// Retrieve and generate
$result = $rag_service->query('What is Drupal?', [
  'context_sources' => ['node:article', 'node:page'],
  'max_contexts' => 5,
]);

Permissions

The module provides the following permissions:

  • Administer HuggingFace settings: Configure module settings
  • Manage HuggingFace models: Import, update, delete models
  • Execute HuggingFace pipelines: Run model inference
  • View HuggingFace analytics: Access usage statistics

Architecture

Service Layer

  • ai_agent_huggingface.service - Main HuggingFace integration service
  • ai_agent_huggingface.model_discovery - Model discovery and synchronization
  • ai_agent_huggingface.pipeline_manager - Pipeline execution management
  • ai_agent_huggingface.orchestration - AI workflow orchestration
  • ai_agent_huggingface.rag - RAG pipeline service

Entity Types

  • HuggingFace Model: Configuration entities for imported models
  • Custom content entities for training datasets and results

Plugin Types

  • AiAgent Plugin: HuggingFace agent implementation
  • AiProvider Plugin: HuggingFace provider with tokenization
  • ECA Action Plugins: Fine-tuning, orchestration actions
  • ECA Condition Plugins: GPU availability, model status checks
  • ECA Event Plugins: Model lifecycle events
  • Queue Worker Plugins: Async workflow execution
  • AdvancedQueue Job Types: Model download jobs

Testing

See TESTING.md for comprehensive testing documentation.

# Run all tests
vendor/bin/phpunit -c web/core modules/custom/ai_agent_huggingface

# Run specific test suite
vendor/bin/phpunit modules/custom/ai_agent_huggingface/tests/src/Unit
vendor/bin/phpunit modules/custom/ai_agent_huggingface/tests/src/Kernel
vendor/bin/phpunit modules/custom/ai_agent_huggingface/tests/src/Functional

Troubleshooting

API Connection Issues

Model Loading Failures

  • Ensure model ID is correct and model exists on HuggingFace Hub
  • Check model access permissions (some models require approval)
  • Verify sufficient memory for model loading

Performance Issues

  • Enable caching for frequently used models
  • Configure quota limits to prevent API overuse
  • Use background queue processing for heavy operations

Support

Contributing

Contributions are welcome! Please follow Drupal coding standards and include tests for new features.

License

GPL-2.0-or-later

Credits

Related Modules