README

🚀 Hybrid MySQL-MongoDB synchronization for high-performance Laravel microservices

Query MongoDB with automatic MySQL fallback, relationship auto-detection, field selection, eager loading, and production-ready security.

📋 Table of Contents

Features
Requirements
Installation
Configuration
Quick Start
Usage Guide
API Reference
Performance
Security
Troubleshooting
License

✨ Features

🎯 Core Features

✅ Hybrid Queries: MongoDB-first with automatic MySQL fallback
✅ Auto-Detection: Automatic Eloquent relationship detection via Reflection
✅ Field Selection: Reduce payload by 50%+ with selective field loading
✅ Eager Loading: Eliminate N+1 queries (6 queries → 2 queries)
✅ Type Safety: Full PHP 8.2+ type hints and strict types
✅ Zero Config: Works out-of-the-box with sensible defaults

🔒 Security (P1 Fixes)

✅ NoSQL Injection Prevention: Operator whitelist validation
✅ Input Sanitization: Auto-removal of dangerous characters ($, {}, etc.)
✅ DoS Mitigation: Maximum 1000 records per query
✅ Sensitive Data Protection: Auto-removal of passwords/tokens from API responses
✅ Field Validation: Blocks MongoDB operators in field names

⚡ Performance (P2 Optimizations)

✅ Configurable Cache: File, Redis, Database, Memcached support
✅ Batch Loading: Eager loading reduces queries by 50-70%
✅ Persistent Cache: Relationship metadata cached across requests
✅ Memory Efficient: < 1MB memory usage per request
✅ Query Optimization: Automatic index hints and projections

🛠️ Developer Experience

✅ Fluent API: Chainable query builder
✅ Facades: Laravel-style facades for clean code
✅ Logging: Comprehensive error logging and debugging
✅ PSR-4: Modern PHP standards

📦 Requirements

Component	Version
PHP	^8.2
Laravel	^11.0 \| ^12.0
MongoDB	^5.0
MySQL	^8.0
mongodb/laravel-mongodb	^5.0

🚀 Installation

1. Install via Composer

composer require neutrino-labs/laravel-mongodb-sync

2. Publish Configuration

php artisan vendor:publish --tag=mongodb-sync-config

3. Configure Environment

# MongoDB Connection
MONGODB_CONNECTION=mongodb
MONGODB_HOST=127.0.0.1
MONGODB_PORT=27017
MONGODB_DATABASE=your_database
MONGODB_USERNAME=your_username
MONGODB_PASSWORD=your_password

# MySQL Fallback
MONGODB_MYSQL_FALLBACK=true

# Cache Strategy (file, redis, database, memcached)
MONGODB_CACHE_DRIVER=file

# Cache TTL (seconds)
MONGODB_RELATION_CACHE_TTL=3600

⚙️ Configuration

config/mongodb-sync.php

<?php

return [
    // MongoDB connection from config/database.php
    'connection' => env('MONGODB_CONNECTION', 'mongodb'),

    // Enable automatic MySQL fallback
    'mysql_fallback' => env('MONGODB_MYSQL_FALLBACK', true),

    // Cache driver: file, redis, database, memcached, array
    'cache_driver' => env('MONGODB_CACHE_DRIVER', 'file'),

    // Relationship cache TTL (seconds)
    'relation_cache_ttl' => env('MONGODB_RELATION_CACHE_TTL', 3600),

    // Auto-detect Eloquent relationships
    'auto_detect_relations' => env('MONGODB_AUTO_DETECT_RELATIONS', true),

    // Model to collection mapping
    'collection_map' => [
        \App\Models\User::class => 'users',
        \App\Models\Post::class => 'posts',
        // Add your models here
    ],

    // Field selection per relationship (reduces payload by 50%+)
    'relation_select' => [
        'posts' => ['id', 'title', 'status', 'created_at'],
        'user' => ['id', 'name', 'email'],
    ],
];

🎯 Quick Start

Basic Query

use Neutrino\MongoDBSync\Services\MongoDBQueryService;

$mongo = new MongoDBQueryService('users');
$users = $mongo->where('status', '=', 'active')->get();

With Relationships

use Neutrino\MongoDBSync\Services\HybridQueryService;
use App\Models\User;

$hybrid = new HybridQueryService();
$user = $hybrid->findWithRelations(
    collection: 'users',
    modelClass: User::class,
    field: 'id',
    value: 1
);

// Auto-loads: $user->posts, $user->profile, etc.

Search API

use Neutrino\MongoDBSync\Services\SearchService;
use Illuminate\Http\Request;

$searchService = new SearchService($hybridQuery);
return $searchService->search(
    collection: 'users',
    modelClass: User::class,
    request: $request,
    perPage: 15
);

📖 Usage Guide

Basic Queries

Simple Queries

use Neutrino\MongoDBSync\Services\MongoDBQueryService;

$mongo = new MongoDBQueryService('users');

// Find by ID
$user = $mongo->find(1);

// Where clause
$users = $mongo->where('age', '>', 18)->get();

// Multiple conditions
$users = $mongo
    ->where('status', '=', 'active')
    ->where('age', '>=', 18)
    ->get();

// Select specific fields
$users = $mongo
    ->select(['name', 'email', 'created_at'])
    ->get();

// Limit and offset
$users = $mongo->limit(10)->offset(20)->get();

// Order by
$users = $mongo->orderBy('created_at', 'desc')->get();

Supported Operators

Operator	Description	Example
`=`	Equal	`where('age', '=', 25)`
`!=`, `<>`	Not equal	`where('status', '!=', 'banned')`
`>`	Greater than	`where('age', '>', 18)`
`>=`	Greater or equal	`where('score', '>=', 100)`
`<`	Less than	`where('price', '<', 50)`
`<=`	Less or equal	`where('stock', '<=', 10)`
`like`	Pattern match	`where('name', 'like', 'John%')`
`in`	In array	`where('id', 'in', [1,2,3])`
`nin`	Not in array	`where('status', 'nin', ['banned'])`
`gt`, `gte`, `lt`, `lte`	MongoDB native	`where('age', 'gt', 18)`

MySQL Fallback

use App\Models\User;

$mongo = new MongoDBQueryService('users');
$mongo->withMySQLFallback(User::class);

$user = $mongo->find(1);
// Tries MongoDB first, falls back to MySQL if not found

Relationships

Auto-Detection

use Neutrino\MongoDBSync\Services\HybridQueryService;
use App\Models\User;

$hybrid = new HybridQueryService();

// Automatically detects: posts(), profile(), comments(), etc.
$user = $hybrid->findWithRelations(
    collection: 'users',
    modelClass: User::class,
    field: 'id',
    value: 1
);

echo $user->name;
foreach ($user->posts as $post) {
    echo $post->title;
}

Load into Collection

$users = User::limit(10)->get();

$usersWithRelations = $hybrid->loadRelationsIntoCollection(
    records: $users,
    collection: 'users',
    modelClass: User::class
);

Field Selection

Reduce payload size by 50%+ by selecting only needed fields.

Global Configuration

// config/mongodb-sync.php
'relation_select' => [
    'posts' => ['id', 'title', 'status', 'created_at'],
    'user' => ['id', 'name', 'email', 'avatar'],
],

Per-Endpoint Override

class UserController extends Controller
{
    public function index()
    {
        return $this->searchService->search(
            collection: 'users',
            modelClass: User::class,
            request: $request,
            perPage: 15,
            relationSelect: [
                'posts' => ['id', 'title', 'status'], // Less fields for list
            ]
        );
    }

    public function show($id)
    {
        return $this->searchService->findById(
            collection: 'users',
            modelClass: User::class,
            id: $id,
            relationSelect: [
                'posts' => ['id', 'title', 'content', 'status', 'created_at'], // More fields for detail
            ]
        );
    }
}

Results

Before: 2,356 bytes (10 fields per post)
After:    992 bytes (4 fields per post)
Reduction: 57.9% 🎉

Search & Filters

Basic Search

use Neutrino\MongoDBSync\Services\SearchService;

$searchService = new SearchService($hybridQuery);

// GET /api/users?page=1&per_page=15
$response = $searchService->search(
    collection: 'users',
    modelClass: User::class,
    request: $request,
    perPage: 15
);

Advanced Filters

// GET /api/users?filters[0][field]=status&filters[0][operator]==&filters[0][value]=active
$request = new Request([
    'filters' => [
        ['field' => 'status', 'operator' => '=', 'value' => 'active'],
        ['field' => 'age', 'operator' => '>', 'value' => 18],
    ],
    'page' => 1,
    'per_page' => 15,
]);

$response = $searchService->search('users', User::class, $request);

Response Format

{
  "success": true,
  "data": {
    "items": [
      {
        "id": 1,
        "name": "John Doe",
        "email": "john@example.com",
        "posts": [
          {"id": 1, "title": "Post 1", "status": "published"}
        ]
      }
    ],
    "pagination": {
      "total": 100,
      "per_page": 15,
      "current_page": 1,
      "last_page": 7,
      "from": 1,
      "to": 15
    }
  }
}

Caching

Configuration

# File cache (default, no dependencies)
MONGODB_CACHE_DRIVER=file

# Redis cache (high performance)
MONGODB_CACHE_DRIVER=redis

# Database cache (persistent)
MONGODB_CACHE_DRIVER=database

Manual Cache Control

$hybrid = new HybridQueryService();

// Clear cache for specific model
$hybrid->clearRelationshipCache(User::class);

// Clear all relationship cache
$hybrid->clearRelationshipCache();

Cache Benefits

1st call: 1.08ms (with Reflection)
2nd call: 0.05ms (from cache)
Improvement: 95% faster 🚀

📚 API Reference

MongoDBQueryService

new MongoDBQueryService(string $collection)

// Query Building
->where(string $field, string $operator, mixed $value): self
->select(array $fields): self
->limit(int $limit): self
->offset(int $offset): self
->orderBy(string $field, string $direction = 'asc'): self

// Execution
->get(): array
->first(): ?object
->find(mixed $id): ?object
->count(): int

// MySQL Fallback
->withMySQLFallback(string $modelClass): self

HybridQueryService

new HybridQueryService()

// Find with relationships
->findWithRelations(
    string $collection,
    string $modelClass,
    string $field,
    mixed $value,
    array $relations = [],
    array $eloquentRelations = [],
    bool $withTrashed = false
): ?object

// Load relationships into collection
->loadRelationsIntoCollection(
    $records,
    string $collection,
    string $modelClass,
    array $relations = [],
    array $eloquentRelations = [],
    ?array $relationSelect = null
)

// Detect relationships
->detectRelationships(string $modelClass): array

// Cache control
->clearRelationshipCache(?string $modelClass = null): void

SearchService

new SearchService(HybridQueryService $hybridQuery)

// Search with pagination
->search(
    string $collection,
    ?string $modelClass,
    Request $request,
    int $perPage = 15,
    ?array $relationSelect = null
): JsonResponse

// Find by ID
->findById(
    string $collection,
    ?string $modelClass,
    int $id,
    bool $withRelations = true,
    ?array $relationSelect = null
): JsonResponse

// Find by field
->findBy(
    string $collection,
    ?string $modelClass,
    string $field,
    mixed $value,
    bool $withRelations = true
): JsonResponse

⚡ Performance

Benchmarks

Operation	Before	After	Improvement
N+1 Queries	6 queries	2 queries	67% reduction
Payload Size	2,356 bytes	992 bytes	57% reduction
Cache Hit	1.08ms	0.05ms	95% faster
Memory Usage	~5MB	<1MB	80% reduction

Best Practices

1. Use Field Selection

'relation_select' => [
    'posts' => ['id', 'title', 'status', 'created_at'],
],

2. Enable Caching

MONGODB_CACHE_DRIVER=redis # Use Redis in production
MONGODB_RELATION_CACHE_TTL=3600

3. Batch Operations

// ✅ Good: 2 queries with eager loading
$usersWithRelations = $hybrid->loadRelationsIntoCollection($users, 'users', User::class);

4. Index MongoDB Collections

// MongoDB Shell
db.users.createIndex({ "status": 1, "created_at": -1 })
db.posts.createIndex({ "user_id": 1 })

🔒 Security

Built-in Protections

1. NoSQL Injection Prevention

// ❌ Blocked
$mongo->where('email', '$where', 'malicious code'); // InvalidArgumentException

// ✅ Safe
$mongo->where('email', '=', 'john@example.com');

2. Field Name Validation

// ❌ Blocked: field names can't start with $
$mongo->where('$ne', '=', 'value');

// ✅ Safe
$mongo->where('status', '=', 'active');

3. DoS Protection

// Automatically limited to 1000 records max
$mongo->limit(5000)->get(); // Returns max 1000

4. Sensitive Data Removal

// Automatically removes from API responses:
// - password, remember_token, api_token
// - access_token, secret, secret_key

5. Input Sanitization

// Auto-removes dangerous characters: $, { }
// Limits string length to 500 chars
// Limits arrays to 1000 items

Security Checklist

✅ Operator whitelist (15 safe operators)
✅ Field name validation (no $ prefix)
✅ Array size limits (max 1000 items)
✅ Result limits (max 1000 records)
✅ Value sanitization
✅ Sensitive field removal
✅ Comprehensive logging

🐛 Troubleshooting

Relationships Not Loading

// Clear cache
$hybrid->clearRelationshipCache(User::class);

// Check model mapping in config/mongodb-sync.php
'collection_map' => [
    \App\Models\Post::class => 'posts', // ← Add this
],

Cache Not Working

# Use 'file' if Redis not available
MONGODB_CACHE_DRIVER=file

# Clear cache
php artisan config:clear
php artisan cache:clear

N+1 Queries

// Use eager loading
$users = $hybrid->loadRelationsIntoCollection($users, 'users', User::class);

// Enable query logging
DB::enableQueryLog();
dd(DB::getQueryLog());

📝 Changelog

[3.0.0] - 2026-01-30

Added

✨ Configurable cache strategy
✨ Eager loading (N+1 fix)
✨ Field selection in relationships (50%+ payload reduction)
🔒 NoSQL injection prevention
🔒 Input sanitization
🔒 DoS protection
⚡ Persistent cache

Changed

🔄 Refactored relationship loading
🔄 Enhanced security validations

Fixed

🐛 Cache driver compatibility
🐛 N+1 query problem
🐛 Memory leaks

📄 License

This package is open source software licensed under the MIT License. You are free to use, modify, and distribute it.

👥 Authors

Neutrino Soluções em Tecnologia
🌐 https://neutrino.dev.br
📧 neutrino@neutrino.dev.br

Made with ❤️ by Neutrino Soluções em Tecnologia

neutrino-labs / laravel-mongodb-sync

Maintainers

Details