trunglv/magento2-concurrent_indexer

Magento2 concurrent indexer with spatie/fork

dev-main 2024-08-15 08:21 UTC

Requires

• php: >=8.0
• spatie/fork: *

OSL-3.0, AFL-3.0

• Luu Van Trung - Betagento

README

Purposes:

Magento 2 already supports multithreading for some indexer processes, but not all of them—specifically, the Product Flat Indexer. The purpose of this module is to:

1. Enable the Product Flat Indexer to run concurrently.

2. Replace the default multithreading functionality in Magento Core with the spatie/fork module. Although I haven't benchmarked it yet, I have a feeling that spatie/fork is faster and seems to be more modern and professional in terms of coding.

Installation

composer require trunglv/magento2-concurrent_indexer

Config

In etc/env.php, we should define MAGE_INDEXER_THREADS_COUNT (Magento2 Core) and a new one, BETA_CONCURRENT_INDEXER_THREADS_ENABLE.

!!!!If you want to use "spatie/fork" , you should enable BETA_CONCURRENT_INDEXER_THREADS_ENABLE

...
'MAGE_INDEXER_THREADS_COUNT' => 3,
'BETA_CONCURRENT_INDEXER_THREADS_ENABLE' => 1

How is multithreading in the Indexer implemented by Magento 2 Core?

CLASS \Magento\Catalog\Model\Indexer\Category\Product\Action

/**
     * Run reindexation
     *
     * @return void
     */
    protected function reindex(): void
    {
        $userFunctions = [];

        foreach ($this->storeManager->getStores() as $store) {
            if ($this->getPathFromCategoryId($store->getRootCategoryId())) {
                $userFunctions[$store->getId()] = function () use ($store) {
                    $this->reindexStore($store);
                };
            }
        }

        $this->processManager->execute($userFunctions);
    }

CLASS \Magento\Indexer\Model\ProcessManager

/**
     * Execute user functions
     *
     * @param \Traversable $userFunctions
     */
    public function execute($userFunctions)
    {
        if ($this->threadsCount > 1 && $this->isCanBeParalleled() && !$this->isSetupMode() && PHP_SAPI == 'cli') {
            $this->multiThreadsExecute($userFunctions);
        } else {
            $this->simpleThreadExecute($userFunctions);
        }
    }

How is multithreading in the Indexer implemented by my module?

CLASS \Betagento\ConcurrentIndexer\Plugin\Indexer\AroundProcessManager

if ($this->config->isEnabled() && $this->config->getThreadCount() > 1 && $this->config->isCanBeParalleled() && !$this->config->isSetupMode() && PHP_SAPI == 'cli') {

    \Spatie\Fork\Fork::new()->concurrent($this->config->getThreadCount())->run(...$userFunctions);
            return;
}

@trunglv Skype : [e-mail] - email : [e-mail]

vendor/bin/phpstan analyse app/code/Betagento/ConcurrentIndexer/ --level 9
 5/5 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%


                                                                                                                        
 [OK] No errors