Search by

conneqt / module-customer-assortment

epartment

N/A

Package info

git.dev.epartment.nl/conneqt/m2/customer-assortment-module

Type:magento2-module

pkg:composer/conneqt/module-customer-assortment

Statistics

Installs: 3 127

Dependents: 0

Suggesters: 0


README

What this module does

conneqt/module-customer-assortment adds customer-specific product visibility to Magento 2.

In short, the module lets you assign products to a customer-specific assortment and then uses that assortment to:

  • filter catalog search results
  • block direct access to product detail pages when a product is not allowed
  • support guest visibility rules for selected products and categories
  • expose REST endpoints for synchronizing assortments from an external system
  • optionally keep product flags and parent-product relations in sync through Magento's message queue

This is useful when not every customer should see the same catalog, or when an ERP/PIM/integration decides which SKUs a customer may buy or search for.

Release history is in CHANGELOG.md.

How it works

At a high level, the module works in four layers:

  1. Assortment storage
    Customer-to-product assignments are stored in the conneqt_customer_assortment table declared in etc/db_schema.xml.

  2. Sync and maintenance
    The helpers behind the REST API create, update, remove, and check assortment rows. When enabled, they also publish product update messages so Magento can update derived product state asynchronously.

  3. Storefront enforcement
    Magento search and product-view flows are intercepted so shoppers only see products they are allowed to access.

  4. Admin and configuration
    The module adds configuration fields and a customer admin tab so assortment behavior can be controlled and reviewed from the backend.

Request flow overview

1. Assortment data is stored per customer

The main table is etc/db_schema.xml:

  • customer_id: the external/customer identifier used by the module
  • product_sku: the Magento SKU that belongs to the customer assortment
  • external_id: optional external reference from an upstream system
  • personal_sku: optional customer-specific alias that can be searched

The module also stores customer-specific search terms in conneqt_customer_search_query.

Column types, keys and the index each query path relies on are documented under Database schema; read it before changing a column width or adding an index.

2. API calls or internal code update the assortment

The main write/read helpers are:

REST routes are declared in etc/webapi.xml, while dependency wiring for API implementations lives in etc/di.xml.

3. Product visibility is enforced on the storefront

The most important visibility flow is configured in etc/di.xml:

4. Search is extended with assortment-specific data

The module also extends Magento search behavior:

5. Queue consumers keep product metadata in sync

If automatic updates are enabled, Helper/ProductAdd.php publishes messages to the conneqt.assortment.product.update topic.

Queue configuration is defined in:

The queue handler Model/Queue/Handler/Handler.php can:

  • automatically toggle the customer_specific product attribute
  • automatically add parent products for configurable, bundle, and grouped products

Installation

Install the module with Composer:

composer require conneqt/module-customer-assortment

Then enable and upgrade Magento as usual:

bin/magento module:enable Conneqt_CustomerAssortment
bin/magento setup:upgrade
bin/magento cache:flush

If you use the asynchronous product update flow, make sure the Magento queue consumer for conneqt.assortment.product.update is running in your environment.

Requirements

  • PHP >= 8.1
  • magento/framework
  • magento/module-catalog-search ^102.0
  • cweagans/composer-patches
  • conneqt/m2-base >=1.0

These constraints are declared in composer.json.

Configuration

Configuration is available in the Magento admin at:

Stores → Configuration → Conneqt → Customer Assortment

The fields are declared in etc/adminhtml/system.xml.

Important settings

  • Customer Id Attribute
    Selects which customer attribute the module uses as the assortment identifier. This makes it possible to use an ERP/customer number instead of Magento's internal customer entity ID.

  • Automatically adjust customer specific
    When enabled, product visibility metadata is recalculated after assortment changes so products can be marked as customer-specific automatically.

  • Automatically add parent product to a customers assortment
    When enabled, parent products of assigned simple products can be added automatically through the queue handler.

  • No assortment full access
    Lets logged-in customers without any assortment rows see the full catalog. Guests are still handled by the guest visibility rules.

Product and category attributes added by this module

The data patches in Setup/Patch/Data create the attributes used by the visibility logic:

These attributes are part of the guest and customer-specific visibility checks performed in search and on the product detail page.

Admin UI

The module adds assortment-related admin UI for customer management:

Important files at a glance

FilePurpose
etc/db_schema.xmlDeclares the assortment and customer search query tables.
etc/di.xmlWires API preferences, Magento overrides, and plugins.
etc/webapi.xmlDeclares the REST endpoints for external integrations.
etc/adminhtml/system.xmlAdds the module configuration section in admin.
Helper/ProductAdd.phpMain entry point for inserting/updating assortment rows and publishing queue updates.
Helper/CustomerAssortment.phpReads current-customer assortment state from HTTP context and storage helpers.
Plugin/ElasticsearchPlugin.phpRestricts search results to allowed products and guest-visible items.
Plugin/ProductDetailBlockPlugin.phpBlocks direct PDP access for products outside the active shopper's assortment.
Model/Queue/Handler/Handler.phpProcesses async product updates after assortment changes.
view/adminhtml/ui_component/assortment_listing.xmlDefines the customer assortment grid in admin.

API reference

The REST API documentation has been moved to a separate file:

That file contains the endpoint list, request payloads, and response structure for:

  • add products
  • remove products
  • remove all products
  • get products
  • check whether a customer has a product

Search query integration and Mirasvit (read before touching the search plugins)

The search plugins (Plugin/ElasticsearchPlugin.php, Plugin/OpenSearchPlugin.php) run beforeQuery on the search client and rewrite the raw engine query array. Two clauses are injected: the assortment visibility conditions (always) and the personal-SKU nested clause (logged-in only). The personal-SKU clause depends on the shape of the query another module built, which makes it fragile. Know these facts:

  • The personal-SKU clause requires a query_string clause in bool.must. Only Mirasvit Search Ultimate's query builder (Mirasvit\SearchUltimate\SearchElastic\SearchAdapter\QueryBuilder) produces query_string. Plain core Magento builds match/match_phrase clauses instead, so without Mirasvit the personal-SKU clause is silently never addedgetSearchQuery() returns '' and addPersonalSku() no-ops. There is no error and no log line. If this module is ever used on a shop without Mirasvit Search Ultimate, getSearchQuery() must be extended to also extract the term from core's match clauses.
  • Mirasvit wraps the shopper's term in noise characters: searching 89.0689.065.0013 produces "query": "((((*89.0689.065.0013*))))". When extracting the raw term, strip all wrapping characters with a single order-independent charlist: trim(trim($searchTerm), "()\"'* \t\n\r\0\x0B"). Do not trim * and ( in separate sequential calls — trimming stars before parens leaves the inner stars in place, and escapeWildcardValue() then escapes them, producing a wildcard (*\*term\**) that requires literal asterisks and never matches. This exact bug shipped in 1.2.5 and made personal-SKU search return nothing — and it stayed unnoticed for a long time because a project-level searchable attribute containing the same alias values masked it (see Troubleshooting below).
  • Mirasvit score rules wrap the whole query in script_score. OpenSearchPlugin handles both shapes (getMustQuery()/getMustQueryByValue()); ElasticsearchPlugin assumes a plain bool root and would corrupt a script_score query. If Mirasvit score rules are activated on a shop using the ES7/ES8 client, port the OpenSearch handling first.
  • Mirasvit injects _misc subfields (copy_to: _search) into indexed fields, including the nested personal_skus entries. Personal-SKU text can therefore also match through generic full-text search (for every visitor, unscoped). This can make the customer-scoped nested clause look like it works when it doesn't. When verifying this module's search behavior, test with a logged-in customer AND a guest, and confirm the guest gets no personal-SKU match.
  • Debugging recipe: capture the real query with the ES index slowlog — PUT <index>/_settings {"index.search.slowlog.threshold.query.trace":"0ms"}, run one storefront search (mind FPC: add a throwaway query param), read the slowlog, then set the threshold back to -1. Code reading is not reliable here because the query shape is config/data-driven.

Indexed personal_skus data has its own trap: nothing invalidates catalogsearch_fulltext when assortment rows change (this module has no mview/indexer subscription), so personal SKUs added after the last full reindex are absent from the index until the affected products are reindexed.

Troubleshooting personal-SKU search

Work through these in order — the cheap data checks come first because in practice the cause is usually data or configuration, not code. This is especially true when the same code behaves differently between two environments.

  1. Rule out other match paths first. Check whether the shop has searchable product attributes that contain the same alias values — e.g. a legacy "customer/partner catalog numbers" text attribute filled by an ERP import (SELECT ea.attribute_code FROM catalog_eav_attribute cea JOIN eav_attribute ea ON ea.attribute_id = cea.attribute_id WHERE cea.is_searchable = 1 and inspect suspicious text attributes' values). Such an attribute matches personal SKUs through plain full-text search for every visitor, unscoped, which (a) masks a broken nested clause, (b) makes search behavior differ between environments whose attribute data differs even though code and config are identical, and (c) is usually a data leak that should be made non-searchable once this module's scoped search works.
  2. Check the right index. Every store view has its own index (<prefix>_product_<storeId>). Confirm which store id the storefront you are testing actually uses before inspecting documents — inspecting a healthy sibling store's index proves nothing.
  3. Check the document. GET <index>/_search with {"query":{"term":{"sku.keyword":"<sku>"}}} and _source: ["personal_skus"]. The entries must contain the expected customer_id value — which is the configured customer_id_attribute value as a string, not necessarily the Magento entity id. Remember the staleness trap above: rows added after the last reindex are not in the document.
  4. Check the mapping. personal_skus must be type: nested (forced by the Improved* client overrides). The mapping is only written at index creation, so the first install needs a full catalogsearch_fulltext reindex; a dynamic (object) mapping makes every nested query fail.
  5. Replay the module's clause manually. Run the nested term customer_id + wildcard personal_skus.sku query with curl against the exact index. If the manual query hits but the storefront doesn't, the module never added the clause (or built it wrong) — capture the live query with the slowlog recipe above and compare.
  6. Test hygiene. Use a product that is Visible Individually (configurable/grouped children never appear in results regardless of assortment state); test logged-in AND guest; bypass full-page cache with a throwaway query parameter (&nc=123) — a cached results page never hits the search engine, so code changes appear to have no effect.

Database schema

Both tables are declared in etc/db_schema.xml, with the matching entries in etc/db_schema_whitelist.json. Column widths are deliberately tight: on shops with large assortments conneqt_customer_assortment reaches millions of rows, and a declared VARCHAR width is charged in full against temporary tables and sort buffers even though InnoDB stores the value at its actual length on disk.

conneqt_customer_assortment

ColumnTypeNotes
entity_idint unsigned, auto-incrementSurrogate primary key. Used as the row identifier by the admin grid (view/adminhtml/ui_component/assortment_listing.xml) and by Model/ResourceModel/Post::deleteMultiple().
customer_idvarchar(64)Value of the attribute configured at conneqt_customer_assortment/index/customer_id_attribute, stored as a string — not necessarily Magento's customer entity id.
product_skuvarchar(64)Bounded by catalog_product_entity.sku, which Magento itself declares as varchar(64).
external_idvarchar(64)Free-form reference supplied by the upstream system over REST; not validated by this module.
personal_skuvarchar(64)Customer-specific SKU alias; same value class as product_sku.
created_at / updated_attimestampupdated_at is also written explicitly at Helper/ProductAdd.php:155.

Every varchar width above was chosen for a specific reason — read Why these column widths before changing one.

Keys:

KeyColumnsServes
PRIMARYentity_idRow identity for the admin grid and bulk delete.
CONNEQT_CUSTOMER_ASSORTMENT_CUSTOMER_ID_PRODUCT_SKU (unique)customer_id, product_skuEnforces one row per customer/SKU pair, and serves every customer-first lookup: ProductGet::get(), ProductGet::getSkuOnly(), ProductHas, ProductRemove, ProductAdd::addSingle().
CONNEQT_CUSTOMER_ASSORTMENT_PRODUCT_SKUproduct_skuProduct-first lookups. customer_id leads the unique key, so that key cannot serve a product_sku-only filter. Without this index Helper/ProductGet.php:49 (getPersonalSkus(), called once per product on every catalogsearch_fulltext reindex) scans the whole table, as do CustomerGet::get()/getIdOnly() and the orphan check at Helper/ProductRemove.php:40.

customer_id is not a foreign key and cannot be one — it holds the value of a configurable attribute, not an entity id — so rows are not removed when a customer is deleted. The same applies to product_sku and deleted products. See M2 below.

conneqt_customer_search_query

A per-customer variant of Magento's core search_query table, populated by Model/ResourceModel/CustomerSearchQuery.php. customer_id here is Magento's customer entity id (int unsigned, nullable for guests) — it is read from HTTP context at Helper/CustomerAssortment.php:97, not from the configurable assortment identifier. query_text and redirect keep the core table's varchar(255) (see Why these column widths); display_in_terms, is_active and is_processed are tinyint flags — they only ever hold 0 or 1, and tinyint is half the width of the smallint they replaced.

KeyColumnsServes
PRIMARYquery_idRow identity.
CONNEQT_CUSTOMER_SEARCH_QUERY_QUERY_TEXT_STORE_ID_POPULARITYquery_text, customer_id, store_id, popularityloadByQueryText() and getQueryId(), which filter on all three of query_text, store_id and customer_id.
CONNEQT_CUSTOMER_SEARCH_QUERY_IS_PROCESSEDis_processedBatch processing of unprocessed terms.
CONNEQT_CUSTOMER_SEARCH_QUERY_STORE_ID_POPULARITYstore_id, popularityPopular-terms listings, and — because store_id leads it — the store_id foreign key.

A separate single-column store_id index used to exist. It was a redundant left prefix of CONNEQT_CUSTOMER_SEARCH_QUERY_STORE_ID_POPULARITY and has been removed; the foreign key remains satisfied by that composite index.

Why these column widths

Recorded so the next optimisation pass does not have to re-derive this — and does not narrow these columns further on the assumption that nobody thought about it. The widths were set under ticket IMM-205, which proposed varchar(32) for customer_id and external_id and varchar(64) for the SKU columns. Two of the four landed wider than proposed, deliberately.

The distinction that drives every row below is whether a column's maximum length is hard (bounded by something in Magento or in this module, so a longer value cannot physically arrive) or soft (bounded by nothing — the column width is the only thing standing between an upstream system and the database).

ColumnWidthBoundWhere the bound comes fromWhy not narrower
product_sku64HardMagento declares catalog_product_entity.sku as varchar(64). No value that can exist as a SKU in the catalog can exceed it.Nothing left to take. This is the platform's own bound, and narrowing below it would reject SKUs the catalog accepts.
personal_sku64Soft, by conventionSame value class as product_sku — a customer-specific SKU alias supplied by the upstream system.The aliases are ERP article numbers rather than free text, but this module never validates them against catalog_product_entity, so the catalog's 64 is a convention here, not a guarantee. Holding it equal to product_sku keeps the two from ever disagreeing about what a SKU can look like.
customer_id64SoftHolds the value of whatever attribute is configured at conneqt_customer_assortment/index/customer_id_attribute.The deliberate deviation from the ticket's 32. The attribute is configurable, and Config/Field/Source/CustomerAttributes.php offers every text customer attribute in the admin dropdown — email included. A shop keyed on email would abort the ALTER at 32, and would keep failing on every longer address afterwards. The setting can also be changed after the column is narrowed, so the width has to survive a configuration nobody has chosen yet.
external_id64SoftFree-form reference accepted over REST at Helper/ProductAdd.php:148; never validated or length-checked by this module.Also 64 rather than 32, same reasoning: the only thing bounding this column is MySQL itself. A longer reference from the upstream system should not fail the sync endpoint. Narrowing is defensible if the ERP's reference format is known and fixed — measure live data first (next section).
query_text255Hard, by core parityCore search_query.query_text is varchar(255). What actually truncates a term before it reaches this column is catalog/search/max_query_length (Magento default 128), applied in QueryFactory::getPreparedQueryText() at Model/QueryFactory.php.Narrowing below the configured max query length silently truncates legitimate search terms. Keeping core's width means a term stored by core and by this module behave identically, and the config can be raised without a schema change.
redirect255Hard, by core parityCore search_query.redirect. It holds a URL.Core parity, and URLs are not short.

Two rules that follow, for whoever optimises next:

  • 64 is the floor for the SKU columns, not a target. product_sku is already at Magento's own bound; personal_sku is deliberately pinned to it.
  • customer_id and external_id can only be narrowed against live data, never against the schema, because nothing in the module bounds them. And a measurement of today's data is not a guarantee about tomorrow's: customer_id_attribute is an admin setting, and external_id comes from outside. If you narrow them, you are betting on configuration and upstream behaviour staying put.

The price of that safety is small: under utf8mb4, 32 fits in a one-byte length prefix (32 × 4 = 128 bytes) while 64 does not (256 bytes), so choosing 64 costs one byte per row per column — in the clustered index and once more in the unique index. On ten million rows that is roughly 20 MB, against the risk of a failed deploy or a configuration the module can no longer support. See What the column widths do and do not save for why the declared width barely affects on-disk size in the first place, and where the real gain is.

Applying the schema change to an existing installation

Narrowing a VARCHAR forces MySQL to rebuild the table (ALGORITHM=COPY), and the new product_sku index is built in the same pass. On a multi-gigabyte conneqt_customer_assortment this is not a quick operation and it blocks writes for its duration.

  1. Check that no value exceeds the new widths before deploying. With STRICT_TRANS_TABLES active — which Magento requires — an over-long value aborts the ALTER rather than truncating silently, so this is a deploy blocker, not a data-loss risk:

    SELECT
        MAX(CHAR_LENGTH(customer_id))  AS max_customer_id,
        MAX(CHAR_LENGTH(product_sku))  AS max_product_sku,
        MAX(CHAR_LENGTH(external_id))  AS max_external_id,
        MAX(CHAR_LENGTH(personal_sku)) AS max_personal_sku
    FROM conneqt_customer_assortment;
    

    Every result must be <= 64. If customer_id is the one that overflows, the shop has customer_id_attribute pointed at a long attribute — email is selectable in the admin dropdown (Config/Field/Source/CustomerAttributes.php offers every text customer attribute) — in which case raise the declared width rather than truncating data.

  2. Pause the assortment sync — the upstream REST feed and the conneqt.assortment.product.update consumer — for the duration of the upgrade.

  3. Run bin/magento setup:upgrade, then bin/magento setup:di:compile and bin/magento cache:flush.

  4. A reindex is not required: the search index mapping is unchanged.

What the column widths do and do not save

Worth stating precisely, because the intuition is usually wrong:

  • On disk, almost nothing. InnoDB stores a VARCHAR at its actual length, never padded to the declared maximum. varchar(255)varchar(64) saves roughly one byte per row, and only where the declared maximum now fits in 255 bytes — under utf8mb4 that means 63 characters or fewer, so varchar(64) in fact keeps the two-byte length prefix.
  • In query memory, a great deal. MySQL allocates the full declared width for in-memory temporary tables and sort buffers. The admin flows at Ui/DataProvider/Product/AssortmentDataProvider.php:73 and Block/Adminhtml/Edit/Tab/View/Assortment.php:60 join this table into a product collection and then group the result. Four utf8mb4 varchar(255) columns reserve roughly 4 KB per temporary-table row; at varchar(64) that is roughly 1 KB — often the difference between an in-memory temporary table and one spilled to disk.
  • The table file does shrink — but because of the rebuild, not the widths. The sync deletes and re-inserts rows in bulk (Helper/ProductAdd.php cleanup, ProductRemove::removeAll()), which leaves InnoDB pages heavily fragmented. The ALTER in step 3 rebuilds the tablespace and returns the free pages to the filesystem. (Unverified: how much is reclaimed depends on the shop's churn and cannot be predicted from the schema.)

Known issues & improvement points

Medium

M1. The surrogate primary key stores the natural key twiceetc/db_schema.xml

  • What: conneqt_customer_assortment has an auto-increment entity_id primary key plus a unique key on (customer_id, product_sku). InnoDB therefore holds customer_id and product_sku once in the clustered index and again, together with the primary key, in the secondary unique index.
  • Why it matters: this is the single largest available space saving on the table — on the order of a third of it — and it also costs a second B-tree write on every insert performed by the sync. Making (customer_id, product_sku) the primary key and dropping entity_id removes the duplicate entirely.
  • Suggested fix: not applied here because entity_id is load-bearing outside the schema: requestFieldName/primaryFieldName/indexField in view/adminhtml/ui_component/assortment_listing.xml, the row link in Ui/Component/Listing/Columns/CustomerActions.php:21, _init('conneqt_customer_assortment', 'entity_id') in Model/ResourceModel/Post.php:15, and the bulk delete at Model/ResourceModel/Post.php:82. All of those must move to the composite key first. Worth a ticket of its own.

M2. Orphan rows are never removedetc/db_schema.xml

  • What: neither customer_id nor product_sku can carry a foreign key — the first holds the value of a configurable attribute, the second a SKU string — so deleting a customer or a product leaves its assortment rows behind.
  • Why it matters: on a long-lived shop this is dead weight in the largest table, and orphan rows are still read by getSkuOnly() on every logged-in category page.
  • Suggested fix: a scheduled cleanup that deletes rows whose product_sku has no match in catalog_product_entity, and rows whose customer_id has no match in the configured customer attribute. Measure first — the row count tells you whether it is worth the cron.

M3. entity_id can exhaust its range under a churning syncetc/db_schema.xml

  • What: entity_id is int unsigned (maximum 4,294,967,295). The cleanup path in Helper/ProductAdd.php deletes and re-inserts assortment rows in bulk, and deleted auto-increment values are never reused.
  • Why it matters: a full re-sync of a multi-million-row assortment burns millions of values per run. At that rate the ceiling is reachable, and hitting it makes every insert fail.
  • Suggested fix: monitor with SELECT AUTO_INCREMENT FROM information_schema.TABLES WHERE TABLE_NAME = 'conneqt_customer_assortment'. Widening to bigint costs eight bytes per row in two indexes and is the wrong answer if M1 is done instead, since that removes the column altogether.

Low

L1. The personal-SKU filter is OR'd, so empty strings pass itHelper/ProductGet.php:53, Helper/ProductGet.php:72

  • What: addFieldToFilter('personal_sku', [['notnull' => true], ['neq' => '']]) passes a list of conditions for one field, which Magento joins with OR, producing personal_sku IS NOT NULL OR personal_sku != ''. Since != '' is unknown for NULL, the clause collapses to IS NOT NULL and empty-string aliases are not excluded.
  • Why it matters: empty personal SKUs are written into the personal_skus nested field of the search index, where they are inert but add index size and noise.
  • Suggested fix: use two addFieldToFilter('personal_sku', ...) calls, which Magento joins with AND, or a single ['gt' => ''] condition.

Audited and clean

  • conneqt_customer_search_query datatypes match their sources: store_id is smallint unsigned like store.store_id, customer_id is int unsigned like customer_entity.entity_id, and query_text/redirect keep the varchar(255) of core search_query (Magento's own maximum query length is configurable up to that width, so narrowing them would truncate legitimate terms).
  • num_results and popularity are int unsigned; both are unbounded counters, so no narrower type is safe.
  • created_at is not dead — Block/Adminhtml/Edit/Tab/View/Assortment.php:46 uses it as the grid's default sort.
  • The store_id foreign key on conneqt_customer_search_query survives the removal of the single-column store_id index, because CONNEQT_CUSTOMER_SEARCH_QUERY_STORE_ID_POPULARITY has store_id as its leading column and MySQL accepts that as the required index.
  • The legacy customer_id foreign key to customer_entity is absent from etc/db_schema.xml but retained in etc/db_schema_whitelist.json; that entry must stay so installations still carrying the constraint are allowed to drop it.

Limitations and notes

This module focuses on catalog search and direct product access. Some storefront areas may still need custom integration if they load products outside the standard search and product-view flows.

In particular, review and test:

  • custom product blocks or widgets
  • related products / upsells / cross-sells
  • layered navigation counts
  • configurable and grouped child-product visibility in custom themes
  • compatibility with third-party search, merchandising, or personalization modules

Also note that the module's customerId can be mapped to a custom customer attribute, so it does not have to match Magento's default internal customer entity ID.