georgebohnisch/laravel-elasticsearch-query-builder

:Query Elasticsearch by using Eloquent

Maintainers

Details

github.com/georgebohnisch/Laravel-Elasticsearch-Query-Builder

Homepage

Source

Installs: 112

Dependents: 0

Suggesters: 0

Security: 0

Stars: 0

Watchers: 1

Forks: 8

1.0.3 2019-09-03 19:20 UTC

Requires

Requires (Dev)

GPL-3.0-only dac2c75adcfe3aa3dfe55f9b6c002983ed3624ed

  • Shisun(Leo) Xia <shisun.xia.woop@hotmail.com>

elasticsearchlaravelLaravelElasticsearchQueryBuilder

This package is not auto-updated.

Last update: 2024-04-26 05:12:22 UTC

README

Query Elasticsearch by using Eloquent

This Laravel package is developed to simplify the process of querying Elasticsearch. Eloquent is a powerful tool to access and manipulate data in RDB. However, it is not designed to query no-sql DB like Elasticsearch. This package is made to fill the gap between the most popular ORM in Laravel and Elasticsearch.

The package only relies on the official PHP Elasticsearch package(v6).

PHP version

The package is developed and tested under PHP v7.1. It should be also compatible with v7.*. Please email me if you find any compatibility issue.

Elasticsearch version

The package is developed and tested under Elasticsearch v6.*. It should be also compatible with v5.*. Please email me if you find any compatibility issue. It is confirmed that the package does not support versions before v5.*.

Installation

  • Add the following to your composer.json file:

    "shisun/laravel-elasticsearch-query-builder": "dev-master"

  • Run composer update in terminal

  • Add the package to your app.php file in the config folder:

    'providers' => [
    ...
    Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilderServiceProvider::class,
]

  • If your Elasticsearch is not accessible at localhost:9200 then you have to publish the config file by running:

    $ php artisan vendor:publish --provider="Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilderServiceProvider"

  • (Optional) Change the ES_HOSTS in config/laravel-elasticsearch-query-builder.php to the address of your Elasticsearch

  • Use with Eloquent Model

    • Add EsTrait to your model 
      use Shisun\LaravelElasticsearchQueryBuilder\EsTrait;

class User extends Model
{
    use EsTrait;
}
    • Enable validation(Optional) 
      use Shisun\LaravelElasticsearchQueryBuilder\EsTrait;
class User extends Model
{
   use EsTrait;
   public $mappingProperties;
   public function __construct($attributes = []) {
       parent::__construct($attributes);
   
       $this->mappingProperties = [
           'id' => [
               'type' => 'integer',
           ],
           'name' => [
               'type' => 'text',
                   'index' => true,
           ],
           'email' => [
               'type' => 'keyword'
           ],
           'address' => [
               'type' => 'text'
           ],
           'float_example' => [
               'type' => 'float'
           ],
           'multi_fields_example' => [
               'type' => 'text',
               'fields' => [
                   'raw' => [
                       'type' => 'keyword'
                   ]
               ]
           ],
           'created_at' => [
               'type' => 'date',
               'format' => 'yyyy-MM-dd HH:mm:ss',
           ],
           'some_relations' => [
               'type' => 'nested',
               'properties' => [
                   ....
               ]
           ]
       ];
   }
}
    • Set model-level Index and Type name. Add follow two functions to your model.(Optional) 
      public function getIndexName() {
    return 'users';
}
// type name will be set to index name if this function is not defined.
public function getTypeName() {
    return 'users';
}

  • Use without Eloquent

    use Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilder as Builder;
$query = (new Builder())->setOptions([
    'index_name' => 'users',
    'type_name'  => 'users'
])->where('name', 'Leo')->get()->toArray();
// or if you only want to generate the query without getting results
$query = (new Builder())->where('name', 'Leo')->getBody()

Usage

Table of Contents

Init & Configs

__construct

  • You do not need to use this function if you use Eloquent

  • Parameters

    Name Required Type Default Description
    model Eloquent,null null

  • Output

    self

  • Examples

    1. basic example 
      use Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilder as Builder;
$builder = new Builder();

setOptions

  • You do not need to use this function if you use Eloquent

  • Parameters

    Name Required Type Default Description
    options Y array

  • Output

    self

  • Examples

    1. Basic 
      use Shisun\LaravelElasticsearchQueryBuilder\LaravelElasticsearchQueryBuilder as Builder;
$builder = (new Builder())->setOptions([
    'index_name' => 'users',
    'type_name'  => 'users',
    'validation' => false,
    'mapping_properties' => [check the example in the installation section]
]);

Query Clauses

where

  • Parameters

    Name Required Type Default Description
    column Y callable,string
    operator string null =,>,<,<=,>=,!=,*
    value mixed null
    or bool false
    boost bool,int false The weight of the column

  • Output

    self

  • Examples

    1. = can be ignored 
      User::es()->where('id', 1)->first()
    2. * is the wildcard operator 
      // Find the user whose name starts with 'Leo'
User::es()->where('name', '*', 'Leo*')->first()
    3. column can be a function 
      User::es()->where(function($q) {
    $q->orWhere(...)->orWhere(...);
})->get()->toArray()

whereNull

  • Parameters

    Name Required Type Default Description
    column Y string Column or relation

  • Output

    self

  • Examples

    1. Find all users with no name 
      User::es()->whereNull('name')->get()
    2. Find all users who don't have address 
      User::es()->whereNull('Addresses')->get()

whereNotNull

  • Parameters

    Name Required Type Default Description
    column Y string Column or relation

  • Output

    self

  • Examples

    1. Find all users with a name 
      User::es()->whereNotNull('name')->get()
    2. Find all users who have at least one address 
      User::es()->whereNotNull('Addresses')->get()

orWhere

  • Parameters

    Name Required Type Default Description
    column Y callable,string
    operator string null =,>,<,<=,>=,like,!=,*
    value mixed null
    boost bool,int false The weight of the column

  • Output

    self

  • Examples

    1. = can be ignored 
      User::es()->orWhere('id', 1)->first()
    2. column can be a function 
      User::es()->orWhere(function($q) {
    $q->where(...)->where(...);
})->limit(1)->get()->toArray()

whereMatch

  • It is used to make fuzzy text search. This function should only be applied on text fields.

  • Parameters

    Name Required Type Default Description
    column Y string
    value mixed null
    options array [] matchquery options. Check elasticsearch Docs for references

  • Output

    self

  • Examples

    1. without options 
      User::es()->whereMatch('email', 'shisun@')->first()
    2. with options 
      User::es()->whereMatch('email', 'shisun@', [
       'query' => 'this will be overrided by $value',
       'operator' => 'and',
       'zero_terms_query' => 'all'
     ])->first()

whereDoesntMatch

  • Parameters

    Name Required Type Default Description
    column Y string
    value mixed null
    options array [] matchquery options. Check elasticsearch Docs for references

  • Output

    self

  • Examples

    1. without options 
      User::es()->whereDoesntMatch('email', 'shisun@')->first()
    2. with options 
      User::es()->whereDoesntMatch('email', 'shisun@', [
       'query' => 'this will be overrided by $value',
       'operator' => 'and',
       'zero_terms_query' => 'all'
     ])->first()

orWhereMatch

  • Parameters

    Name Required Type Default Description
    column Y string
    value mixed null
    options array [] matchquery options. Check elasticsearch Docs for references

  • Output

    self

  • Examples

    1. without options 
      User::es()->orWhereMatch('email', 'shisun@')->first()
    2. with options 
      User::es()->orWhereMatch('email', 'shisun@', [
       'query' => 'this will be overrided by $value',
       'operator' => 'and',
       'zero_terms_query' => 'all'
     ])->first()

whereHas

  • Parameters

    Name Required Type Default Description
    relation Y string, callable Must be capitalized
    closure callable, null null from and to cannot be both null
    or boolean null for internal use. See orWhereHas
    boost boolean,integer false Used to adjust the weight of the condition

  • Output

    self

  • Examples

    1. basic example 
      // find all users with active someRelations
User::es()->whereHas('someRelations', function($q) {
    $q->where('status', 'active');
})->first();
    2. first parameter can be a function 
      // find all users with either active someRelations or red someOtherRelations
// Note: the parent 'whereHas' is interchangable with 'where' clause
User::es()->whereHas(function($q) {
    $q->orWhereHas('someRelations', function($k) {
         $k->where('status', 'active');
    })->orWhereHas('someOtherRelations', function($k) {
         $k->where('color', 'red');
    });
})->first();

orWhereHas

  • Parameters

    Name Required Type Default Description
    relation Y string, callable Must be capitalized
    closure callable, null null from and to cannot be both null
    boost boolean,integer false Used to adjust the weight of the condition

  • Output

    self

  • Examples

    1. basic example 
      // find all users with either active someRelations or red someOtherRelations
// Note: the parent 'whereHas' is interchangable with 'where' clause
User::es()->whereHas(function($q) {
    $q->orWhereHas('someRelations', function($k) {
         $k->where('status', 'active');
    })->orWhereHas('someOtherRelations', function($k) {
         $k->where('color', 'red');
    });
})->first();

whereHasNull

  • Parameters

    Name Required Type Default Description
    relation Y string Must be capitalized
    closure null, callable null

  • Output

    self

  • Examples

    1. basic example 
      // find all users with no someRelation
User::es()->whereHasNull('someRelations')->first();
    2. with sub query 
      // find all users with no someRelation with id = 1
User::es()->whereHasNull('someRelations', function($q) {
    $q->where('id', 1);
})->first();

orWhereHasNull

  • Parameters

    Name Required Type Default Description
    relation Y string Must be capitalized
    closure null, callable null

  • Output

    self

  • Examples

    1. basic example 
      // find all users either with no someRelation or named as 'Leo'
User::es()->where(function($q) {
    $q->orWhereHasNull('someRelations')->orWhere('name', 'Leo');
})->first();

whereIn

  • Parameters

    Name Required Type Default Description
    column Y string
    values Y array

  • Output

    self

  • Examples

    1. basic example 
      // find all users with pending or active status
User::es()->whereIn('status', ['active', 'pending'])->get();

orWhereIn

  • Parameters

    Name Required Type Default Description
    column Y string
    values Y array

  • Output

    self

  • Examples

    1. basic example 
      // find all users with either pending/active status or with name Leo
User::es()->where(function($q) {
    $q->orWhereIn('status', ['active', 'pending'])->orWhere('name', 'Leo');
})->get();

whereNotIn

  • Parameters

    Name Required Type Default Description
    column Y string
    values Y array

  • Output

    self

  • Examples

    1. basic example 
      // find all users that are not in pending or active status
User::es()->whereNotIn('status', ['active', 'pending'])->get();

orWhereNotIn

  • Parameters

    Name Required Type Default Description
    column Y string
    values Y array

  • Output

    self

  • Examples

    1. basic example 
      // find all users with either not in pending/active status or with name Leo
User::es()->where(function($q) {
    $q->orWhereNotIn('status', ['active', 'pending'])->orWhere('name', 'Leo');
})->get();

whereBetween

  • Parameters

    Name Required Type Default Description
    column Y string
    from numeric null from and to cannot be both null
    to numeric null

  • Output

    self

  • Examples

    1. basic example 
      User::es()->whereBetween('id', 1, 5)->first()

orWhereBetween

  • Parameters

    Name Required Type Default Description
    column Y string
    from numeric null from and to cannot be both null
    to numeric null

  • Output

    self

  • Examples

    1. basic example 
      User::es()->orWhereBetween('id', 1, 5)->first()

Order

orderBy

  • Parameters

    Name Required Type Default Description
    column Y string
    order numeric asc either asc or desc
    script array,boolean false

  • Output

    self

  • Examples

    1. basic example 
      User::es()->orderBy('id', 'desc')->get()
    2. script 
      // If the category of the item is Laptop then use discount_price for ordering. Otherwise, use listing_price.
Item::es()->orderBy('id', 'desc', 
            ['lang' => 'painless', 'source' => "if(doc['category'].value == 'Laptops') {return doc['discount_price'].value;} else {return doc['listing_price'].value;}"])
      ->get()

Pagination

page

  • Parameters

    Name Required Type Default Description
    page Y integer
    per_page Y integer

  • Output

    self

  • Examples

    1. basic example 
      // get the first page. 25 users per page.
User::es()->page(1, 25)->get()

limit

  • Parameters

    Name Required Type Default Description
    limit Y integer

  • Output

    self

  • Examples

    1. basic example 
      // get 25 users.
User::es()->limit(25)->get()

offset

  • Parameters

    Name Required Type Default Description
    limit Y integer

  • Output

    self

  • Examples

    1. basic example 
      // skip first 25 users.
User::es()->offset(25)->get()

Aggregations

aggregate

  • Parameters

    Name Required Type Default Description
    name Y string
    agg Y callable

  • Output

    self

  • Examples

    1. basic example 
      Item::es()->aggregate('group_items_by_categories', function($q) {
    $q->groupBy('category_id');
})->get()->aggregations();
    2. with filters 
      // get all active and pending items. But only aggregate on the filtered items that are also red.
// Note: aggregate clause is only applied on the filtered items.
Item::es()->whereIn('status', ['active', 'pending'])->aggregate('categories', function($q) {
    $q->where('color', 'red')->aggregate('group_by', function($k) {
        $k->groupBy('category_id');
    });
})->get()->aggregations();
// this returns
// [
//     'categories' => [
//         'doc_count' => 50,
//         'group_by'  => [
//             'doc_count_error_upper_bound' => 0,
//             'sum_other_doc_count'         => 8,
//             'buckets' => [
//                 [
//                     'key' => 1,
//                     'doc_count' => 15
//                 ],
//                 ...
//             ]
//         ]
//     ]
//  ]
// the 'key' in buckets is one of the category_id

aggregateAll

  • Parameters

    Name Required Type Default Description
    name Y string
    agg Y callable

  • Output

    self

  • Examples

    1. with filters 
      // get all active and pending items. And aggregate on all red items.
// Note: aggregateAll clause is applied on all items regardless other queries.
Item::es()->whereIn('status', ['active', 'pending'])->aggregateAll('categories', function($q) {
    $q->where('color', 'red')->aggregate('group_by', function($k) {
        $k->groupBy('category_id');
    });
})->get()->aggregations();
// this returns
// [
//     'categories' => [
//         'doc_count' => 50,
//         'group_by'  => [
//             'doc_count_error_upper_bound' => 0,
//             'sum_other_doc_count'         => 8,
//             'buckets' => [
//                 [
//                     'key' => 1,
//                     'doc_count' => 15
//                 ],
//                 ...
//             ]
//         ]
//     ]
//  ]
// the 'key' in buckets is one of the category_id

aggregateOn

  • Parameters

    Name Required Type Default Description
    relation Y string Must be capitalized
    agg Y callable
    custom_name string null

  • Output

    self

  • Examples

    1. basic example 
      Item::es()->aggregateOn('someRelations', function($q) {
    $q->min('id');
})->get()->aggregations();
// this returns
// [
//     'some_relations' => [
//         'doc_count' => 50,
//         'min_id'  => [
//             'value' => 1
//         ]
//     ]
//  ]
// 'some_relations' will be replaced if custom_name is provided

groupBy

  • Parameters

    Name Required Type Default Description
    column Y string
    size integer 10 The limit of number of groups. Default to 10 groups

  • Output

    self

  • Check aggregate clause for examples

min

  • Parameters

    Name Required Type Default Description
    column Y string
    custom_name string null

  • Output

    self

  • Examples

    1. basic example 
      Item::es()->min('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->min('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'min_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'min_' + column

max

  • Parameters

    Name Required Type Default Description
    column Y string
    custom_name string null
    missing_value numeric null This value will be used to replace null values

  • Output

    self

  • Examples

    1. basic example 
      Item::es()->max('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->max('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'max_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'max_' + column

avg

  • Parameters

    Name Required Type Default Description
    column Y string
    custom_name string null
    missing_value numeric null This value will be used to replace null values

  • Output

    self

  • Examples

    1. basic example 
      Item::es()->avg('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->avg('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'avg_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'avg_' + column

sum

  • Parameters

    Name Required Type Default Description
    column string Empty
    custom_name string null
    missing_value numeric null This value will be used to replace null values
    script array null Check official Doc as reference

  • Output

    self

  • Examples

    1. basic example 
      Item::es()->sum('id')->get()->aggregations();
// or
Item::es()->aggregate('test', function($q) {
    $q->sum('id');
})->get()->aggregations();
// They both return. Note: the 'test' in the second example is ignored.
// [
//     'sum_id' => [
//         'value' => 1
//     ]
//  ]
// 'min_id' will be replaced if custom_name is provided. The format of the name is 'sum_' + column

Other Filters

minScore

  • Parameters

    Name Required Type Default Description
    score Y numeric Only get items with _score greater than this value

  • Output

    self

  • Examples

    1. basic example 
      User::es()->whereMatch('name', 'Leo', ['operator' => 'and'])->minScore(5)->get()

with

  • Parameters

    Name Required Type Default Description
    relations Y list include these relations in results. All relations are included by default.

  • Output

    self

  • Examples

    1. basic example 
      User::es()->with('Addresses', 'Company')->get()

withOut

  • Parameters

    Name Required Type Default Description
    relations Y list Exclude these relations in results. All relations are included by default.

  • Output

    self

  • Examples

    1. basic example 
      User::es()->withOut('Addresses', 'Company')->get()

Execute Query

get

scroll

  • Parameters

    Name Required Type Default Description
    scroll_alive string 5m This specifies the life span of the scroll session.
    scroll_size integer 500 This specifies the number of records retrieved from Elasticsearch per batch.
    json boolean false If encode the results from each batch in JSON format. Fetching data and put the data in PHP array can consume a lot of memory. This option can greatly reduce the memory usage.

  • Output

    array

  • Examples

    1. basic example 
      User::es()->scroll()

count

  • Output

    integer

  • Examples

    1. basic example 
      User::es()->count()

first

  • Parameters

    Name Required Type Default Description
    eloquent boolean false get result in array or Eloquent. Returns array if not provided.

  • Output

    array|Eloquent

  • Examples

    1. basic example 
      User::es()->first()

find

  • Parameters

    Name Required Type Default Description
    key Y mixed

  • Output

    array

  • Examples

    1. basic example 
      // get the user with id = 5
// Note: the key_name can be set by setKeyName or setOptions. The key_name is 'id' by default
User::es()->find(5)

delete

  • Delete a record from elasticsearch. Returns false if the record does not exist.

  • Parameters

    Name Required Type Default Description
    key Y mixed

  • Output

    array

  • Examples

    1. basic example 
      // delete the user with id = 5
// Note: the key_name can be set by setKeyName or setOptions. The key_name is 'id' by default
User::es()->delete(5)

Results Manipulation

toArray

  • Output

    array

toEloquent

Warning: This function does not work if you don't use the package with Eloquent.

  • Output

    Eloquent

rawResults

Get raw results from Elasticsearch

  • Output

    array

aggregations

Get aggregation results

  • Output

    array

getAggregationBuckets

This is a helper function to get buckets from the aggregation specified by agg_name

  • Parameters

    Name Required Type Default Description
    agg_name Y mixed
    agg mixed null

  • Output

    array

  • Examples

    1. basic example 
      // if the aggregations are
// this returns
// [
//     'categories' => [
//         'doc_count' => 50,
//         'group_by'  => [
//             'doc_count_error_upper_bound' => 0,
//             'sum_other_doc_count'         => 8,
//             'buckets' => [
//                 [
//                     'key' => 1,
//                     'doc_count' => 15
//                 ],
//                 ...
//             ]
//         ]
//     ]
//  ]
// Then you can use getAggregationBuckets('categories') to get the buckets array

paginate

Returns pagination information

  • Parameters

    Name Required Type Default Description
    per_page mixed The value can be ignored if per_page is set by limit() or page(). Otherwise, it's required.

  • Output

    array

  • Example

    1. basic
    [
  "pages" => [
    1,
    2,
    3,
    4,
    5,
  ],
  "rows" => 165,
  "active" => 1,
  "totalpages" => 7.0,
  "prev" => false,
  "next" => true,
  "per_page" => 25
]

Direct Query Output

getQuery

Returns the query part of the body

getBody

Returns the body

getAggs

Returns the aggregation part of the body

Release History

Meta

Shisun(Leo) Xia - shisun.xia@hotmail.com

Distributed under the GNU V3 license. See LICENSE for more information.

https://github.com/ShisunXia/Laravel-Elasticsearch-Query-Builder