Requirements

  • cURL is installed and enabled
  • Version Matrix
  • ElasticSearch Version PHP Version Required
    7.x 7.1+
    6.x 7.0+
    5.x 5.5+
    2.x 5.5+
  • Please read this aritcle for How to Upgrade Your Website PHP Version
  • How to check PHP version is running on your website
    • From the Backend of Joomla site
    • Go to System → System Information: this screen contains all system information you may need, the Joomla! Version currently running, and also PHP, database versions and so on.
    • system-infomation.jpg

Configure Searchly service

Searchly.com provides powerfull features to fulfill your Elasticsearch deployment with a reasonable price. On this section, we will guide you how to create and configure Elasticsearch service on Searchly.

If you already have Elasticsearch service installed on your server or use Elasticsearch from another provider, you can use that one and skip this step. However, you need to manage permission to access your service by yourself.

Create Searchly account

  • Open http://www.searchly.com/
  • Select a plan that suits your needs
  • It has a free "Starter" plan that enables you to create up to 10 Search spaces, and each can create 2 indices with 20MB of storage that can index up to 5K documents.
  • Sign up and verify your account.

Create Search Space

  • Once your account is verified, you can login its dashboard.
  • You will be redirected to the page where you can create Search Space for the first time login.
  • searchly-1-create-search-space.jpg
  • Type a name for Search Space, select Region, and select version 7 (recommend), version 6, version 5 or version 2 for Elasticsearch version.
  • Choose Create

Create Indices

  • Searchly will grant access to specific indices. Hence, you need to create indices first, then create access key and grant access to them.
  • searchly-1-manage-indices.jpg
  • Select Dashboard > Indices to open the Indices Manager page where you can create an index.
  • Chose New Index
  • On Create New Index form Enter index name
  • searchly-1-create-index.jpg
  • Choose Create button

Create Access Keys and Set up Access Policy

You need to create 2 different Access Keys to index, one is for index request in back-end and one is for search on front-end. Select ACL > Access Keys to open the Access Keys Manager page where you can create an access keys.

searchly-2-manage-access-key.jpg

Create Private Access Key for Index request

  • Choose Create New Access Key
  • searchly-3-create-private-access-key-for-indexing-data.jpg
  • Type Access Key Name
  • Uncheck for Read Only option
  • Select indices that you grant access for.
  • Choose Create

Create Public Access Key for Search request

  • Choose Create New Access Key
  • searchly-4-create-public-access-key-for-searching.jpg
  • Type Access Key Name
  • Check for Read Only option
  • Select indices that you grant access for.
  • Choose Create

Get Connection Urls

  • searchly-5-manage-keys.jpg
  • Click on Details button to open page where you can view details about your Access Key and get Connection Url
  • searchly-5-get-connection-url.jpg

Configure Integration

  • Copy Connection URLs of Private Access Key and Public Access Key and enter them correspondingly to Endpoint and Search Endpoint params on Configuration form of Geek Elasticsearch component.
  • Enter index name what you created to Index param.
  • searchly-6-config-service.jpg
  • Save changes.

Configure Amazon ElasticSearch Service

If you already have Elasticsearch service installed on your server or use Elasticsearch from another provider, you can use that one and skip this step. However, you need to manage permission to access your service by yourself.
On this documentation, we will guide you to create Amazon Elasticsearch service and manage its access policy.

If you are looking for an alternative provider of hosted search solution which is away from AWS, Searchly.com is an option that offer you a good service with a reasonable price.

Create AWS Account

  • If you already have an AWS account, you can skip this step.
  • When you sign up for Amazon Web Services (AWS), your AWS account is automatically signed up for all services in AWS, including AWS Elasticsearch Service (Amazon ES).
  • To create an AWS Account
  • Open http://aws.amazon.com/, and then choose Create an AWS Account.
  • Follow the instructions to sign up. You need to enter payment information before using Amazon ES. There are no sign-up fees, and charges are not incurred until you create a Elasticsearch domain. You have to pay for the resources that you use only.

Create IAM Account

Services in AWS including  Amazon ES, require you to provide credentials when you access them, so that the service can determine whether you have permissions to access the resources owned by that service. You can create access keys for your AWS account to access the API. However, we don't recommend you to access AWS using the credentials for your AWS account. We recommend that you use AWS Identity and Access Management (IAM) instead.

To create an IAM user.

  • Go to Console Home page
  • Under Security & Identity, choose Identity & Access Management
  • create-iam-user-0.jpg
  • Or you can directly sign in to the Identity & Access Management (IAM) console at https://console.aws.amazon.com/iam/.
  • On navigation pane, choose Users -> Create New Users.
  • create-iam-user-2.jpg
  • In input text 1, type a username and check the checkbox Generate an access key for each user
  • Choose Create
  • Save Access key ID + Access Secret Key to the safe place on your computer, or you can click on Download Credentials button to download csv file.
  • create-iam-user-3.jpg
  • Choose Close to return back to list of users page
  • On  list of users, choose the name (not the checkbox) of the user you  created, you can use Search box to search for username.
  • create-iam-user-4.jpg
  • On user detail page, Save User Amazon Resource Name (User ARN), you can also create another Access Key in this page
  • create-iam-user-5.jpg

Create ElasticSearch Domain

  • Go to https://aws.amazon.com
  • Sign in to the console with your AWS account
  • Select Region where you want to create Elasticsearch Service in.
  • You can select region from Dropdown menu item (next to Support menu item) on the top menu
  • select-a-region.jpg
  • Tips: you should select regions near your customers to reduce data latency of your applications.
  • Under Analytics, choose Elasticsearch Service.
  • choose-elasticsearch-service.jpg

Step 1. Choose Deployment Type

Choose Deployment Type and select the Elasticsearch version for your domain.

create-aws-domain-step-1.jpg

Step 2. Configure Domain

  • Domain Name: On the Define domain page, for Domain name, type a name for the domain. The name must start with a lowercase letter and must be between 3 and 28 characters. Valid characters are a-z (lowercase only), 0-9, and - (hyphen).
  • For Instance type, choose an instance type for the Amazon ES domain.
  • AWS Free tier
  • Choose Next

create-aws-domain-step-2.jpg

Step 3. Configure access and security

  • Network Connection: Choose Public Access
  • Select Custom Access Policy, then use a Custom Policy Builder below to add element with following settings:
    • Column 1: Select IAM User
    • Column 2: Enter IAM User ARN (created above)
    • Column 3: Select Allow
  • Choose Next

create-aws-domain-step-3.jpg

Step 4. Review

Review the information, and then choose Confirm to create a domain.

create-aws-domain-step-4.jpg

Step 5. Get AWS ES Endpoint

  • Wait for about 10 minutes for the domain to be initialized.
  • Once your AWS Elasticsearch domain is created successfully, you can get Endpoint URL of Elasticsearch domain.

create-aws-domain-step-5.jpg

Step 6. Modify Access Policy

  • In the Step 3. Configure access and security, we configured access to allow IAM User to have full access to domain. But in our case, everyone can also search data from front-end of website, so we will have to edit the access policy to do that.
  • Choose Actions > Modify Access Policy

create-aws-domain-step-6.jpg

  • Copy and paste the JSON code below to replace the content in the textbox Add or edit the access policy
  • Replace the [IAM_ARN] with your IAM user ARN, and replace [ELASTICSEARCH_ARN] with Elasticsearch ARN.
  • Choose Submit to save changes.

Updated on May 28th, 2021:

AWS no longer enables anonymous access to the domain unless you add an IP-based condition to limit access from specific IPs or IP range. So, you will have to add IP-based condition to Access Policy code to grant HTTP GET and HTTP POST requests from your web server to Elasticsearch domain.

In the sample Access Policy code below, it will enable access from the specified IP range: 192.0.2.0 to 192.0.2.24
You have to replace this IP address with public IP address of your web server.

If you don't know what is public IP address of your website, you can create the file checkip.php, add content below to created file, then upload that file to root folder your Joomla website and open it on your browser, you will find an IP displayed there.

Note: Since the Elasticsearch domain will only allow access from IP of your website, therefor you will need to enable the setting "Send Indirect Request" in confirguration page of component.

<?php
echo file_get_contents('https://checkip.amazonaws.com/');
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "[IAM_ARN]"
      },
      "Action": "es:*",
      "Resource": "[ELASTICSEARCH_ARN]/*"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "es:ESHttpGet",
        "es:ESHttpPost"
      ],
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [
            "192.0.2.0/24"
          ]
        }
      },
      "Resource": "[ELASTICSEARCH_ARN]/*"
    }
  ]
}

create-aws-domain-step-7.jpg

Avoiding Unexpected Charges

Install Component

  • Download install package from our download section here
  • Install it as normally via Joomla installer, it will install Geek Elasticsearch component and its all sub extensions.

Configuration

  • Configure component
  • How to access.
  • Click on Options button at any screen of Geek Elasticsearch admin to access its configuration form.

ElasticSearch Service

Setting Description

Elasticsearch Version

Select a version of Elasticsearch that you use. Depend on Elasticsearch service that you use, It can be a version of Elasticsearch installed on your server, or of AWS Elasticsearch Domain or of Searchly Search Space.

Check Version Matrix first to select the ElasticSearch version that matches the version of PHP installed on your server.

Endpoint

An endpoint is a URL that is the entry point for an Elasticsearch service.
- If you use AWS Elasticsearch service, you can get Endpoint url here.
- If you use Searchly service, It is a Connection URL for index request.

Search Endpoint

For Elasticsearch service that provide different endpoint urls for search request and index request, such as Searchly.
If you use Searchly service, you can get Connection URL for search request here.
If you use other Elasticsearch services, you can leave this field blank.

Index

Name of index where your data will be stored in.

NOTE: an index is like a “database” in a relational database, you must use different index for each sites if you have multiple ones.

AWS Region

Select a AWS region where you create an ElasticSearch service in - See Create Elasticsearch Domain section (for AWS ElasticSearch service only).

If you use other-AWS ElasticSearch service, you can ignore this setting.

Username

Enter the username for the user. If you use AWS ElasticSearch service, it can be a username of your AWS account or Access Key Id of IAM user.

Password

Enter the password for the user. If you use AWS ElasticSearch service, it can be a password of your AWS account or Secret Access Key of IAM user.

Send Indirect Request (Send search request through server script) By default, the search module will send direct AJAX request to ElasticSearch service.
But if ElasticSearch service is behind a Firewall and it only allows access from specific IPs, then the direct ajax request from client won't work.
In this case, you need to enable this setting, then the search module will send ajax request to your server, after then the server script will send request to your ElasticSearch service.
NOTE: If you enable this setting, you should create a new menu item with type “Geek ElasticSearch > Search Endpoint”. This will help to create SEF url for search endpoint.
create-menu-item.jpg
create-menu-item-2.jpg

Index and search

  • options-tab-2.jpg
Setting Description

Last run index time

The last time when index runs successfully.

Indexer Batchsize

The batch size controls how many items are processed per batch. Large batch sizes require lots of memory whereas small batch sizes require less memory but execute more requests which tends to take longer.

Title Text Weight Multiplier

The multiplier is used to control how much influence matching text has on the overall relevance score of a search result. A multiplier is considered in relationship to the other multipliers. The title text comes from the title of the content.

Body Text Weight Multiplier

The multiplier is used to control how much influence matching text has on the overall relevance score of a search result. A multiplier is considered in relationship to the other multipliers. The body text comes from the summary and/or body of the content.

Path Text Weight Multiplier

The multiplier is used to control how much influence matching text has on the overall relevance score of a search result. A multiplier is considered in relationship to the other multipliers. The path text comes from the SEF URL of the content.

Enable Logging

Enable this option to create a log file in your site's logs folder during the index process. This file is useful for troubleshooting issues with the index process. It is recommended that logging be disabled unless necessary.

Search Analytics

  • options-tab-2.jpg
Setting Description

Collect Search Terms

Enable this option to collect search terms submitted by users. Then you can filter and analyze search terms on this page.

Permissions

  • options-tab-3.jpg

Enable indexing plugins

  • The Indexing plugins are which the logic is built to search for items from specific component and return them to Indexer to update the ElasticSearch index. Without these plugins, data from a component cannot be indexed and processed during searching from the Geek ElasticSearch search form.
  • On Dashboard of component, click on Manage button on Indexing Plugins section to open plugin manager pages where you can enable / disable plugins associated with components that you want their data will be indexed in Elasticsearch.
  • IMPORTANT NOTE!: enables only indexing plugins for 3rd components what are installed on your site.
  • manage-indexing-plugins.jpg

Disable Finder (Smart Search) plugins

  • Because we will use Geek Elasticsearch to replace Joomla Smart Search,  it is no longer need to update Joomla! Smart Search index and you can disable all Finder plugins but Smart Search - Geek ElasticSearch to ignore unnecessary process.
  • disable-finder-plugins.jpg

Trigger Events Plugins

  • These plugins provide functions which are triggered during the content creation process of Joomla! components and immediately update the ElasticSearch index.

Run Indexer

  • Once you completely setup Elasticsearch service and configure component as well as enable indexing plugins, you need to run indexer to index data in Elasticsearch.
  • There are some ways and options to run indexer:

Run full index:

  • Click on Index button on toolbar at any screen of component. It will re-index all content of your site.

Run incremental index:

  • Click on Incremental Index button on toolbar at any screen of component. It will index only new content and update content from Last indexing time run .

Setup cron job:

  • Geek ElasticSearch supports Trigger Events Plugins that help ElasticSearch index up-to-date automatically whenever content items are amended. However, there are some 3rd components not support these Trigger Events plugins to trigger its content creation process or there are some circumstances which you need to re-run the indexer.
  • You can do this manually using Index / Incremental Index toolbar button on the Geek ElasticSearch admin page. You can re-index content automatically by running indexer as a command-line application. This makes it particularly convenient to run the indexer from a cron job.
  • All hostings support to create cron job, but there might be difference of how to create. If you don’t know how to do that, you can ask your hosting provider. The Create a Cron Job form is look like image  below
  • cron-tab-1.jpg
  • You need to specify the time or times when the job is run and the command to run.
  • For the command, you can copy it from Dashboard page on Geek Elasticsearch component admin, it is look like as below
  • /path/to/php -f /home/www/demo/cli/geekelasticsearch_indexer.php
  • Replace the /path/to/php with path to the PHP 5.5.0+ (Because application requires PHP 5.5.0+ to run). Ask your hosting provider for this path if you do not know where this path is.
  • NOTE: crontab will always run incremental index.

Manage indexed content

  • This screen shows a list of all the content items indexed in ElasticSearch.
  • manage-indexed-content.jpg
  • Index - runs the Geek Elasticsearch Indexer. A small popup window will appear with a progress bar which illustrates the indexing process working through the content on your site. Do not close this popup window until the process is finished. On sites which have  large amount of content it may take a long time for the indexing (tens of minutes).
    NOTE: Geek Elasticsearch indexer can be run from the command-line interface (CLI) if required. See
    Setting up cron job section for more detail.
  • Incremental Index - runs the Geek Elasticsearch Indexer, but it only update the index for those content items that have changed since the index was last updated.
  • Publish - makes selected items available to visitors on your website.
  • Unpublish - makes the selected items unavailable to visitors on your website.
  • Delete - deletes the selected content items. Works with one or multiple content items selected. Deleting a content item from Elasticsearch only deletes it from the index and does not affect the content item itself.
  • Clear Index -  you use this feature when you need to completely clear all the existing index entries before completely rebuilding the index.

Create and publish module

  • Go to modules manager page.
  • Create new instance of Module Geek Elasticsearch.
  • Update settings for Search icon, search form and search result to suit your need.
  • Publish and assign module to module position on your template.

Update Index Settings

On this screen, the left column shows the current settings information of the index, and the right column shows the form where you can add and put settings to your index.

"Update Index Settings" is an advanced feature that enables to take full advantage of Elasticsearch’s abilities. It enables you to configure the analyzer for the index, to change Character Filters, Tokenizer and Token Filters to meet the requirements of search that your website needs.

You have to add settings in JSON format from the code editor, then click on the "Put Settings" button to send an updated request of index settings. If your settings arevalid and updated successfully, you will see the new updated settings on the left column.

This is an advanced feature, in order to use this feature, we assume that you have basic knowledge of configuring Elasticsearch. So if you have no idea how to use this feature, please do not put any update settings to avoid unexpecting errors.

settings-information.jpg
put-settings.jpg

Some example of analyzers

Elasticsearch provides some built-in analyzers for your reference. You can copy sample settings from the guide pages to update your index settings.

Please NOTE that Geek ElasticSearch does not support specify analyzer for the fields, it uses default analyzer for all text fields by default. So instead of creating custom analyzers, you need to update the default analyzer, and new settings will be applied to all fields without a need for updating mapping.

For more details, when you copy settings from the Elasticsearch guide page, you need to replace the name of the custom analyzer to “default”. For example if the sample settings code is:

"analyzer": {
   "rebuilt_arabic": {
     ...
  }
}

You need to replace the name of analyzer that is "rebuilt_arabic" to "default" as follows:

"analyzer": {
   "default": {
     ...
  }
}

Language Analyzers

Language Analyzers are a set of analyzers aimed to analyzing specific language texts. The following types are supported: arabic, armenian, basque, bengali, brazilian, bulgarian, catalan, cjk, czech, danish, dutch, english, estonian, finnish, french, galician, german, greek, hindi, hungarian, indonesian, irish, italian, latvian, lithuanian, norwegian, persian, portuguese, romanian, russian, sorani, spanish, swedish, turkish, thai.
https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html

ASCII Folding Token Filter

This Token Filter converts alphabetic, numeric, and symbolic characters that are not in the Basic Latin Unicode block (first 127 ASCII characters) to their ASCII equivalent, if one exists. For example, the filter changes à to a.
https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-asciifolding-tokenfilter.html

This means that you can search accent-insensitive, for example if the document contains the text:
Vous êtes le Phénix des hôtes de ces bois.
Then you can use searchwords below to find it:
etes, Phenix, hotes
You can use the settings below to enable this feature:

{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "tokenizer": "standard",
          "filter": [ "asciifolding" ]
        }
      }
    }
  }
}

Lowercase token filter

The Token Filter changes token text to lowercase. This will enable the implementation of incase-sensitive search
For example, you can use the lowercase filter to change THE Lazy DoG to the lazy dog.
In addition to the default filter, the lowercase token filter provides access to Lucene’s language-specific lowercase filters for Greek, Irish, and Turkish.
https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lowercase-tokenfilter.html

You can use the settings below to enable this feature:

{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "tokenizer": "standard",
          "filter": [ "lowercase" ]
        }
      }
    }
  }
}

You can also use many token filters together as follows:

{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "tokenizer": "standard",
          "filter": [ "lowercase", "asciifolding" ]
        }
      }
    }
  }
}

NOTE:
After the index settings updated successfully, You might need to re-index data to get new changes applied in documents.

IMPORTANT NOTE:
Configuring Elasticsearch is out of our support scope. So, you need to update your index settings by yourself. If you need us for this help, please submit a request for Elasticsearch - installation service to This email address is being protected from spambots. You need JavaScript enabled to view it., we will check and reply you for a cost estimation cost of thetime that we need to spend.

Search Analytics

  • This screen enables you to filter and analyze users' search terms to better understand what they are looking for on site.
  • Note: You need to enable Collecting search terms setting on configuration page to get search terms collected. This setting is disabled by default.
  • search-analytics.jpg