AWS elasticsearch tutorial

AWS Elasticsearch Tutorial

Amazon Web Services / By

AWS Elasticsearch service intends to help you deploy, scale & operate Elasticsearch over the AWS cloud. If you already have a clean and crisp knowledge of the concepts of Elasticsearch, then you are definitely willing to get started with it. But, it is important for you to know the right approach steps to ensure that you get the job done efficiently.

This is the AWS Elasticsearch tutorial to help you understand the process of creating and configuring an Elasticsearch domain by using the dedicated Amazon service. With these steps, you will get a clear idea of how to get the domain online quickly. So, follow this tutorial till the end!

Recommended: Read out our previous article “What is AWS Elasticsearch” to learn the basics! 

Creating a Domain within Amazon Elasticsearch Service

Amazon ES (Elasticsearch) Domain is synonymous with the Elasticsearch cluster. Domains are represented as the clusters that come with settings, instance counts, storage resources, and instance types that are specified by you. Go ahead and create your Amazon Elasticsearch Service domain from the dedicated console, AWS SDKs or AWS CLI. The steps to create the Amazon Elasticsearch Service domain are:

  1. Visit the official website of AWS services, and choose ‘Sign in to Console.’ 
  2. Select the ‘Analytics’ tab and choose the option ‘Elasticsearch Service.’ 
  3. Now, go ahead and choose the tab, ‘Create a new domain.’ 
  4. Now, choose the type of deployment that you wish to prefer by selecting the tab ‘Development and Testing.’ 
  5. Choose the latest Elasticsearch version and click on ‘Next.’ 
  6. Give a name to your’ domain.’ 
  7. You can ignore the ‘custom endpoint’ setting.
  8. Move ahead to the ‘Data Nodes’ section and select instance type as ‘t3.small.elasticsearch’ with a default value for one node.
  9. Rest all of the settings are not essential as of now, and choose Next. 
  10. Preferably use a public access domain by selecting ‘Public Access’ under ‘Network Configuration.’ 
  11. For implementing access control, you can choose the ‘Create master user’ option and add the user name & password. 
  12. You will then encounter SAML and Amazon Cognito authentication, which you can ignore for now. 
  13. Under the ‘Domain Access Policy,’ you need to select ‘Allow Open Access to the Domain.’ (For this tutorial, the access control is handed over to fined-grained control and not the access policy, for which it is public access domain)
  14. The encryption settings should be kept at default, and then click on ‘Next.’
  15. Ignore the tag’s settings and click on ‘Next.’ 
  16. Now, confirm the configuration of the domain and select ‘Confirm.’ Wait for 10 to 15 minutes as the domain initializes. It might take a bit longer time, depending upon the selected configurations. 
  17. Once the domain is created, take note of its endpoint.

Uploading Data to Amazon Elasticsearch Service for Indexing Purpose

After you have created your domain, you can now upload data onto it, with the help of most programming languages or preferably command line. You can use any of the HTTP tools embedded within the Amazon ES Service. For instance, you can use curl, Postman, or dev console. Here are the steps that will help you master the process of uploading data to the Amazon ES service for the indexing needs:

  1. Putting the Document into Index

You need to run a specified command for adding a single document to the created Amazon ES Service domain. Irrespective of which HTTP tool you use, you need to make the HTTP call for creating an index within a new document. The sample HTTP call is:

PUT /vegetables/_doc/1
{
“name”:”carrot”,
“color”:”orange”
}

This example here specifies that you are using the dev console within Kibana as your HTTP tool. You need to adapt different URLs and credentials for other tools that you use. After calling, the endpoint creates the Index and puts a single document into the Index ID 1. The Index in this example has been given the name ‘vegetables.’ 

The _doc part within this example of index creation call represents the document type. It is preferred that you create an index for each type of document. But, it is possible only if you create an ID for the documents within Amazon Elasticsearch Service.

  1. Creating Auto-generated IDs

For creating the Amazon Elasticsearch Service ID for your documents, you need to use the command POST instead of PUT.

POST /veggies/_doc
{
“name”:”beet”,
“color”:”red”,
“classification”:”root”
}

Under this call, you are creating an index with the name ‘veggies’ and adds the document onto it. It also creates an ID for the document that will eradicate the necessity of adding any specific number ID to the call. It is because you are already creating a document with the system-generated ID, for which you are not requested to give one.

  1. Updating Document with POST

When you are using the command HTTP POST with an identifier, you can seek an update for the existing document. First, choose an ID and create a document. Let’s take the example of ID number 42.

POST /veggies/_doc/42
{
“name”:”sugar-beet”,
“color”:”red”,
“classification”:”bark”
}

Once the document is created, use the ID to then update the document, with the next call:

POST /veggies/_doc/42
{
“name”:”sugar-beet”,
“color”:”red”,
“classification”:”root”
}

Now, the command entered into the call will update the document with the newest value of classification entered by the use. In the case of this example, the classification value is ‘root.’

In case you are trying to upload a document that is not available within the Index, then Amazon ES Service creates it for you. It concludes that you need to use the commands PUT for creating a document with a specific ID, POST for updating the document with a specified ID, and also create a document by using the auto-generated ID when a user doesn’t provide one.

  1. Executing Bulk Actions

The above three steps highlighted the basic document creation and uploading it onto Amazon ES Service for indexing. But, with the use of bulk API, you can upload a huge bunch of data to the Amazon ES Service for indexing.

Using the _bulk API, you can call several actions on single or multiple indexes at once. You can perform several creating, updating, and deleting tasks at once. Hence, it will eventually speed up the operations with a basic formula that is:

POST /_bulk
<action_meta>\n
<action_data>\n
<action_meta>\n
<action_data>\n

Every action that you desire to execute will use two JSON lines. In the first, you will have to provide the description of your action or enter metadata. In the second line, you will have to enter the data! Every action or part is separated by entering the command for a new line (\n).

When taken together, the meta & data work collectively to represent a singular action and execute within a bulk operation. Hence, this is how you can enter bulk data onto the Amazon Elasticsearch Service for indexing.

Searching Documents from the Command Line

Searching is yet another important event within the implementation of Elasticsearch. You will be using the Elasticsearch search API for carrying the lookout for documents within the service. You can also use Kibana for searching the documents within the domain.

If you use the command line, domain name as ‘movies,’ the curl HTTP tool, and the search document as ‘mars,’ then you will have to enter this call:

curl -XGET -u ‘master-user:master-user-password’ ‘domain-endpoint/movies/_search?q=mars&pretty=true’

If you are searching the documents with the use of Kibana, you can go ahead with some specified steps:

  • Point the browser to the Kibana plugin for the Amazon ES domain that you created.
  • Login by using your username & password.
  • Configure at least one index pattern, as Kibana uses them to identify the indices that you wish to analyze. 
  • Go to the main menu of Kibana, select ‘Stack Management,’ choose ‘Index Patterns,’ and choose ‘Create Index Pattern.’ 
  • Move onto the next step, and you can see document fields of the created Index. 
  • Now, head back to the Kibana main menu again, and look for the ‘Discover’ tab. 
  • Enter the document name that you want to search, and press enter. 

Deleting a Domain

For deleting a domain, you just need to follow few steps in a specific order, which includes:

  • Sign in to the Amazon ES console
  • Go to the ‘My Domains’ section, and select the domain you want to delete.
  • Select ‘Actions’ and choose ‘Delete Domain.’
  • Confirm the deletion by selecting ‘Delete the Domain,’ and then go ahead and click on ‘Delete.’ 

Final Words

Now, as you have known the process of starting with Amazon Elasticsearch Service, you can enter the world of in-depth operations within Amazon Elasticsearch. Once you master the procedure of creating a domain, uploading the document, searching the document, and deleting the domain, you can proceed upon learning the management of indices within the domain and other such advanced features.

AWS Elasticsearch is easily accessible and is highly scalable. As it allows integration with all other AWS services, the users get the flexibility to collaborate all of them to function together and efficiently. This AWS Elasticsearch tutorial has covered detailed steps on executing each of the processes to get you started with the Amazon Elasticsearch Service.

About Pavan Gumaste

Pavan Rao is a programmer / Developer by Profession and Cloud Computing Professional by choice with in-depth knowledge in AWS, Azure, Google Cloud Platform. He helps the organisation figure out what to build, ensure successful delivery, and incorporate user learning to improve the strategy and product further.

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top