ugurcandede / elasticsearch-document-updater

This Python script is designed to perform bulk Elasticsearch data update operations.

ElasticSearch Bulk Document Updater Script

This Python script is designed to perform bulk Elasticsearch data update operations. It accomplishes the following tasks:

1. Reads document data from a JSON file.
2. Parallelizes document updates.
3. Sends each update operation to Elasticsearch in bulk.

Usage:

• This script reads document data from a JSON file and performs document updates on Elasticsearch.
• The JSON file should have the following format:

  {
    "organizationId1": ["requesterId1", "requesterId2", ...],
    "organizationId2": ["requesterId3", "requesterId4", ...],
    ...
  }

• After the execution, it prints start and end times along with the processing time.

Requirements:

• Python 3.6 or higher
• Elasticsearch 7.6.2 or higher
• requests library for Python
• result.json file containing the document data to be updated. (This file should be in the same directory as the script)

Generate result.json file:

This query can be used to generate the result.json file from the database.

SELECT json_agg(json_build_array(organization_id, user_ids))
FROM (SELECT au.organization_id, jsonb_agg(DISTINCT au.id) AS user_ids
      FROM ticket t
               LEFT JOIN app_user au ON t.requester_id = au.id
      WHERE au.organization_id IS NOT NULL
      GROUP BY au.organization_id
      ) subquery;

How to run:

• Install the requests and argparse libraries for Python.
  • or run the following command:
  • pip install -r requirements.txt
• Run the script with the following command:
  • tenantId: The tenantId to be updated.
  • env: The environment to be updated.

python script.py --tenantId develop --env local

---

Don't forget to update environment_addresses for each environment address.