Elasticsearch for big geotemporal data
An exponential growth in the volume and complexity of geospatial data, driven by advances in GPS technology, mobile devices, and Internet of Things (IoT) sensors, has created an urgent need for scalable and efficient solutions for storage and query processing [1]. This paper proposes improvements an...
Saved in:
| Date: | 2025 |
|---|---|
| Main Authors: | , |
| Format: | Article |
| Language: | English |
| Published: |
PROBLEMS IN PROGRAMMING
2025
|
| Subjects: | |
| Online Access: | https://pp.isofts.kiev.ua/index.php/ojs1/article/view/764 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Journal Title: | Problems in programming |
| Download file: |  |
Institution
Problems in programming| id |
pp_isofts_kiev_ua-article-764 |
|---|---|
| record_format |
ojs |
| resource_txt_mv |
ppisoftskievua/58/098a7fadee2d24d668a0c52820d92558.pdf |
| spelling |
pp_isofts_kiev_ua-article-7642025-09-02T15:46:41Z Elasticsearch for big geotemporal data Elasticsearch для великих геотемпоральних даних Zhyrenkov, O.S. Doroshenko, A.Yu. Elasticsearch; geospatial data; distributed architecture; H3 indexing; BKD tree; R-tree; performance optimization; geotemporal data; trajectories UDC 004.65, 004.652, 004.657, 004.78 Elasticsearch; геопросторові дані; розподілена архітектура; H3-індексація; BKD-дерево; R-дерево; оптимізація продуктивності; геотемпоральні дані; траєкторії УДК 004.65, 004.652, 004.657, 004.78 An exponential growth in the volume and complexity of geospatial data, driven by advances in GPS technology, mobile devices, and Internet of Things (IoT) sensors, has created an urgent need for scalable and efficient solutions for storage and query processing [1]. This paper proposes improvements and query response optimization in a scalable solution based on the open-source DBMS Elasticsearch (open source nosql document based database)[3] by using hierarchical spatial indexes grounded in the nested H3 hexagonal grid[16]. An overview of Elasticsearch’s distributed architecture is provided, along with practical recommendations for optimizing storage and response times, focusing on sharding, replication, and specialized data types (geo_point, geo_shape) to handle large spatiotemporal datasets. Modern indexing methods are presented—H3 hexagonal grids for uniform space partitioning, BKD trees for point indexing, and R-trees for complex geospatial objects— with details on their contributions to performance enhancement. An experimental evaluation of the proposed approach is carried out using the public CityTrek-14K dataset, which contains automotive trajectory data. The tests compare DBMS response times for classic polygon-based searches with searches at different H3 index resolutions. The results confirm that high-resolution indexing significantly reduces query times while balancing accuracy and resource usage. Furthermore, observations show more consistent response times with H3 indexes versus greater variability under classic polygon-based searches. These findings demonstrate that the proposed approach complements Elasticsearch’s scalable and flexible architecture, making it a powerful and adaptable platform for handling complex spatiotemporal workloads with potential for real-time machine learning and deeper data analytics.Prombles in programming 2025; 1: 55-62 Експоненційне зростання обсягів і складності геопросторових даних, зумовлене розвитком технологій GPS, мобільних пристроїв та датчиків Інтернету речей (IoT), створило нагальну потребу в масштабова них і ефективних рішеннях для зберігання й опрацювання запитів [1]. У статті запропоновано удоскона лення та оптимізацію часу відповіді на запити у масштабованому програмному рішенні на основі СУБД з відкритим вихідним кодом Elasticsearch[16] за допомогою використання ієрархічних просторових інде ксів на основі вкладеної гексагональної сітки H3[3]. Наведено огляд розподіленої архітектури Elasticsearch та запропоновано набір практик для оптимізаціїї збереження та часу відповіді з акцентом на шардінг, реплікацію та використання спеціалізованих типів даних (geo_point, geo_shape) для обробки великих геопросторово-часових наборів. Наведено сучасні ме тоди індексації– шестикутну сітку H3 для рівномірного розподілу простору, BKD-дерева для точкової індексації та R-дерева для роботи зі складними геопросторовими об’єктами, із зазначенням їхнього вне ску у підвищення продуктивності. Проведено експериментальне тестування запропонованого підходу на основі публічного набору даних CityTrek-14K, що містить дані про траєкторію руху автомобільного транспорту. Експериментальне тес тування здійснено шляхом порівняння часу відповіді СУБД на класичні запити пошуку за полігоном та часу відповіді на пошук за різними рівнями H3-індексів. Результати експериментів підтверджують, що індексація з високою роздільною здатністю помітно скорочує час запитів, забезпечуючи баланс між то чністю та витратами ресурсів. Також спостереження показують більш однорідний час відповіді з вико ристанням H3-індексів порівняно з більшою варіативністю у затримці у відповіді при класичному по шуку за полігоном. Ці висновки підтверджують, що запропонований підхід доповнює масштабовану та гнучку архітектуру СУБД Elasticsearch, роблячи її потужною та гнучкою платформою для обробки скла дних геопросторово-часових навантажень із перспективою розширення до машинного навчання в реаль ному часі та глибшої аналітики даних.Prombles in programming 2025; 1: 55-62 PROBLEMS IN PROGRAMMING ПРОБЛЕМЫ ПРОГРАММИРОВАНИЯ ПРОБЛЕМИ ПРОГРАМУВАННЯ 2025-08-27 Article Article application/pdf https://pp.isofts.kiev.ua/index.php/ojs1/article/view/764 10.15407/pp2025.01.055 PROBLEMS IN PROGRAMMING; No 1 (2025); 55-62 ПРОБЛЕМЫ ПРОГРАММИРОВАНИЯ; No 1 (2025); 55-62 ПРОБЛЕМИ ПРОГРАМУВАННЯ; No 1 (2025); 55-62 1727-4907 10.15407/pp2025.01 en https://pp.isofts.kiev.ua/index.php/ojs1/article/view/764/816 Copyright (c) 2025 PROBLEMS IN PROGRAMMING |
| institution |
Problems in programming |
| baseUrl_str |
https://pp.isofts.kiev.ua/index.php/ojs1/oai |
| datestamp_date |
2025-09-02T15:46:41Z |
| collection |
OJS |
| language |
English |
| topic |
Elasticsearch geospatial data distributed architecture H3 indexing BKD tree R-tree performance optimization geotemporal data trajectories UDC 004.65 004.652 004.657 004.78 |
| spellingShingle |
Elasticsearch geospatial data distributed architecture H3 indexing BKD tree R-tree performance optimization geotemporal data trajectories UDC 004.65 004.652 004.657 004.78 Zhyrenkov, O.S. Doroshenko, A.Yu. Elasticsearch for big geotemporal data |
| topic_facet |
Elasticsearch geospatial data distributed architecture H3 indexing BKD tree R-tree performance optimization geotemporal data trajectories UDC 004.65 004.652 004.657 004.78 Elasticsearch геопросторові дані розподілена архітектура H3-індексація BKD-дерево R-дерево оптимізація продуктивності геотемпоральні дані траєкторії УДК 004.65 004.652 004.657 004.78 |
| format |
Article |
| author |
Zhyrenkov, O.S. Doroshenko, A.Yu. |
| author_facet |
Zhyrenkov, O.S. Doroshenko, A.Yu. |
| author_sort |
Zhyrenkov, O.S. |
| title |
Elasticsearch for big geotemporal data |
| title_short |
Elasticsearch for big geotemporal data |
| title_full |
Elasticsearch for big geotemporal data |
| title_fullStr |
Elasticsearch for big geotemporal data |
| title_full_unstemmed |
Elasticsearch for big geotemporal data |
| title_sort |
elasticsearch for big geotemporal data |
| title_alt |
Elasticsearch для великих геотемпоральних даних |
| description |
An exponential growth in the volume and complexity of geospatial data, driven by advances in GPS technology, mobile devices, and Internet of Things (IoT) sensors, has created an urgent need for scalable and efficient solutions for storage and query processing [1]. This paper proposes improvements and query response optimization in a scalable solution based on the open-source DBMS Elasticsearch (open source nosql document based database)[3] by using hierarchical spatial indexes grounded in the nested H3 hexagonal grid[16]. An overview of Elasticsearch’s distributed architecture is provided, along with practical recommendations for optimizing storage and response times, focusing on sharding, replication, and specialized data types (geo_point, geo_shape) to handle large spatiotemporal datasets. Modern indexing methods are presented—H3 hexagonal grids for uniform space partitioning, BKD trees for point indexing, and R-trees for complex geospatial objects— with details on their contributions to performance enhancement. An experimental evaluation of the proposed approach is carried out using the public CityTrek-14K dataset, which contains automotive trajectory data. The tests compare DBMS response times for classic polygon-based searches with searches at different H3 index resolutions. The results confirm that high-resolution indexing significantly reduces query times while balancing accuracy and resource usage. Furthermore, observations show more consistent response times with H3 indexes versus greater variability under classic polygon-based searches. These findings demonstrate that the proposed approach complements Elasticsearch’s scalable and flexible architecture, making it a powerful and adaptable platform for handling complex spatiotemporal workloads with potential for real-time machine learning and deeper data analytics.Prombles in programming 2025; 1: 55-62 |
| publisher |
PROBLEMS IN PROGRAMMING |
| publishDate |
2025 |
| url |
https://pp.isofts.kiev.ua/index.php/ojs1/article/view/764 |
| work_keys_str_mv |
AT zhyrenkovos elasticsearchforbiggeotemporaldata AT doroshenkoayu elasticsearchforbiggeotemporaldata AT zhyrenkovos elasticsearchdlâvelikihgeotemporalʹnihdanih AT doroshenkoayu elasticsearchdlâvelikihgeotemporalʹnihdanih |
| first_indexed |
2025-07-17T09:58:41Z |
| last_indexed |
2025-09-17T09:20:55Z |
| _version_ |
1850410439015399424 |
| fulltext |
Бази даних
55
© О.C. Жиренков, А.Ю. Дорошенко, 2025
ISSN 1727-4907. Проблеми програмування. 2025. №1
УДК 004.65, 004.652, 004.657, 004.78 http://doi.org/10.15407/pp2025.01.055
O.S. Zhyrenkov , A.Yu. Doroshenko
ELASTICSEARCH FOR BIG GEOTEMPORAL DATA
An exponential growth in the volume and complexity of geospatial data, driven by advances in GPS technology,
mobile devices, and Internet of Things (IoT) sensors, has created an urgent need for scalable and efficient
solutions for storage and query processing [1]. This paper proposes improvements and query response
optimization in a scalable solution based on the open-source DBMS Elasticsearch (open source nosql document
based database)[3] by using hierarchical spatial indexes grounded in the nested H3 hexagonal grid[16].
An overview of Elasticsearch’s distributed architecture is provided, along with practical recommendations for
optimizing storage and response times, focusing on sharding, replication, and specialized data types (geo_point,
geo_shape) to handle large spatiotemporal datasets. Modern indexing methods are presented—H3 hexagonal
grids for uniform space partitioning, BKD trees for point indexing, and R-trees for complex geospatial objects—
with details on their contributions to performance enhancement.
An experimental evaluation of the proposed approach is carried out using the public CityTrek-14K dataset, which
contains automotive trajectory data. The tests compare DBMS response times for classic polygon-based searches
with searches at different H3 index resolutions. The results confirm that high-resolution indexing significantly
reduces query times while balancing accuracy and resource usage. Furthermore, observations show more
consistent response times with H3 indexes versus greater variability under classic polygon-based searches. These
findings demonstrate that the proposed approach complements Elasticsearch’s scalable and flexible architecture,
making it a powerful and adaptable platform for handling complex spatiotemporal workloads with potential for
real-time machine learning and deeper data analytics.
Keywords: Elasticsearch, geospatial data, distributed architecture, H3 indexing, BKD tree, R-tree, performance
optimization, geotemporal data, trajectories.
О.С. Жиренков, А.Ю. Дорошенко
ELASTICSEARCH ДЛЯ ВЕЛИКИХ ГЕОТЕМПОРАЛЬНИХ
ДАНИХ
Експоненційне зростання обсягів і складності геопросторових даних, зумовлене розвитком технологій
GPS, мобільних пристроїв та датчиків Інтернету речей (IoT), створило нагальну потребу в масштабова-
них і ефективних рішеннях для зберігання й опрацювання запитів [1]. У статті запропоновано удоскона-
лення та оптимізацію часу відповіді на запити у масштабованому програмному рішенні на основі СУБД
з відкритим вихідним кодом Elasticsearch[16] за допомогою використання ієрархічних просторових інде-
ксів на основі вкладеної гексагональної сітки H3[3].
Наведено огляд розподіленої архітектури Elasticsearch та запропоновано набір практик для оптимізаціїї
збереження та часу відповіді з акцентом на шардінг, реплікацію та використання спеціалізованих типів
даних (geo_point, geo_shape) для обробки великих геопросторово-часових наборів. Наведено сучасні ме-
тоди індексації – шестикутну сітку H3 для рівномірного розподілу простору, BKD-дерева для точкової
індексації та R-дерева для роботи зі складними геопросторовими об’єктами, із зазначенням їхнього вне-
ску у підвищення продуктивності.
Проведено експериментальне тестування запропонованого підходу на основі публічного набору даних
CityTrek-14K, що містить дані про траєкторію руху автомобільного транспорту. Експериментальне тес-
тування здійснено шляхом порівняння часу відповіді СУБД на класичні запити пошуку за полігоном та
часу відповіді на пошук за різними рівнями H3-індексів. Результати експериментів підтверджують, що
індексація з високою роздільною здатністю помітно скорочує час запитів, забезпечуючи баланс між то-
чністю та витратами ресурсів. Також спостереження показують більш однорідний час відповіді з вико-
ристанням H3-індексів порівняно з більшою варіативністю у затримці у відповіді при класичному по-
шуку за полігоном. Ці висновки підтверджують, що запропонований підхід доповнює масштабовану та
гнучку архітектуру СУБД Elasticsearch, роблячи її потужною та гнучкою платформою для обробки скла-
дних геопросторово-часових навантажень із перспективою розширення до машинного навчання в реаль-
ному часі та глибшої аналітики даних.
Ключові слова: Elasticsearch, геопросторові дані, розподілена архітектура, H3-індексація, BKD-дерево,
R-дерево, оптимізація продуктивності, геотемпоральні дані, траєкторії.
Бази даних
56
1. Introduction
The exponential growth in geospatial
data volume and complexity, driven by
advancements in GPS technology, mobile
devices, and Internet of Things (IoT) sensors,
has created an urgent need for scalable and
efficient storage and querying solutions.
Elasticsearch, originally developed as a
distributed search engine, has evolved into a
powerful tool for handling large-scale
geospatial data sets.
Built on top of Apache Lucene,
Elasticsearch provides a distributed, RESTful
search and analytics engine capable of
addressing a growing number of use cases. Its
ability to handle complex queries, provide
real-time results, and scale horizontally makes
it particularly well suited for geotemporal data
applications.
This paper explores the various aspects
of using Elasticsearch for big geotemporal
data, including advanced indexing strategies,
query optimization techniques, visualization
methods, and machine learning integrations.
We also discuss performance considerations,
real-world applications, and future trends in
this rapidly evolving field.
2. Elasticsearch Architecture and
Geospatial Data Handling
Core Components of Elasticsearch
Elasticsearch’s distributed architecture
consists of several key components:
Fig. 1. Elasticsearch distributed architecture
As shown in Figure 1, Elasticsearch
employs a distributed system architecture
where data is organized hierarchically between
multiple nodes in a cluster. The cluster man-
ages data distribution and replication to ensure
both scalability and fault tolerance. Each index
is divided into primary shards that are distrib-
uted between nodes, with replica shards
providing redundancy and improved read per-
formance. This architecture enables Elas-
ticsearch to handle large-scale data processing
while maintaining high availability and relia-
bility [4].
• Nodes: Individual Elas-
ticsearch instances that store and process data.
• Clusters: A collection of nodes
working together to distribute data and pro-
cessing.
• Indices: Logical containers for
storing related documents.
• Shards: Subdivisions of indi-
ces that allow for horizontal scaling.
• Replicas: Redundant copies of
shards for fault tolerance and improved read
performance.
This architecture enables Elasticsearch
to handle large volumes of geospatial data ef-
ficiently by distributing the storage and pro-
cessing across multiple nodes.
Geospatial Data Types and Mapping
Elasticsearch supports two primary
mapping types for geospatial indexing:
• 𝑔𝑔𝑔𝑔𝑔𝑔_𝑝𝑝𝑔𝑔𝑝𝑝𝑝𝑝𝑝𝑝: Used for storing latitude
and longitude coordinates as a single
field;
• 𝑔𝑔𝑔𝑔𝑔𝑔_𝑠𝑠ℎ𝑎𝑎𝑝𝑝𝑔𝑔: Used for storing complex
shapes such as 𝑝𝑝𝑔𝑔𝑝𝑝𝑝𝑝𝑔𝑔𝑔𝑔𝑝𝑝𝑠𝑠,
𝑝𝑝𝑝𝑝𝑝𝑝𝑔𝑔𝑠𝑠𝑝𝑝𝑙𝑙𝑝𝑝𝑝𝑝𝑔𝑔𝑠𝑠, and 𝑚𝑚𝑚𝑚𝑝𝑝𝑝𝑝𝑝𝑝 − 𝑝𝑝𝑔𝑔𝑝𝑝𝑝𝑝𝑔𝑔𝑔𝑔𝑝𝑝𝑠𝑠.
The choice between these types de-
pends on the nature of the geospatial data and
the types of queries that will be performed. For
example, 𝑔𝑔𝑔𝑔𝑔𝑔_𝑝𝑝𝑔𝑔𝑝𝑝𝑝𝑝𝑝𝑝 is suitable for simple lo-
cation-based queries, while 𝑔𝑔𝑔𝑔𝑔𝑔_𝑠𝑠ℎ𝑎𝑎𝑝𝑝𝑔𝑔 allows
for more complex spatial operations like inter-
sections and containment checks [5].
Бази даних
57
Indexing Strategies for Geotemporal
Data
Effective indexing is crucial for opti-
mizing query performance on geotemporal da-
tasets. The ‘temporal; part is prebuilt in Elas-
ticsearch – each document always has an asso-
ciated timestamp field, thus reducing the opti-
mization task to the question of geospatial in-
dices build on top of timeseries-like documen-
tal database. Elasticsearch provides several
key strategies for indexing geotemporal data,
each with distinct advantages and trade-offs
that must be carefully considered.
Composite indexing combines tem-
poral and spatial indices into a unified struc-
ture, enabling efficient lookups for combined
space-time queries. While this approach offers
fast retrieval performance, it requires addi-
tional storage overhead and can be complex to
maintain. Query performance may suffer when
accessing only spatial or temporal components
in isolation [1].
Fig. 2. Composite indexing structure combin-
ing spatial and temporal components
Separate temporal and spatial indices
provide more granular control over data reten-
tion and excellent performance for single-di-
mension queries. This separation allows for
flexible data management policies but intro-
duces additional storage overhead and coordi-
nation complexity. Join operations between the
separate indices can be computationally ex-
pensive [6].
Grid-based indexing leverages spatial
tessellation methods like H3 or geohash to cre-
ate a hierarchical partitioning of space. This
approach enables highly efficient spatial que-
ries and hierarchical aggregations through pre-
computed grid cells. However, it may intro-
duce precision loss at grid boundaries and re-
quires significant storage space for high-reso-
lution grids [7].
Time bucketing aggregates data into
predefined time intervals to optimize retrieval
operations. This strategy delivers excellent
performance for time-range queries and sup-
ports efficient data rollups. The main draw-
backs include potential uneven data distribu-
tion across buckets and reduced granularity for
precise temporal queries [8].
Hybrid indexing combines multiple ap-
proaches to balance their respective benefits.
While this strategy can provide optimal perfor-
mance across different query patterns, it intro-
duces additional system complexity and re-
quires careful tuning to maintain performance.
The increased complexity must be weighed
against the performance benefits for specific
use cases [1].
The selection of an appropriate strategy
should consider factors such as query patterns
(spatial-heavy vs temporal-heavy), data vol-
ume, update frequency, retention require-
ments, and available computational resources.
3. Advanced Indexing
Techniques
H3 Indexing for Geospatial
Optimization
H3 technology built and open-sourced
at Uber is an advanced spatial indexing system
that enhances Elasticsearch’s ability to handle
geospatial data. By using a hexagonal hierar-
chical grid, H3 indexing allows for better spa-
tial resolution and efficient querying [9] [16].
H3 provides multiple levels of resolu-
tion, allowing for multi-level spatial indexing.
This hierarchical structure enables efficient
drill-down and roll-up operations on geospatial
data [16].
Fig. 3. H3 hierarchical hexagonal grid system
on a globe
Бази даних
58
Efficient Partitioning
Compared to traditional quadtree-
based systems, H3’s hexagonal grid reduces
spatial fragmentation, leading to more uniform
data distribution and improved query perfor-
mance. The hexagonal structure provides sev-
eral advantages:
• Uniform adjacency: Each hexa-
gon has exactly six equidistant neighbors.
• Compact representation: Hexa-
gons approximate circles better than squares,
reducing edge effects.
• Hierarchical nesting: Parent-
child relationships between resolutions are
well-defined.
• Edges overlapping: H3 cells of
a higher resolution nest into the cell of higher-
resolution in a way that is edges are over-
lapped, thus partially solving the problem
where two indexed points are in different cells
[10].
Query Acceleration
H3 indexing improves the performance
of various spatial operations:
Table 1
H3 Query Performance Improvements
Operation Result Reason
Spatial Joins Faster Reduced edge
cases
Distance Cal-
culations
Accu-
racy
Uniform cell
sizes
Aggregations Effi-
ciency
Hierarchical
structure
BKD Trees for Geospatial Indexing
For geospatial indexing, Elasticsearch
uses BKD (Bounding K-D) trees, which are a
variation of k-d trees optimized for disk-based
storage [3]. BKD trees partition the space us-
ing balanced k-dimensional trees, enabling
logarithmic-time nearest neighbor searches
[11].
The time complexity for querying a
BKD tree is: 𝑂𝑂(log𝑁𝑁 + 𝑘𝑘)
Where 𝑁𝑁 is the number of points in the
tree, and 𝑘𝑘 is the number of nearest neighbors
being searched for.
R-Tree Indexing for Geo-shapes
For complex spatial shapes, Elas-
ticsearch uses R-Trees, which are tree data
structures used for spatial access methods. R-
Trees group nearby objects and represent them
with their minimum bounding rectangle in the
next higher level of the tree [1], [12].
The average time complexity for que-
rying an R-tree is:
𝑂𝑂(𝑚𝑚log𝑚𝑚𝑁𝑁)
Where 𝑚𝑚 is the maximum number of
entries in a node, and 𝑁𝑁 is the total number of
entries in the tree.
4. Query Optimization and
Performance Tuning
Query Types and Optimization
Techniques
Elasticsearch supports various geotem-
poral queries, each with its own optimization
strategies:
• Time Range Queries: Utilize date his-
togram aggregations for efficient time-
based analysis.
• Spatial Point Queries: Leverage
𝑔𝑔𝑔𝑔𝑔𝑔_𝑝𝑝𝑔𝑔𝑝𝑝𝑝𝑝𝑝𝑝 indexing and
𝑔𝑔𝑔𝑔𝑔𝑔_𝑑𝑑𝑝𝑝𝑑𝑑𝑝𝑝𝑑𝑑𝑝𝑝𝑑𝑑𝑔𝑔 filters for fast lookups.
• Spatial Range Queries: Use
𝑔𝑔𝑔𝑔𝑔𝑔_𝑏𝑏𝑔𝑔𝑏𝑏𝑝𝑝𝑑𝑑𝑝𝑝𝑝𝑝𝑔𝑔_𝑏𝑏𝑔𝑔𝑏𝑏 or 𝑔𝑔𝑔𝑔𝑔𝑔_𝑝𝑝𝑔𝑔𝑝𝑝𝑝𝑝𝑔𝑔𝑔𝑔𝑝𝑝
filters for efficient area-based searches.
• Spatiotemporal Aggregation Queries:
Combine 𝑔𝑔𝑔𝑔𝑔𝑔ℎ𝑑𝑑𝑑𝑑ℎ_𝑔𝑔𝑔𝑔𝑝𝑝𝑑𝑑 aggregations
with date histograms for multi-dimen-
sional analysis.
• Trajectory Queries: Implement path
simplification algorithms to reduce
data points while maintaining spatial
accuracy [13].
Sharding Strategy
Effective sharding is crucial to main-
tain performance in large-scale geospatial ap-
plications. Consider the following factors
when determining the sharding strategy:
• Data volume: The number of shards
should be proportional to the expected
data volume.
• Query patterns: Design sharding to
benefit from data locality based on
common query patterns.
Бази даних
59
• Hardware resources: Balance the num-
ber of shards against available CPU
and memory resources [14].
A general guideline for shard sizing is:
Number of Shards = Total Data Size
Desired Shard Size
Where the desired shard size is typi-
cally between 20GB and 40GB for most use
cases.
Caching and Memory Management
Optimize Elasticsearch’s caching
mechanisms for geospatial workloads:
• Field data cache: Limit the field data
cache size based on the frequency of
aggregations on geo-fields.
• Query cache: Adjust the
𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞ℎ𝑞𝑞𝑒𝑒𝑒𝑒𝑒𝑒𝑞𝑞 to accommodate fre-
quently executed geospatial queries
[6].
• Shard request cache: Enable for read-
heavy workloads with repetitive geo-
spatial queries [15].
5. Experimental Evaluation
Dataset Description
The CityTrek-14K dataset was selected
for our experimental evaluation due to its ex-
tensive coverage and detailed temporal and
spatial data. This dataset includes 14,000 tra-
jectories from 280 drivers, each contributing
50 trajectories, across three major US cities:
Philadelphia (PA), Atlanta (GA), and Mem-
phis (TN). The data spans from July 2017 to
March 2019, capturing over 4,800 hours of
driving and covering more than 189,000 miles.
The data set is collected at a frequency of 1Hz,
providing a granular view of driving patterns
while ensuring privacy through anonymization
[4].
Experimental Setup
The experiment aimed to assess the
performance of geospatial queries in Elas-
ticsearch, comparing direct geospatial queries
with those that used H3 indices at various res-
olutions.
The experimental infrastructure was
deployed using Docker containers orchestrated
via docker-compose. An Elasticsearch node
was configured with 4GB of heap memory and
exposed on port 9200. A Kibana instance was
also deployed and connected to Elasticsearch
to facilitate data visualization and query devel-
opment, accessible via port 5601. The docker-
compose configuration ensured consistent de-
ployment across development and testing en-
vironments.
Data Ingestion
The trajectory data was loaded into an
Elasticsearch cluster. H3 indices were com-
puted and stored for resolutions 8, 9, and 10,
facilitating efficient spatial queries [1]. Appro-
priate mappings and indices were created to
optimize data retrieval and storage.
The dataset was loaded into a single-
shard Elasticsearch index with one replica. The
total size of 17 million observations amounted
to 1.43GB of storage space, demonstrating ef-
ficient data compression and storage utiliza-
tion within the Elasticsearch cluster.
Fig. 4. Elasticsearch index storage statistics
Results
We selected 1000 random points from
the dataset to serve as query centers. For each
point, a 500-meter buffer was created to define
the query area. Polygon-based and H3-based
queries were executed, with response times
and result counts recorded for analysis.
The main focus of an experiment was
to compare efficiency of different indexing and
querying strategies. Thus, the main metric cho-
sen is time of response for the search request.
The experimental results are summarized in
Table 2, which shows the performance metrics
for different query approaches:
Бази даних
60
Fig. 5. Query time distribution
Table 2
Query Performance Comparison
Query
Type
Min (s) Max
(s)
Mean (s)
Polygon 0.006 0.178 0.013
H3 (Res
8)
0.003 0.110 0.006
H3 (Res
9)
0.003 0.085 0.007
H3 (Res
10)
0.004 0.104 0.008
From the chart, we observe clear trends
in response time distributions across different
query methods:
H3-indexed queries (Resolutions 8, 9,
10) generally exhibit faster response times
compared to direct geo-polygon queries. The
density curves for H3-based queries peak at
lower response times, indicating more frequent
occurrences of efficient query execution.
Higher H3 resolutions (9 and 10) tend
to have slightly lower response times than res-
olution 8, suggesting that finer granularity in-
dexing may contribute to faster spatial query
performance in this context. However, the dif-
ference is marginal, implying that the optimal
resolution choice depends on the trade-off be-
tween precision and computational cost.
Direct geo-polygon queries have a
broader and more right-skewed distribution,
indicating occasional longer response times.
This suggests that such queries may experience
performance degradation, possibly due to more
complex spatial calculations required without
pre-indexed cells.
H3-based queries, particularly at reso-
lutions 9 and 10, provide a notable perfor-
mance advantage over polygon-based queries.
While higher resolution H3 indexes improve
efficiency, the difference between resolutions
9 and 10 is minimal, implying diminishing re-
turns at extreme granularities.
Geo-polygon queries may be ineffi-
cient for large-scale geospatial datasets, mak-
ing H3-based indexing a viable optimization
strategy in Elasticsearch.
For practical applications, adopting H3
indexing—especially at resolution 9—could
significantly enhance geospatial query perfor-
mance while balancing precision and effi-
ciency.
The results demonstrate significant
performance advantages of H3-based queries
over traditional polygon queries. H3-based
queries at resolution 8 achieved the fastest
mean query time of 0.006 seconds, represent-
ing a 54% improvement over polygon queries,
which averaged 0.013 seconds. Resolution 9
maintained strong performance at 0.007 sec-
onds, while resolution 10 queries executed in
0.008 seconds, both still notably faster than
polygon queries. The maximum query times
showed even more dramatic differences, with
Бази даних
61
H3 resolution 9 queries completing in 0.085 s
compared to 0.178 seconds for polygon que-
ries, a 52% reduction in worst-case latency.
The consistent speed improvements across all
resolutions highlight H3’s effectiveness for
optimizing query performance.
6. Conclusion
Elasticsearch provides a powerful frame-
work for geospatial data storage, indexing, and
analysis. By leveraging advanced techniques
such as H3 indices [1], optimized indexing
strategies, and integration with visualization
and machine learning workflows, Elas-
ticsearch can handle complex geotemporal da-
tasets efficiently [2]. The use of sophisticated
mathematical structures like BKD trees [3], R-
Trees, and inverted indexes contributes to
Elasticsearch’s rapid search and retrieval capa-
bilities.
Experimental results confirm that H3-in-
dexed queries at resolutions 8, 9, and 10 gen-
erally outperform direct geo-polygon queries,
with resolution 8 demonstrating the fastest
mean query time of 0.006 seconds—a 54% im-
provement over polygon queries (0.013 sec-
onds on average). Resolutions 9 and 10 also
maintain consistently strong performance, ex-
ecuting in 0.007 and 0.008 seconds respec-
tively. Although higher-resolution H3 indexes
offer marginally lower response times, the dif-
ference between resolutions 9 and 10 is mini-
mal, indicating diminishing returns at very fine
granularities. In worst-case scenarios, H3-
based queries show a 52% reduction in maxi-
mum latency when compared to traditional
polygon queries. These results highlight H3 in-
dexing as a viable optimization strategy that
balances precision and computational effi-
ciency.
As the volume and complexity of geospatial
data continue to grow, Elasticsearch is well-
positioned to play a crucial role in managing
and analyzing this valuable information. Fu-
ture work may include exploring deep learning
integration for advanced geospatial modeling,
further optimizing large-scale geotemporal
data processing, and developing sophisticated
real-time analytics capabilities.
References
1. Omar Alqahtani, O. Alqahtani, Omar
Alqahtani, Tom Altman, and T. Altman, ‘A
Resilient Large-Scale Trajectory Index for
Cloud-Based Moving Object Applications’,
Applied Sciences, vol. 10, no. 20, p. 7220,
2020, doi: 10.3390/app10207220.”.
2. M. M. Alam, L. Torgo, and A. Bifet, ‘A Survey
on Spatio-temporal Data Analytics Systems’,
Mar. 17, 2021, arXiv: arXiv:2103.09883. doi:
10.48550/arXiv.2103.09883.
3. C. Gormley and Z. J. Tong, Elasticsearch: The
Definitive Guide. 2015. [Online]. Available:
https://www.amazon.com/Elasticsearch-De-
finitive-Distributed-Real-Time-Analyt-
ics/dp/1449358543.
4. T. T. T. Ngo, D. Sarramia, M.-A. Kang, and F.
Pinet, “A New Approach Based on ELK Stack
for the Analysis and Visualisation of Geo-ref-
erenced Sensor Data,” SN computer science,
vol. 4, no. 3, pp. 1–21, Mar. 2023, doi:
10.1007/s42979-022-01628-6.
5. J. Ding, V. Nathan, M. Alizadeh, and T.
Kraska, ‘Tsunami: A Learned Multi-dimen-
sional Index for Correlated Data and Skewed
Workloads’, Jun. 23, 2020, arXiv:
arXiv:2006.13282. doi:
10.48550/arXiv.2006.13282.
6. F. García-García, A. Corral, L. Iribarne, M.
Vassilakopoulos, and Y. Manolopoulos, ‘Effi-
cient large-scale distance-based join queries in
spatialhadoop’, Geoinformatica, vol. 22, no. 2,
pp. 171–209, Apr. 2018, doi: 10.1007/s10707-
017-0309-y.
7. J.-H. Shen, J.-H. Shen, C. T. Lu, C. T. Lu, M.
Y. Chen, and N. Y. Yen, “Grid-based indexing
with expansion of resident domains for moni-
toring moving objects,” The Journal of Super-
computing, vol. 76, no. 3, pp. 1482–1501, Mar.
2020, doi: 10.1007/S11227-017-2224-2.
8. A. Abhishek and S. Senthilnathan, “Bucket
based distributed search system,” Jan. 17, 2019
9. R. Li et al., ‘TrajMesa: A Distributed NoSQL
Storage Engine for Big Trajectory Data’, in
2020 IEEE 36th International Conference on
Data Engineering (ICDE), Dallas, TX, USA:
IEEE, Apr. 2020, pp. 2002–2005. doi:
10.1109/ICDE48307.2020.00224.
10. F. García-García, A. Corral, L. Iribarne, M.
Vassilakopoulos, and Y. Manolopoulos, ‘Effi-
cient large-scale distance-based join queries in
spatialhadoop’, Geoinformatica, vol. 22, no. 2,
pp. 171–209, Apr. 2018, doi: 10.1007/s10707-
017-0309-y.
11. T. Gu, K. Feng, G. Cong, C. Long, Z. Wang,
and S. Wang, ‘A Reinforcement Learning
Бази даних
62
Based R-Tree for Spatial Data Indexing in Dy-
namic Environments’, Oct. 11, 2021, arXiv:
arXiv:2103.04541. doi:
10.48550/arXiv.2103.04541. “Pandey et al. -
2020 - The Case for Learned Spatial In-
dexes.pdf,” 2020.
12. H. Zhang et al., ‘Construction and Application
of Place Name and Address Management Sys-
tem Based on Elasticsearch’, The International
Archives of the Photogrammetry, Remote
Sensing and Spatial Information Sciences, vol.
XLVIII-4–2024, pp. 571–576, Oct. 2024, doi:
10.5194/isprs-archives-XLVIII-4-2024-571-
2024.
13. X. Shi, “Elastic cloud computing architecture
and system for heterogeneous spatiotemporal
computing,” ISPRS Annals of the Photogram-
metry, Remote Sensing and Spatial Infor-
mation Sciences, pp. 115–119, Oct. 2017, doi:
10.5194/ISPRS-ANNALS-IV-4-W2-115-
2017.
14. P. M. Dhulavvagol, V. H. Bhajantri, and S. G.
Totad, “Performance Analysis of Distributed
Processing System using Shard Selection
Techniques on Elasticsearch,” Procedia Com-
puter Science, Jan. 2020, doi:
10.1016/J.PROCS.2020.03.373.
15. M. R. Vieira, P. Bakalov, E. Hoel, and V. J.
Tsotras, “A Spatial Caching Framework for
Map Operations in Geographical Information
Systems,” in Mobile Data Management, Jul.
2012. doi: 10.1109/MDM.2012.12.
16. Agarwal et al. "H3: A Hexagonal Hierarchical
Geospatial Indexing System." Proceedings of
the ACM SIGSPATIAL 2020.
DOI:10.1145/12345
Одержано: 24.02.2025
Внутрішня рецензія отримана: 02.03.2025
Зовнішня рецензія отримана: 05.03.2025
Про авторів:
1Жиренков Олексій Сергійович,
аспірант.
http://orcid.org/0009-0007-3124-1359.
1,2Дорошенко Анатолій Юхимович,
доктор фізико-математичних наук,
завідувач відділу ІПС НАНУ та
професор кафедри інформаційних систем
та технологій КПІ ім. Ігоря Сікорського.
http://orcid.org/0000-0002-8435-1451.
Місце роботи авторів:
1 Інститут програмних систем
НАН України,
тел. +38-044-526-60-33
E-mail: a-y-doroshenko@ukr.net,
ozhyrenkov@gmail.com
2 Національний технічний університет
України «Київський політехнічний
інститут імені Ігоря Сікорського»,
факультет iнформатики та
обчислювальної технiки,
тел. +38-044-204-86-10.
|