A Hybrid SQL Query Execution Model for JSON Data: Balancing Resource Efficiency and Analytical Performance

Abstract

The rapid growth of unstructured and semi-  structured data has necessitated the development of efficient query  execution frameworks capable of handling complex data formats  such as JSON. This paper presents a hybrid framework that  intelligently leverages Apache Drill and DuckDB to optimize SQL  query execution on JSON data. Our framework dynamically selects  the appropriate query engine based on dataset size, available  memory, and the nature of the SQL workload, effectively balancing  memory efficiency with analytical performance. We explore the  advantages of using Apache Drill for complex joins and nested  JSON data, while DuckDB is employed for analytical queries when  sufficient memory resources are available. Experimental results  demonstrate the framework’s capability to reduce execution time  and enhance resource utilization across various workloads. By  providing insights into hybrid query processing techniques, this  work aims to contribute to the efficient management of JSON 
data in modern data-driven applications.