Chapter 34. Built-in Analytical Platform

Table of Contents

34.1. Architecture
34.1.1. Data Export and Import
34.1.2. File Catalogue and Process Automation
34.2. Considerations and Limitations
34.3. Deployment Recommendations
34.3.1. Impact on OLTP Workload from Analytical Queries
34.3.2. Requirements for Data Security and OLAP Workload Distribution
34.3.3. Available Network, Storage, and Computational Resources
34.4. Capacity Planning
34.4.1. Total Amount of OLAP Data in the Storage
34.4.2. Amount of Data Processed by a Standard Analytical Query
34.4.3. Required Analytical Query Execution Time
34.4.4. Number of Analytical Queries Executed per Unit of Time
34.5. Storage for Analytical Files
34.5.1. Parquet Files
34.5.2. Directory Structure
34.6. Data Type Correspondence in Postgres Pro, DuckDB, and Parquet
34.7. Using the Built-in Analytical Platform
34.7.1. Getting Started
34.7.2. Configuring Security for Accessing OLAP Resources
34.7.3. Exporting OLAP Data to a Storage
34.7.4. Creating a View for Accessing Parquet Files
34.7.5. Configuring Computational Resource Limits
34.7.6. Basic Scenario of an Analyst's work
34.8. Benchmark Test Results
34.8.1. Preparation of Test Data and Tables
34.8.2. TCP-H Benchmark Test Results
34.8.3. TCP-DS Benchmark Test Results
34.8.4. Conclusions

Built-in analytical platform is a Postgres Pro solution designed to operate with OLAP (Online Analytical Processing) workloads. The key component of the platform is pgpro_duckdb, an extension that allows building a modern analytical platform within a Postgres Pro instance without any additional tools.

Modern analytical platforms usually include several independent components, which allows for greater flexibility and scalability. The main components of the system are the query execution engine, data storage, and catalog of analytical tables. The DuckDB pluggable engine supports vectorized query execution and columnar data formats. It can access both Postgres Pro tables and external storages.

The built-in analytical platform offers the following advantages:

Note

The pgpro_duckdb extension is currently in an experimental phase.