v4.8.0 Release Notes

Release date: September, 2026

Version: v4.8.0

Important

Upgrade classification: Mandatory

This release fixes critical defects that can cause queries to return incorrect results without raising an error, data to become corrupted, and clusters to become unavailable. Upgrade as soon as possible.

SynxDB v4.8.0 adds time-series data support, extends Apache Iceberg tables with partitioning, schema evolution, and historical snapshot queries, and lets you use gpupgrade to upgrade a cluster that contains PAX tables. Read the upgrade notes before you upgrade.

• Time-series data: Create time-series tables with the time_series extension, and use continuous aggregates, compression of historical data, and background jobs.

• Data federation and lakehouse integration: Iceberg tables support partitioning by column and historical snapshot queries, tables in a builtin catalog support schema evolution, and tables in an external catalog can cache metadata; ORC and Parquet foreign tables can cache file metadata.

• Query processing and optimization: Parquet foreign tables support vectorized scans.

• Storage: When a PAX table fetches sparse rows through an index, it decompresses only the chunks that hold the matching rows.

• Tools and utilities: gpupgrade can upgrade a cluster that contains PAX tables and rebuilds bitmap indexes automatically.

New features

Database

Category	Feature	User documents
Time-series data	The time_series extension creates time-series tables that split data into chunks by time automatically.	Create a time-series table
Time-series data	Continuous aggregates incrementally materialize summaries grouped into time buckets.	Use continuous aggregates
Time-series data	You can compress historical data in a time-series table into a columnar format.	Compress historical data
Time-series data	Background jobs refresh continuous aggregates and compress historical data on a schedule set by policies.	Configure background jobs and policies
Data federation and lakehouse integration	Iceberg tables support partitioning by column.	Partition by column
Data federation and lakehouse integration	Iceberg tables in a builtin catalog support schema evolution through ALTER TABLE.	Evolve the schema
Data federation and lakehouse integration	You can query the historical data of an Iceberg table by snapshot ID or by point in time.	Query a historical snapshot
Data federation and lakehouse integration	Iceberg tables in an external catalog can cache metadata through the metadata_cache option.	Control metadata freshness for external-catalog tables
Data federation and lakehouse integration	ORC and Parquet foreign tables can cache file metadata.	Cache the metadata of ORC and Parquet files
Query processing and optimization	Parquet foreign tables support vectorized scans.	Use vectorized scans on foreign tables
Storage	When a PAX table fetches sparse rows through an index, it decompresses only the chunks that hold the matching rows.	Speed up sparse fetches that go through an index
Tools and utilities	gpupgrade can upgrade a cluster that contains PAX tables and rebuilds bitmap indexes automatically.	Upgrade with gpupgrade, Confirm the index rebuild

New feature details

Time-series data

• Time-series tables: A time-series table splits data into chunks by time automatically, so a query that filters on a time range scans only the matching chunks instead of the whole table. Data that arrives continuously over time and is queried mostly for a recent period, such as device monitoring data, application metrics, logs, and IoT sensor readings, can go directly into a time-series table. Compared with a regular table that you partition by time yourself, you do not need to create partitions in advance or keep adding new ones as time passes. With time_bucket, time_bucket_gapfill, locf, and interpolate, you can compute statistics by time bucket in SQL and fill in results for time periods that have no data.

  See Create a time-series table.

• Continuous aggregates: A continuous aggregate materializes summaries grouped into time buckets incrementally, so newly written data only adds its own share of the computation. Minute-level and hour-level monitoring reports and dashboards no longer rescan the raw data every time, and the cost of summarizing no longer grows with the total amount of data. In real-time mode, query results also include the most recent data that has not been materialized yet, so newly written data appears in reports right away. When you need stable response times, you can switch to returning only the materialized data. One time-series table can keep several summaries at different granularities, such as hourly and daily.

  See Use continuous aggregates.

• Compression of historical data: You can compress historical chunks that no longer receive writes into a columnar format. This usually reduces their storage to a fraction of the original size and makes aggregate analysis on historical data more efficient. Time-series data often has to be kept for months or years, while detailed queries focus on recent data. Keeping recent data in row format and compressing historical data into columnar format balances write efficiency against the cost of long-term retention. Compressed data also replicates to the mirrors, so it survives a failover.

  See Compress historical data.

• Background jobs and policies: You can configure continuous aggregate refreshes and historical data compression as policies, which background jobs built into the database run on a schedule, without an external scheduler. These maintenance tasks run with the database, so operators no longer maintain extra scripts and scheduled tasks, and you can query the execution history of each job directly in the database. You can also register your own stored procedures as scheduled jobs for routine maintenance such as data cleanup.

  See Configure background jobs and policies.

Data federation and lakehouse integration

• Partition Iceberg tables by column: When you create an Iceberg table, you can name partition columns with PARTITION BY, and the table stores its data files separately by the values of those columns. When engines such as Spark and Trino share the same data, they can prune files by partition, so queries that filter on dimensions such as region or date read less data. The partition information is recorded in the standard Iceberg partition spec. It creates no parent or child tables, and you still use the table as a single table in SynxDB. Fact tables loaded in daily batches and business data archived by region can declare their partitions directly at table creation.

  See Partition by column.

• Schema evolution for Iceberg tables: An Iceberg table managed by a builtin catalog supports adding, dropping, and renaming columns and widening column types through ALTER TABLE. The change touches only metadata and does not rewrite existing data files, so it completes immediately even on a large table. When business fields change, for example when you add a field to an existing table or widen a counter column from integer to bigint to hold larger values, you do not need to rebuild the table or migrate data.

  See Evolve the schema.

• Historical snapshot queries: You can list the historical snapshots of an Iceberg table and read the data as it was at a given snapshot ID or point in time. This helps you check the data at a specific moment, investigate accidental writes or deletes, or reproduce a historical report without affecting the current data. You can filter and aggregate the results like a regular table, or join them with the current data to find the rows that changed between two points in time. Even if the table structure changed later, the query still returns the column definitions as they were at the time of the snapshot.

  See Query a historical snapshot.

• Metadata freshness control for external-catalog tables: An Iceberg table attached to an existing table in an external catalog, such as Hive Metastore or Polaris, can use the metadata_cache table option to set how long SynxDB reuses the metadata. When the external catalog is remote, queries do not need a round trip to the catalog every time, which suits analytical queries that can accept a delay of a few minutes. Write operations always use the latest metadata and do not overwrite data that other engines committed. By default, the option disables caching. Set it only for tables that you query often and that do not need the freshest data.

  See Control metadata freshness for external-catalog tables.

• File metadata cache for ORC and Parquet: When you turn on datalake.enable_file_meta_cache, ORC and Parquet foreign tables cache the metadata of data files, so opening the same file again does not read the metadata from the object storage or HDFS. When you query the same set of files repeatedly, this reduces access to the external storage. The parameter is off by default. You can turn it on in a session as needed, and the segment log shows whether each file hits the cache.

  See Cache the metadata of ORC and Parquet files.

Query processing and optimization

• Vectorized scans on Parquet foreign tables: With vectorization enabled, Parquet files on object storage or HDFS can also use vectorized execution through foreign tables, the same as ORC foreign tables. Parquet is the most common file format in a data lake. You can run ad hoc analysis directly in SynxDB on data that engines such as Spark wrote, or join it with tables in the database, and benefit from vectorized execution without loading the data first. When a query references a column that is not supported, the whole query falls back to row-based execution and returns the same results, so turning on vectorization keeps query results correct.

  See Use vectorized scans on foreign tables.

Storage

• Faster sparse fetches through an index on PAX tables: When writing data, a PAX table splits compressed columns into smaller chunks. When an index reads a few scattered rows, the scan decompresses only the chunks that hold the matching rows instead of the whole group. The same PAX table often serves both analytical queries that scan the whole table and index lookups that fetch a few detail rows by order number or device ID, and the latter now decompress less data. The optimization is on by default and applies to newly written data. For a table that serves mostly index point lookups, you can lower the number of rows per chunk to decompress even less data.

  See Speed up sparse fetches that go through an index.

Tools and utilities

• Upgrade clusters that contain PAX tables with gpupgrade: You can now upgrade a cluster that uses PAX tables with gpupgrade, so you no longer need to export and reload the data of those tables separately. The upgrade also rebuilds bitmap indexes automatically, which removes the step of rebuilding them one by one and the risk of wrong query results from a missed rebuild. After the upgrade, you can confirm the rebuild with the queries in the documentation.

  See Upgrade with gpupgrade and Confirm the index rebuild.

Product change information

Upgrade notes

Note the following when you upgrade from v4.7 or earlier to v4.8. For the complete upgrade procedure, see Upgrade with gpupgrade.

• The catalog version has changed in this release, so you cannot upgrade by replacing the binaries alone. Use gpupgrade instead. If you replace only the binaries, the cluster reports database files are incompatible with server at startup. See Upgrade the cluster.

• The upgrade does not update installed extensions. Run the update_extensions.sql that pg_upgrade generates to update datalake_fdw to 1.1. Otherwise, features that depend on 1.1, such as historical snapshot queries on Iceberg tables, are unavailable. After the update, if you need to run ALTER COLUMN TYPE on an Iceberg table, create one function manually in each database. See Update extensions.

• In earlier releases, a query run through a cursor, a PL/pgSQL FOR ... IN SELECT loop, or a parameterized statement could leave duplicate primary keys in a heap table distributed by its primary key, or lose inserted rows, when it ran concurrently with subtransactions in other sessions. This release fixes the problem, but data that is already corrupted does not recover on its own. If your cluster runs this kind of workload, check the heap tables distributed by their primary keys for duplicate keys after the upgrade. Compare the result of count(*) with that of count(DISTINCT <primary_key_column>). If they differ, the table has duplicate rows. You can also run REINDEX on the primary key index, which reports a uniqueness violation when duplicate rows exist. Do not check with GROUP BY ... HAVING count(*) > 1 under GPORCA. GPORCA relies on the uniqueness of the primary key to skip this check, so the query finds no duplicate rows.

Behavior changes

• New data written to a PAX table is now compressed in chunks of a fixed row count by default, which speeds up sparse fetches through an index and increases storage slightly. Existing data is unaffected. To keep the original format for new data, turn off pax.enable_chunk_index before writing. See Speed up sparse fetches that go through an index.

• With vectorization enabled, a query in which every window has a PARTITION BY now uses sort-based window aggregation by default. If the query also carries a Top-K qualification, Partition Top-K first prunes each partition down to its candidate rows. Previously, the optimizer could choose hash-based window aggregation for such a query. The new vector.enable_partition_topk parameter controls this behavior. See Execution methods for window functions.

• An explicit DISTRIBUTED clause in CREATE FOREIGN TABLE now takes effect. Previously, the database silently ignored the clause. As a result, a writable external table migrated from CREATE EXTERNAL TABLE writes in parallel from all segments again, and a table definition exported by pg_dump matches the original table after restore.

GUC configuration parameters

Newly added GUCs

The following configuration parameters are added:

• datalake.enable_file_meta_cache: default off. Caches the metadata of data files for ORC and Parquet foreign tables in gopher. See Configuration parameters.

• datalake.file_meta_max_size: default 512kB, range [64kB, 2097151kB]. Sets the maximum size of metadata that can be cached for a single file. See Configuration parameters.

• datalake.external_table_text_block_size: default 32MB, range [64kB, 1GB]. Sets the block size that the text parser uses to split a large file when datalake.external_table_new_text is on. The value must be larger than the longest record in the file. See Configuration parameters.

• pax.enable_chunk_index: default on. Controls whether PAX splits compressed columns into chunks of a fixed row count and compresses each chunk separately when writing data, which speeds up sparse fetches through an index. See Configuration parameters.

• pax.chunk_index_rows: default 8192, range [1024, 131072]. Sets the number of rows per chunk when pax.enable_chunk_index is on. See Configuration parameters.

• vector.enable_partition_topk: default on. With vectorization enabled, makes a query in which every window has a PARTITION BY use sort-based window aggregation. See Configuration parameters.

• vector.enable_window_hash_agg: default on. With vectorization enabled, controls whether the optimizer can use hash-based window aggregation for window functions. See Configuration parameters.

• vector.partition_topk_require_topk_qual: default off. When on, generates a Partition Top-K plan for a window query only if the query carries a Top-K qualification. See Configuration parameters.

The following time_series.* parameters are available only after you preload the time_series extension through shared_preload_libraries:

• time_series.bgw_enabled: default on. Controls whether the background job scheduler of the time_series extension is enabled. See Configuration parameters.

• time_series.bgw_launcher_poll_time: default 60000 (ms). Sets how often the launcher process checks which databases have the extension installed. See Configuration parameters.

• time_series.bgw_log_level: default warning. Sets the log level of the background job subsystem. See Configuration parameters.

• time_series.bgw_scheduler_restart_time: default -1 (s). Sets how long the postmaster waits before it restarts a scheduler process that exited abnormally. -1 leaves recovery to the launcher. See Configuration parameters.

• time_series.cagg_default_max_retries: default -1. Sets the default number of retries for a continuous aggregate refresh job. -1 means no limit. See Configuration parameters.

• time_series.cagg_default_max_runtime: default 0. Sets the default maximum run time of a continuous aggregate refresh job. 0 means no limit. See Configuration parameters.

• time_series.chunk_append_max_chunks: default 64. Sets the maximum number of child plans that a ChunkAppend plan node can contain. See Configuration parameters.

• time_series.debug_bgw_scheduler_exit_status: default 0. A debugging parameter that sets the status code a scheduler process returns when it exits. See Configuration parameters.

• time_series.enable_cagg_create: default on. Controls whether you can create continuous aggregates. See Configuration parameters.

• time_series.enable_chunk_append: default on. Controls whether a query that sorts by time and has a LIMIT gets a ChunkAppend plan node. See Configuration parameters.

• time_series.materializations_per_refresh_window: default 10. Sets the number of materialization batches within a single refresh window of a continuous aggregate. See Configuration parameters.

• time_series.max_background_workers: default 16. Sets the maximum total number of background processes that the time_series extension can use. See Configuration parameters.

• time_series.max_jobs_per_scheduler: default 16. Sets the maximum number of job processes that a single scheduler process can start at the same time. See Configuration parameters.

• time_series.shutdown_bgw_scheduler: default off. When you set it to on and reload the configuration, the scheduler process exits and restarts automatically after about one minute. See Configuration parameters.

Changed GUCs

The following configuration parameter has changed:

• vector.pool_threads: a value of 1 is now treated as 0, which disables the thread pool. For threaded execution, set it to 2 or higher. See Configuration parameters.

Improvements

Storage and access methods

• Bitmap scans and index scans on PAX tables now decompress only the columns that the query references, which lowers the cost of queries that read a few columns of a wide table.

Security

• Upgraded dependencies with known vulnerabilities in the datalake Java components, including Jackson, Netty, and Tomcat, and removed dependencies that are no longer maintained, which greatly reduces the high-severity alerts in security scans.

Bug fixes

Data federation and lakehouse integration

• Fixed a problem where, with datalake.external_table_new_text on, reading a CSV or text foreign table larger than 32 MB silently truncated one row and dropped the next one at every block boundary.

• Fixed a problem where, when an HDFS volume specified only server_name in SQL, segments could not get the NameNode and other connection settings from gphdfs.conf. You could create the Iceberg table but could not read or write it, and the gopher service on the segments exited. An incomplete volume configuration in a builtin catalog now fails at table creation.

• Fixed a gphdfs:// foreign table binding to the wrong foreign data wrapper (FDW), which crashed the coordinator on SELECT.

• Fixed a namespace resolution error at commit time on every INSERT, UPDATE, or DELETE against an Iceberg table in an external catalog when the table did not specify namespace and the catalog did not set default_namespace.

Query optimizer and executor

• Fixed intermittent crashes of query executor (QE) processes in a co-located parallel hash join, in cases such as ORDER BY ... LIMIT and materialized view refreshes.

• Fixed stability problems in vectorized execution, including a race in a ShareInputScan within one slice, a double release of an internal structure, and a query deadlock when vector.pool_threads is 1.

• Fixed leftover spill files from a ShareInputScan under vectorized execution, and a resource leak when a transaction aborts.

• Fixed a problem where vectorized execution left behind internal resources for every plan it ran. With long-lived connections such as those from a connection pool, the memory of a backend process no longer grows with the number of vectorized queries.

Storage and access methods

• Fixed a problem where VACUUM FULL, CLUSTER, or a table-rewriting ALTER TABLE on a PAX table wrote a non-zero relfrozenxid, which kept the datfrozenxid of the database from advancing and moved it toward transaction ID wraparound protection.

• Fixed an out-of-memory error on a PAX table that showed only an internal function name. The error now states that memory ran out and how many bytes were requested.

• Fixed a problem where the database treated a custom access method that shares the AO table handler as non-AO, which made VACUUM FREEZE on such a table abort.

Processes and concurrency

• Fixed a problem where a query run through a cursor or a parameterized statement (for example, a prepared statement in JDBC, psycopg2, or ODBC) could not see subtransactions still in progress in other sessions, which could leave duplicate primary keys in a heap table distributed by its primary key, or lose inserted rows. Data corrupted before the fix does not recover on its own. For how to check for it, see Upgrade notes.

• Fixed intermittent query failures with the error global xmin > xmin under workloads with concurrent rollbacks.