## E.24. Release 10.4

Release date: 2018-05-10

This release contains a variety of fixes from 10.3. For information about new features in major release 10, see Section E.28.

### E.24.1. Migration to Version 10.4

A dump/restore is not required for those running 10.X.

However, if you use the `adminpack` extension, you should update it as per the first changelog entry below.

Also, if the function marking mistakes mentioned in the second and third changelog entries below affect you, you will want to take steps to correct your database catalogs.

Also, if you are upgrading from a version earlier than 10.3, see Section E.25.

### E.24.2. Changes

- Remove public execute privilege from `contrib/adminpack`'s `pg_logfile_rotate()` function (Stephen Frost)

  `pg_logfile_rotate()` is a deprecated wrapper for the core function `pg_rotate_logfile()`. When that function was changed to rely on SQL privileges for access control rather than a hard-coded superuser check, `pg_logfile_rotate()` should have been updated as well, but the need for this was missed. Hence, if `adminpack` is installed, any user could request a logfile rotation, creating a minor security issue.

  After installing this update, administrators should update `adminpack` by performing `ALTER EXTENSION adminpack UPDATE` in each database in which `adminpack` is installed. (CVE-2018-1115)

- Fix incorrect volatility markings on a few built-in functions (Thomas Munro, Tom Lane)

  The functions `query_to_xml`, `cursor_to_xml`, `cursor_to_xmlschema`, `query_to_xmlschema`, and `query_to_xml_and_xmlschema` should be marked volatile because they execute user-supplied queries that might contain volatile operations. They were not, leading to a risk of incorrect query optimization. This has been repaired for new installations by correcting the initial catalog data, but existing installations will continue to contain the incorrect markings. Practical use of these functions seems to pose little hazard, but in case of trouble, it can be fixed by manually updating these functions' `pg_proc` entries, for example `ALTER FUNCTION pg_catalog.query_to_xml(text, boolean, boolean, text) VOLATILE`. (Note that that will need to be done in each database of the installation.) Another option is to pg_upgrade the database to a version containing the corrected initial data.

- Fix incorrect parallel-safety markings on a few built-in functions (Thomas Munro, Tom Lane)

  The functions `brin_summarize_new_values`, `brin_summarize_range`, `brin_desummarize_range`, `gin_clean_pending_list`, `cursor_to_xml`, `cursor_to_xmlschema`, `ts_rewrite`, `ts_stat`, `binary_upgrade_create_empty_extension`, and `pg_import_system_collations` should be marked parallel-unsafe; some because they perform database modifications directly, and others because they execute user-supplied queries that might do so. They were marked parallel-restricted instead, leading to a risk of unexpected query errors. This has been repaired for new installations by correcting the initial catalog data, but existing installations will continue to contain the incorrect markings. Practical use of these functions seems to pose little hazard unless `force_parallel_mode` is turned on. In case of trouble, it can be fixed by manually updating these functions' `pg_proc` entries, for example `ALTER FUNCTION pg_catalog.brin_summarize_new_values(regclass) PARALLEL UNSAFE`. (Note that that will need to be done in each database of the installation.) Another option is to pg_upgrade the database to a version containing the corrected initial data.

- Avoid re-using TOAST value OIDs that match dead-but-not-yet-vacuumed TOAST entries (Pavan Deolasee)

  Once the OID counter has wrapped around, it's possible to assign a TOAST value whose OID matches a previously deleted entry in the same TOAST table. If that entry were not yet vacuumed away, this resulted in “unexpected chunk number 0 (expected 1) for toast value `nnnnn`” errors, which would persist until the dead entry was removed by `VACUUM`. Fix by not selecting such OIDs when creating a new TOAST entry.

- Correctly enforce any `CHECK` constraints on individual partitions during `COPY` to a partitioned table (Etsuro Fujita)

  Previously, only constraints declared for the partitioned table as a whole were checked.

- Accept `TRUE` and `FALSE` as partition bound values (Amit Langote)

  Previously, only string-literal values were accepted for a boolean partitioning column. But then pg_dump would print such values as `TRUE` or `FALSE`, leading to dump/reload failures.

- Fix memory management for partition key comparison functions (Álvaro Herrera, Amit Langote)

  This error could lead to crashes when using user-defined operator classes for partition keys.

- Fix possible crash when a query inserts tuples in several partitions of a partitioned table, and those partitions don't have identical row types (Etsuro Fujita, Amit Langote)

- Change `ANALYZE`'s algorithm for updating `pg_class`.`reltuples` (David Gould)

  Previously, pages not actually scanned by `ANALYZE` were assumed to retain their old tuple density. In a large table where `ANALYZE` samples only a small fraction of the pages, this meant that the overall tuple density estimate could not change very much, so that `reltuples` would change nearly proportionally to changes in the table's physical size (`relpages`) regardless of what was actually happening in the table. This has been observed to result in `reltuples` becoming so much larger than reality as to effectively shut off autovacuuming. To fix, assume that `ANALYZE`'s sample is a statistically unbiased sample of the table (as it should be), and just extrapolate the density observed within those pages to the whole table.

- Include extended-statistics objects in the set of table properties duplicated by `CREATE TABLE ... LIKE ... INCLUDING ALL` (David Rowley)

  Also add an `INCLUDING STATISTICS` option, to allow finer-grained control over whether this happens.

- Fix `CREATE TABLE ... LIKE` with `bigint` identity columns (Peter Eisentraut)

  On platforms where `long` is 32 bits (which includes 64-bit Windows as well as most 32-bit machines), copied sequence parameters would be truncated to 32 bits.

- Avoid deadlocks in concurrent `CREATE INDEX CONCURRENTLY` commands that are run under `SERIALIZABLE` or `REPEATABLE READ` transaction isolation (Tom Lane)

- Fix possible slow execution of `REFRESH MATERIALIZED VIEW CONCURRENTLY` (Thomas Munro)

- Fix `UPDATE/DELETE ... WHERE CURRENT OF` to not fail when the referenced cursor uses an index-only-scan plan (Yugo Nagata, Tom Lane)

- Fix incorrect planning of join clauses pushed into parameterized paths (Andrew Gierth, Tom Lane)

  This error could result in misclassifying a condition as a “join filter” for an outer join when it should be a plain “filter” condition, leading to incorrect join output.

- Fix possibly incorrect generation of an index-only-scan plan when the same table column appears in multiple index columns, and only some of those index columns use operator classes that can return the column value (Kyotaro Horiguchi)

- Fix misoptimization of `CHECK` constraints having provably-NULL subclauses of top-level `AND`/`OR` conditions (Tom Lane, Dean Rasheed)

  This could, for example, allow constraint exclusion to exclude a child table that should not be excluded from a query.

- Prevent planner crash when a query has multiple `GROUPING SETS`, none of which can be implemented by sorting (Andrew Gierth)

- Fix executor crash due to double free in some `GROUPING SETS` usages (Peter Geoghegan)

- Fix misexecution of self-joins on transition tables (Thomas Munro)

- Avoid crash if a table rewrite event trigger is added concurrently with a command that could call such a trigger (Álvaro Herrera, Andrew Gierth, Tom Lane)

- Avoid failure if a query-cancel or session-termination interrupt occurs while committing a prepared transaction (Stas Kelvich)

- Fix query-lifespan memory leakage in repeatedly executed hash joins (Tom Lane)

- Fix possible leak or double free of visibility map buffer pins (Amit Kapila)

- Avoid spuriously marking pages as all-visible (Dan Wood, Pavan Deolasee, Álvaro Herrera)

  This could happen if some tuples were locked (but not deleted). While queries would still function correctly, vacuum would normally ignore such pages, with the long-term effect that the tuples were never frozen. In recent releases this would eventually result in errors such as “found multixact `nnnnn` from before relminmxid `nnnnn`”.

- Fix overly strict sanity check in `heap_prepare_freeze_tuple` (Álvaro Herrera)

  This could result in incorrect “cannot freeze committed xmax” failures in databases that have been pg_upgrade'd from 9.2 or earlier.

- Prevent dangling-pointer dereference when a C-coded before-update row trigger returns the “old” tuple (Rushabh Lathia)

- Reduce locking during autovacuum worker scheduling (Jeff Janes)

  The previous behavior caused drastic loss of potential worker concurrency in databases with many tables.

- Ensure client hostname is copied while copying `pg_stat_activity` data to local memory (Edmund Horner)

  Previously the supposedly-local snapshot contained a pointer into shared memory, allowing the client hostname column to change unexpectedly if any existing session disconnected.

- Handle `pg_stat_activity` information for auxiliary processes correctly (Edmund Horner)

  The `application_name`, `client_hostname`, and `query` fields might show incorrect data for such processes.

- Fix incorrect processing of multiple compound affixes in `ispell` dictionaries (Arthur Zakirov)

- Fix collation-aware searches (that is, indexscans using inequality operators) in SP-GiST indexes on text columns (Tom Lane)

  Such searches would return the wrong set of rows in most non-C locales.

- Prevent query-lifespan memory leakage with SP-GiST operator classes that use traversal values (Anton Dignös)

- Count the number of index tuples correctly during initial build of an SP-GiST index (Tomas Vondra)

  Previously, the tuple count was reported to be the same as that of the underlying table, which is wrong if the index is partial.

- Count the number of index tuples correctly during vacuuming of a GiST index (Andrey Borodin)

  Previously it reported the estimated number of heap tuples, which might be inaccurate, and is certainly wrong if the index is partial.

- Fix a corner case where a streaming standby gets stuck at a WAL continuation record (Kyotaro Horiguchi)

- In logical decoding, avoid possible double processing of WAL data when a walsender restarts (Craig Ringer)

- Fix logical replication to not assume that type OIDs match between the local and remote servers (Masahiko Sawada)

- Allow `scalarltsel` and `scalargtsel` to be used on non-core datatypes (Tomas Vondra)

- Reduce libpq's memory consumption when a server error is reported after a large amount of query output has been collected (Tom Lane)

  Discard the previous output before, not after, processing the error message. On some platforms, notably Linux, this can make a difference in the application's subsequent memory footprint.

- Fix double-free crashes in ecpg (Patrick Krecker, Jeevan Ladhe)

- Fix ecpg to handle `long long int` variables correctly in MSVC builds (Michael Meskes, Andrew Gierth)

- Fix mis-quoting of values for list-valued GUC variables in dumps (Michael Paquier, Tom Lane)

  The `local_preload_libraries`, `session_preload_libraries`, `shared_preload_libraries`, and `temp_tablespaces` variables were not correctly quoted in pg_dump output. This would cause problems if settings for these variables appeared in `CREATE FUNCTION ... SET` or `ALTER DATABASE/ROLE ... SET` clauses.

- Fix pg_recvlogical to not fail against pre-v10 PostgreSQL servers (Michael Paquier)

  A previous fix caused pg_recvlogical to issue a command regardless of server version, but it should only be issued to v10 and later servers.

- Ensure that pg_rewind deletes files on the target server if they are deleted from the source server during the run (Takayuki Tsunakawa)

  Failure to do this could result in data inconsistency on the target, particularly if the file in question is a WAL segment.

- Fix pg_rewind to handle tables in non-default tablespaces correctly (Takayuki Tsunakawa)

- Fix overflow handling in PL/pgSQL integer `FOR` loops (Tom Lane)

  The previous coding failed to detect overflow of the loop variable on some non-gcc compilers, leading to an infinite loop.

- Adjust PL/Python regression tests to pass under Python 3.7 (Peter Eisentraut)

- Support testing PL/Python and related modules when building with Python 3 and MSVC (Andrew Dunstan)

- Fix errors in initial build of `contrib/bloom` indexes (Tomas Vondra, Tom Lane)

  Fix possible omission of the table's last tuple from the index. Count the number of index tuples correctly, in case it is a partial index.

- Rename internal `b64_encode` and `b64_decode` functions to avoid conflict with Solaris 11.4 built-in functions (Rainer Orth)

- Sync our copy of the timezone library with IANA tzcode release 2018e (Tom Lane)

  This fixes the zic timezone data compiler to cope with negative daylight-savings offsets. While the PostgreSQL project will not immediately ship such timezone data, zic might be used with timezone data obtained directly from IANA, so it seems prudent to update zic now.

- Update time zone data files to tzdata release 2018d for DST law changes in Palestine and Antarctica (Casey Station), plus historical corrections for Portugal and its colonies, as well as Enderbury, Jamaica, Turks & Caicos Islands, and Uruguay.