## E.241. Release 8.4

Release date: 2009-07-01

### E.241.1. Overview

After many years of development, PostgreSQL has become feature-complete in many areas. This release shows a targeted approach to adding features (e.g., authentication, monitoring, space reuse), and adds capabilities defined in the later SQL standards. The major areas of enhancement are:

- Windowing Functions

- Common Table Expressions and Recursive Queries

- Default and variadic parameters for functions

- Parallel Restore

- Column Permissions

- Per-database locale settings

- Improved hash indexes

- Improved join performance for `EXISTS` and `NOT EXISTS` queries

- Easier-to-use Warm Standby

- Automatic sizing of the Free Space Map

- Visibility Map (greatly reduces vacuum overhead for slowly-changing tables)

- Version-aware psql (backslash commands work against older servers)

- Support SSL certificates for user authentication

- Per-function runtime statistics

- Easy editing of functions in psql

- New contrib modules: pg_stat_statements, auto_explain, citext, btree_gin

The above items are explained in more detail in the sections below.

### E.241.2. Migration to Version 8.4

A dump/restore using pg_dump is required for those wishing to migrate data from any previous release.

Observe the following incompatibilities:

#### E.241.2.1. General

- Use 64-bit integer datetimes by default (Neil Conway)

  Previously this was selected by configure's `--enable-integer-datetimes` option. To retain the old behavior, build with `--disable-integer-datetimes`.

- Remove ipcclean utility command (Bruce)

  The utility only worked on a few platforms. Users should use their operating system tools instead.

#### E.241.2.2. Server Settings

- Change default setting for `log_min_messages` to `warning` (previously it was `notice`) to reduce log file volume (Tom)

- Change default setting for `max_prepared_transactions` to zero (previously it was 5) (Tom)

- Make `debug_print_parse`, `debug_print_rewritten`, and `debug_print_plan` output appear at `LOG` message level, not `DEBUG1` as formerly (Tom)

- Make `debug_pretty_print` default to `on` (Tom)

- Remove `explain_pretty_print` parameter (no longer needed) (Tom)

- Make `log_temp_files` settable by superusers only, like other logging options (Simon Riggs)

- Remove automatic appending of the epoch timestamp when no `%` escapes are present in `log_filename` (Robert Haas)

  This change was made because some users wanted a fixed log filename, for use with an external log rotation tool.

- Remove `log_restartpoints` from `recovery.conf`; instead use `log_checkpoints` (Simon)

- Remove `krb_realm` and `krb_server_hostname`; these are now set in `pg_hba.conf` instead (Magnus)

- There are also significant changes in `pg_hba.conf`, as described below.

#### E.241.2.3. Queries

- Change `TRUNCATE` and `LOCK` to apply to child tables of the specified table(s) (Peter)

  These commands now accept an `ONLY` option that prevents processing child tables; this option must be used if the old behavior is needed.

- `SELECT DISTINCT` and `UNION`/`INTERSECT`/`EXCEPT` no longer always produce sorted output (Tom)

  Previously, these types of queries always removed duplicate rows by means of Sort/Unique processing (i.e., sort then remove adjacent duplicates). Now they can be implemented by hashing, which will not produce sorted output. If an application relied on the output being in sorted order, the recommended fix is to add an `ORDER BY` clause. As a short-term workaround, the previous behavior can be restored by disabling `enable_hashagg`, but that is a very performance-expensive fix. `SELECT DISTINCT ON` never uses hashing, however, so its behavior is unchanged.

- Force child tables to inherit `CHECK` constraints from parents (Alex Hunsaker, Nikhil Sontakke, Tom)

  Formerly it was possible to drop such a constraint from a child table, allowing rows that violate the constraint to be visible when scanning the parent table. This was deemed inconsistent, as well as contrary to SQL standard.

- Disallow negative `LIMIT` or `OFFSET` values, rather than treating them as zero (Simon)

- Disallow `LOCK TABLE` outside a transaction block (Tom)

  Such an operation is useless because the lock would be released immediately.

- Sequences now contain an additional `start_value` column (Zoltan Boszormenyi)

  This supports `ALTER SEQUENCE ... RESTART`.

#### E.241.2.4. Functions and Operators

- Make `numeric` zero raised to a fractional power return `0`, rather than throwing an error, and make `numeric` zero raised to the zero power return `1`, rather than error (Bruce)

  This matches the longstanding `float8` behavior.

- Allow unary minus of floating-point values to produce minus zero (Tom)

  The changed behavior is more IEEE-standard compliant.

- Throw an error if an escape character is the last character in a `LIKE` pattern (i.e., it has nothing to escape) (Tom)

  Previously, such an escape character was silently ignored, thus possibly masking application logic errors.

- Remove `~=~` and `~<>~` operators formerly used for `LIKE` index comparisons (Tom)

  Pattern indexes now use the regular equality operator.

- `xpath()` now passes its arguments to libxml without any changes (Andrew)

  This means that the XML argument must be a well-formed XML document. The previous coding attempted to allow XML fragments, but it did not work well.

- Make `xmlelement()` format attribute values just like content values (Peter)

  Previously, attribute values were formatted according to the normal SQL output behavior, which is sometimes at odds with XML rules.

- Rewrite memory management for libxml-using functions (Tom)

  This change should avoid some compatibility problems with use of libxml in PL/Perl and other add-on code.

- Adopt a faster algorithm for hash functions (Kenneth Marshall, based on work of Bob Jenkins)

  Many of the built-in hash functions now deliver different results on little-endian and big-endian platforms.

##### E.241.2.4.1. Temporal Functions and Operators

- `DateStyle` no longer controls `interval` output formatting; instead there is a new variable `IntervalStyle` (Ron Mayer)

- Improve consistency of handling of fractional seconds in `timestamp` and `interval` output (Ron Mayer)

  This may result in displaying a different number of fractional digits than before, or rounding instead of truncating.

- Make `to_char()`'s localized month/day names depend on `LC_TIME`, not `LC_MESSAGES` (Euler Taveira de Oliveira)

- Cause `to_date()` and `to_timestamp()` to more consistently report errors for invalid input (Brendan Jurd)

  Previous versions would often ignore or silently misread input that did not match the format string. Such cases will now result in an error.

- Fix `to_timestamp()` to not require upper/lower case matching for meridian (`AM`/`PM`) and era (`BC`/`AD`) format designations (Brendan Jurd)

  For example, input value `ad` now matches the format string `AD`.

### E.241.3. Changes

Below you will find a detailed account of the changes between PostgreSQL 8.4 and the previous major release.

#### E.241.3.1. Performance

- Improve optimizer statistics calculations (Jan Urbanski, Tom)

  In particular, estimates for full-text-search operators are greatly improved.

- Allow `SELECT DISTINCT` and `UNION`/`INTERSECT`/`EXCEPT` to use hashing (Tom)

  This means that these types of queries no longer automatically produce sorted output.

- Create explicit concepts of semi-joins and anti-joins (Tom)

  This work formalizes our previous ad-hoc treatment of `IN (SELECT ...)` clauses, and extends it to `EXISTS` and `NOT EXISTS` clauses. It should result in significantly better planning of `EXISTS` and `NOT EXISTS` queries. In general, logically equivalent `IN` and `EXISTS` clauses should now have similar performance, whereas previously `IN` often won.

- Improve optimization of sub-selects beneath outer joins (Tom)

  Formerly, a sub-select or view could not be optimized very well if it appeared within the nullable side of an outer join and contained non-strict expressions (for instance, constants) in its result list.

- Improve the performance of `text_position()` and related functions by using Boyer-Moore-Horspool searching (David Rowley)

  This is particularly helpful for long search patterns.

- Reduce I/O load of writing the statistics collection file by writing the file only when requested (Martin Pihlak)

- Improve performance for bulk inserts (Robert Haas, Simon)

- Increase the default value of `default_statistics_target` from `10` to `100` (Greg Sabino Mullane, Tom)

  The maximum value was also increased from `1000` to `10000`.

- Perform `constraint_exclusion` checking by default in queries involving inheritance or `UNION ALL` (Tom)

  A new `constraint_exclusion` setting, `partition`, was added to specify this behavior.

- Allow I/O read-ahead for bitmap index scans (Greg Stark)

  The amount of read-ahead is controlled by `effective_io_concurrency`. This feature is available only if the kernel has `posix_fadvise()` support.

- Inline simple set-returning SQL functions in `FROM` clauses (Richard Rowell)

- Improve performance of multi-batch hash joins by providing a special case for join key values that are especially common in the outer relation (Bryce Cutt, Ramon Lawrence)

- Reduce volume of temporary data in multi-batch hash joins by suppressing “physical tlist” optimization (Michael Henderson, Ramon Lawrence)

- Avoid waiting for idle-in-transaction sessions during `CREATE INDEX CONCURRENTLY` (Simon)

- Improve performance of shared cache invalidation (Tom)

#### E.241.3.2. Server

##### E.241.3.2.1. Settings

- Convert many `postgresql.conf` settings to enumerated values so that `pg_settings` can display the valid values (Magnus)

- Add `cursor_tuple_fraction` parameter to control the fraction of a cursor's rows that the planner assumes will be fetched (Robert Hell)

- Allow underscores in the names of custom variable classes in `postgresql.conf` (Tom)

##### E.241.3.2.2. Authentication and security

- Remove support for the (insecure) `crypt` authentication method (Magnus)

  This effectively obsoletes pre-PostgreSQL 7.2 client libraries, as there is no longer any non-plaintext password method that they can use.

- Support regular expressions in `pg_ident.conf` (Magnus)

- Allow Kerberos/GSSAPI parameters to be changed without restarting the postmaster (Magnus)

- Support SSL certificate chains in server certificate file (Andrew Gierth)

  Including the full certificate chain makes the client able to verify the certificate without having all intermediate CA certificates present in the local store, which is often the case for commercial CAs.

- Report appropriate error message for combination of `MD5` authentication and `db_user_namespace` enabled (Bruce)

##### E.241.3.2.3. `pg_hba.conf`

- Change all authentication options to use `name=value` syntax (Magnus)

  This makes incompatible changes to the `ldap`, `pam` and `ident` authentication methods. All `pg_hba.conf` entries with these methods need to be rewritten using the new format.

- Remove the `ident sameuser` option, instead making that behavior the default if no usermap is specified (Magnus)

- Allow a usermap parameter for all external authentication methods (Magnus)

  Previously a usermap was only supported for `ident` authentication.

- Add `clientcert` option to control requesting of a client certificate (Magnus)

  Previously this was controlled by the presence of a root certificate file in the server's data directory.

- Add `cert` authentication method to allow user authentication via SSL certificates (Magnus)

  Previously SSL certificates could only verify that the client had access to a certificate, not authenticate a user.

- Allow `krb5`, `gssapi` and `sspi` realm and `krb5` host settings to be specified in `pg_hba.conf` (Magnus)

  These override the settings in `postgresql.conf`.

- Add `include_realm` parameter for `krb5`, `gssapi`, and `sspi` methods (Magnus)

  This allows identical usernames from different realms to be authenticated as different database users using usermaps.

- Parse `pg_hba.conf` fully when it is loaded, so that errors are reported immediately (Magnus)

  Previously, most errors in the file wouldn't be detected until clients tried to connect, so an erroneous file could render the system unusable. With the new behavior, if an error is detected during reload then the bad file is rejected and the postmaster continues to use its old copy.

- Show all parsing errors in `pg_hba.conf` instead of aborting after the first one (Selena Deckelmann)

- Support `ident` authentication over Unix-domain sockets on Solaris (Garick Hamlin)

##### E.241.3.2.4. Continuous Archiving

- Provide an option to `pg_start_backup()` to force its implied checkpoint to finish as quickly as possible (Tom)

  The default behavior avoids excess I/O consumption, but that is pointless if no concurrent query activity is going on.

- Make `pg_stop_backup()` wait for modified WAL files to be archived (Simon)

  This guarantees that the backup is valid at the time `pg_stop_backup()` completes.

- When archiving is enabled, rotate the last WAL segment at shutdown so that all transactions can be archived immediately (Guillaume Smet, Heikki)

- Delay “smart” shutdown while a continuous archiving base backup is in progress (Laurenz Albe)

- Cancel a continuous archiving base backup if “fast” shutdown is requested (Laurenz Albe)

- Allow `recovery.conf` boolean variables to take the same range of string values as `postgresql.conf` boolean variables (Bruce)

##### E.241.3.2.5. Monitoring

- Add `pg_conf_load_time()` to report when the PostgreSQL configuration files were last loaded (George Gensure)

- Add `pg_terminate_backend()` to safely terminate a backend (the `SIGTERM` signal works also) (Tom, Bruce)

  While it's always been possible to `SIGTERM` a single backend, this was previously considered unsupported; and testing of the case found some bugs that are now fixed.

- Add ability to track user-defined functions' call counts and runtimes (Martin Pihlak)

  Function statistics appear in a new system view, `pg_stat_user_functions`. Tracking is controlled by the new parameter `track_functions`.

- Allow specification of the maximum query string size in `pg_stat_activity` via new `track_activity_query_size` parameter (Thomas Lee)

- Increase the maximum line length sent to syslog, in hopes of improving performance (Tom)

- Add read-only configuration variables `segment_size`, `wal_block_size`, and `wal_segment_size` (Bernd Helmle)

- When reporting a deadlock, report the text of all queries involved in the deadlock to the server log (Itagaki Takahiro)

- Add `pg_stat_get_activity(pid)` function to return information about a specific process id (Magnus)

- Allow the location of the server's statistics file to be specified via `stats_temp_directory` (Magnus)

  This allows the statistics file to be placed in a RAM-resident directory to reduce I/O requirements. On startup/shutdown, the file is copied to its traditional location (`$PGDATA/global/`) so it is preserved across restarts.

#### E.241.3.3. Queries

- Add support for `WINDOW` functions (Hitoshi Harada)

- Add support for `WITH` clauses (CTEs), including `WITH RECURSIVE` (Yoshiyuki Asaba, Tatsuo Ishii, Tom)

- Add `TABLE` command (Peter)

  `TABLE tablename` is a SQL standard short-hand for `SELECT * FROM tablename`.

- Allow `AS` to be optional when specifying a `SELECT` (or `RETURNING`) column output label (Hiroshi Saito)

  This works so long as the column label is not any PostgreSQL keyword; otherwise `AS` is still needed.

- Support set-returning functions in `SELECT` result lists even for functions that return their result via a tuplestore (Tom)

  In particular, this means that functions written in PL/pgSQL and other PL languages can now be called this way.

- Support set-returning functions in the output of aggregation and grouping queries (Tom)

- Allow `SELECT FOR UPDATE`/`SHARE` to work on inheritance trees (Tom)

- Add infrastructure for SQL/MED (Martin Pihlak, Peter)

  There are no remote or external SQL/MED capabilities yet, but this change provides a standardized and future-proof system for managing connection information for modules like `dblink` and `plproxy`.

- Invalidate cached plans when referenced schemas, functions, operators, or operator classes are modified (Martin Pihlak, Tom)

  This improves the system's ability to respond to on-the-fly DDL changes.

- Allow comparison of composite types and allow arrays of anonymous composite types (Tom)

  This allows constructs such as `row(1, 1.1) = any (array[row(7, 7.7), row(1, 1.0)])`. This is particularly useful in recursive queries.

- Add support for Unicode string literal and identifier specifications using code points, e.g. `U&'d\0061t\+000061'` (Peter)

- Reject `\000` in string literals and `COPY` data (Tom)

  Previously, this was accepted but had the effect of terminating the string contents.

- Improve the parser's ability to report error locations (Tom)

  An error location is now reported for many semantic errors, such as mismatched datatypes, that previously could not be localized.

##### E.241.3.3.1. `TRUNCATE`

- Support statement-level `ON TRUNCATE` triggers (Simon)

- Add `RESTART`/`CONTINUE IDENTITY` options for `TRUNCATE TABLE` (Zoltan Boszormenyi)

  The start value of a sequence can be changed by `ALTER SEQUENCE START WITH`.

- Allow `TRUNCATE tab1, tab1` to succeed (Bruce)

- Add a separate `TRUNCATE` permission (Robert Haas)

##### E.241.3.3.2. `EXPLAIN`

- Make `EXPLAIN VERBOSE` show the output columns of each plan node (Tom)

  Previously `EXPLAIN VERBOSE` output an internal representation of the query plan. (That behavior is now available via `debug_print_plan`.)

- Make `EXPLAIN` identify subplans and initplans with individual labels (Tom)

- Make `EXPLAIN` honor `debug_print_plan` (Tom)

- Allow `EXPLAIN` on `CREATE TABLE AS` (Peter)

##### E.241.3.3.3. `LIMIT`/`OFFSET`

- Allow sub-selects in `LIMIT` and `OFFSET` (Tom)

- Add SQL-standard syntax for `LIMIT`/`OFFSET` capabilities (Peter)

  To wit, `OFFSET num {ROW|ROWS} FETCH {FIRST|NEXT} [num] {ROW|ROWS} ONLY`.

#### E.241.3.4. Object Manipulation

- Add support for column-level privileges (Stephen Frost, KaiGai Kohei)

- Refactor multi-object `DROP` operations to reduce the need for `CASCADE` (Alex Hunsaker)

  For example, if table `B` has a dependency on table `A`, the command `DROP TABLE A, B` no longer requires the `CASCADE` option.

- Fix various problems with concurrent `DROP` commands by ensuring that locks are taken before we begin to drop dependencies of an object (Tom)

- Improve reporting of dependencies during `DROP` commands (Tom)

- Add `WITH [NO] DATA` clause to `CREATE TABLE AS`, per the SQL standard (Peter, Tom)

- Add support for user-defined I/O conversion casts (Heikki)

- Allow `CREATE AGGREGATE` to use an `internal` transition datatype (Tom)

- Add `LIKE` clause to `CREATE TYPE` (Tom)

  This simplifies creation of data types that use the same internal representation as an existing type.

- Allow specification of the type category and “preferred” status for user-defined base types (Tom)

  This allows more control over the coercion behavior of user-defined types.

- Allow `CREATE OR REPLACE VIEW` to add columns to the end of a view (Robert Haas)

##### E.241.3.4.1. `ALTER`

- Add `ALTER TYPE RENAME` (Petr Jelinek)

- Add `ALTER SEQUENCE ... RESTART` (with no parameter) to reset a sequence to its initial value (Zoltan Boszormenyi)

- Modify the `ALTER TABLE` syntax to allow all reasonable combinations for tables, indexes, sequences, and views (Tom)

  This change allows the following new syntaxes:

  - `ALTER SEQUENCE OWNER TO`

  - `ALTER VIEW ALTER COLUMN SET/DROP DEFAULT`

  - `ALTER VIEW OWNER TO`

  - `ALTER VIEW SET SCHEMA`

  There is no actual new functionality here, but formerly you had to say `ALTER TABLE` to do these things, which was confusing.

- Add support for the syntax `ALTER TABLE ... ALTER COLUMN ... SET DATA TYPE` (Peter)

  This is SQL-standard syntax for functionality that was already supported.

- Make `ALTER TABLE SET WITHOUT OIDS` rewrite the table to physically remove `OID` values (Tom)

  Also, add `ALTER TABLE SET WITH OIDS` to rewrite the table to add `OID`s.

##### E.241.3.4.2. Database Manipulation

- Improve reporting of `CREATE`/`DROP`/`RENAME DATABASE` failure when uncommitted prepared transactions are the cause (Tom)

- Make `LC_COLLATE` and `LC_CTYPE` into per-database settings (Radek Strnad, Heikki)

  This makes collation similar to encoding, which was always configurable per database.

- Improve checks that the database encoding, collation (`LC_COLLATE`), and character classes (`LC_CTYPE`) match (Heikki, Tom)

  Note in particular that a new database's encoding and locale settings can be changed only when copying from `template0`. This prevents possibly copying data that doesn't match the settings.

- Add `ALTER DATABASE SET TABLESPACE` to move a database to a new tablespace (Guillaume Lelarge, Bernd Helmle)

#### E.241.3.5. Utility Operations

- Add a `VERBOSE` option to the `CLUSTER` command and clusterdb (Jim Cox)

- Decrease memory requirements for recording pending trigger events (Tom)

##### E.241.3.5.1. Indexes

- Dramatically improve the speed of building and accessing hash indexes (Tom Raney, Shreya Bhargava)

  This allows hash indexes to be sometimes faster than btree indexes. However, hash indexes are still not crash-safe.

- Make hash indexes store only the hash code, not the full value of the indexed column (Xiao Meng)

  This greatly reduces the size of hash indexes for long indexed values, improving performance.

- Implement fast update option for GIN indexes (Teodor, Oleg)

  This option greatly improves update speed at a small penalty in search speed.

- `xxx_pattern_ops` indexes can now be used for simple equality comparisons, not only for `LIKE` (Tom)

##### E.241.3.5.2. Full Text Indexes

- Remove the requirement to use `@@@` when doing GIN weighted lookups on full text indexes (Tom, Teodor)

  The normal `@@` text search operator can be used instead.

- Add an optimizer selectivity function for `@@` text search operations (Jan Urbanski)

- Allow prefix matching in full text searches (Teodor Sigaev, Oleg Bartunov)

- Support multi-column GIN indexes (Teodor Sigaev)

- Improve support for Nepali language and Devanagari alphabet (Teodor)

##### E.241.3.5.3. `VACUUM`

- Track free space in separate per-relation “fork” files (Heikki)

  Free space discovered by `VACUUM` is now recorded in `*_fsm` files, rather than in a fixed-sized shared memory area. The `max_fsm_pages` and `max_fsm_relations` settings have been removed, greatly simplifying administration of free space management.

- Add a visibility map to track pages that do not require vacuuming (Heikki)

  This allows `VACUUM` to avoid scanning all of a table when only a portion of the table needs vacuuming. The visibility map is stored in per-relation “fork” files.

- Add `vacuum_freeze_table_age` parameter to control when `VACUUM` should ignore the visibility map and do a full table scan to freeze tuples (Heikki)

- Track transaction snapshots more carefully (Alvaro)

  This improves `VACUUM`'s ability to reclaim space in the presence of long-running transactions.

- Add ability to specify per-relation autovacuum and TOAST parameters in `CREATE TABLE` (Alvaro, Euler Taveira de Oliveira)

  Autovacuum options used to be stored in a system table.

- Add `--freeze` option to vacuumdb (Bruce)

#### E.241.3.6. Data Types

- Add a `CaseSensitive` option for text search synonym dictionaries (Simon)

- Improve the precision of `NUMERIC` division (Tom)

- Add basic arithmetic operators for `int2` with `int8` (Tom)

  This eliminates the need for explicit casting in some situations.

- Allow `UUID` input to accept an optional hyphen after every fourth digit (Robert Haas)

- Allow `on`/`off` as input for the boolean data type (Itagaki Takahiro)

- Allow spaces around `NaN` in the input string for type `numeric` (Sam Mason)

##### E.241.3.6.1. Temporal Data Types

- Reject year `0 BC` and years `000` and `0000` (Tom)

  Previously these were interpreted as `1 BC`. (Note: years `0` and `00` are still assumed to be the year 2000.)

- Include `SGT` (Singapore time) in the default list of known time zone abbreviations (Tom)

- Support `infinity` and `-infinity` as values of type `date` (Tom)

- Make parsing of `interval` literals more standard-compliant (Tom, Ron Mayer)

  For example, `INTERVAL '1' YEAR` now does what it's supposed to.

- Allow `interval` fractional-seconds precision to be specified after the `second` keyword, for SQL standard compliance (Tom)

  Formerly the precision had to be specified after the keyword `interval`. (For backwards compatibility, this syntax is still supported, though deprecated.) Data type definitions will now be output using the standard format.

- Support the IS0 8601 `interval` syntax (Ron Mayer, Kevin Grittner)

  For example, `INTERVAL 'P1Y2M3DT4H5M6.7S'` is now supported.

- Add `IntervalStyle` parameter which controls how `interval` values are output (Ron Mayer)

  Valid values are: `postgres`, `postgres_verbose`, `sql_standard`, `iso_8601`. This setting also controls the handling of negative `interval` input when only some fields have positive/negative designations.

- Improve consistency of handling of fractional seconds in `timestamp` and `interval` output (Ron Mayer)

##### E.241.3.6.2. Arrays

- Improve the handling of casts applied to `ARRAY[]` constructs, such as `ARRAY[...]::integer[]` (Brendan Jurd)

  Formerly PostgreSQL attempted to determine a data type for the `ARRAY[]` construct without reference to the ensuing cast. This could fail unnecessarily in many cases, in particular when the `ARRAY[]` construct was empty or contained only ambiguous entries such as `NULL`. Now the cast is consulted to determine the type that the array elements must be.

- Make SQL-syntax `ARRAY` dimensions optional to match the SQL standard (Peter)

- Add `array_ndims()` to return the number of dimensions of an array (Robert Haas)

- Add `array_length()` to return the length of an array for a specified dimension (Jim Nasby, Robert Haas, Peter Eisentraut)

- Add aggregate function `array_agg()`, which returns all aggregated values as a single array (Robert Haas, Jeff Davis, Peter)

- Add `unnest()`, which converts an array to individual row values (Tom)

  This is the opposite of `array_agg()`.

- Add `array_fill()` to create arrays initialized with a value (Pavel Stehule)

- Add `generate_subscripts()` to simplify generating the range of an array's subscripts (Pavel Stehule)

##### E.241.3.6.3. Wide-Value Storage (TOAST)

- Consider TOAST compression on values as short as 32 bytes (previously 256 bytes) (Greg Stark)

- Require 25% minimum space savings before using TOAST compression (previously 20% for small values and any-savings-at-all for large values) (Greg)

- Improve TOAST heuristics for rows that have a mix of large and small toastable fields, so that we prefer to push large values out of line and don't compress small values unnecessarily (Greg, Tom)

#### E.241.3.7. Functions

- Document that `setseed()` allows values from `-1` to `1` (not just `0` to `1`), and enforce the valid range (Kris Jurka)

- Add server-side function `lo_import(filename, oid)` (Tatsuo)

- Add `quote_nullable()`, which behaves like `quote_literal()` but returns the string `NULL` for a null argument (Brendan Jurd)

- Improve full text search `headline()` function to allow extracting several fragments of text (Sushant Sinha)

- Add `suppress_redundant_updates_trigger()` trigger function to avoid overhead for non-data-changing updates (Andrew)

- Add `div(numeric, numeric)` to perform `numeric` division without rounding (Tom)

- Add `timestamp` and `timestamptz` versions of `generate_series()` (Hitoshi Harada)

##### E.241.3.7.1. Object Information Functions

- Implement `current_query()` for use by functions that need to know the currently running query (Tomas Doran)

- Add `pg_get_keywords()` to return a list of the parser keywords (Dave Page)

- Add `pg_get_functiondef()` to see a function's definition (Abhijit Menon-Sen)

- Allow the second argument of `pg_get_expr()` to be zero when deparsing an expression that does not contain variables (Tom)

- Modify `pg_relation_size()` to use `regclass` (Heikki)

  `pg_relation_size(data_type_name)` no longer works.

- Add `boot_val` and `reset_val` columns to `pg_settings` output (Greg Smith)

- Add source file name and line number columns to `pg_settings` output for variables set in a configuration file (Magnus, Alvaro)

  For security reasons, these columns are only visible to superusers.

- Add support for `CURRENT_CATALOG`, `CURRENT_SCHEMA`, `SET CATALOG`, `SET SCHEMA` (Peter)

  These provide SQL-standard syntax for existing features.

- Add `pg_typeof()` which returns the data type of any value (Brendan Jurd)

- Make `version()` return information about whether the server is a 32- or 64-bit binary (Bruce)

- Fix the behavior of information schema columns `is_insertable_into` and `is_updatable` to be consistent (Peter)

- Improve the behavior of information schema `datetime_precision` columns (Peter)

  These columns now show zero for `date` columns, and 6 (the default precision) for `time`, `timestamp`, and `interval` without a declared precision, rather than showing null as formerly.

- Convert remaining builtin set-returning functions to use `OUT` parameters (Jaime Casanova)

  This makes it possible to call these functions without specifying a column list: `pg_show_all_settings()`, `pg_lock_status()`, `pg_prepared_xact()`, `pg_prepared_statement()`, `pg_cursor()`

- Make `pg_*_is_visible()` and `has_*_privilege()` functions return `NULL` for invalid OIDs, rather than reporting an error (Tom)

- Extend `has_*_privilege()` functions to allow inquiring about the OR of multiple privileges in one call (Stephen Frost, Tom)

- Add `has_column_privilege()` and `has_any_column_privilege()` functions (Stephen Frost, Tom)

##### E.241.3.7.2. Function Creation

- Support variadic functions (functions with a variable number of arguments) (Pavel Stehule)

  Only trailing arguments can be optional, and they all must be of the same data type.

- Support default values for function arguments (Pavel Stehule)

- Add `CREATE FUNCTION ... RETURNS TABLE` clause (Pavel Stehule)

- Allow SQL-language functions to return the output of an `INSERT`/`UPDATE`/`DELETE` `RETURNING` clause (Tom)

##### E.241.3.7.3. PL/pgSQL Server-Side Language

- Support `EXECUTE USING` for easier insertion of data values into a dynamic query string (Pavel Stehule)

- Allow looping over the results of a cursor using a `FOR` loop (Pavel Stehule)

- Support `RETURN QUERY EXECUTE` (Pavel Stehule)

- Improve the `RAISE` command (Pavel Stehule)

  - Support `DETAIL` and `HINT` fields

  - Support specification of the `SQLSTATE` error code

  - Support an exception name parameter

  - Allow `RAISE` without parameters in an exception block to re-throw the current error

- Allow specification of `SQLSTATE` codes in `EXCEPTION` lists (Pavel Stehule)

  This is useful for handling custom `SQLSTATE` codes.

- Support the `CASE` statement (Pavel Stehule)

- Make `RETURN QUERY` set the special `FOUND` and `GET DIAGNOSTICS` `ROW_COUNT` variables (Pavel Stehule)

- Make `FETCH` and `MOVE` set the `GET DIAGNOSTICS` `ROW_COUNT` variable (Andrew Gierth)

- Make `EXIT` without a label always exit the innermost loop (Tom)

  Formerly, if there were a `BEGIN` block more closely nested than any loop, it would exit that block instead. The new behavior matches Oracle(TM) and is also what was previously stated by our own documentation.

- Make processing of string literals and nested block comments match the main SQL parser's processing (Tom)

  In particular, the format string in `RAISE` now works the same as any other string literal, including being subject to `standard_conforming_strings`. This change also fixes other cases in which valid commands would fail when `standard_conforming_strings` is on.

- Avoid memory leakage when the same function is called at varying exception-block nesting depths (Tom)

#### E.241.3.8. Client Applications

- Fix `pg_ctl restart` to preserve command-line arguments (Bruce)

- Add `-w`/`--no-password` option that prevents password prompting in all utilities that have a `-W`/`--password` option (Peter)

- Remove `-q` (quiet) option of createdb, createuser, dropdb, dropuser (Peter)

  These options have had no effect since PostgreSQL 8.3.

##### E.241.3.8.1. psql

- Remove verbose startup banner; now just suggest `help` (Joshua Drake)

- Make `help` show common backslash commands (Greg Sabino Mullane)

- Add `\pset format wrapped` mode to wrap output to the screen width, or file/pipe output too if `\pset columns` is set (Bryce Nesbitt)

- Allow all supported spellings of boolean values in `\pset`, rather than just `on` and `off` (Bruce)

  Formerly, any string other than “off” was silently taken to mean `true`. psql will now complain about unrecognized spellings (but still take them as `true`).

- Use the pager for wide output (Bruce)

- Require a space between a one-letter backslash command and its first argument (Bernd Helmle)

  This removes a historical source of ambiguity.

- Improve tab completion support for schema-qualified and quoted identifiers (Greg Sabino Mullane)

- Add optional `on`/`off` argument for `\timing` (David Fetter)

- Display access control rights on multiple lines (Brendan Jurd, Andreas Scherbaum)

- Make `\l` show database access privileges (Andrew Gilligan)

- Make `\l+` show database sizes, if permissions allow (Andrew Gilligan)

- Add the `\ef` command to edit function definitions (Abhijit Menon-Sen)

##### E.241.3.8.2. psql \d* commands

- Make `\d*` commands that do not have a pattern argument show system objects only if the `S` modifier is specified (Greg Sabino Mullane, Bruce)

  The former behavior was inconsistent across different variants of `\d`, and in most cases it provided no easy way to see just user objects.

- Improve `\d*` commands to work with older PostgreSQL server versions (back to 7.4), not only the current server version (Guillaume Lelarge)

- Make `\d` show foreign-key constraints that reference the selected table (Kenneth D'Souza)

- Make `\d` on a sequence show its column values (Euler Taveira de Oliveira)

- Add column storage type and other relation options to the `\d+` display (Gregory Stark, Euler Taveira de Oliveira)

- Show relation size in `\dt+` output (Dickson S. Guedes)

- Show the possible values of `enum` types in `\dT+` (David Fetter)

- Allow `\dC` to accept a wildcard pattern, which matches either datatype involved in the cast (Tom)

- Add a function type column to `\df`'s output, and add options to list only selected types of functions (David Fetter)

- Make `\df` not hide functions that take or return type `cstring` (Tom)

  Previously, such functions were hidden because most of them are datatype I/O functions, which were deemed uninteresting. The new policy about hiding system functions by default makes this wart unnecessary.

##### E.241.3.8.3. pg_dump

- Add a `--no-tablespaces` option to pg_dump/pg_dumpall/pg_restore so that dumps can be restored to clusters that have non-matching tablespace layouts (Gavin Roy)

- Remove `-d` and `-D` options from pg_dump and pg_dumpall (Tom)

  These options were too frequently confused with the option to select a database name in other PostgreSQL client applications. The functionality is still available, but you must now spell out the long option name `--inserts` or `--column-inserts`.

- Remove `-i`/`--ignore-version` option from pg_dump and pg_dumpall (Tom)

  Use of this option does not throw an error, but it has no effect. This option was removed because the version checks are necessary for safety.

- Disable `statement_timeout` during dump and restore (Joshua Drake)

- Add pg_dump/pg_dumpall option `--lock-wait-timeout` (David Gould)

  This allows dumps to fail if unable to acquire a shared lock within the specified amount of time.

- Reorder pg_dump `--data-only` output to dump tables referenced by foreign keys before the referencing tables (Tom)

  This allows data loads when foreign keys are already present. If circular references make a safe ordering impossible, a `NOTICE` is issued.

- Allow pg_dump, pg_dumpall, and pg_restore to use a specified role (Benedek László)

- Allow pg_restore to use multiple concurrent connections to do the restore (Andrew)

  The number of concurrent connections is controlled by the option `--jobs`. This is supported only for custom-format archives.

#### E.241.3.9. Programming Tools

##### E.241.3.9.1. libpq

- Allow the `OID` to be specified when importing a large object, via new function `lo_import_with_oid()` (Tatsuo)

- Add “events” support (Andrew Chernow, Merlin Moncure)

  This adds the ability to register callbacks to manage private data associated with `PGconn` and `PGresult` objects.

- Improve error handling to allow the return of multiple error messages as multi-line error reports (Magnus)

- Make `PQexecParams()` and related functions return `PGRES_EMPTY_QUERY` for an empty query (Tom)

  They previously returned `PGRES_COMMAND_OK`.

- Document how to avoid the overhead of `WSACleanup()` on Windows (Andrew Chernow)

- Do not rely on Kerberos tickets to determine the default database username (Magnus)

  Previously, a Kerberos-capable build of libpq would use the principal name from any available Kerberos ticket as default database username, even if the connection wasn't using Kerberos authentication. This was deemed inconsistent and confusing. The default username is now determined the same way with or without Kerberos. Note however that the database username must still match the ticket when Kerberos authentication is used.

##### E.241.3.9.2. libpq SSL (Secure Sockets Layer) support

- Fix certificate validation for SSL connections (Magnus)

  libpq now supports verifying both the certificate and the name of the server when making SSL connections. If a root certificate is not available to use for verification, SSL connections will fail. The `sslmode` parameter is used to enable certificate verification and set the level of checking. The default is still not to do any verification, allowing connections to SSL-enabled servers without requiring a root certificate on the client.

- Support wildcard server certificates (Magnus)

  If a certificate CN starts with `*`, it will be treated as a wildcard when matching the hostname, allowing the use of the same certificate for multiple servers.

- Allow the file locations for client certificates to be specified (Mark Woodward, Alvaro, Magnus)

- Add a `PQinitOpenSSL` function to allow greater control over OpenSSL/libcrypto initialization (Andrew Chernow)

- Make libpq unregister its OpenSSL callbacks when no database connections remain open (Bruce, Magnus, Russell Smith)

  This is required for applications that unload the libpq library, otherwise invalid OpenSSL callbacks will remain.

##### E.241.3.9.3. ecpg

- Add localization support for messages (Euler Taveira de Oliveira)

- ecpg parser is now automatically generated from the server parser (Michael)

  Previously the ecpg parser was hand-maintained.

##### E.241.3.9.4. Server Programming Interface (SPI)

- Add support for single-use plans with out-of-line parameters (Tom)

- Add new `SPI_OK_REWRITTEN` return code for `SPI_execute()` (Heikki)

  This is used when a command is rewritten to another type of command.

- Remove unnecessary inclusions from `executor/spi.h` (Tom)

  SPI-using modules might need to add some `#include` lines if they were depending on `spi.h` to include things for them.

#### E.241.3.10. Build Options

- Update build system to use Autoconf 2.61 (Peter)

- Require GNU bison for source code builds (Peter)

  This has effectively been required for several years, but now there is no infrastructure claiming to support other parser tools.

- Add pg_config `--htmldir` option (Peter)

- Pass `float4` by value inside the server (Zoltan Boszormenyi)

  Add configure option `--disable-float4-byval` to use the old behavior. External C functions that use old-style (version 0) call convention and pass or return `float4` values will be broken by this change, so you may need the configure option if you have such functions and don't want to update them.

- Pass `float8`, `int8`, and related datatypes by value inside the server on 64-bit platforms (Zoltan Boszormenyi)

  Add configure option `--disable-float8-byval` to use the old behavior. As above, this change might break old-style external C functions.

- Add configure options `--with-segsize`, `--with-blocksize`, `--with-wal-blocksize`, `--with-wal-segsize` (Zdenek Kotala, Tom)

  This simplifies build-time control over several constants that previously could only be changed by editing `pg_config_manual.h`.

- Allow threaded builds on Solaris 2.5 (Bruce)

- Use the system's `getopt_long()` on Solaris (Zdenek Kotala, Tom)

  This makes option processing more consistent with what Solaris users expect.

- Add support for the Sun Studio compiler on Linux (Julius Stroffek)

- Append the major version number to the backend gettext domain, and the `soname` major version number to libraries' gettext domain (Peter)

  This simplifies parallel installations of multiple versions.

- Add support for code coverage testing with gcov (Michelle Caisse)

- Allow out-of-tree builds on Mingw and Cygwin (Richard Evans)

- Fix the use of Mingw as a cross-compiling source platform (Peter)

#### E.241.3.11. Source Code

- Support 64-bit time zone data files (Heikki)

  This adds support for daylight saving time (DST) calculations beyond the year 2038.

- Deprecate use of platform's `time_t` data type (Tom)

  Some platforms have migrated to 64-bit `time_t`, some have not, and Windows can't make up its mind what it's doing. Define `pg_time_t` to have the same meaning as `time_t`, but always be 64 bits (unless the platform has no 64-bit integer type), and use that type in all module APIs and on-disk data formats.

- Fix bug in handling of the time zone database when cross-compiling (Richard Evans)

- Link backend object files in one step, rather than in stages (Peter)

- Improve gettext support to allow better translation of plurals (Peter)

- Add message translation support to the PL languages (Alvaro, Peter)

- Add more DTrace probes (Robert Lor)

- Enable DTrace support on macOS Leopard and other non-Solaris platforms (Robert Lor)

- Simplify and standardize conversions between C strings and `text` datums, by providing common functions for the purpose (Brendan Jurd, Tom)

- Clean up the `include/catalog/` header files so that frontend programs can include them without including `postgres.h` (Zdenek Kotala)

- Make `name` char-aligned, and suppress zero-padding of `name` entries in indexes (Tom)

- Recover better if dynamically-loaded code executes `exit()` (Tom)

- Add a hook to let plug-ins monitor the executor (Itagaki Takahiro)

- Add a hook to allow the planner's statistics lookup behavior to be overridden (Simon Riggs)

- Add `shmem_startup_hook()` for custom shared memory requirements (Tom)

- Replace the index access method `amgetmulti` entry point with `amgetbitmap`, and extend the API for `amgettuple` to support run-time determination of operator lossiness (Heikki, Tom, Teodor)

  The API for GIN and GiST opclass `consistent` functions has been extended as well.

- Add support for partial-match searches in GIN indexes (Teodor Sigaev, Oleg Bartunov)

- Replace `pg_class` column `reltriggers` with boolean `relhastriggers` (Simon)

  Also remove unused `pg_class` columns `relukeys`, `relfkeys`, and `relrefs`.

- Add a `relistemp` column to `pg_class` to ease identification of temporary tables (Tom)

- Move platform FAQs into the main documentation (Peter)

- Prevent parser input files from being built with any conflicts (Peter)

- Add support for the `KOI8U` (Ukrainian) encoding (Peter)

- Add Japanese message translations (Japan PostgreSQL Users Group)

  This used to be maintained as a separate project.

- Fix problem when setting `LC_MESSAGES` on MSVC-built systems (Hiroshi Inoue, Hiroshi Saito, Magnus)

#### E.241.3.12. Contrib

- Add `contrib/auto_explain` to automatically run `EXPLAIN` on queries exceeding a specified duration (Itagaki Takahiro, Tom)

- Add `contrib/btree_gin` to allow GIN indexes to handle more datatypes (Oleg, Teodor)

- Add `contrib/citext` to provide a case-insensitive, multibyte-aware text data type (David Wheeler)

- Add `contrib/pg_stat_statements` for server-wide tracking of statement execution statistics (Itagaki Takahiro)

- Add duration and query mode options to `contrib/pgbench` (Itagaki Takahiro)

- Make `contrib/pgbench` use table names `pgbench_accounts`, `pgbench_branches`, `pgbench_history`, and `pgbench_tellers`, rather than just `accounts`, `branches`, `history`, and `tellers` (Tom)

  This is to reduce the risk of accidentally destroying real data by running pgbench.

- Fix `contrib/pgstattuple` to handle tables and indexes with over 2 billion pages (Tatsuhito Kasahara)

- In `contrib/fuzzystrmatch`, add a version of the Levenshtein string-distance function that allows the user to specify the costs of insertion, deletion, and substitution (Volkan Yazici)

- Make `contrib/ltree` support multibyte encodings (laser)

- Enable `contrib/dblink` to use connection information stored in the SQL/MED catalogs (Joe Conway)

- Improve `contrib/dblink`'s reporting of errors from the remote server (Joe Conway)

- Make `contrib/dblink` set `client_encoding` to match the local database's encoding (Joe Conway)

  This prevents encoding problems when communicating with a remote database that uses a different encoding.

- Make sure `contrib/dblink` uses a password supplied by the user, and not accidentally taken from the server's `.pgpass` file (Joe Conway)

  This is a minor security enhancement.

- Add `fsm_page_contents()` to `contrib/pageinspect` (Heikki)

- Modify `get_raw_page()` to support free space map (`*_fsm`) files. Also update `contrib/pg_freespacemap`.

- Add support for multibyte encodings to `contrib/pg_trgm` (Teodor)

- Rewrite `contrib/intagg` to use new functions `array_agg()` and `unnest()` (Tom)

- Make `contrib/pg_standby` recover all available WAL before failover (Fujii Masao, Simon, Heikki)

  To make this work safely, you now need to set the new `recovery_end_command` option in `recovery.conf` to clean up the trigger file after failover. pg_standby will no longer remove the trigger file itself.

- `contrib/pg_standby`'s `-l` option is now a no-op, because it is unsafe to use a symlink (Simon)