Default email bcb2f287e1 Project import generated by Copybara.

GitOrigin-RevId: d603719ec6e294f034936c0d0dc06f689d91b6c3

2024-06-20 20:27:18 +05:30

15 KiB

Raw Blame History

PostgreSQL

Source: {file}modules/services/databases/postgresql.nix

Upstream documentation: https://www.postgresql.org/docs/

PostgreSQL is an advanced, free relational database.

Configuring

To enable PostgreSQL, add the following to your {file}configuration.nix:

{
  services.postgresql.enable = true;
  services.postgresql.package = pkgs.postgresql_15;
}

Note that you are required to specify the desired version of PostgreSQL (e.g. pkgs.postgresql_15). Since upgrading your PostgreSQL version requires a database dump and reload (see below), NixOS cannot provide a default value for such as the most recent release of PostgreSQL.

By default, PostgreSQL stores its databases in {file}/var/lib/postgresql/$psqlSchema. You can override this using , e.g.

{
  services.postgresql.dataDir = "/data/postgresql";
}

Initializing

As of NixOS 23.11, services.postgresql.ensureUsers.*.ensurePermissions has been deprecated, after a change to default permissions in PostgreSQL 15 invalidated most of its previous use cases:

In psql < 15, ALL PRIVILEGES used to include CREATE TABLE, where in psql >= 15 that would be a separate permission
psql >= 15 instead gives only the database owner create permissions
Even on psql < 15 (or databases migrated to >= 15), it is recommended to manually assign permissions along these lines
- https://www.postgresql.org/docs/release/15.0/
- https://www.postgresql.org/docs/15/ddl-schemas.html#DDL-SCHEMAS-PRIV

Assigning ownership

Usually, the database owner should be a database user of the same name. This can be done with services.postgresql.ensureUsers.*.ensureDBOwnership = true;.

If the database user name equals the connecting system user name, postgres by default will accept a passwordless connection via unix domain socket. This makes it possible to run many postgres-backed services without creating any database secrets at all

Assigning extra permissions

For many cases, it will be enough to have the database user be the owner. Until services.postgresql.ensureUsers.*.ensurePermissions has been re-thought, if more users need access to the database, please use one of the following approaches:

WARNING: services.postgresql.initialScript is not recommended for ensurePermissions replacement, as that is only run on first start of PostgreSQL.

NOTE: all of these methods may be obsoleted, when ensure* is reworked, but it is expected that they will stay viable for running database migrations.

NOTE: please make sure that any added migrations are idempotent (re-runnable).

as superuser

Advantage: compatible with postgres < 15, because it's run as the database superuser postgres.

in database `postStart`

Disadvantage: need to take care of ordering yourself. In this example, mkAfter ensures that permissions are assigned after any databases from ensureDatabases and extraUser1 from ensureUsers are already created.

  {
    systemd.services.postgresql.postStart = lib.mkAfter ''
      $PSQL service1 -c 'GRANT SELECT ON ALL TABLES IN SCHEMA public TO "extraUser1"'
      $PSQL service1 -c 'GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO "extraUser1"'
      # ....
    '';
  }

in intermediate oneshot service

  {
    systemd.services."migrate-service1-db1" = {
      serviceConfig.Type = "oneshot";
      requiredBy = "service1.service";
      before = "service1.service";
      after = "postgresql.service";
      serviceConfig.User = "postgres";
      environment.PSQL = "psql --port=${toString services.postgresql.settings.port}";
      path = [ postgresql ];
      script = ''
        $PSQL service1 -c 'GRANT SELECT ON ALL TABLES IN SCHEMA public TO "extraUser1"'
        $PSQL service1 -c 'GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO "extraUser1"'
        # ....
      '';
    };
  }

as service user

Advantage: re-uses systemd's dependency ordering;

Disadvantage: relies on service user having grant permission. To be combined with ensureDBOwnership.

in service `preStart`

  {
    environment.PSQL = "psql --port=${toString services.postgresql.settings.port}";
    path = [ postgresql ];
    systemd.services."service1".preStart = ''
      $PSQL -c 'GRANT SELECT ON ALL TABLES IN SCHEMA public TO "extraUser1"'
      $PSQL -c 'GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO "extraUser1"'
      # ....
    '';
  }

in intermediate oneshot service

  {
    systemd.services."migrate-service1-db1" = {
      serviceConfig.Type = "oneshot";
      requiredBy = "service1.service";
      before = "service1.service";
      after = "postgresql.service";
      serviceConfig.User = "service1";
      environment.PSQL = "psql --port=${toString services.postgresql.settings.port}";
      path = [ postgresql ];
      script = ''
        $PSQL -c 'GRANT SELECT ON ALL TABLES IN SCHEMA public TO "extraUser1"'
        $PSQL -c 'GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO "extraUser1"'
        # ....
      '';
    };
  }

Upgrading

::: {.note} The steps below demonstrate how to upgrade from an older version to pkgs.postgresql_13. These instructions are also applicable to other versions. :::

Major PostgreSQL upgrades require a downtime and a few imperative steps to be called. This is the case because each major version has some internal changes in the databases' state during major releases. Because of that, NixOS places the state into {file}/var/lib/postgresql/<version> where each version can be obtained like this:

$ nix-instantiate --eval -A postgresql_13.psqlSchema
"13"

For an upgrade, a script like this can be used to simplify the process:

{ config, pkgs, ... }:
{
  environment.systemPackages = [
    (let
      # XXX specify the postgresql package you'd like to upgrade to.
      # Do not forget to list the extensions you need.
      newPostgres = pkgs.postgresql_13.withPackages (pp: [
        # pp.plv8
      ]);
    in pkgs.writeScriptBin "upgrade-pg-cluster" ''
      set -eux
      # XXX it's perhaps advisable to stop all services that depend on postgresql
      systemctl stop postgresql

      export NEWDATA="/var/lib/postgresql/${newPostgres.psqlSchema}"

      export NEWBIN="${newPostgres}/bin"

      export OLDDATA="${config.services.postgresql.dataDir}"
      export OLDBIN="${config.services.postgresql.package}/bin"

      install -d -m 0700 -o postgres -g postgres "$NEWDATA"
      cd "$NEWDATA"
      sudo -u postgres $NEWBIN/initdb -D "$NEWDATA"

      sudo -u postgres $NEWBIN/pg_upgrade \
        --old-datadir "$OLDDATA" --new-datadir "$NEWDATA" \
        --old-bindir $OLDBIN --new-bindir $NEWBIN \
        "$@"
    '')
  ];
}

The upgrade process is:

Rebuild nixos configuration with the configuration above added to your {file}configuration.nix. Alternatively, add that into separate file and reference it in imports list.
Login as root (sudo su -)
Run upgrade-pg-cluster. It will stop old postgresql, initialize a new one and migrate the old one to the new one. You may supply arguments like --jobs 4 and --link to speedup migration process. See https://www.postgresql.org/docs/current/pgupgrade.html for details.
Change postgresql package in NixOS configuration to the one you were upgrading to via . Rebuild NixOS. This should start new postgres using upgraded data directory and all services you stopped during the upgrade.
After the upgrade it's advisable to analyze the new cluster.
- For PostgreSQL ≥ 14, use the vacuumdb command printed by the upgrades script.
- For PostgreSQL < 14, run (as su -l postgres in the , in this example {file}/var/lib/postgresql/13):
```
$ ./analyze_new_cluster.sh
```
::: {.warning} The next step removes the old state-directory! :::
```
$ ./delete_old_cluster.sh
```

Versioning and End-of-Life

PostgreSQL's versioning policy is described here. TLDR:

Each major version is supported for 5 years.
Every three months there will be a new minor release, containing bug and security fixes.
For criticial/security fixes there could be more minor releases inbetween. This happens very infrequently.
After five years, a final minor version is released. This usually happens in early November.
After that a version is considered end-of-life (EOL).
Around February each year is the first time an EOL-release will not have received regular updates anymore.

Technically, we'd not want to have EOL'ed packages in a stable NixOS release, which is to be supported until one month after the previous release. Thus, with NixOS' release schedule in May and November, the oldest PostgreSQL version in nixpkgs would have to be supported until December. It could be argued that a soon-to-be-EOL-ed version should thus be removed in May for the .05 release already. But since new security vulnerabilities are first disclosed in Februrary of the following year, we agreed on keeping the oldest PostgreSQL major version around one more cycle in #310580.

Thus:

In September/October the new major version will be released and added to nixos-unstable.
In November the last minor version for the oldest major will be released.
Both the current stable .05 release and nixos-unstable should be updated to the latest minor.
In November, before branch-off for the .11 release, the EOL-ed major will be removed from nixos-unstable.

This leaves a small gap of a couple of weeks after the latest minor release and the end of our support window for the .05 release, in which there could be an emergency release to other major versions of PostgreSQL - but not the oldest major we have in that branch. In that case: If we can't trivially patch the issue, we will mark the package/version as insecure immediately.

Options

A complete list of options for the PostgreSQL module may be found here.

Plugins

Plugins collection for each PostgreSQL version can be accessed with .pkgs. For example, for pkgs.postgresql_15 package, its plugin collection is accessed by pkgs.postgresql_15.pkgs:

$ nix repl '<nixpkgs>'

Loading '<nixpkgs>'...
Added 10574 variables.

nix-repl> postgresql_15.pkgs.<TAB><TAB>
postgresql_15.pkgs.cstore_fdw        postgresql_15.pkgs.pg_repack
postgresql_15.pkgs.pg_auto_failover  postgresql_15.pkgs.pg_safeupdate
postgresql_15.pkgs.pg_bigm           postgresql_15.pkgs.pg_similarity
postgresql_15.pkgs.pg_cron           postgresql_15.pkgs.pg_topn
postgresql_15.pkgs.pg_hll            postgresql_15.pkgs.pgjwt
postgresql_15.pkgs.pg_partman        postgresql_15.pkgs.pgroonga
...

To add plugins via NixOS configuration, set services.postgresql.extraPlugins:

{
  services.postgresql.package = pkgs.postgresql_12;
  services.postgresql.extraPlugins = ps: with ps; [
    pg_repack
    postgis
  ];
}

You can build custom PostgreSQL-with-plugins (to be used outside of NixOS) using function .withPackages. For example, creating a custom PostgreSQL package in an overlay can look like:

self: super: {
  postgresql_custom = self.postgresql_12.withPackages (ps: [
    ps.pg_repack
    ps.postgis
  ]);
}

Here's a recipe on how to override a particular plugin through an overlay:

self: super: {
  postgresql_15 = super.postgresql_15// {
    pkgs = super.postgresql_15.pkgs // {
      pg_repack = super.postgresql_15.pkgs.pg_repack.overrideAttrs (_: {
        name = "pg_repack-v20181024";
        src = self.fetchzip {
          url = "https://github.com/reorg/pg_repack/archive/923fa2f3c709a506e111cc963034bf2fd127aa00.tar.gz";
          sha256 = "17k6hq9xaax87yz79j773qyigm4fwk8z4zh5cyp6z0sxnwfqxxw5";
        };
      });
    };
  };
}

JIT (Just-In-Time compilation)

JIT-support in the PostgreSQL package is disabled by default because of the ~300MiB closure-size increase from the LLVM dependency. It can be optionally enabled in PostgreSQL with the following config option:

{
  services.postgresql.enableJIT = true;
}

This makes sure that the jit-setting is set to on and a PostgreSQL package with JIT enabled is used. Further tweaking of the JIT compiler, e.g. setting a different query cost threshold via jit_above_cost can be done manually via services.postgresql.settings.

The attribute-names of JIT-enabled PostgreSQL packages are suffixed with _jit, i.e. for each pkgs.postgresql (and pkgs.postgresql_<major>) in nixpkgs there's also a pkgs.postgresql_jit (and pkgs.postgresql_<major>_jit). Alternatively, a JIT-enabled variant can be derived from a given postgresql package via postgresql.withJIT. This is also useful if it's not clear which attribute from nixpkgs was originally used (e.g. when working with config.services.postgresql.package or if the package was modified via an overlay) since all modifications are propagated to withJIT. I.e.

with import <nixpkgs> {
  overlays = [
    (self: super: {
      postgresql = super.postgresql.overrideAttrs (_: { pname = "foobar"; });
    })
  ];
};
postgresql.withJIT.pname

evaluates to "foobar".

15 KiB Raw Blame History

PostgreSQL

Configuring

Initializing

Assigning ownership

Assigning extra permissions

as superuser

in database postStart

in intermediate oneshot service

as service user

in service preStart

in intermediate oneshot service

Upgrading

Versioning and End-of-Life

Options

Plugins

JIT (Just-In-Time compilation)

15 KiB

Raw Blame History

in database `postStart`

in service `preStart`