diff --git a/doc/languages-frameworks/beam.section.md b/doc/languages-frameworks/beam.section.md index 7b643efa30d8..70ad1591be01 100644 --- a/doc/languages-frameworks/beam.section.md +++ b/doc/languages-frameworks/beam.section.md @@ -6,46 +6,68 @@ In this document and related Nix expressions, we use the term, _BEAM_, to descri ## Available versions and deprecations schedule {#available-versions-and-deprecations-schedule} +### Erlang OTP {#erlang} + +Nixpkgs follows upstream Erlang in their [support lifecycle](https://erlang.org/download/otp_versions_tree.html) and keeps up to the last 3 released versions of Erlang available. Due to upstream and NixOS release timings, this may mean removal of the oldest release prior to upstream fully dropping support. + ### Elixir {#elixir} -Nixpkgs follows the [official elixir deprecation schedule](https://hexdocs.pm/elixir/compatibility-and-deprecations.html) and keeps the last 5 released versions of Elixir available. +Nixpkgs follows the [official elixir deprecation schedule](https://hexdocs.pm/elixir/compatibility-and-deprecations.html) and keeps up to the last 5 released versions of Elixir available. ## Structure {#beam-structure} -All BEAM-related expressions are available via the top-level `beam` attribute, which includes: +All BEAM-related expressions are available via top-level package sets. It is recommended to work with a single package set to ensure consistent versions. -- `interpreters`: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (`beam.interpreters.erlang_22`, etc), Elixir (`beam.interpreters.elixir`) and LFE (Lisp Flavoured Erlang) (`beam.interpreters.lfe`). +- `beamPackages` - default OTP version +- `beamMinimalPackages` - default OTP version, without wxwidgets, which saves ~1GB in closure size -- `packages`: a set of package builders (Mix and rebar3), each compiled with a specific Erlang/OTP version, e.g. `beam.packages.erlang22`. +There are also OTP version specific package sets, e.g. for OTP 28: -The default Erlang compiler, defined by `beam.interpreters.erlang`, is aliased as `erlang`. The default BEAM package set is defined by `beam.packages.erlang` and aliased at the top level as `beamPackages`. +- `beam28Packages` +- `beamMinimal28Packages` -To create a package builder built with a custom Erlang version, use the lambda, `beam.packagesWith`, which accepts an Erlang/OTP derivation and produces a package builder similar to `beam.packages.erlang`. +Inside each package set are: -Many Erlang/OTP distributions available in `beam.interpreters` have versions with ODBC and/or Java enabled or without wx (no observer support). For example, there's `beam.interpreters.erlang_22_odbc_javac`, which corresponds to `beam.interpreters.erlang_22` and `beam.interpreters.erlang_22_nox`, which corresponds to `beam.interpreters.erlang_22`. +- erlang itself (version comes from package set) +- interpreters: elixir (multiple versions, e.g. elixir_1_18) and lfe +- packages: rebar3, hex, etc +- builders: mixRelease, buildRebar3, etc +- hooks: for composing builders and packages -## Build Tools {#build-tools} +To use a non-default Elixir it's important to keep the rest of the package set consistent, so it's recommended to use `.extend`. This ensures that builders like `mixRelease`, `fetchMixDeps`, and `buildMix` all pick up the overridden Elixir: -### Rebar3 {#build-tools-rebar3} +```nix +let + beamPackages = beam27Packages.extend (self: super: { elixir = self.elixir_1_18; }); +in +beamPackages.mixRelease { + # ... +} +``` -We provide a version of Rebar3, under `rebar3`. We also provide a helper to fetch Rebar3 dependencies from a lockfile under `fetchRebar3Deps`. +## Build Tools {#beam-build-tools} -We also provide a version on Rebar3 with plugins included, under `rebar3WithPlugins`. This package is a function which takes two arguments: `plugins`, a list of nix derivations to include as plugins (loaded only when specified in `rebar.config`), and `globalPlugins`, which should always be loaded by rebar3. Example: `rebar3WithPlugins { globalPlugins = [beamPackages.pc]; }`. +### Rebar3 {#beam-build-tools-rebar3} + +We provide a version of Rebar3, under `beamPackages.rebar3`. We also provide a helper to fetch Rebar3 dependencies from a lockfile under `beamPackages.fetchRebar3Deps`. + +We also provide a version on Rebar3 with plugins included, under `beamPackages.rebar3WithPlugins`. This package is a function which takes two arguments: `plugins`, a list of nix derivations to include as plugins (loaded only when specified in `rebar.config`), and `globalPlugins`, which should always be loaded by rebar3. Example: `beamPackages.rebar3WithPlugins { globalPlugins = [beamPackages.pc]; }`. When adding a new plugin it is important that the `name` attribute is the same as the atom used by rebar3 to refer to the plugin. -### Mix & Erlang.mk {#build-tools-other} +### Erlang.mk {#beam-build-tools-erlangmk} Erlang.mk works exactly as expected. There is a bootstrap process that needs to be run, which is supported by the `buildErlangMk` derivation. -For Elixir applications use `mixRelease` to make a release. See examples for more details. +### Mix {#beam-build-tools-mix} -There is also a `buildMix` helper, whose behavior is closer to that of `buildErlangMk` and `buildRebar3`. The primary difference is that mixRelease makes a release, while buildMix only builds the package, making it useful for libraries and other dependencies. +For Elixir applications that use [mix release](https://hexdocs.pm/mix/Mix.Release.html), use the `mixRelease` builder to make a release. See examples for more details. + +There is also a `buildMix` helper, whose behavior is closer to that of `buildErlangMk` and `buildRebar3`. The primary difference is that `mixRelease` makes a release, while `buildMix` only builds the package, which is more useful for libraries and other dependencies. ## How to Install BEAM Packages {#how-to-install-beam-packages} -BEAM builders are not registered at the top level, because they are not relevant to the vast majority of Nix users. -To use any of those builders into your environment, refer to them by their attribute path under `beamPackages`, e.g. `beamPackages.rebar3`: +To use any of these builders in your environment, refer to them by their attribute path under `beamPackages` (or another BEAM package set), e.g. `beamPackages.rebar3`: ::: {.example #ex-beam-ephemeral-shell} # Ephemeral shell @@ -75,35 +97,39 @@ pkgs.mkShell { packages = [ pkgs.beamPackages.rebar3 ]; } #### Rebar3 Packages {#rebar3-packages} -The Nix function, `buildRebar3`, defined in `beam.packages.erlang.buildRebar3` and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. - -If a package needs to compile native code via Rebar3's port compilation mechanism, add `compilePort = true;` to the derivation. +The builder `beamPackages.buildRebar3` can be used to build a derivation that understands how to build a Rebar3 project. #### Erlang.mk Packages {#erlang-mk-packages} -Erlang.mk functions similarly to Rebar3, except we use `buildErlangMk` instead of `buildRebar3`. +Erlang.mk functions similarly to Rebar3, except we use `beamPackages.buildErlangMk` instead of `beamPackages.buildRebar3`. + +If a package needs to compile native code via Erlang.mk's port compilation mechanism, add `compilePorts = true;` to the derivation. + +### Elixir Applications {#packaging-elixir-applications} #### Mix Packages {#mix-packages} -`mixRelease` is used to make a release in the mix sense. Dependencies will need to be fetched with `fetchMixDeps` and passed to it. +`beamPackages.mixRelease` is used to make a release in the mix sense. Dependencies will need to be fetched with `beamPackages.fetchMixDeps` and passed to it. #### mixRelease - Elixir Phoenix example {#mix-release-elixir-phoenix-example} -there are 3 steps: frontend dependencies (javascript), backend dependencies (elixir), and the final derivation that puts both of those together +There are 3 steps: frontend dependencies (javascript), backend dependencies (elixir), and the final derivation that puts both of those together. ##### mixRelease - Frontend dependencies (javascript) {#mix-release-javascript-deps} -For phoenix projects, inside of Nixpkgs you can either use `fetchYarnDeps` or `buildNpmPackage`. An example with `fetchYarnDeps` can be found [here](https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/pl/plausible/package.nix). An example with `fetchYarnDeps` will follow. To package something outside of nixpkgs, you have alternatives like [npmlock2nix](https://github.com/nix-community/npmlock2nix) or [nix-npm-buildpackage](https://github.com/serokell/nix-npm-buildpackage) +For phoenix projects, inside of Nixpkgs you can either use `fetchYarnDeps` or `buildNpmPackage`. An example with `buildNpmPackage` can be found [here](https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/pl/plausible/package.nix), and an example with `fetchYarnDeps` can be found [here](https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/pi/pinchflat/package.nix). ##### mixRelease - backend dependencies (mix) {#mix-release-mix-deps} -There are 2 ways to package backend dependencies. With mix2nix and with a fixed-output-derivation (FOD). +There are 2 ways to package backend dependencies: either per-dependency mix2nix or with a fixed-output-derivation (FOD). + +When writing an elixir project targeting `mixRelease`, you can also consider using [deps_nix](https://github.com/code-supply/deps_nix) with `mixNixDeps`. `deps_nix` supports git dependencies, but is intended to be added to the project's `mix.exs` directly. ###### mix2nix {#mix2nix} `mix2nix` is a cli tool available in Nixpkgs. It will generate a Nix expression from a `mix.lock` file. It is quite standard in the 2nix tool series. -Note that currently mix2nix can't handle git dependencies inside the mix.lock file. If you have git dependencies, you can either add them manually (see [example](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/pleroma/default.nix#L20)) or use the FOD method. +Note that currently mix2nix can't handle git dependencies inside the mix.lock file. If you have git dependencies, you can either add them manually (see [example](https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/pl/pleroma/package.nix)) or use the FOD method. The advantage of using mix2nix is that nix will know your whole dependency graph. On a dependency update, this won't trigger a full rebuild and download of all the dependencies, where FOD will do so. @@ -151,7 +177,7 @@ You will need to run the build process once to fix the hash to correspond to you ###### FOD {#fixed-output-derivation} -A fixed output derivation will download mix dependencies from the internet. To ensure reproducibility, a hash will be supplied. Note that mix is relatively reproducible. An FOD generating a different hash on each run hasn't been observed (as opposed to npm where the chances are relatively high). See [elixir-ls](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/beam-modules/elixir-ls/default.nix) for a usage example of FOD. +A fixed output derivation will download mix dependencies from the internet. To ensure reproducibility, a hash will be supplied. Note that mix is relatively reproducible. An FOD generating a different hash on each run hasn't been observed (as opposed to npm where the chances are relatively high). See [akkoma](https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/ak/akkoma/package.nix) for a usage example of FOD. Practical steps @@ -176,12 +202,11 @@ Note that if after you've replaced the value, nix suggests another hash, then mi Here is how your `default.nix` file would look for a Phoenix project. ```nix -with import { }; - +{ + # beam27Packages or beam29Packages is available if you need a particular version + beamPackages, +}: let - # beam.interpreters.erlang_26 is available if you need a particular version - packages = beam.packagesWith beam.interpreters.erlang; - pname = "your_project"; version = "0.0.1"; @@ -191,7 +216,7 @@ let }; # if using mix2nix you can use the mixNixDeps attribute - mixFodDeps = packages.fetchMixDeps { + mixFodDeps = beamPackages.fetchMixDeps { pname = "mix-deps-${pname}"; inherit src version; # nix will complain and tell you the right value to replace this with @@ -200,11 +225,8 @@ let # if you have build time environment variables add them here MY_ENV_VAR = "my_value"; }; - - nodeDependencies = (pkgs.callPackage ./assets/default.nix { }).shell.nodeDependencies; - in -packages.mixRelease { +beamPackages.mixRelease { inherit src pname @@ -215,9 +237,6 @@ packages.mixRelease { MY_ENV_VAR = "my_value"; postBuild = '' - ln -sf ${nodeDependencies}/lib/node_modules assets/node_modules - npm run deploy --prefix ./assets - # for external task you need a workaround for the no deps check flag # https://github.com/phoenixframework/phoenix/issues/2690 mix do deps.loadpaths --no-deps-check, phx.digest @@ -229,7 +248,7 @@ packages.mixRelease { Setup will require the following steps: - Move your secrets to runtime environment variables. For more information refer to the [runtime.exs docs](https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-runtime-configuration). On a fresh Phoenix build that would mean that both `DATABASE_URL` and `SECRET_KEY` need to be moved to `runtime.exs`. -- `cd assets` and `nix-shell -p node2nix --run "node2nix --development"` will generate a Nix expression containing your frontend dependencies +- Generate a Nix expression for your frontend dependencies using `fetchNpmDeps`/`buildNpmPackage` or `fetchYarnDeps`, depending on whether the project uses npm or yarn - commit and push those changes - you can now `nix-build .` - To run the release, set the `RELEASE_TMP` environment variable to a directory that your program has write access to. It will be used to store the BEAM settings. @@ -248,7 +267,7 @@ in your project with the following }: let - release = pkgs.callPackage ./default.nix; + release = pkgs.callPackage ./default.nix { }; release_name = "app"; working_directory = "/home/app"; in @@ -320,9 +339,10 @@ Usually, we need to create a `shell.nix` file and do our development inside the with pkgs; let - elixir = beam.packages.erlang_27.elixir_1_18; + # pin OTP via beam27Packages/beam28Packages/... and Elixir via .extend + beamPackages = beam27Packages.extend (self: super: { elixir = self.elixir_1_18; }); in -mkShell { buildInputs = [ elixir ]; } +mkShell { buildInputs = [ beamPackages.elixir ]; } ``` ### Using an overlay {#beam-using-overlays} @@ -337,7 +357,7 @@ let self: super: { elixir_1_18 = super.elixir_1_18.override { version = "1.18.1"; - sha256 = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; + hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; }; } ); @@ -355,18 +375,17 @@ Here is an example `shell.nix`. with import { }; let + # pin OTP via beam27Packages/beam28Packages/... and Elixir via .extend + beamPackages = beam27Packages.extend (self: super: { elixir = self.elixir_1_18; }); + # define packages to install basePackages = [ git - # replace with beam.packages.erlang.elixir_1_18 if you need - beam.packages.erlang.elixir + beamPackages.elixir nodejs postgresql_14 - # only used for frontend dependencies - # you are free to use yarn2nix as well - nodePackages.node2nix # formatting js file - nodePackages.prettier + prettier ]; inputs = basePackages ++ lib.optionals stdenv.hostPlatform.isLinux [ inotify-tools ]; @@ -379,13 +398,13 @@ let export HEX_HOME=$PWD/.nix-mix # make hex from Nixpkgs available # `mix local.hex` will install hex into MIX_HOME and should take precedence - export MIX_PATH="${beam.packages.erlang.hex}/lib/erlang/lib/hex/ebin" + export MIX_PATH="${beamPackages.hex}/lib/erlang/lib/hex/ebin" export PATH=$MIX_HOME/bin:$HEX_HOME/bin:$PATH export LANG=C.UTF-8 # keep your shell history in iex export ERL_AFLAGS="-kernel shell_history enabled" - # postges related + # postgres related # keep all your db data in a folder inside the project export PGDATA="$PWD/db" diff --git a/doc/redirects.json b/doc/redirects.json index aa3fab88f28a..de13226671e4 100644 --- a/doc/redirects.json +++ b/doc/redirects.json @@ -3052,19 +3052,28 @@ "available-versions-and-deprecations-schedule": [ "index.html#available-versions-and-deprecations-schedule" ], + "erlang": [ + "index.html#erlang" + ], "elixir": [ "index.html#elixir" ], "beam-structure": [ "index.html#beam-structure" ], - "build-tools": [ + "beam-build-tools": [ + "index.html#beam-build-tools", "index.html#build-tools" ], - "build-tools-rebar3": [ + "beam-build-tools-rebar3": [ + "index.html#beam-build-tools-rebar3", "index.html#build-tools-rebar3" ], - "build-tools-other": [ + "beam-build-tools-erlangmk": [ + "index.html#beam-build-tools-erlangmk" + ], + "beam-build-tools-mix": [ + "index.html#beam-build-tools-mix", "index.html#build-tools-other" ], "how-to-install-beam-packages": [ @@ -3082,6 +3091,9 @@ "packaging-erlang-applications": [ "index.html#packaging-erlang-applications" ], + "packaging-elixir-applications": [ + "index.html#packaging-elixir-applications" + ], "rebar3-packages": [ "index.html#rebar3-packages" ],