# Support for specific programming languages

The standard build environment makes it easy to build typical Autotools-based packages with very little code. Any other kind of package can be accomodated by overriding the appropriate phases of `stdenv`. However, there are specialised functions in Nixpkgs to easily build packages for other programming languages, such as Perl or Haskell. These are described in this chapter.

## Perl

Nixpkgs provides a function `buildPerlPackage`, a generic package builder function for any Perl package that has a standard `Makefile.PL`. It’s implemented in `pkgs/development/perl-modules/generic`.

Perl packages from CPAN are defined in `pkgs/top-level/perl-packages.nix`, rather than `pkgs/all-packages.nix`. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from `perl-packages.nix`. However, more complicated packages should be put in a separate file, typically in `pkgs/development/perl-modules`. Here is an example of the former:

    ClassC3 = buildPerlPackage rec {
      name = "Class-C3-0.21";
      src = fetchurl {
        url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz";
        sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz";
      };
    };

Note the use of `mirror://cpan/`, and the `${name}` in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in `all-packages.nix` through the variable `perlPackages`. For instance, if you have a package that needs `ClassC3`, you would typically write

    foo = import ../path/to/foo.nix {
      inherit stdenv fetchurl ...;
      inherit (perlPackages) ClassC3;
    };

in `all-packages.nix`. You can test building a Perl package as follows:

    $ nix-build -A perlPackages.ClassC3

`buildPerlPackage` adds `perl-` to the start of the name attribute, so the package above is actually called `perl-Class-C3-0.21`. So to install it, you can say:

    $ nix-env -i perl-Class-C3

(Of course you can also install using the attribute name: `nix-env -i -A perlPackages.ClassC3`.)

So what does `buildPerlPackage` do? It does the following:

1. In the configure phase, it calls `perl Makefile.PL` to generate a Makefile. You can set the variable `makeMakerFlags` to pass flags to `Makefile.PL`

2. It adds the contents of the `PERL5LIB` environment variable to `#! .../bin/perl` line of Perl scripts as `-Idir` flags. This ensures that a script can find its dependencies.

3. In the fixup phase, it writes the propagated build inputs (`propagatedBuildInputs`) to the file `$out/nix-support/propagated-user-env-packages`. `nix-env` recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies.

`buildPerlPackage` is built on top of `stdenv`, so everything can be customised in the usual way. For instance, the `BerkeleyDB` module has a `preConfigure` hook to generate a configuration file used by `Makefile.PL`:

    { buildPerlPackage, fetchurl, db }:

    buildPerlPackage rec {
      name = "BerkeleyDB-0.36";

      src = fetchurl {
        url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz";
        sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1";
      };

      preConfigure = ''
        echo "LIB = ${db}/lib" > config.in
        echo "INCLUDE = ${db}/include" >> config.in
      '';
    }

Dependencies on other Perl packages can be specified in the `buildInputs` and `propagatedBuildInputs` attributes. If something is exclusively a build-time dependency, use `buildInputs`; if it’s (also) a runtime dependency, use `propagatedBuildInputs`. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules:

    ClassC3Componentised = buildPerlPackage rec {
      name = "Class-C3-Componentised-1.0004";
      src = fetchurl {
        url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz";
        sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1";
      };
      propagatedBuildInputs = [
        ClassC3 ClassInspector TestException MROCompat
      ];
    };

### Generation from CPAN

Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program `nix-generate-from-cpan`, which can be installed as follows:

    $ nix-env -i nix-generate-from-cpan

This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example:

    $ nix-generate-from-cpan XML::Simple
      XMLSimple = buildPerlPackage {
        name = "XML-Simple-2.20";
        src = fetchurl {
          url = mirror://cpan/authors/id/G/GR/GRANTM/XML-Simple-2.20.tar.gz;
          sha256 = "5cff13d0802792da1eb45895ce1be461903d98ec97c9c953bc8406af7294434a";
        };
        propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ];
        meta = {
          description = "Easily read/write XML (esp config files)";
          license = "perl";
        };
      };

The output can be pasted into `pkgs/top-level/perl-packages.nix` or wherever else you need it.

## Python

Currently supported interpreters are `python26`, `python27`, `python32`, `python33`, `python34` and `pypy`.

`python` is an alias of `python27` and `python3` is an alias of `python34`.

`python26` and `python27` do not include modules that require external dependencies (to reduce dependency bloat). Following modules need to be added as `buildInput` explicitly:

- `python.modules.bsddb`

- `python.modules.curses`

- `python.modules.curses_panel`

- `python.modules.crypt`

- `python.modules.gdbm`

- `python.modules.sqlite3`

- `python.modules.tkinter`

- `python.modules.readline`

For convenience `python27Full` and `python26Full` are provided with all modules included.

Python packages that use `setuptools` or `distutils`, can be built using the `buildPythonPackage` function as documented below.

All packages depending on any Python interpreter get appended `$out/${python.libPrefix}/site-packages` to `$PYTHONPATH` if such directory exists.

### Useful attributes on interpreters packages:

`libPrefix`

  Name of the folder in `${python}/lib/` for corresponding interpreter.

`interpreter`

  Alias for `${python}/bin/${executable}.`

`buildEnv`

  Function to build python interpreter environments with extra packages bundled together. See for usage and documentation.

`sitePackages`

  Alias for `lib/${libPrefix}/site-packages`.

`executable`

  Name of the interpreter executable, ie `python3.4`.

### `buildPythonPackage` function

The function is implemented in `pkgs/development/python-modules/generic/default.nix`. Example usage:

    twisted = buildPythonPackage {
      name = "twisted-8.1.0";

      src = pkgs.fetchurl {
        url = http://tmrc.mit.edu/mirror/twisted/Twisted/8.1/Twisted-8.1.0.tar.bz2;
        sha256 = "0q25zbr4xzknaghha72mq57kh53qw1bf8csgp63pm9sfi72qhirl";
      };

      propagatedBuildInputs = [ self.ZopeInterface ];

      meta = {
        homepage = http://twistedmatrix.com/;
        description = "Twisted, an event-driven networking engine written in Python";
        license = stdenv.lib.licenses.mit;
      };
    };

Most of Python packages that use `buildPythonPackage` are defined in `pkgs/top-level/python-packages.nix` and generated for each python interpreter separately into attribute sets `python26Packages`, `python27Packages`, `python32Packages`, `python33Packages`, `python34Packages` and `pypyPackages`.

`buildPythonPackage` mainly does four things:

1. In the `configurePhase`, it patches `setup.py` to always include setuptools before distutils for monkeypatching machinery to take place.

2. In the `buildPhase`, it calls `${python.interpreter} setup.py build ...`

3. In the `installPhase`, it calls `${python.interpreter} setup.py install ...`

4. In the `postFixup` phase, `wrapPythonPrograms` bash function is called to wrap all programs in `$out/bin/*` directory to include `$PYTHONPATH` and `$PATH` environment variables.

By default `doCheck = true` is set and tests are run with `${python.interpreter} setup.py test` command in `checkPhase`.

As in Perl, dependencies on other Python packages can be specified in the `buildInputs` and `propagatedBuildInputs` attributes. If something is exclusively a build-time dependency, use `buildInputs`; if it’s (also) a runtime dependency, use `propagatedBuildInputs`.

By default `meta.platforms` is set to the same value as the interpreter unless overriden otherwise.

#### `buildPythonPackage` parameters (all parameters from `mkDerivation` function are still supported)

`namePrefix`

  Prepended text to `${name}` parameter. Defaults to `"python3.3-"` for Python 3.3, etc. Set it to `""` if you're packaging an application or a command line tool.

`disabled`

  If `true`, package is not build for particular python interpreter version. Grep around `pkgs/top-level/python-packages.nix` for examples.

`setupPyInstallFlags`

  List of flags passed to `setup.py install` command.

`setupPyBuildFlags`

  List of flags passed to `setup.py build` command.

`pythonPath`

  List of packages to be added into `$PYTHONPATH`. Packages in `pythonPath` are not propagated into user environment (contrary to `propagatedBuildInputs`).

`preShellHook`

  Hook to execute commands before `shellHook`.

`postShellHook`

  Hook to execute commands after `shellHook`.

`distutilsExtraCfg`

  Extra lines passed to `[easy_install]` section of `distutils.cfg` (acts as global setup.cfg configuration).

`makeWrapperArgs`

  A list of strings. Arguments to be passed to `makeWrapper`, which wraps generated binaries. By default, the arguments to `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling the binary. Additional arguments here can allow a developer to set environment variables which will be available when the binary is run. For example, `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.

### `python.buildEnv` function

Create Python environments using low-level `pkgs.buildEnv` function. Example `default.nix`:

    with import <nixpkgs> {};

    python.buildEnv.override {
      extraLibs = [ pkgs.pythonPackages.pyramid ];
      ignoreCollisions = true;
    }

Running `nix-build` will create `/nix/store/cf1xhjwzmdki7fasgr4kz6di72ykicl5-python-2.7.8-env` with wrapped binaries in `bin/`.

You can also use `env` attribute to create local environments with needed packages installed (somewhat comparable to `virtualenv`). For example, with the following `shell.nix`:

    with import <nixpkgs> {};

    (python3.buildEnv.override {
      extraLibs = with python3Packages;
        [ numpy
          requests
        ];
    }).env

Running `nix-shell` will drop you into a shell where `python` will have specified packages in its path.

#### `python.buildEnv` arguments

`extraLibs`

  List of packages installed inside the environment.

`postBuild`

  Shell command executed after the build of environment.

`ignoreCollisions`

  Ignore file collisions inside the environment (default is `false`).

### Tools

Packages inside nixpkgs are written by hand. However many tools exist in community to help save time. No tool is preferred at the moment.

- python2nix by Vladimir Kirillov

- pypi2nix by Rok Garbas

- pypi2nix by Jaka Hudoklin

### Development

To develop Python packages `buildPythonPackage` has additional logic inside `shellPhase` to run `${python.interpreter} setup.py develop` for the package.

`shellPhase` is executed only if `setup.py` exists.

Given a `default.nix`:

    with import <nixpkgs> {};

    buildPythonPackage {
      name = "myproject";

      buildInputs = with pkgs.pythonPackages; [ pyramid ];

      src = ./.;
    }

Running `nix-shell` with no arguments should give you the environment in which the package would be build with `nix-build`.

Shortcut to setup environments with C headers/libraries and python packages:

    $ nix-shell -p pythonPackages.pyramid zlib libjpeg git

There is a boolean value `lib.inNixShell` set to `true` if nix-shell is invoked.

### FAQ

How to solve circular dependencies?

  If you have packages `A` and `B` that depend on each other, when packaging `B` override package `A` not to depend on `B` as input (and also the other way around).

`install_data / data_files` problems resulting into `error: could not create '/nix/store/6l1bvljpy8gazlsw2aw9skwwp4pmvyxw-python-2.7.8/etc': Permission denied`

  Known bug in setuptools `install_data` does not respect --prefix. Example of such package using the feature is `pkgs/tools/X11/xpra/default.nix`. As workaround install it as an extra `preInstall` step:

      ${python.interpreter} setup.py install_data --install-dir=$out --root=$out
      sed -i '/ = data_files/d' setup.py

Rationale of non-existent global site-packages

  There is no need to have global site-packages in Nix. Each package has isolated dependency tree and installing any python package will only populate `$PATH` inside user environment. See to create self-contained interpreter with a set of packages.

### Contributing guidelines

Following rules are desired to be respected:

- Make sure package builds for all python interpreters. Use `disabled` argument to `buildPythonPackage` to set unsupported interpreters.

- If tests need to be disabled for a package, make sure you leave a comment about reasoning.

- Packages in `pkgs/top-level/python-packages.nix` are sorted quasi-alphabetically to avoid merge conflicts.

## Ruby

There currently is support to bundle applications that are packaged as Ruby gems. The utility "bundix" allows you to write a `Gemfile`, let bundler create a `Gemfile.lock`, and then convert this into a nix expression that contains all Gem dependencies automatically.

For example, to package sensu, we did:

    $ cd pkgs/servers/monitoring
    $ mkdir sensu
    $ cat > Gemfile
    source 'https://rubygems.org'
    gem 'sensu'
    $ bundler package --path /tmp/vendor/bundle
    $ $(nix-build '<nixpkgs>' -A bundix)/bin/bundix
    $ cat > default.nix
    { lib, bundlerEnv, ruby }:

    bundlerEnv {
      name = "sensu-0.17.1";

      inherit ruby;
      gemfile = ./Gemfile;
      lockfile = ./Gemfile.lock;
      gemset = ./gemset.nix;

      meta = with lib; {
        description = "A monitoring framework that aims to be simple, malleable,
    and scalable.";
        homepage    = http://sensuapp.org/;
        license     = with licenses; mit;
        maintainers = with maintainers; [ theuni ];
        platforms   = platforms.unix;
      };
    }

Please check in the `Gemfile`, `Gemfile.lock` and the `gemset.nix` so future updates can be run easily.

## Go

The function `buildGoPackage` builds standard Go packages.

### buildGoPackage

    net = buildGoPackage rec {
      name = "go.net-${rev}";
      goPackagePath = "golang.org/x/net";
      subPackages = [ "ipv4" "ipv6" ];
      rev = "e0403b4e005";
      src = fetchFromGitHub {
        inherit rev;
        owner = "golang";
        repo = "net";
        sha256 = "1g7cjzw4g4301a3yqpbk8n1d4s97sfby2aysl275x04g0zh8jxqp";
      };
      goPackageAliases = [ "code.google.com/p/go.net" ];
      propagatedBuildInputs = [ goPackages.text ];
      buildFlags = "--tags release";
      disabled = isGo13;
    };

is an example expression using buildGoPackage, the following arguments are of special significance to the function:

`goPackagePath` specifies the package's canonical Go import path.

`subPackages` limits the builder from building child packages that have not been listed. If `subPackages` is not specified, all child packages will be built.

In this example only `code.google.com/p/go.net/ipv4` and `code.google.com/p/go.net/ipv4` will be built.

`goPackageAliases` is a list of alternative import paths that are valid for this library. Packages that depend on this library will automatically rename import paths that match any of the aliases to `goPackagePath`.

In this example imports will be renamed from `code.google.com/p/go.net` to `golang.org/x/net` in every package that depend on the `go.net` library.

`propagatedBuildInputs` is where the dependencies of a Go library are listed. Only libraries should list `propagatedBuildInputs`. If a standalone program is being build instead, use `buildInputs`. If a library's tests require additional dependencies that are not propagated, they should be listed in `buildInputs`.

`buildFlags` is a list of flags passed to the go build command.

If `disabled` is `true`, nix will refuse to build this package.

In this example the package will not be built for go 1.3. The `isGo13` is an utility function that returns `true` if go used to build the package has version 1.3.x.

Reusable Go libraries may be found in the `goPackages` set. You can test build a Go package as follows:

    $ nix-build -A goPackages.net

You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc:

    for p in $NIX_PROFILES; do
        GOPATH="$p/share/go:$GOPATH"
    done

To extract dependency information from a Go package in automated way use go2nix.

## Java

Ant-based Java packages are typically built from source as follows:

    stdenv.mkDerivation {
      name = "...";
      src = fetchurl { ... };

      buildInputs = [ jdk ant ];

      buildPhase = "ant";
    }

Note that `jdk` is an alias for the OpenJDK.

JAR files that are intended to be used by other packages should be installed in `$out/share/java`. The OpenJDK has a stdenv setup hook that adds any JARs in the `share/java` directories of the build inputs to the `CLASSPATH` environment variable. For instance, if the package `libfoo` installs a JAR named `foo.jar` in its `share/java` directory, and another package declares the attribute

    buildInputs = [ jdk libfoo ];

then `CLASSPATH` will be set to `/nix/store/...-libfoo/share/java/foo.jar`.

Private JARs should be installed in a location like `$out/share/package-name`.

If your Java package provides a program, you need to generate a wrapper script to run it using the OpenJRE. You can use `makeWrapper` for this:

    buildInputs = [ makeWrapper ];

    installPhase =
      ''
        mkdir -p $out/bin
        makeWrapper ${jre}/bin/java $out/bin/foo \
          --add-flags "-cp $out/share/java/foo.jar org.foo.Main"
      '';

Note the use of `jre`, which is the part of the OpenJDK package that contains the Java Runtime Environment. By using `${jre}/bin/java` instead of `${jdk}/bin/java`, you prevent your package from depending on the JDK at runtime.

It is possible to use a different Java compiler than `javac` from the OpenJDK. For instance, to use the Eclipse Java Compiler:

    buildInputs = [ jre ant ecj ];

(Note that here you don’t need the full JDK as an input, but just the JRE.) The ECJ has a stdenv setup hook that sets some environment variables to cause Ant to use ECJ, but this doesn’t work with all Ant files. Similarly, you can use the GNU Java Compiler:

    buildInputs = [ gcj ant ];

Here, Ant will automatically use `gij` (the GNU Java Runtime) instead of the OpenJRE.

## Lua

Lua packages are built by the `buildLuaPackage` function. This function is implemented in `pkgs/development/lua-modules/generic/default.nix` and works similarly to `buildPerlPackage`. (See for details.)

Lua packages are defined in `pkgs/top-level/lua-packages.nix`. Most of them are simple. For example:

    fileSystem = buildLuaPackage {
      name = "filesystem-1.6.2";
      src = fetchurl {
        url = "https://github.com/keplerproject/luafilesystem/archive/v1_6_2.tar.gz";
        sha256 = "1n8qdwa20ypbrny99vhkmx8q04zd2jjycdb5196xdhgvqzk10abz";
      };
      meta = {
        homepage = "https://github.com/keplerproject/luafilesystem";
        hydraPlatforms = stdenv.lib.platforms.linux;
        maintainers = with maintainers; [ flosse ];
      };
    };

Though, more complicated package should be placed in a seperate file in `pkgs/development/lua-modules`.

Lua packages accept additional parameter `disabled`, which defines the condition of disabling package from luaPackages. For example, if package has `disabled` assigned to `lua.luaversion != "5.1"`, it will not be included in any luaPackages except lua51Packages, making it only be built for lua 5.1.

## Coq

Coq libraries should be installed in `$(out)/lib/coq/${coq.coq-version}/user-contrib/`. Such directories are automatically added to the `$COQPATH` environment variable by the hook defined in the Coq derivation.

Some libraries require OCaml and sometimes also Camlp5. The exact versions that were used to build Coq are saved in the `coq.ocaml` and `coq.camlp5` attributes.

Here is a simple package example. It is a pure Coq library, thus it only depends on Coq. Its `makefile` has been generated using `coq_makefile` so we only have to set the `$COQLIB` variable at install time.

    {stdenv, fetchurl, coq}:
    stdenv.mkDerivation {
      src = fetchurl {
        url = http://coq.inria.fr/pylons/contribs/files/Karatsuba/v8.4/Karatsuba.tar.gz;
        sha256 = "0ymfpv4v49k4fm63nq6gcl1hbnnxrvjjp7yzc4973n49b853c5b1";
      };

      name = "coq-karatsuba";

      buildInputs = [ coq ];

      installFlags = "COQLIB=$(out)/lib/coq/${coq.coq-version}/";
    }