Scope:
- Combination of
- Textual matches of "baseNameOf (toString"
- Redundant toString calls I've found with my latest
"lazy paths" nix branch as they force lazy fetches into
the store. More info and new PR soon.
- Only cases I believe are worthwhile or easily determined
I've determined the validity by
- testing llvmPackages instantiation
- figuring out which types can pass into any particular
toString call - "human fuzzy type checker"
Behavior considerations by type:
- `path`: converted back to a string *without* context
`baseNameOf` does not copy things to the store on its own,
equivalent to its behavior for string inputs
- `null`: converted to `""` -> may be valid input!
ok if "" would not have been acceptable anyway
- `string` itself: passed through identically -> trivial
- `attrset` with `outPath`: same coercion as built into
the `baseNameOf` function -> trivial
- other atomic types: generally not sensible inputs to
`baseNameOf` -> fuzzy but true
- concrete types start with uppercase: Int, String, Bool, Derivation,
etc.
- type variables start with lowercase: a, b, etc.
- list:
- use `[x]` for homogeneous lists instead of `List x` or `[ x ]`
- use `List` for heterogeneous lists (not that common in `lib`)
- attr:
- use `AttrSet` for a generic attribute set type
- use `{ key1 :: Type1; key2 :: Type2; ... }` for adding signatures
for known attribute names and types
- use `{ key1 = value1; key2 = value2; ... }` for adding attributes
with known literals
- end with an ellipsis (`...`) if the set can contain unknown
attributes
- use `{ [String] :: x }` if all the attributes has the same type `x`
- prefer `Any` over `a` if the latter is not reused
This patch adds `lib.repoRevToName` function that generalizes away most of the
code used for derivation name generation by `fetch*` functions (`fetchzip`,
`fetchFromGitHub`, etc, except those which are delayed until latter commits
for mass-rebuild reasons).
It's first argument controls how the resulting name will look (see below).
Since `lib` has no equivalent of Nixpkgs' `config`, this patch adds
`config.fetchedSourceNameDefault` option to Nixpkgs and then re-exposes
`lib.repoRevToName config.fetchedSourceNameDefault` expression as
`pkgs.repoRevToNameMaybe` which is then used in `fetch*` derivations.
The result is that different values of `config.fetchedSourceNameDefault` now
control how the `src` derivations produced by `fetch*` functions are to be
named, e.g.:
- `fetchedSourceNameDefault = "source"` (the default):
```
$ nix-instantiate -A fuse.src
/nix/store/<hash>-source.drv
```
- `fetchedSourceNameDefault = "versioned"`:
```
$ nix-instantiate -A fuse.src
/nix/store/<hash>-libfuse-2.9.9-source.drv
```
- `fetchedSourceNameDefault = "full"`:
```
$ nix-instantiate -A fuse.src
/nix/store/<hash>-libfuse-2.9.9-github-source.drv
```
See the documentation of `config.fetchedSourceNameDefault` for more info.
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.
Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.
A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.
This commit was automatically created and can be verified using
nix-build https://github.com/infinisil/treewide-nixpkgs-reformat-script/archive/a08b3a4d199c6124ac5b36a889d9099b4383463f.tar.gz \
--argstr baseRev b32a094368
result/bin/apply-formatting $NIXPKGS_PATH
so that it doesn't make the manual build fail
> Apparently this is related to the combination of this new function not getting exported from the file, while still getting documented.
This requires us to avoid the `tryEval` + `throw` combination,
because throw is strict in its error message, and we don't want
to drop our single clue when `commitIdFromGitRepo` is used
incorrectly.
It gives a warning on the lazy-trees branch of Nix
(NixOS/nix#6530)
"warning: applying 'toString' to path '...' and then accessing it is deprecated, at '...'"
'else toString (/. + "${base}/${path}");' at line 183 may still cause a warning but i don't know how
to reach that codepath and test so im leaving it untouched
changing it to 'else /. + "${base}/${path}";'
caused this error
```
error: a string that refers to a store path cannot be appended to a path
at /home/systems/nixpkgs/lib/sources.nix:183:20:
182| then path
183| else /. + "${base}/${path}";
| ^
184| in if pathIsRegularFile path
```
Change comment type so than nixdoc picks them up into Nixpkgs manual.
Also improve phrasing a bit and move stuff around so that it is formatted better.
I'm working on a project that involves running a virtual machine
monitor program, which creates a control socket in the project
directory (because it doesn't make sense to put it anywhere else).
This obviously isn't part of the source of my program, so I think
cleanSource should filter it out.
When in the presence of worktrees, it happens that /commondir has a
trailing slash.
In these circumstances, it can lead to `lib.pathType` being passed paths
like `/foo/bar/.git/`, which in turn lead to
`error: attribute '.git' missing`.
With this change, we now make sure send properly-formatted paths to all
other functions.
This, in particular, fixes running NixOS tests on worktrees created by
libgit2 on my machine. (Worktrees created by git itself appear to not
hit the issue.)
split comes from builtins, not lib.
error: attribute 'split' missing, at /nix/path/nixpkgs/lib/sources.nix:4:4
(use '--show-trace' to show detailed location information)
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.
Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.
If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.
Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
Currently, not providing `name` to `cleanSourceWith` will use the name
of the imported directory. However, a common case is for this to be the
top level of some repository. In that case, the name will be the name of
the checkout on the current machine, which is not necessarily
reproducible across different settings, and can lead to e.g. cache
misses in CI.
This is documented in the comment on `cleanSourceWith`, but this does
not stop it being a subtle trap for users.
There are different tradeoffs in each case:
1. If `cleanSourceWith` defaults to `"source"`, then we may end up with a
user not knowing what directory a source store path corresponds to.
However, it being called "unnamed" may give them a clue that there is a
way for them to name it, and lead them to the definition of the
function, which has a clear `name` parameter.
2. If `cleanSoureWith` defaults to the directory name, then a user may face
occasional loss of caching, which is hard to notice, and hard to track
down. Tracking it down likely requires use of more advanced tools like
`nix-diff`, and reading the source of a lot of nix code.
I think the downside of the status quo is worse.
This is really another iteration of
https://github.com/NixOS/nix/issues/1305: that led to adding the `name`
argument in the first place, this just makes us use a better default
`name`.
Before c9214c394b and
9d396d2e42 if .git is symlink the version
would gracefully default to no git revision. With those changes an
exception is thrown instead.
This introduces a new function `pathIsGitRepo` that checks if
`commitIdFromGitRepo` fails without error so we don't have to
reimplement this logic again and can fail gracefully.
lib.commitIdFromGitRepo now resolves the refs from the
parent repository in case the supplied path is a file
containing the path to said repository. this adds support
for git-worktree and things alike. see gitrepository-layout(5).
this also:
- adds a new boolean function lib.pathIsRegularFile to
check whether a path is a regular file
- patches lib.revisionWithDefault and
the revision and versionSuffix attributes in
config.system.nixos in order to support git-worktrees
This change is API-compatible and hash-compatible with the previous
version.
At first I considered to write a rename function too, but adding
it name to cleanSourceWith was a no-brainer for ease of use. It
turns out that a rename function isn't any more useful than
cleanSourceWith.
To avoid having to write the identity predicate when renaming,
the filter is now optional.
builtins.path is supported since Nix 2.0 which is required by nixpkgs
`sourceByRegex src regexes` should include a source file if one of the
regular expressions `regexes` matches the path of that file relative
to `src`.
However to compute this relative path `sourceByRegex` uses:
```
relPath = lib.removePrefix (toString src + "/") (toString path);
```
Note that `toString path` evaluates to an absolute file somewhere
under `src` and not under `/nix/store`.
The problem is that this doesn't work if `src` is a `cleanSourceWith`
invocation as well because `toString src` will then evaluate to
`src.outPath` which will evaluate to `builtins.filterSource ...` which
evaluates to a path in `/nix/store` which is not a prefix of `path`.
The solution is to replace `src` with `origSrc` where
```
origSrc = if isFiltered then src.origSrc else src;
isFiltered = src ? _isLibCleanSourceWith;
```
Test this by executing the following from the nixpkgs repo:
```
(cat << 'EOI'
let
pkgs = import ./. {};
in pkgs.runCommand "test-sourceByRegex" {
test_sourceByRegex =
let
src1 = pkgs.lib.sourceByRegex ./. [ "^test-sourceByRegex.nix$" ];
src2 = pkgs.lib.sourceByRegex src1 [ "^test-sourceByRegex.nix$" ];
in src2 + "/test-sourceByRegex.nix";
} ''
cp $test_sourceByRegex $out
''
EOI
) > test-sourceByRegex.nix
nix-build test-sourceByRegex.nix
```
