A while back we introduced image_name_is_valid() for validating image
file names. It's more liberal than hostname_is_valid() in many ways (and
allows version suffixes and such). Since importd deals in offline images
(as opposed to machined otherwise which deals in running machines),
let's hence use the right helper to validate the identifiers.
This adds "Ex" versions of all bus calls import implements, that make
two changes:
1. A "class" parameter is added that allows choosing between
machine/sysext/confext/portable images to download. Depending on the
chose class the target directory is selected differently (i.e. not
just /var/lib/machines/, but alternatively /var/lib/portables/,
/var/lib/extensions/, /var/lib/confexts/.
2. The boolean flags are replaced by a 64bit flags parameter.
The two enums are mostly the same, the former is just an extension of
the latter. Let's merge them, to simplify things. This is particularly
useful as we then can reuse this systematically as D-Bus method call
flags too, in a generic fashion that works for both imports and pulls
the same.
Pretty much just renaming of flags.
Sometimes it makes sense to hard kill a client if we die. Let's hence
add a third FORK_DEATHSIG flag for this purpose: FORK_DEATHSIG_SIGKILL.
To make things less confusing this also renames FORK_DEATHSIG to
FORK_DEATHSIG_SIGTERM to make clear it sends SIGTERM. We already had
FORK_DEATHSIG_SIGINT, hence this makes things nicely symmetric.
A bunch of users are switched over for FORK_DEATHSIG_SIGKILL where we
know it's safe to abort things abruptly. This should make some kernel
cases more robust, since we cannot get confused by signal masks or such.
While we are at it, also fix a bunch of bugs where we didn't take
FORK_DEATHSIG_SIGINT into account in safe_fork()
We use it for more than just pipe() arrays. For example also for
socketpair(). Hence let's give it a generic name.
Also add EBADF_TRIPLET to mirror this for things like
stdin/stdout/stderr arrays, which we use a bunch of times.
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.
For consistency, 'de' is used everywhere for the var name.
rearrange_stdio() invalidates specified fds even on failure, which means
we should always invalidate the fds we pass in no matter what. Let's
make this explicit by using TAKE_FD() for that everywhere.
Note that in many places we such invalidation doesnt get us much
behaviour-wise, since we don't use the variables anymore later. But
TAKE_FD() in a way is also documentation, it encodes explicitly that the
fds are invalidated here, so I think it's a good thing to always make
this explicit here.
This does what the previous commit did for systemd-import the same way
for systemd-pull.
It also adds one more thing: the checksum validation is extended, in
addition of doing SHA256SUMS/gpg verification it is now possible to
immediately specify a hash value on the command line that the download
needs to match. This is particularly useful in --direct mode as we can
download/decompress/unpack arbitrary files and check the hash of the
downloaded file on-the-fly.
Let's be paranoid and do something useful if we operate with empty
haystack/needle. This doesn't actually fix anything, as the places as
far as I can see check for non-emptyness already beforehand, but I will
sleep safer at night, if we don't even allow the trap to be fallen in,
ever, even if the code is changed sooner or later.
Follow-up for 133b34f69a where this was
forgotten.
While we are at it, bring the parameters into the same order as we
declare them in the PullRaw/PullTar objects, i.e. match them to the
canonical order.
We already had support for downlading a .nspawn and a .roothash file,
let's make the set complete, and also download .verity + roothash.p7s if
it exists, as nspawn consumes that.
Since there are now four kinds of additional resources to acquire, let's
introduce a PullFlags flags value for this instead of separate 'bool'
variables, it's just too many to always pass those around on the
function parameter list.
Previously the PullJob object took internal care of rerequested the
SHA256SUMS file, if requesting <image>.sha256 didn't work. This was a
weird a non-abstraction only used when actually getting the checksum
files.
Let's move this out of the PullJob, so that it is generic again, and
does roughly the same stuff for all resources it is used for: let's
define a generic .on_not_found() handler that can be set on a PullJob
object, and is called whenever with see HTTP 404, and may be used to
provide a new URL to try if the first didn't work.
This is also preparation for later work to support PKCS#7 signatures
instead of gpg signatures, where a similar logic is needed, and we thus
should have a generic infrastructure place.
This gets rid of the VerificationStyle field in the PullJob object:
instead of storing this non-generic field we just derive the same
information from the URL itself, which is safe, since we generated it
ourselves earlier.
Whenever we invoke external, foreign code from code that has
RLIMIT_NOFILE's soft limit bumped to high values, revert it to 1024
first. This is a safety precaution for compatibility with programs using
select() which cannot operate with fds > 1024.
This commit adds the call to rlimit_nofile_safe() to all invocations of
exec{v,ve,l}() and friends that either are in code that we know runs
with RLIMIT_NOFILE bumped up (which is PID 1 and all journal code for
starters) or that is part of shared code that might end up there.
The calls are placed as early as we can in processes invoking a flavour
of execve(), but after the last time we do fd manipulations, so that we
can still take benefit of the high fd limits for that.
Ideally, coccinelle would strip unnecessary braces too. But I do not see any
option in coccinelle for this, so instead, I edited the patch text using
search&replace to remove the braces. Unfortunately this is not fully automatic,
in particular it didn't deal well with if-else-if-else blocks and ifdefs, so
there is an increased likelikehood be some bugs in such spots.
I also removed part of the patch that coccinelle generated for udev, where we
returns -1 for failure. This should be fixed independently.
These lines are generally out-of-date, incomplete and unnecessary. With
SPDX and git repository much more accurate and fine grained information
about licensing and authorship is available, hence let's drop the
per-file copyright notice. Of course, removing copyright lines of others
is problematic, hence this commit only removes my own lines and leaves
all others untouched. It might be nicer if sooner or later those could
go away too, making git the only and accurate source of authorship
information.
This part of the copyright blurb stems from the GPL use recommendations:
https://www.gnu.org/licenses/gpl-howto.en.html
The concept appears to originate in times where version control was per
file, instead of per tree, and was a way to glue the files together.
Ultimately, we nowadays don't live in that world anymore, and this
information is entirely useless anyway, as people are very welcome to
copy these files into any projects they like, and they shouldn't have to
change bits that are part of our copyright header for that.
hence, let's just get rid of this old cruft, and shorten our codebase a
bit.
Files which are installed as-is (any .service and other unit files, .conf
files, .policy files, etc), are left as is. My assumption is that SPDX
identifiers are not yet that well known, so it's better to retain the
extended header to avoid any doubt.
I also kept any copyright lines. We can probably remove them, but it'd nice to
obtain explicit acks from all involved authors before doing that.
This macro will read a pointer of any type, return it, and set the
pointer to NULL. This is useful as an explicit concept of passing
ownership of a memory area between pointers.
This takes inspiration from Rust:
https://doc.rust-lang.org/std/option/enum.Option.html#method.take
and was suggested by Alan Jenkins (@sourcejedi).
It drops ~160 lines of code from our codebase, which makes me like it.
Also, I think it clarifies passing of ownership, and thus helps
readability a bit (at least for the initiated who know the new macro)
When we check the exit status of a subprocess, let's compare it with
EXIT_SUCCESS rather than 0 when looking for success.
This clarifies in code what kind of variable we are looking at and what
we are doing.
This renames wait_for_terminate_and_warn() to
wait_for_terminate_and_check(), and adds a flags parameter, that
controls how much to log: there's one flag that means we log about
abnormal stuff, and another one that controls whether we log about
non-zero exit codes. Finally, there's a shortcut flag value for logging
in both cases, as that's what we usually use.
All callers are accordingly updated. At three occasions duplicate logging
is removed, i.e. where the old function was called but logged in the
caller, too.
This adds a new safe_fork() wrapper around fork() and makes use of it
everywhere. The new wrapper does a couple of things we previously did
manually and separately in a safer, more correct and automatic way:
1. Optionally resets signal handlers/mask in the child
2. Sets a name on all processes we fork off right after forking off (and
the patch assigns useful names for all processes we fork off now,
following a systematic naming scheme: always enclosed in () – in order
to indicate that these are not proper, exec()ed processes, but only
forked off children, and if the process is long-running with only our
own code, without execve()'ing something else, it gets am "sd-" prefix.)
3. Optionally closes all file descriptors in the child
4. Optionally sets a PR_SET_DEATHSIG to SIGTERM in the child, in a safe
way so that the parent dying before this happens being handled
safely.
5. Optionally reopens the logs
6. Optionally connects stdin/stdout/stderr to /dev/null
7. Debug logs about the forked off processes.
We are using the same pattern at various places: call dup2() on an fd,
and close the old fd, usually in combination with some O_CLOEXEC
fiddling. Let's add a little helper for this, and port a few obvious
cases over.
