Turns out that the original way we did things was quite broken, as it
skipped a _lot_ of code. This was because we just threw everything into
one pile and tried to spatch it, but this made Coccinelle sad, like when
man page examples redefined some of our macros, causing typedef
conflicts.
For example, with a minimal reproducer that defines a cleanup macro in
two source files, Coccinelle has no issues when spatch-ing each one
separately:
$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci main.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: main.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
SPECIAL NAMES: adding _cleanup_free_ as a attribute
$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci
logcontrol-example.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: logcontrol-example.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
But when you try to spatch both of them at once, Coccinelle starts
complaining and skipping the "bad" code:
$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci main.c logcontrol-example.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: main.c logcontrol-example.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
SPECIAL NAMES: adding _cleanup_free_ as a attribute
remapping: _cleanup_ to an ident in macro name
ERROR-RECOV: found sync end of #define, line 44
parsing pass2: try again
ERROR-RECOV: found sync end of #define, line 44
parse error
= File "logcontrol-example.c", line 44, column 21, charpos = 1719
around = '__attribute__',
whole content = #define _cleanup_(f) __attribute__((cleanup(f)))
badcount: 2
bad: #include <systemd/sd-journal.h>
bad:
BAD:!!!!! #define _cleanup_(f) __attribute__((cleanup(f)))
This was, unfortunately, hidden as it is visible only with
--verbose-parsing (or --parse-error-msg).
Another issue was how we handled includes. The original way of throwing
them into the pile of source files doesn't really work, leading up to
similar issues as above. The better way is to let Coccinelle properly
resolve all includes by telling it where to find our own include files
(basically the same thing we already do during compilation).
After fixing all this, Coccinelle now has a chance to process much more
of our code (there are still some issues in more complex macros, but
that requires further investigation). However, there's a huge downside
from all of this - doing a _proper_ code analysis is surprisingly time
and resource heavy; meaning that processing just one Coccinelle rule now
takes 15 - 30 minutes.
To make this slightly less painful, Coccinelle supports caching the
generated ASTs, which actually helps a lot - it gets the runtime of one
rule from 15 - 30 minutes down to ~1 minute. It, of course, has its own
downside - the cache is _really_ big (ATTOW the cache takes ~15 GiB).
However, even with the aggressive AST caching you're still looking at
~1 hour for one full Coccinelle run, which is a bit annoying, but I
guess that's the price of doing things _properly_ (but I'll definitely
look into ways of further optimizing this).
As the result is a bit funky (but still valid), i.e.:
static inline void iovec_done_erase(struct iovec *iovec) {
assert(iovec);
- iovec->iov_base = erase_and_free(iovec->iov_base);
- iovec->iov_len = 0;
+ *iovec = IOVEC_MAKE(erase_and_free(iovec->iov_base), 0);
}
Since the empty_to_null() function was "macrofied", we need to use a bit
of black magic to make Coccinelle avoid running the transformation on
the macro itself.
Follow-up to ef2409cbde.
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
RUN_WITH_UMASK was initially conceived for spawning externals progs with the
umask set. But nowadays we use it various syscalls and stuff that doesn't "run"
anything, so the "RUN_" prefix has outlived its usefulness.
This was a trivial wrapper that didn't provide any added value. With more
complicated structures like strvs, hashmaps, sets, and arrays, it is possible
to have an empty container. But in case of a list, the list is empty only when
the head is missing.
Also, we generally want the positive condition, so we replace many
if (!LIST_IS_EMPTY(x)) with just if (x).
Let's define two helpers strdupa_safe() + strndupa_safe() which do the
same as their non-safe counterparts, except that they abort if called
with allocations larger than ALLOCA_MAX.
This should ensure that all our alloca() based allocations are subject
to this limit.
afaics glibc offers three alloca() based APIs: alloca() itself,
strndupa() + strdupa(). With this we have now replacements for all of
them, that take the limit into account.
* flag-set.cocci: perform the transformation only if the second
argument is a constant
* sd-journal/lookup3.c: skip the cocci completely for this file, since
it's not "ours"
* strjoina.cocci: skip the transformation on the "test_strjoina" test,
since it intentionally tests the "incorrect" expression we're trying to
transform (the same thing was already done in strjoin.cocci)
to drop braces around single-line if statements. Also, prefix it with
zz- so it runs as the last one, so it's able to fix stuff tweaked by
previous transformations.
Coccinelle can't do this automagically and requires we supply it
respective header files. Unfortunately, the option for this
(--macro-file=) can be used only once, so let's create our own
macro file by collecting macros needed for the semantic parser
to be happy.
The original issue with this transformation was that we were replacing
the whole if statement instead of just the expression inside. That
caused the code to be weirdly formatted, as Coccinelle put a new block
around each replaced if statement.
This version replaces just the inner expression if it's in its incorrect
form, otherwise it just accepts it (to avoid recursion).
My former dumb me didn't read the documentation properly, so with the
introduction of custom isomorphisms I caused two issues:
1) Masked all standard isomorphisms defined by Coccinelle
2) Replace the original issue with a completely new one
Patch contains a coccinelle script, but it only works in some cases. Many
parts were converted by hand.
Note: I did not fix errors in return value handing. This will be done separate
to keep the patch comprehensible. No functional change is intended in this
patch.
Some transformations generate results we don't want to keep, so
let's disable such transformations for specific files.
Also, disable const-strlen.cocci everywhere, as the STRLEN macro has a
pretty limited scope, so the transformation generates false positives in
most cases.
There's no point in running these transformation for certain files,
mainly anything from src/boot/efi and src/shared/linux, as this code
doesn't have access to our internal utility functions
For example, following transformation:
- isempty(s) ? NULL : s
+ empty_to_null(s)
would get applied to the empty_to_null function itself as well,
causing an infinite recursion, like:
--- src/basic/string-util.h
+++ /tmp/cocci-output-307-9f76e6-string-util.h
@@ -50,11 +50,11 @@ static inline bool isempty(const char *p
}
static inline const char *empty_to_null(const char *p) {
- return isempty(p) ? NULL : p;
+ return empty_to_null(p);
}
Let's avoid that by checking the current match position
For example, the following transformation:
- sizeof(s)-1
+ STRLEN(s)
would replace sizeof by STRLEN even in the STRLEN macro definition
itself, which generates following nonsensical patch:
--- src/basic/macro.h
+++ /tmp/cocci-output-8753-b50773-macro.h
@@ -182,7 +182,7 @@ static inline unsigned long ALIGN_POWER2
* Contrary to strlen(), this is a constant expression.
* @x: a string literal.
*/
-#define STRLEN(x) (sizeof(""x"") - 1)
+#define STRLEN(x) (STRLEN("" x ""))
/*
* container_of - cast a member of a structure out to the containing structure
Let's exclude the macro itself from the transformation to avoid this
The `coccinelle/take-fd.cocci` transformation file attempts to rewrite
r = fd;
fd = -1;
to
r = TAKE_FD(fd);
Unfortunately, using `identifier` or `idexpression` as a metavariable
type in this case wouldn't match more complex location descriptions,
like:
x->fd = fd
fd = -1;
Using 'expression' metavariable type generates false positives,
as you can't specify scope of such expression. The only real example
from the current codebase is the global 'errno' variable, which results
in following patch generated by `spatch`:
--- src/basic/errno-util.h
+++ /tmp/cocci-output-28263-971baa-errno-util.h
@@ -15,8 +15,7 @@ static inline void _reset_errno_(int *sa
#define UNPROTECT_ERRNO \
do { \
- errno = _saved_errno_; \
- _saved_errno_ = -1; \
+ errno = TAKE_FD(_saved_errno_); \
} while (false)
static inline int negative_errno(void) {
Let's explicitly state that the matched expression should not equal
'errno' to avoid this. It's not particularly nice, but it should be
enough, at least for now.
Coccinelle needs a custom isomorphism file with rules (isomorphisms) how
to correctly rewrite conditions with explicit NULL checks (i.e.
if (ptr == NULL)) to their shorter form (i.e. if (!ptr)). Coccinelle
already contains such isomorphisms in its default .iso file, however,
they're in the opposite direction, which results in useless output from
coccinelle/equals-null.cocci.
With this fix, `spatch` should no longer report patches like:
@@ -628,8 +628,9 @@ static int path_deserialize_item(Unit *u
f = path_result_from_string(value);
if (f < 0)
log_unit_debug(u, "Failed to parse result value: %s", value);
- else if (f != PATH_SUCCESS)
- p->result = f;
+ else {if (f != PATH_SUCCESS)
+ p->result = f;
+ }
} else
log_unit_debug(u, "Unknown serialization key: %s", key);
This is partially a refactoring, but also makes many more places use
unlocked operations implicitly, i.e. all users of fopen_temporary().
AFAICT, the uses are always for short-lived files which are not shared
externally, and are just used within the same context. Locking is not
necessary.
