Missing unix/linux/posix file open option

What I would like is a file open option for "create replacement file".

The idea is that this makes a new inode in the same mount point as the target filename, which has no actual directory entry, but on close it replaces the directory entry of the specified path with the new file.

Why?

There are many situations where you want to make a new file to replace an existing file, but want the change from old to new file to be atomic at the file system level.

Something reading the half written new file is bad.

So you make a new temporary file, and then finally use rename() to replace the original, but that is only atomic if you are on the same mount point. You cannot simply use a mktemp() in /tmp for that as the target file may not be on the same mount point as /tmp. So you make a file with a dot prefix and some suffix or something. Messy. And needs cleaning up if you crash.

Easy?

It really would not be hard, I am sure, for the underlying filing system to support this as a file open mode. The atomic replacement of the directory entry is already a thing in rename(), and the idea of a file not in a directory but using space while open is easy - make a file and unlink it before closing. So the underlying mechanisms for this exist.

The only caveat, as a really useful extra feature, would be if not a "clean" close() call, i.e. a close because code aborted or exited without calling close(), it would *not* replace the original, just lose the new file as not in a directory. This helps cover the crash case, and always cleans up inherently.

This would be so useful.

And, of course, make gcc use this for making binaries!!!

Just to be clear, I am not suggesting "buffering" the whole file. The system to have an open file not in a directory already exists. One can create a file and open it and unlink it and still write to it, on to "disk". That is the "buffering" here. Just atomically either lose the new file (if crash, as would happen if you did that) or replace the directory entry with it and lose the old file, on clean close() call.

You can nearly do it!

Thanks for all the feedback, it is close... open() explains a _GNU_SOURCE specific option O_TMPFILE. This allows you to create an unnamed file which will vanish when you close, even if you crash. It then explains you can use a slightly convoluted call to linkat() to name the file before you close it. This nearly does the job, but not quite.

1. The open() call needs a directory so it knows file system. Annoyingly you cannot pass the filename you want and have it work out the directory. It has to be the directory, meaning you have to faff about getting the directory from the file name. Not a big faff, obviously.
2. The linkat() call needs CAP_DAC_READ_SEARCH set. There is a convoluted way to using /proc/ otherwise. More faff. Also, given that there is a documented way around the limitation, why is it dependant on a capability in the first place?!
3. The linkat() call expects the new filename not to exist.

This does allow a file to be atomically created as a complete file, with no temp files if you crash. But this last point is the show stopper as it means you have to unlink() first, leaving a small window where the file does not exist. That or you link to a temporary file name and use rename() which puts us almost back where we started, albeit with a smaller window for leaving a temp file behind.

The obvious fix would be a new flag to linkat() to allow replacing the new file. That or allow AT_EMPTY_PATH in renameat2().

popt

Linux has a really good library called popt. It parses command line arguments.

I use it all the time, and it allows a variety of arguments to be cleanly handled, with different types, default values, help text, and so on.

For a long time it has bugged me, and I assumed it was a bug, that it would leak a small amount of memory. This shows using valgrind. But as it is a one-off leak on a command line I have not bothered too much.

However, following some discussions re memory free tidying on one command recently I thought I would try and get my code 100% leak free - always a good aim.

I have found the problem.

popt malloc's any POPT_ARG_STRING values that it sets!

Now, I was not surprised to find some malloc'd memory, but was surprised at this. I was also surprised that poptFreeContext() does not free all allocated memory, these values are for the user to free.

What concerned me is that I did not know this, and the manual pages are not entirely clear. They do actually explain that poptGetOptArg() returns final arg of poptGetNextOpt(), and "The calling function is responsible for deallocating this string.". It does not seem to make clear that all arguments returned or stored for POPT_ARG_STRING are malloc'd. But that seems to be the case.

Now I know, I can ensure I free all string variables at the end of my program. Or can I?

What of defaults?

Well, damn, this is fun - some variables have defaults (the help text shows them even, if you ask it too), so you can set a default and the use popt, and have a string that is supplier, or the default.

This is useful, until you start freeing all your strings at the end, as these are typically const char assigned, and so not something you can free.

The only way to be clean is assign your defaults using strdup(), so you can always free, whether default or stored by popt.

Why does it do it?

What puzzles me is why it is done in the first place. The arguments are null terminated strings in argv[], so could be referenced directly. Even with a -x or --xname= prefix, an offset in to the argv[] value could be returned.

It is worse!

It seems poptGetArg() also alloc's strings as well, and the manual is totally silent on that point.

Arg! And worse, poptGetArg() is const char*, which is not valid for free() even!

No, No, that does not work

OK, I really tried, honest.

I set my defaults to be strdup() based so they are always safe to free() whether using default or supplied value.

What does that do? Well, if not using the default, the new supplied value is set, and malloc'd, so the free() is still safe. But what of the original default, that is now lost, not free'd, so a bloody leak!

I cannot see any clean way to do this - popt is basically broken.

Arrrg!