utmp implementation #33

dslm4515 · 2020-10-20T04:47:33Z

The Musl Libc does not have utmp functionality built-in like glibc.

Perhaps skarnet’s utmps should be built before coreutils? Without it, some commands like last will report /dev/null/utmp is not a directory

The text was updated successfully, but these errors were encountered:

firasuke · 2020-10-20T10:09:09Z

Well here's the thing with utmp, utmpx and skarnet's utmps:

utmp.h has been deprecated in 2001 in favor of utmpx.h
On Glibc systems the former utmp.h is just an alias (or a symlink) for the latter utmpx.h
On musl libc, utmps can be used to provide utmpx functionality - but not utmp (This means that applications like procps-ng either don't work properly (w shows nothing under musl with default no-op implementation), or fails to build (utmps provides utmpx.h but no utmp.h)
utmps also uses a daemon as the only authority to manage the utmp and wtmp data, which is an extra service to consider
you'll also need to remove the stub header utmpx.h (and maybe even utmp.h) provided by musl if you plan on using skarnet's utmps
You'll also need to patch multiple software to support utmpx.h instead of utmp.h since utmps only provides the latter. Here's a list of packages using utmp h (that might require patching) that I compiled based on my experiments in glaucus:
- autoconf
- toybox (some pending tools)
- gcc (in libsanitizer)
- shadow
- musl
- util-linux (for agetty)
- procps-ng (for w and whattime)

From the musl wiki FAQ:

Q: Why is the utmp/wtmp functionality only implemented as stubs?

if the feature is implemented, you need to take additional measures to protect your user’s privacy

in order to use the utmp/wtmp feature, you need a suid/sgid binary to modify the database, which opens the door for security issues:

if you compromise those binaries, you can inject arbitrary data into the db, that other programs might interpret in exploitable ways

that’s a HUGE risk to pay for the sake of a basically-useless and possibly-harmful “feature”

The only distribution I know of that has utmps fully working with s6, musl and many other packages is Adélie Linux, so be sure to check their patches and the way they're implementing it.

For further reading:

Hope that helps.

dslm4515 · 2020-10-20T15:18:56Z

@firasuke , oh no, that helps a lot!

I just stumbled into an APKBUILD from Adélie Linux that shows utmps and skarnet being used while building shadow:

LIBS="-lutmps -lskarnet" ./configure ...

Its good to know I can check Adélie Linux to see how can get a utmpx implementation working.

Now I know why when build utmps after MLFS, it's not used at all.

firasuke · 2020-10-20T17:11:43Z

Yup, there's also that.

Don't forget to examine the resulting executables using your ldd.

dslm4515 · 2020-10-30T23:59:45Z

Looks like I found my first hurdle: inetutils.

Can't decide if i should drop inetutils. Personally, I use ifconfig instead of ip... maybe its time for me to learn to use ip

make[2]: Entering directory '/sources/inetutils-1.9.4/libinetutils'
  CC       logwtmpko.o
logwtmpko.c:35:1: error: conflicting types for 'logwtmp'
   35 | logwtmp (char *line, char *name, char *host)
      | ^~~~~~~
In file included from /usr/include/utmpx.h:11,
                 from logwtmpko.c:24:
/usr/include/utmps/utmpx.h:64:13: note: previous declaration of 'logwtmp' was here
   64 | extern void logwtmp (char const *, char const *, char const *) ;
      |             ^~~~~~~
make[2]: *** [Makefile:1468: logwtmpko.o] Error 1
make[2]: Leaving directory '/sources/inetutils-1.9.4/libinetutils'
make[1]: *** [Makefile:1491: all-recursive] Error 1
make[1]: Leaving directory '/sources/inetutils-1.9.4'
make: *** [Makefile:1428: all] Error 2

Created a patch to fix mismatch of types. Compile error goes away. Patch will be featured in MLFS-8.00

dslm4515 · 2020-11-07T19:03:58Z

Completed mlfs-8.00 with utmp/utmpx implementation. Not sure how to test. elogind now no longer has error Failed to create inotify on /dev/null/utmp, ignoring: Not a directory

firasuke · 2020-11-08T05:47:45Z

Nice, now hit w and see if it reports back any user(s).

firasuke · 2020-11-29T12:21:26Z

Any updates on this? Did you try what I mentioned in the previous comment?

dslm4515 · 2020-12-05T05:14:20Z

Sorry, I had some strange issues:
I am currently rebuilding mlfs-8.00 on i686, since I decided not only to add utmps, but also replace pkg-config with pkgconf and gettext with gettext-tiny... and building mlfs-8.00 on x86_64. I built musl-1.2.1 fine on the i686 system, but had compile time errors on the x86_64 system. Both systems are using a copy of the same patches, scripts, and sources.

The i686 system had already built the final system Musl libc and working on bzip2. The x86_64 system just finished it's tool chain and entered chroot. When building musl, one of the patches from Adelie Linux failed. Strange as same patch was used in the i686 build... So i try to compile Musl on the i686 system and patch also failed. Not sure why hours ago, patch did not fail. In the end, i had to modify the patch from Adelie Linux to salvage the build.

Now back to your question:

# w
 23:11:25 up 6 days, 11:44,  0 users,  load average: 0.41, 0.29, 0.25
USER     TTY        LOGIN@   IDLE   JCPU   PCPU WHAT

^ is this expected output?

dslm4515 · 2020-12-05T05:16:50Z

... same output if i try same command on my Samsung Chromebook3 (GalliumOS 3.1)

firasuke · 2020-12-05T09:57:04Z

That's the known issue, w doesn't show available users.

Try running w on any working Linux system (also on Adélie) and compare outputs.

dslm4515 · 2020-12-05T22:20:56Z

Just for reference, I booted an Artix Linux LiveCD and got a different output:

artix:[artix]:~$ w
 22:10:10 up 14 min, 1 user, load average:0.00, 0.22, 0.43
USER     TTY        LOGIN@    IDLE    JCPU    PCPU    WHAT
artix    tty1      21:59      0.00s   0.12s   0.01s   w

Artix uses Glibc though.

I'll try a liveCD of Adelie Linux next

dslm4515 · 2020-12-07T21:33:55Z

I didnt want to waste a blank DVD to burn a 719MB LiveCD of Adelie Linux, so installed Adelie Linux on an old Thinkpad x61s. Yes, Adelie does output users when try w.

Looks like, my bootscripts are not correctly running utmps?

firasuke · 2020-12-08T02:24:07Z

Unfortunately, that seems to be the case.

firasuke · 2020-12-10T08:12:52Z

Any updates to this issue?

dslm4515 · 2020-12-11T21:12:40Z

Its still on my to-do list.

For my MLFS builds, i use pkgtools. I'm currently creating build scripts because some packages, like bzip2 have a broken install target. As a workaround, I manually copy files to my DESTDIR directory.

Once i get the scripts done, I will just quickly update the scripts as I update MLFS repo. This should speed up building and testing. So far I'm at 54 scripts done with 39 to go.

firasuke · 2020-12-11T21:41:55Z

In my honest opinion, and based on my experience, Artix currently has the best s6 + s6-rc implementation, and I'd suggest that you spend some time understanding how they've implemented it. They also seem to have the closest vanilla implementation compared to the likes of Adelie (which uses OpenRC) and Obarun (which literally added an entire layer over s6/s6-rc).

The only thing that Artix lacks is musl and utmps, and I think combining the knowledge gained from understanding both implemenations can get a working s6/s6+rc on musl with utmps fully working.

The first glaucus release also used s6/s6+rc on musl (with utmps available and being linked to, but still not working), but I'm working on fixing that now.

dslm4515 · 2020-12-11T22:02:52Z

I tried to look into Obarun's 66 toolset/utilities but... couldn't figure out where the sources are and how to extract 66 from Obarun to use in in MLFS. It boasts that it makes it easier to work with S6+S6-rc.

If I remembered correctly, my bootscripts are loosely based on what I understood from Obarun.

I might scrap the Adelie Linux install on my x61 and install Artix for more s6+s6-rc research.

If curious:
One of the reasons I abandoned runit was I could never boot with my own boot scripts from scratch. Last time I build MLFS with runit, I had to use inthecloud247's repo [https://github.com/inthecloud247/runit-for-lfs] which has not been updated since 2014. It was like runit would only read scripts made by inthecloud247 or by Void Linux devs.... then again, s6+s6-rc boots in parallel and i could never find a way to do the same with runit

firasuke · 2020-12-11T22:52:37Z

I'd say Obarun's implementation is kinda their own thing now, even their s6/s6-rc stuff were archived and are no longer maintained in favor of 66.

I recommend you study Artix's and Adelie's implementations more carefully. I think even alpine has s6/s6-rc packages now (but I believe they're not intended to be run as PID 1, but they're still considered valuable resources.

Both Adelie's and Alpine's s6/s6-rc packages are maintained by Laurent Bercot himself as well.

dslm4515 · 2020-12-14T11:13:56Z

I rewrote my bootscripts by copying and modifying bootscripts form Artix and service scripts from Adelie Linux.

My MLFS build boots after so many attempts but still no users listed for w.

dslm4515 · 2020-12-14T17:14:44Z

Ugh. I forgot to check apkbuilds of Adelie ! I missed a configuration option —enable-utmps

Adelie Linux stores the utmps ‘server socket’ in /run/utmps...for my build is empty but i forgot to check under Adelie Linux

firasuke · 2020-12-14T19:56:18Z

Oh it's a good thing that you checked.

Artix doesn't use utmps (because it has glibc), and w grabs its information from /run/utmp which is created via the cleanup oneshot from s6-scripts (/etc/s6/sv/cleanup/up):

install -m0664 -o root -g utmp /dev/null /run/utmp

But this shouldn't be needed with utmps because utmpd/wtmpd are provided.

dslm4515 · 2020-12-15T00:00:26Z

For Adelie,
/run/utmps has .utmpd-socket And .wtmpd .socketAnd utmp (Which holds the data for w?)

My build boots with that directory empty. I will check the configure options for all s6 packages and try again... current state of bootscripts does yield an error with udevadm so I might just rewrite from scratch instead of merging Artix’s boot scripts and Adelie’s utmps service scripts

firasuke · 2020-12-26T21:16:55Z

I'm very pleased with what we've accomplished here.

Hopefully, we can tackle more problems in the future and come up with solutions for them.

dslm4515 · 2021-01-18T21:16:30Z

I enabled utmp in elogind and applied Adelie linux's patch to have source look for utmpx.h instead of utmp.h

Now I got this warning during logins:
Failed to set utmp path to /run/utmps/utmp: function not implemented

Not sure if elogind needs more patching as w still functions as intended

firasuke · 2021-05-14T08:07:23Z

For future reference:

Apparently musl is planning on providing non-stub utmp backends (according to their roadmap).

dslm4515 · 2021-05-28T21:59:30Z

Strange. I forgot to patch the usr/include/paths.h of musl and w still functions correctly!

#define _PATH_UTMP     "/dev/null/utmp"
#define _PATH_WTMP     "/dev/null/wtmp"

System is using musl version 1.2.2

firasuke · 2021-05-29T00:52:16Z

Strange. I forgot to patch the usr/include/paths.h of musl and w still functions correctly!

What!? Are you sure about that?

dslm4515 · 2021-05-29T01:52:15Z

Yeah. While building elogind for CMLFS, I noticed elogind was attempting to access /dev/null/utmp instead of /run/utmps/utmp... and I confirmed it by checking /usr/include/paths.h.

I also know w was working because when I first booted a CMLFS build I had errors for utmps. W wasn’t working since utmps was using the wrong dynamic linker. After recompiling utmps, w was working.

firasuke · 2021-06-11T20:37:11Z

Apparently a WTMPX file was added:

https://www.mail-archive.com/[email protected]/msg01560.html

Upon checking include/utmps/utmpx.h:

#define WTMPX_FILE "/var/log/wtmp"

What are your thoughts?

firasuke · 2021-06-11T21:18:15Z

According to the NEWS file it was added starting from 0.1.0.2:

WTMPX_FILE, which is unused by utmps, now points to /var/log/wtmp
so that programs that want to read that file (e.g. lastlog) can
access wtmp without patching. Or with less patching.

dslm4515 · 2021-06-11T21:49:42Z

I am currently building MLFS with updated sources.... including utmps-0.1.0.2.

So i will try to skip some patching and see how effective this change may be

firasuke · 2021-06-11T21:54:20Z

I am currently building MLFS with updated sources.... including utmps-0.1.0.2.

So i will try to skip some patching and see how effective this change may be

Good luck with that, I still think we need to patch src/include/utmps/utmpx.h though...

firasuke · 2021-06-11T22:02:50Z

Oh also:

The wtmpd example now uses /var/log/wtmp instead of /run/utmps/wtmp
since wtmp is supposed to survive across reboots.

dslm4515 · 2021-06-11T22:51:08Z

You are right, still need to patch those paths in musl libc headers... perhaps change paths to:

#define _PATH_UTMP      "/var/log/utmp"
#define _PATH_WTMP      "/var/log/wtmp"

dslm4515 · 2021-06-17T19:33:49Z

I rebuilt MLFS with latest SW as of June 2021... I accidentally packaged the whole rootfs instead of just my stage directory (where i fakeroot install)... which bricked the MLFS build.... nothing works (first bash couldnt resolved symbols then no commands and be found) unless i prefix every command with LD_LIBRARY_PATH=/lib:/usr/lib. I tried to re-create my build from the packages i made, but same issue. So I decided to scrap the build, and re-compile everything.

I get to util-linux and build fails. Same build instructions as before (no issue from what i can remember):

  CC       login-utils/lslogins-lslogins.o
In file included from login-utils/lslogins.c:39:
/usr/include/lastlog.h:1:10: fatal error: utmp.h: No such file or directory
    1 | #include <utmp.h>
      |          ^~~~~~~~
compilation terminated.
make[2]: *** [Makefile:11388: login-utils/lslogins-lslogins.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/sources/util-linux-2.36.2'
make[1]: *** [Makefile:13801: all-recursive] Error 1
make[1]: Leaving directory '/sources/util-linux-2.36.2'
make: *** [Makefile:5901: all] Error 2
(mlfs chroot) root:/sources/util-linux-2.36.2#

Not sure why util-linux needs utmp.h. Check the source (of util-linux):

#include <lastlog.h>

Then i check that header, as it belongs to musl (version 1.2.2 in this build):

 #include <utmp.h>

wth. I removed that header, as i thought I am using skarnet's utmps to provide utmpx and not use musl's utmp.h

So i check lastlog.h from musl 1.2.1 source:

#ifndef _LASTLOG_H
#define _LASTLOG_H

#include <sys/types.h>

#ifdef __cplusplus
extern "C" {
#endif

#define UT_HOSTSIZE 256
#define UT_LINESIZE 32

struct lastlog {
        time_t ll_time;
        char ll_line[UT_LINESIZE];
        char ll_host[UT_HOSTSIZE];
};

#ifdef __cplusplus
}
#endif

#endif

I thought musl only had stubs for utmp because it's hard to secure an implementation. Starting with musl-1.2.2, should I not remove headers utmp.h and utmpx.h? Skarnet's utmps installs it's headers in /usr/include/utmps

dslm4515 · 2021-06-17T19:47:50Z

Looking back at the beginning of this discussion, i should only need to remove musl's utmpx.h header in favor of skarnet's utmps utmpx.h...

firasuke · 2024-02-09T15:29:42Z

@dslm4515 So it seems that this is still an issue... After Y2038 many upstream software developers are moving with systemd's logind and disabling utmp/utmpx entirely, which is making it harder to maintain this.

What are your thoughts on this? Also is w (from util-linux) and login (from shadow) still working? Should we consider different alternatives that work with utmps?

firasuke · 2024-02-09T17:14:28Z

It seems that newer versions of shadow (starting from 4.14.0) have disabled utmpx entirely regardless of whether a distro is using utmps or not and they have switched to logind. This means that login is no longer storing utmp entries.

I opened an issue upstream to track this. Perhaps we can have support for utmps as well.

dslm4515 · 2024-02-09T19:06:04Z

Thank you for bringing it up.

I still have issues getting wtmp and/or utmp working. File itself is empty. But 'w' still works (somewhat-- lists users via ssh sessions but not on ttys? Maybe forgot to turn on support in shadow for utmpx?).

S6-Linux-init no longer has a built in boot script for running utmps daemon, so I'm trying to figure it out. S6+S6-RC booting appears complex to me... tempted to back to runnit.

But I do need to develop test(s) to make sure utmp/wtmp works.

For me, I like to stay away from systemd. I've been using seatd and turnstile instead of logind/elogind for a year or two so far.

firasuke · 2024-02-09T20:02:16Z

Thank you for bringing it up.

No worries, thought I'd ask for your input on this.

I still have issues getting wtmp and/or utmp working. File itself is empty. But 'w' still works (somewhat-- lists users via ssh sessions but not on ttys? Maybe forgot to turn on support in shadow for utmpx?).

shadow no longer has an option for utmpx (--enable-utmpx got removed).

S6-Linux-init no longer has a built in boot script for running utmps daemon, so I'm trying to figure it out. S6+S6-RC booting appears complex to me... tempted to back to runnit.

True, utmpd service was removed and was left to the user to implement. You can check the examples/ directory under the project utmps and see how you can get it to work.

For me, I like to stay away from systemd. I've been using seatd and turnstile instead of logind/elogind for a year or two so far.

I believe Chimera is moving in the right direction. I have to say your projects are somewhat similar.

dslm4515 · 2024-02-09T22:24:23Z

I believe Chimera is moving in the right direction. I have to say your projects are somewhat similar.

I would like to think, I developed CMLFS first before Chimera, but of course that doesnt matter. lol
BUT I owe Chimera Linux a lot of thanks. Without them, I probably would be stuck on LLVM 12 or 13.

shadow no longer has an option for utmpx (--enable-utmpx got removed).

I think, I noticed that with by latest build of CMLFS. I can't remember. I'm using that build now to update BMLFS and see what breaks. I am behind on the 'base' repo, MLFS.

you can check the examples/ directory under the project utmps and see how you can get it to work.

Yes,I did check that... I haven't saved my boot scripts to a git repo yet. But seems my script seems to work as w does report users on ssh sessions. I have to figure out more tests to see if my boot script for utmps is doing its job

dslm4515 · 2024-02-09T22:27:11Z

Worse case scenario, I may end up [in far future] installing systemd/logind in /opt/systemd as a fallback... kind of how I still use libreSSL but have openSSL in /opt/openssl for software like Python 3.10+ that do not support libreSSL without heavy patching.

dslm4515 added enhancement New feature or request question Further information is requested labels Oct 20, 2020

dslm4515 closed this as completed Dec 26, 2020

dslm4515 reopened this Jun 17, 2021

dslm4515 closed this as completed Jul 11, 2021

dslm4515 self-assigned this Feb 9, 2024

dslm4515 reopened this Feb 9, 2024

utmp implementation #33

utmp implementation #33

Comments

dslm4515 commented Oct 20, 2020

firasuke commented Oct 20, 2020 • edited Loading

dslm4515 commented Oct 20, 2020

firasuke commented Oct 20, 2020 • edited Loading

dslm4515 commented Oct 30, 2020

dslm4515 commented Nov 7, 2020

firasuke commented Nov 8, 2020

firasuke commented Nov 29, 2020

dslm4515 commented Dec 5, 2020

dslm4515 commented Dec 5, 2020

firasuke commented Dec 5, 2020

dslm4515 commented Dec 5, 2020

dslm4515 commented Dec 7, 2020

firasuke commented Dec 8, 2020

firasuke commented Dec 10, 2020

dslm4515 commented Dec 11, 2020

firasuke commented Dec 11, 2020 • edited Loading

dslm4515 commented Dec 11, 2020

firasuke commented Dec 11, 2020 • edited Loading

dslm4515 commented Dec 14, 2020 • edited Loading

dslm4515 commented Dec 14, 2020

firasuke commented Dec 14, 2020 • edited Loading

dslm4515 commented Dec 15, 2020 • edited Loading

firasuke commented Dec 26, 2020

dslm4515 commented Jan 18, 2021

firasuke commented May 14, 2021

dslm4515 commented May 28, 2021

firasuke commented May 29, 2021

dslm4515 commented May 29, 2021 • edited Loading

firasuke commented Jun 11, 2021

firasuke commented Jun 11, 2021

dslm4515 commented Jun 11, 2021

firasuke commented Jun 11, 2021

firasuke commented Jun 11, 2021

dslm4515 commented Jun 11, 2021

dslm4515 commented Jun 17, 2021

dslm4515 commented Jun 17, 2021

firasuke commented Feb 9, 2024

firasuke commented Feb 9, 2024

dslm4515 commented Feb 9, 2024

firasuke commented Feb 9, 2024

dslm4515 commented Feb 9, 2024 • edited Loading

dslm4515 commented Feb 9, 2024

firasuke commented Oct 20, 2020 •

edited

Loading

firasuke commented Oct 20, 2020 •

edited

Loading

firasuke commented Dec 11, 2020 •

edited

Loading

firasuke commented Dec 11, 2020 •

edited

Loading

dslm4515 commented Dec 14, 2020 •

edited

Loading

firasuke commented Dec 14, 2020 •

edited

Loading

dslm4515 commented Dec 15, 2020 •

edited

Loading

dslm4515 commented May 29, 2021 •

edited

Loading

dslm4515 commented Feb 9, 2024 •

edited

Loading