Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge FreeBSD 2023-04-21 #1721

Merged
merged 460 commits into from
Jun 21, 2023
Merged

Merge FreeBSD 2023-04-21 #1721

merged 460 commits into from
Jun 21, 2023

Conversation

bsdjhb
Copy link
Collaborator

@bsdjhb bsdjhb commented Jun 21, 2023

PR for CI

bsdimp and others added 30 commits April 20, 2023 16:18
Have more accruate comments. While #if, #else, etc are copied to the
header files, lines that don't start with # are not.  And #include files
are only output to sysinc (which winds up at the front of init_sysent.c
which seems a bit odd). This is all radically undocumented, and likely
has drifted somewhat from 4.4BSD and what other systems do (they've
drifted too, fwiw).

Sponsored by:		Netflix
Move the setting of TARGET_SPEC_VARS to local.sys.env.mk
so meta.sys.mk can do the processing, and include local.meta.sys.mk
later.

Move the setting of GENDIRDEPS_FILTER*_VARS from local.gendirdeps.mk
to local.meta.sys.mk so we can automatically set DEP_* at level 1+
to avoid syntax errors when DEP_* variables are used in conditionals
in Makefile.depend files.

Update gendirdeps.mk just to get the documentation about the above.
No functional change.

local.dirdeps.mk be more careful about adding to DIRDEPS to avoid
unnecessary overhead, and introducing cycles in the graph.
Also set DEP_MACHINE_CPUARCH.

Reviewed by:	stevek
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D39737
These are the changes since the last update (copy-pasted from the
release notes for Chelsio Unified Wire v3.18.0.0):

====================
Version : 1.27.3.0
Date    : 04/07/2023

Fixes
-----
BASE:
- Fixed a hang if module eeprom reads gives invalid data.
- KR backlplane no-fec link problem fixed.
OFLD:
- iscsi ddp errors fixed.
- iwarp connection abort in rare cases causing NIC traffic hang fixed.

ENHANCEMENTS
------------
BASE:
- Cisco GLC-TE 1G modules support added.

====================
Version : 1.27.1.0
Date    : 12/02/2022

Fixes
-----
BASE:
- memwrite dsgl cannot be used for T5.
OFLD:
- Enabled FCoE in SO adapters.
- TOE-TLS crash fixed.
- iscsi hang fixed.

MFC after:	2 weeks
Sponsored by:	Chelsio Communications
When not doing tree walks, it is bad for sub-dirs to depend on
parents.  Move the generation of opt_osname.h to distextract
and have others that need that depend on it.

In usr.sbin/bsdinstall use SUBDIR_DEPEND_ so tree walking still works.

Reviewed by:	obrien
Differential Revision:	https://reviews.freebsd.org/D39742
Forgot to remove this in 559b94a.

Fixes:		559b94a
Sponsored by:	Netflix
Back in 4.3BSD, the system call table wasn't generated, and there was an
entry:
        "4.2 sigreturn",        /* 139 = old 4.2 sigreturn */
This got converted to
139     OBSOL   0 4.2 sigreturn
in 4.3 RENO. Since it was obsolete, nothing bad happened. In fact,
there was code in makeyscalls.sh to cope:
        {       comment = $4
                for (i = 5; i <= NF; i++)
                        comment = comment " " $i
                if (NF < 5)
                        $5 = $4
        }
so the generated comment in syscalls.c was almost correct:
        "obs_4.2",                      /* 139 = obsolete 4.2 sigreturn */
a bug that we have to this very day, despite makesyscalls.sh being
rewritten in lua.

However, this historical wart is the only place in our current
syscalls.master file where we have an extra field for the 'not
generated' class of system calls. Remove the historical wart so that the
re-write of makesyscalls.lua can be simpler (so, I hope, qemu's bsd-user
can large swathes of code automatically generated too). This should help
make things more understandable (changes to simplify makesyscalls.lue
aren't quite debugged, so have to wait for another day).

There's 3 different obsolete sigreturns (but only 1 that was ever in
FreeBSD 2.x and newer).

Sponsored by:		Netflix
The 4.2 sigreturn was a bit of a enima so the 4.2 was remove. Regenerate
to cope the very minor changes in comments and one string.

Sponsored by:		Netflix
Commit 3e0856b updated
__sg_alloc_table_from_pages to use the same API as linux, but modified
the loop condition when going over the pages in a sg list. Part of the
change included moving the sg_next call out of the for loop and into the
body, which causes an off by one error when traversing the list. Since
sg_next is called before the loop body it will skip the first element
and read one past the last element.

This caused panics when running PRIME with nvidia-drm as the off-by-one
issue causes a NULL dereference.

Reviewed by:	bz, hselasky
Differential Revision:	https://reviews.freebsd.org/D39628
Fixes:	3e0856b ("linuxkpi: Fix `sg_alloc_table_from_pages()` to have the same API as Linux")
Before the commit 6cc44223cb6717795afdac4348bbe7e2a968a07d the
field event_mask was fully copied to the EventMasks field.
After this commit the event_mask (uint8_t) is 4 times casted to
EventMask (uint32_t). Because of that 24 bits of each event_mask array
is lost.

This commits brings back simple copying of field, and after words
converting 32 bits field to the requested endian.

I don't think we need more sophisticated method,
as the array is of size 4 (for 32 bits version).

Reviewed by:	imp
MFC after:	1 week
Sponsored by:	Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D39562
Pointed out by:	imp
Sponsored by:   Klara Inc.
* Align 'on <interface>' parameter with the BNF, so use 'on <ifspec>'
* Clarify etherprotospec BNF, to make it clearer that only numbers are
  supported.

Suggested by:	Christian McDonald
Sponsored by:	Rubicon Communications, LLC ("Netgate")
The other stacks it turns out actually expect the output to be called and can become stuck if it is
not. This is because they run there timer code from there and the input routine does not always
assure a timer is running. The real longterm fix here might be to go into the other stacks (rack and bbr)
and make sure that a timer is running after input if you don't do output.. as well as call the timer functions.
This would cut down on calls from hpts. But I think its too dramatic of a change for the immediate time.

Reviewed by: tuexen, glebius
Sponsored by: Netflix Inc
Differential Revision:https://reviews.freebsd.org/D39738
Reported by:	Jenkins
Fixes:	080e56a ("dtrace: expose dtrace_instr_size() to userland and implement it for riscv")
There's no need to quote the # here. Inside of regexp, it's not treated
like a comment from an awk perspective. And inside if '' it's not
treated as special by the shell. gawk also warns.

Sponsored by:		Netflix
If it doesn't exist, we don't need it in the table...

Sponsored by:		Netflix
Need to regen freebsd32 as well when sys/kern/syscalls.master is
updated.

Sponsored by:		Netflix
A few Makefiles overrode LINKS and MLINKS when WITH_LLVM_BINUTILS was
set, which caused some llvm- prefixed tools to disappear.  One such case
was llvm-ranlib, which some ports invoke explicitly.

Use += when adding to LINKS and MLINKS under WITH_LLVM_BINUTILS.

PR:		270955, 270956, 270959
Submitted by:	jbeich
Reviewed by:	arichardson
Fixes: 021385a ("Add WITH_LLVM_BINUTILS to install LLVM binuti...")
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39747
Sponsored by:	The FreeBSD Foundation
Add the remaining bus_space*read*_8 functions conditionally for
only arm64 in order to not break KASAN builds with new code using
one of them.

Suggested by:	markj
Reviewed by:	markj
MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D39581
There is one data corruption problem reported and fixed upstream, not
cherry-picked here yet.

On top of it the following fires under load:
        VERIFY(zil_replaying(zfsvfs->z_log, tx));

The patch which introduced the entire machinery is a revert candidate,
but as the machinery came with a dedicated feature flag, doing so would
render affected pools read-only at best. To be figured out.

As a temporary bandaid at least stop the active usage.
Note this patch does not make the feature disappear from zpool upgrade.

Sponsored by:	Rubicon Communications, LLC ("Netgate")
This reverts commit 5198511.

It results in data corruption, see:
openzfs/zfs#14753

Sponsored by:	Rubicon Communications, LLC ("Netgate")
ccb_h.status has two parts: the actual status and some addition bits to
indicate additional information. It must be masked before comparing
against completion codes. Add new inline function cam_ccb_success to
simplify this to test whether or not the request succeeded. Most of the
code already does this, but a few places don't (the rest likely should
be converted to use cam_ccb_status and/or cam_ccb_success, but that's
for another day). This caused at least one bug in recognizing devices
behind a SATA port multiplexer, though some of these checks were
fine with the special knowledge of the code paths involved.

PR:			270459
Sponsored by:		Netflix
MFC After:		1 week (and maybe a EN requst)
Reviewed by:		ken, mav
Differential Revision:	https://reviews.freebsd.org/D39572
Sponsored by:	Rubicon Communications, LLC ("Netgate")
This reverts commit 1d35493. It was the wrong fix. 757fc66 has
the proper fix to include stdbool for userland.

Sponsored by:		Netflix
Usually the kernel is loaded later, but there are circumstances where it
could have been loaded earlier than changing BEs.  Unload anything that
is already there so that we know we're using artifacts from the proper
environment.

PR:		265471
Reviewed by:	kevans
MFC after:	3 days
brooksdavis and others added 28 commits June 20, 2023 17:46
NETLINK was added along side NETLINK_SUPPORT to control building netlink
specific programs, but it has no consumers so remove it for now.

Reviewed by:	melifaro
Differential Revision:	https://reviews.freebsd.org/D39678
Run makeman and verify that src.conf.5 has been updated if
required and that there are no missing definition files.

Reviewed by:	emaste
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D39682
protopr does not support reading from a core anymore.
So don't state that it can.

Reviewed by:		glebius, rscheff, rrs
MFC after:		1 week
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D39688
PR:		209441
Sponsored by:	The FreeBSD Foundation
x["y"] can be written as x.y, which looks better and is a more typical
lua idiom.

Sponsored by:		Netflix
Reviewed by:		kevans
Differential Revision:	https://reviews.freebsd.org/D39709
Find the few places where we use 'sysxxx' and use "sysxxx" instead to be
more consistent.

Sponsored by:		Netflix
luacheck pointed out two minor issues: line isn't declared as a global,
so declare it local. Also remove an unused parameter.

Suggested by:		kevans
Sponsored by:		Netflix
Have more accruate comments. While #if, #else, etc are copied to the
header files, lines that don't start with # are not.  And #include files
are only output to sysinc (which winds up at the front of init_sysent.c
which seems a bit odd). This is all radically undocumented, and likely
has drifted somewhat from 4.4BSD and what other systems do (they've
drifted too, fwiw).

Sponsored by:		Netflix
Move the setting of TARGET_SPEC_VARS to local.sys.env.mk
so meta.sys.mk can do the processing, and include local.meta.sys.mk
later.

Move the setting of GENDIRDEPS_FILTER*_VARS from local.gendirdeps.mk
to local.meta.sys.mk so we can automatically set DEP_* at level 1+
to avoid syntax errors when DEP_* variables are used in conditionals
in Makefile.depend files.

Update gendirdeps.mk just to get the documentation about the above.
No functional change.

local.dirdeps.mk be more careful about adding to DIRDEPS to avoid
unnecessary overhead, and introducing cycles in the graph.
Also set DEP_MACHINE_CPUARCH.

Reviewed by:	stevek
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D39737
These are the changes since the last update (copy-pasted from the
release notes for Chelsio Unified Wire v3.18.0.0):

====================
Version : 1.27.3.0
Date    : 04/07/2023

Fixes
-----
BASE:
- Fixed a hang if module eeprom reads gives invalid data.
- KR backlplane no-fec link problem fixed.
OFLD:
- iscsi ddp errors fixed.
- iwarp connection abort in rare cases causing NIC traffic hang fixed.

ENHANCEMENTS
------------
BASE:
- Cisco GLC-TE 1G modules support added.

====================
Version : 1.27.1.0
Date    : 12/02/2022

Fixes
-----
BASE:
- memwrite dsgl cannot be used for T5.
OFLD:
- Enabled FCoE in SO adapters.
- TOE-TLS crash fixed.
- iscsi hang fixed.

MFC after:	2 weeks
Sponsored by:	Chelsio Communications
When not doing tree walks, it is bad for sub-dirs to depend on
parents.  Move the generation of opt_osname.h to distextract
and have others that need that depend on it.

In usr.sbin/bsdinstall use SUBDIR_DEPEND_ so tree walking still works.

Reviewed by:	obrien
Differential Revision:	https://reviews.freebsd.org/D39742
Forgot to remove this in 559b94a.

Fixes:		559b94a
Sponsored by:	Netflix
Back in 4.3BSD, the system call table wasn't generated, and there was an
entry:
        "4.2 sigreturn",        /* 139 = old 4.2 sigreturn */
This got converted to
139     OBSOL   0 4.2 sigreturn
in 4.3 RENO. Since it was obsolete, nothing bad happened. In fact,
there was code in makeyscalls.sh to cope:
        {       comment = $4
                for (i = 5; i <= NF; i++)
                        comment = comment " " $i
                if (NF < 5)
                        $5 = $4
        }
so the generated comment in syscalls.c was almost correct:
        "obs_4.2",                      /* 139 = obsolete 4.2 sigreturn */
a bug that we have to this very day, despite makesyscalls.sh being
rewritten in lua.

However, this historical wart is the only place in our current
syscalls.master file where we have an extra field for the 'not
generated' class of system calls. Remove the historical wart so that the
re-write of makesyscalls.lua can be simpler (so, I hope, qemu's bsd-user
can large swathes of code automatically generated too). This should help
make things more understandable (changes to simplify makesyscalls.lue
aren't quite debugged, so have to wait for another day).

There's 3 different obsolete sigreturns (but only 1 that was ever in
FreeBSD 2.x and newer).

Sponsored by:		Netflix
The 4.2 sigreturn was a bit of a enima so the 4.2 was remove. Regenerate
to cope the very minor changes in comments and one string.

Sponsored by:		Netflix
Commit 3e0856b updated
__sg_alloc_table_from_pages to use the same API as linux, but modified
the loop condition when going over the pages in a sg list. Part of the
change included moving the sg_next call out of the for loop and into the
body, which causes an off by one error when traversing the list. Since
sg_next is called before the loop body it will skip the first element
and read one past the last element.

This caused panics when running PRIME with nvidia-drm as the off-by-one
issue causes a NULL dereference.

Reviewed by:	bz, hselasky
Differential Revision:	https://reviews.freebsd.org/D39628
Fixes:	3e0856b ("linuxkpi: Fix `sg_alloc_table_from_pages()` to have the same API as Linux")
Before the commit 6cc44223cb6717795afdac4348bbe7e2a968a07d the
field event_mask was fully copied to the EventMasks field.
After this commit the event_mask (uint8_t) is 4 times casted to
EventMask (uint32_t). Because of that 24 bits of each event_mask array
is lost.

This commits brings back simple copying of field, and after words
converting 32 bits field to the requested endian.

I don't think we need more sophisticated method,
as the array is of size 4 (for 32 bits version).

Reviewed by:	imp
MFC after:	1 week
Sponsored by:	Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D39562
Pointed out by:	imp
Sponsored by:   Klara Inc.
* Align 'on <interface>' parameter with the BNF, so use 'on <ifspec>'
* Clarify etherprotospec BNF, to make it clearer that only numbers are
  supported.

Suggested by:	Christian McDonald
Sponsored by:	Rubicon Communications, LLC ("Netgate")
The other stacks it turns out actually expect the output to be called and can become stuck if it is
not. This is because they run there timer code from there and the input routine does not always
assure a timer is running. The real longterm fix here might be to go into the other stacks (rack and bbr)
and make sure that a timer is running after input if you don't do output.. as well as call the timer functions.
This would cut down on calls from hpts. But I think its too dramatic of a change for the immediate time.

Reviewed by: tuexen, glebius
Sponsored by: Netflix Inc
Differential Revision:https://reviews.freebsd.org/D39738
Reported by:	Jenkins
Fixes:	080e56a ("dtrace: expose dtrace_instr_size() to userland and implement it for riscv")
There's no need to quote the # here. Inside of regexp, it's not treated
like a comment from an awk perspective. And inside if '' it's not
treated as special by the shell. gawk also warns.

Sponsored by:		Netflix
If it doesn't exist, we don't need it in the table...

Sponsored by:		Netflix
Need to regen freebsd32 as well when sys/kern/syscalls.master is
updated.

Sponsored by:		Netflix
A few Makefiles overrode LINKS and MLINKS when WITH_LLVM_BINUTILS was
set, which caused some llvm- prefixed tools to disappear.  One such case
was llvm-ranlib, which some ports invoke explicitly.

Use += when adding to LINKS and MLINKS under WITH_LLVM_BINUTILS.

PR:		270955, 270956, 270959
Submitted by:	jbeich
Reviewed by:	arichardson
Fixes: 021385a ("Add WITH_LLVM_BINUTILS to install LLVM binuti...")
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39747
Sponsored by:	The FreeBSD Foundation
@bsdjhb bsdjhb merged commit 9ff0037 into CTSRD-CHERI:dev Jun 21, 2023
20 checks passed
@bsdjhb bsdjhb deleted the merge-freebsd-20230421 branch June 21, 2023 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.