external/virtualization-layer: patch libtirpc126 to limit fd table sizes

With recent systemd versions, process limits on the number of open files
have been raised by a couple of orders of magnitude (K to G). The libtirpc
library, which is unfortunately used by libnvidia-container-tools to implement
some inter-process communication, allocates some arrays of tracking structures based
on the open-files limit (e.g., dtablesze) - leading to memory allocation failures
as it tries to allocate multi-GiB arrays off the heap.

Add a modified patch, back-ported from libtirpc 1.3.2, that caps the array sizes
back down to 1K, to work around this problem for now.  The original patch only
handled the svc client; the modification also limits the array size for the
server side.

Signed-off-by: Matt Madison <matt@madison.systems>
This commit is contained in:
Matt Madison
2021-07-25 06:19:22 -07:00
committed by Matt Madison
parent 8e6b254d02
commit 269181b906
2 changed files with 52 additions and 0 deletions

View File

@@ -0,0 +1,50 @@
From 99f943123d2832cdd0f77c989d82cc8cba26e90b Mon Sep 17 00:00:00 2001
From: Steve Dickson <steved@redhat.com>
Date: Wed, 22 Apr 2020 12:18:43 -0400
Subject: [PATCH] __rpc_dtbsize: rlim_cur instead of rlim_max
In the client code, rlim_max is used to allocate two
arrays used for multithread locking. These arrays are
indexed with open file descriptors.
With some recent changes to systemd, the rlim_max is
now a very large number and no longer represents the
max number of open file descriptors allowed causing
the locking arrays to be huge resulting in processes
being OOM killed.
It turns out the max number opens allowed in a process
is still fairly small (1023) which means rlim_cur (1024)
can be used instead of rlim_max.
Signed-off-by: Steve Dickson <steved@redhat.com>
---
src/rpc_generic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Index: libtirpc-1.2.6/src/rpc_generic.c
===================================================================
--- libtirpc-1.2.6.orig/src/rpc_generic.c
+++ libtirpc-1.2.6/src/rpc_generic.c
@@ -112,7 +112,7 @@ __rpc_dtbsize()
return (tbsize);
}
if (getrlimit(RLIMIT_NOFILE, &rl) == 0) {
- return (tbsize = (int)rl.rlim_max);
+ return (tbsize = (int)(rl.rlim_cur > 1024 ? 1024 : rl.rlim_cur));
}
/*
* Something wrong. I'll try to save face by returning a
Index: libtirpc-1.2.6/src/rpc_dtablesize.c
===================================================================
--- libtirpc-1.2.6.orig/src/rpc_dtablesize.c
+++ libtirpc-1.2.6/src/rpc_dtablesize.c
@@ -42,6 +42,8 @@ _rpc_dtablesize(void)
if (size == 0) {
size = getdtablesize();
+ if (size > 1024)
+ size = 1024;
}
return (size);
}

View File

@@ -10,6 +10,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=f835cce8852481e4b2bbbdd23b5e47f3 \
SRC_URI = "${SOURCEFORGE_MIRROR}/libtirpc/libtirpc-${PV}.tar.bz2" SRC_URI = "${SOURCEFORGE_MIRROR}/libtirpc/libtirpc-${PV}.tar.bz2"
SRC_URI[sha256sum] = "4278e9a5181d5af9cd7885322fdecebc444f9a3da87c526e7d47f7a12a37d1cc" SRC_URI[sha256sum] = "4278e9a5181d5af9cd7885322fdecebc444f9a3da87c526e7d47f7a12a37d1cc"
SRC_URI += "file://0001-__rpc_dtbsize-rlim_cur-instead-of-rlim_max.patch"
S = "${WORKDIR}/libtirpc-${PV}" S = "${WORKDIR}/libtirpc-${PV}"
inherit autotools pkgconfig inherit autotools pkgconfig