[PATCH 0/2] RFC: enable the use of the KEYRING credential cache

[andros-HgOvQuBEEgTQT0dZR+]

From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+***@public.gmane.org>

This is a request for comments patch set. It is the nfs-utils portion of a solution for synchronizing the Kerberos credential and gss_context credential destruction.

The keyring is used as it provides a known place for the Kerberos credentials used by NFS, and built-in kernel callbacks that can be used for kerberos credential and gss context synchronization.

The design is for the auth_rpcgss module to register a new key type, the "gss-ctx" key, whose methods can be used for some gss context managment. This key is added to the kerberos keyring and contains any information needed for Kerberos based gss context managememnt.

Currently, the uid is stored in the gss-ctx key and the destroy method is implemented to destroy all Kerberos gss_contexts for the uid.

This design requires new nfslogin/nfslogout programs to be used. The nfslogin and logout programs do two tasks.

1) enforce the keyring nameing convention required for GSSD to use keyrings.
2) create/destroy/update the gss-ctx key used to synhronize the gss context withthe kerberos credential cache.

There are three pieces to this design: GSSD patches, new (yet to be officially written) nfslogin/nfslogout, and KERNEL pactches.

------------------------
GSSD code:
There are two pieces:

1) Enabling GSSD to use the Kerberos KEYRING credential cache - the two patches
2) New nfslogin and nfslogout programs to be used for kerberos NFS mounts.


This code enables GSSD to use the Kerberos 5 supported KEYRING credential cache.
GSSD: Add keyring ccache for machine credential
GSSD: gssd_setup_krb5_user_keyring_ccache

Motivation:
In order for GSSD to use the Krb5 KEYRING credential type, a naming convention
is required. Use the same credential cache naming convention as used for the
FILE or MEMORY credential cache names, without a path name component:

KEYRING:krb5cc_<UID>

------------------------
NEW EXECUTABLES:
Proposed new executable: nfslogin:

This program will kinit -c KEYRING:krb5cc_<UID>, enforcing the keyring naming
convetion required by GSSD, and then create a gss-ctx key which holds the UID
as a payload, and is placed in the krb5cc_<UID> keyring.


What I do for testing is what nfslogin will do:

# kinit -c KEYRING:krb5cc_<UID>
# nfs-add-key UID krb5cc_<UID>


For destruction, I run:

# kdestroy -c KEYRING:krb5cc_<UID>

The kdestroy removes the krb5cc_<UID> keyring which causes the gss-ctx key to bedestroyed, and the destroy method removes the UID's kerberos gss contexts.

Here is the nfs-add-key userlevel code that creates the gss-ctx key:

#include <sys/types.h>
#include <keyutils.h>
#include <stdio.h>
#include <errno.h>

main (int ac, char **av)
{
char *keyring_name;
int uid;
key_serial_t key_serial, ring_serial;

if (ac != 3) {
fprintf(stderr, "Usage: %s <uid> <keyring file name>\n",
av[0]);
return;
}
uid = atoi(av[1]);
keyring_name = av[2];

fprintf(stderr, "uid: [%d], keyring:[%s]\n", uid, keyring_name);

ring_serial = request_key("keyring", keyring_name ,NULL,
KEY_SPEC_SESSION_KEYRING);
if (ring_serial < 0) {
perror("request_key failed\n");
return;
}
key_serial = add_key("gss-ctx","_nfs_gss_", &uid, sizeof(int),
ring_serial);
if (key_serial < 0) {
perror("add_key failed\n");
return;
}

}

Proposed new executable: nfslogout:

This program will call kdestroy -c KEYRING:krb5cc_<UID>, enforcing the naming
convetion required by GSSD. Destroying the krb5cc_<UID> keyring calls the
destroy callback on the gss-ctx key, which in turn uses the UID to search all
rpc_clnt gss_auth Kerberos mechanism credential caches and marks all found
gss credentials as not uptodate. This results in immediate revocation of
NFS access for the UID.

------------------------
KERNEL code:
The kernel portion of this request for comments, patch:
"SUNRPC: new keyring key_type for gss context destruction at kdestroy"
does the following:

- registers a new key_type "gss-ctx" used by nfslogin/nfslogout. nfslogin stores the uid in the key.
- the gss-ctx destroy method uses the stored uid to search all gss_auth kerberos rpc client credential caches for the uid's gss contexts and marks them as not up to date.

-----------------------
TESTING

Here is the rpc.gssd -K -f -vvv output for mount:

[***@fedora-64-2 andros]# mount -o minorversion=1,sec=krb5 vs1-d2.androsad.fake:/nfs4krb5 /mnt

Note GSSD creates and finds the KEYRING machine credential cache:

handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clntb)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clntb)
process_krb5_upcall: service is '<null>'
Full hostname for 'vs1-d2.androsad.fake' is 'vs1-d2.androsad.fake'
Full hostname for 'fedora-64-2.androsad.fake' is 'fedora-64-2.androsad.fake'
No key table entry found for FEDORA-64-2.ANDROSAD.FAKE$@ANDROSAD.FAKE while getting keytab entry for 'FEDORA-64-2.ANDROSAD.FAKE$@ANDROSAD.FAKE'
No key table entry found for root/fedora-64-2.androsad.fake-***@public.gmane.org while getting keytab entry for 'root/fedora-64-2.androsad.fake-***@public.gmane.org'
Success getting keytab entry for 'nfs/fedora-64-2.androsad.fake-***@public.gmane.org'
INFO: Credentials in CC 'KEYRING:krb5ccmachine_ANDROSAD.FAKE' are good until 1354298952
INFO: Credentials in CC 'KEYRING:krb5ccmachine_ANDROSAD.FAKE' are good until 1354298952
using KEYRING:krb5ccmachine_ANDROSAD.FAKE as credentials cache for machine creds
using environment variable to select krb5 ccache KEYRING:krb5ccmachine_ANDROSAD.FAKE
creating context using fsuid 0 (save_uid 0)
creating tcp client for server vs1-d2.androsad.fake
DEBUG: port already set to 2049
creating context with server nfs-LzjmjWLHeYIgtYNe+sVdLcegTc+***@public.gmane.org
DEBUG: serialize_krb5_ctx: lucid version!
prepare_krb5_rfc1964_buffer: serializing keys with enctype 4 and length 8
doing downcall lifetime_rec 1791
Closing 'gssd' pipe for /var/lib/nfs/rpc_pipefs/nfs/clnta
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnta
Closing 'gssd' pipe for /var/lib/nfs/rpc_pipefs/nfs/clnt9
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt9

Here is the uid 0 keyring display from /proc

***@fedora-64-2 andros]# cat /proc/keys
05df6597 I--Q--- 1 7m 3f010000 0 0 id_legacy uid:root-bHIg9gIR+***@public.gmane.org: 2
06acf7c5 I--Q--- 1 7m 3f010000 0 0 id_legacy gid:root-bHIg9gIR+***@public.gmane.org: 2
085fd14d I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
127383c6 I--Q--- 1 9m 3f010000 0 0 id_legacy uid:nfs-bHIg9gIR+***@public.gmane.org: 3
25fa46e8 I--Q--- 1 7m 3f010000 0 0 id_legacy gid:daemon-bHIg9gIR+***@public.gmane.org: 2
27de7437 I--Q--- 1 9m 3f010000 0 0 id_legacy gid:unixgroup-bHIg9gIR+***@public.gmane.org: 3
2ebec4f3 I------ 1 perm 1f030000 0 0 keyring .dns_resolver: empty
2fd3ad40 I--Q--- 1 perm 1f3f0000 1000 65534 keyring _uid.1000: empty
322e313d I------ 1 perm 1f030000 0 0 keyring .id_resolver: 12/12
344c5dd9 I--Q--- 101 perm 1f3f0000 1000 1000 keyring _ses: 1/4
3a28e301 I--Q--- 1 9m 3f010000 0 0 id_legacy uid:andros-bHIg9gIR+***@public.gmane.org: 5
3a761f99 I--Q--- 1 perm 1f3f0000 0 65534 keyring _uid_ses.0: 1/4
[***@fedora-64-2 andros]#


Here I run "nfslogin", a kinit then nfs-add-key:

***@fedora-64-2 ~]$ kinit -c KEYRING:/krb5cc_1000
Password for andros-***@public.gmane.org:
[***@fedora-64-2 ~]$ cat /proc/keys
05c7977b I--Q--- 1 perm 3b3f0000 1000 1000 user __krb5_princ__: 35
08a57967 I--Q--- 1 perm 3b3f0000 1000 1000 keyring krb5cc_1000: empty
11909595 I--Q--- 1 perm 3b3f0000 1000 1000 user krbtgt/ANDROSAD.FAKE-***@public.gmane.org: 465
2fd3ad40 I--Q--- 1 perm 1f3f0000 1000 65534 keyring _uid.1000: empty
344c5dd9 I--Q--- 101 perm 1f3f0000 1000 1000 keyring _ses: 3/4
3e12b317 I--Q--- 1 perm 3b3f0000 1000 1000 keyring /krb5cc_1000: 2/4

[***@fedora-64-2 ~]$ /usr/local/src/nfs-keyring/nfs-add-key 1000 krb5cc_1000
uid: [1000], keyring:[krb5cc_1000]
[***@fedora-64-2 ~]$ cat /proc/keys
05c7977b I--Q--- 1 perm 3b3f0000 1000 1000 user __krb5_princ__: 35
08a57967 I--Q--- 1 perm 3b3f0000 1000 1000 keyring krb5cc_1000: 1/4
11909595 I--Q--- 1 perm 3b3f0000 1000 1000 user krbtgt/ANDROSAD.FAKE-***@public.gmane.org: 465
2fd3ad40 I--Q--- 1 perm 1f3f0000 1000 65534 keyring _uid.1000: empty
3281f83c I--Q--- 1 perm 3b3b0000 1000 1000 gss-ctx _nfs_gss_: 4
344c5dd9 I--Q--- 101 perm 1f3f0000 1000 1000 keyring _ses: 3/4
3e12b317 I--Q--- 1 perm 3b3f0000 1000 1000 keyring /krb5cc_1000: 2/4

Here I ls the mount point:

[***@fedora-64-2 ~]$ ls /mnt
8.c dir2 fialplease.txt new.txt test
9.c expired4.txt fial.txt ooops.c thisisbad
andros fake.txt heh.c real yech.txt
dir fial1.txt isittrue.txt really
[***@fedora-64-2 ~]$ cat /proc/keys
0009a221 I--Q--- 1 perm 3b3f0000 0 0 user __krb5_princ__: 61
0421323a I--Q--- 1 perm 3b3f0000 1000 1000 user __krb5_princ__: 35
0a941505 I--Q--- 1 perm 3b3f0000 0 0 user krbtgt/ANDROSAD.FAKE-***@public.gmane.org: 516
0f570859 I--Q--- 1 perm 3b3f0000 1000 1000 user krbtgt/ANDROSAD.FAKE-***@public.gmane.org: 465
1f7e3860 I--Q--- 1 perm 3b3b0000 1000 1000 gss-ctx _nfs_gss_: 4
30f2d34e I--Q--- 1 perm 3b3f0000 1000 0 user nfs/vs1-d2.androsad.fake-***@public.gmane.org: 426
3474b7a1 I--Q--- 1 perm 3b3f0000 1000 1000 keyring krb5cc_1000: 4/4
383d58cd I--Q--- 1 perm 3b3f0000 0 0 keyring krb5ccmachine_ANDROSAD.FAKE: 3/4
3a17c360 I--Q--- 101 perm 1f3f0000 1000 1000 keyring _ses: 3/4
3f8e77d2 I--Q--- 1 perm 3b3f0000 0 0 user nfs/vs1-d2.androsad.fake-***@public.gmane.org: 476
3fa78549 I--Q--- 1 perm 1f3f0000 1000 65534 keyring _uid.1000: empty


Here I kdestroy which also destroys the gss_context thus the immediate return of permission denied when trying to access the mount point:
Note that the keys are all gone including the gss-ctx key.

[***@fedora-64-2 ~]$ kdestroy -c KEYRING:krb5cc_1000
[***@fedora-64-2 ~]$ ls /mnt
ls: cannot access /mnt: Permission denied

[***@fedora-64-2 ~]$ cat /proc/keys
0009a221 I--Q--- 1 perm 3b3f0000 0 0 user __krb5_princ__: 61
05a48686 I--Q--- 1 perm 3b3f0000 0 0 keyring krb5cc_1000: empty
0a941505 I--Q--- 1 perm 3b3f0000 0 0 user krbtgt/ANDROSAD.FAKE-***@public.gmane.org: 516
383d58cd I--Q--- 1 perm 3b3f0000 0 0 keyring krb5ccmachine_ANDROSAD.FAKE: 3/4
3a17c360 I--Q--- 101 perm 1f3f0000 1000 1000 keyring _ses: 3/4
3f8e77d2 I--Q--- 1 perm 3b3f0000 0 0 user nfs/vs1-d2.androsad.fake-***@public.gmane.org: 476
3fa78549 I--Q--- 1 perm 1f3f0000 1000 65534 keyring _uid.1000: empty
[***@fedora-64-2 ~]$ kk


Andy Adamson (2):
GSSD: Add keyring ccache for machine credential
GSSD: gssd_setup_krb5_user_keyring_ccache

utils/gssd/gssd.c | 10 +++++++-
utils/gssd/gssd.h | 1 +
utils/gssd/gssd_proc.c | 15 ++++++++++++++
utils/gssd/krb5_util.c | 49 ++++++++++++++++++++++++++++++++++++++++++-----
utils/gssd/krb5_util.h | 1 +
5 files changed, 68 insertions(+), 8 deletions(-)

[andros-HgOvQuBEEgTQT0dZR+]

From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+***@public.gmane.org>

Signed-off-by: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+***@public.gmane.org>
---
utils/gssd/gssd.c | 10 ++++++++--
utils/gssd/gssd.h | 1 +
utils/gssd/krb5_util.c | 18 ++++++++++++------
3 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/utils/gssd/gssd.c b/utils/gssd/gssd.c
index a3292c9..1250e34 100644
--- a/utils/gssd/gssd.c
+++ b/utils/gssd/gssd.c
@@ -60,6 +60,7 @@ char keytabfile[PATH_MAX] = GSSD_DEFAULT_KEYTAB_FILE;
char ccachedir[PATH_MAX] = GSSD_DEFAULT_CRED_DIR ":" GSSD_USER_CRED_DIR;
char *ccachesearch[GSSD_MAX_CCACHE_SEARCH + 1];
int use_memcache = 0;
+int use_keyring = 0;
int root_uses_machine_creds = 1;
unsigned int context_timeout = 0;
char *preferred_realm = NULL;
@@ -85,7 +86,7 @@ sig_hup(int signal)
static void
usage(char *progname)
{
- fprintf(stderr, "usage: %s [-f] [-l] [-M] [-n] [-v] [-r] [-p pipefsdir] [-k keytab] [-d ccachedir] [-t timeout] [-R preferred realm]\n",
+ fprintf(stderr, "usage: %s [-f] [-l] [-K | -M] [-n] [-v] [-r] [-p pipefsdir] [-k keytab] [-d ccachedir] [-t timeout] [-R preferred realm]\n",
progname);
exit(1);
}
@@ -102,16 +103,21 @@ main(int argc, char *argv[])
char *progname;

memset(ccachesearch, 0, sizeof(ccachesearch));
- while ((opt = getopt(argc, argv, "fvrlmnMp:k:d:t:R")) != -1) {
+ while ((opt = getopt(argc, argv, "fvrlKmnMp:k:d:t:R")) != -1) {
switch (opt) {
case 'f':
fg = 1;
break;
+ case 'K':
+ use_keyring = 1;
+ use_memcache = 0;
+ break;
case 'm':
/* Accept but ignore this. Now the default. */
break;
case 'M':
use_memcache = 1;
+ use_keyring = 0;
break;
case 'n':
root_uses_machine_creds = 0;
diff --git a/utils/gssd/gssd.h b/utils/gssd/gssd.h
index 86472a1..168f99c 100644
--- a/utils/gssd/gssd.h
+++ b/utils/gssd/gssd.h
@@ -64,6 +64,7 @@ extern char pipefs_dir[PATH_MAX];
extern char keytabfile[PATH_MAX];
extern char *ccachesearch[];
extern int use_memcache;
+extern int use_keyring;
extern int root_uses_machine_creds;
extern unsigned int context_timeout;
extern char *preferred_realm;
diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
index aeb8f70..8d42e8f 100644
--- a/utils/gssd/krb5_util.c
+++ b/utils/gssd/krb5_util.c
@@ -407,15 +407,21 @@ gssd_get_single_krb5_cred(krb5_context context,
/*
* Initialize cache file which we're going to be using
*/
-
- if (use_memcache)
- cache_type = "MEMORY";
- else
- cache_type = "FILE";
- snprintf(cc_name, sizeof(cc_name), "%s:%s/%s%s_%s",
+ if (use_keyring) {
+ snprintf(cc_name, sizeof(cc_name), "%s:%s%s_%s",
+ "KEYRING",
+ GSSD_DEFAULT_CRED_PREFIX,
+ GSSD_DEFAULT_MACHINE_CRED_SUFFIX, ple->realm);
+ } else {
+ if (use_memcache)
+ cache_type = "MEMORY";
+ else
+ cache_type = "FILE";
+ snprintf(cc_name, sizeof(cc_name), "%s:%s/%s%s_%s",
cache_type,
ccachesearch[0], GSSD_DEFAULT_CRED_PREFIX,
GSSD_DEFAULT_MACHINE_CRED_SUFFIX, ple->realm);
+ }
ple->endtime = my_creds.times.endtime;
if (ple->ccname != NULL)
free(ple->ccname);

[andros-HgOvQuBEEgTQT0dZR+]

From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+***@public.gmane.org>

Signed-off-by: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+***@public.gmane.org>
---
utils/gssd/gssd_proc.c | 15 +++++++++++++++
utils/gssd/krb5_util.c | 31 +++++++++++++++++++++++++++++++
utils/gssd/krb5_util.h | 1 +
3 files changed, 47 insertions(+), 0 deletions(-)

diff --git a/utils/gssd/gssd_proc.c b/utils/gssd/gssd_proc.c
index 97e8f99..e24dbcb 100644
--- a/utils/gssd/gssd_proc.c
+++ b/utils/gssd/gssd_proc.c
@@ -984,6 +984,20 @@ process_krb5_upcall(struct clnt_info *clp, uid_t uid, int fd, char *tgtname,
service ? service : "<null>");
if (uid != 0 || (uid == 0 && root_uses_machine_creds == 0 &&
service == NULL)) {
+ if (use_keyring) {
+ err = gssd_setup_krb5_user_keyring_ccache(uid,
+ clp->servername);
+ if (err == -EKEYEXPIRED)
+ downcall_err = -EKEYEXPIRED;
+ else if (!err)
+ create_resp = create_auth_rpc_client(clp,
+ &rpc_clnt, &auth, uid,
+ AUTHTYPE_KRB5);
+ if (create_resp == 0)
+ goto resp_found;
+
+ }
+
/* Tell krb5 gss which credentials cache to use */
for (dirname = ccachesearch; *dirname != NULL; dirname++) {
err = gssd_setup_krb5_user_gss_ccache(uid, clp->servername, *dirname);
@@ -1055,6 +1069,7 @@ process_krb5_upcall(struct clnt_info *clp, uid_t uid, int fd, char *tgtname,
goto out_return_error;
}

+resp_found:
if (!authgss_get_private_data(auth, &pd)) {
printerr(1, "WARNING: Failed to obtain authentication "
"data for user with uid %d for server %s\n",
diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
index 8d42e8f..ae701a5 100644
--- a/utils/gssd/krb5_util.c
+++ b/utils/gssd/krb5_util.c
@@ -1039,6 +1039,37 @@ err_cache:
/*==========================*/

/*
+ * Attempt to find a KEYRING cache of the form KEYRING:krb5cc_<UID>.
+ *
+ * Returns 0 if a ccache was found, and a non-zero error code otherwise.
+ */
+int
+gssd_setup_krb5_user_keyring_ccache(uid_t uid, char *servername)
+{
+ char buf[MAX_NETOBJ_SZ];
+ char *princ = NULL, *realm = NULL;
+ int err = -EKEYEXPIRED;
+
+ snprintf(buf, sizeof(buf), "%s:%s_%u", "KEYRING",
+ GSSD_DEFAULT_CRED_PREFIX, uid);
+
+ if (!query_krb5_ccache(buf, &princ, &realm)) {
+ printerr(3, "CC '%s' is expired or corrupt\n", buf);
+ goto out;
+ }
+ err = 0;
+ printerr(2, "Using CC '%s' as credentials cache for %s@%s with "
+ "uid %u for server %s\n", buf, princ, realm, uid, servername);
+
+ free(princ);
+ free(realm);
+
+ gssd_set_krb5_ccache_name(buf);
+out:
+ return err;
+}
+
+/*
* Attempt to find the best match for a credentials cache file
* given only a UID. We really need more information, but we
* do the best we can.
diff --git a/utils/gssd/krb5_util.h b/utils/gssd/krb5_util.h
index 9f41625..472b65e 100644
--- a/utils/gssd/krb5_util.h
+++ b/utils/gssd/krb5_util.h
@@ -25,6 +25,7 @@ struct gssd_k5_kt_princ {

int gssd_setup_krb5_user_gss_ccache(uid_t uid, char *servername,
char *dirname);
+int gssd_setup_krb5_user_keyring_ccache(uid_t uid, char *servername);
int gssd_get_krb5_machine_cred_list(char ***list);
void gssd_free_krb5_machine_cred_list(char **list);
void gssd_setup_krb5_machine_gss_ccache(char *servername);