Reviewing what a syscall is

You probably noticed that the name of this site is the opcode for syscall so it seems fitting for the first post to reference them. Before we can get to the exploitation of syscalls we firstly must refresh ourselves on what exactly they are. Within the Win32 API a syscall is the instruction that moves a set of parameters provided by the user into the kernel context which performs the requested action and returns the result back to the program.

To explore this further lets take a standard Win32 API call which may be used by a malicious application to allocate memory within a process to run shellcode. VirtualAlloc is a Win32 api call used to allocate memory. VirtualAlloc calls the Native API function NtAllocateVirtualMemory which then results in the parameters being passed and the appropriate Service Syscall Number (SSN) being called. A couple of things to note here:

  • The specific SSN number can change with both minor and major updates to Windows.
  • Neither the NativeAPI functions or the syscall numbers are officially documented by Windows
  • Not all NativeAPI functions result in a SSN being called.

To illustrate this process visually below is a diagram of the Win32 API function VirtualAlloc being called which results in a call to NtAllocateVirtualMemory and subsequently the SSN number 18 being moved into eax and the instruction syscall being executed. KiSystemCall64 is invoked which functions within the kernel to facilitate the saving of the userland perimeters, finding the relevant function in the SSDT and executing the function within the kernel with the correct parameters. Afterwards the KiSystemCall64 or another exit mechanism will transfer control back to userland (ring 3).

Looks cool but like why though - part 1

I'm going to answer the why in two parts. So firstly lets look at why the technique developed. Historically a potent detection technique used by AV vendors was hooking. In its most simple implementation a hook is the placing of a simple jmp instruction at the function prologue which points to some sort of analysis / inspection code used to assess the parameters of the potentially suspicious function call. So if the malware author called either VirtualAlloc or NtAllocateVirtualMemory the AV solution would be privy to the parameters of the function call. Using the flow chart from earlier I have added in the control flow shift to the AV/EDR:

Now even though I said above that it's not used anymore I still tried to confirm. I wrote two seperate simple calc shellcode injections [VirtualAlloc -> memcpy -> VirtualProtect -> CreateThread] and checked for hooks however as expected I came up empty handed. Still here is an example I found online of ZwCreateProcess being hooked compared to not hooked (credit for the image goes to crypt0ace):

As we can see in the first image the start of the function begins with an immediate jmp instruction. In comparison the unhooked image shows the standard syscall pattern of moving the SSN number into eax and calling 0f05.

Looks cool by like why though - part 2

Now in 2025 what I explained above isn't entirely relevant to todays operational TTPs. Most if not all AV & EDR vendors have primarily moved away from userland techniques and now rely on kernel techniques. These new techniques are somewhat similar to how AMSI works against powershell obfuscation. As in you can use all the obfuscation techniques you want, however at some point the code will have to be decoded and executed and that is what AMSI will see and assess. Likewise malware authors can use all the userland techniques they want however at some point the code enters the kernel and through a combination of kernel-mode callbacks, ETW, minifilter drivers and pattern analysis the actions the malware is performing cannot be hidden.

Sounds not ideal however the problem with seeing everything is that in some ways your seeing nothing. The volume of telemetry is huge and now you are relying on machine learning, pattern analysis and heuristics to find a signal of malicious activity. Even in the 2025 landscape indirect syscalls can still be useful to break up patterns and inhibit the behavioural analysis of EDRs.

Direct vs Indirect syscalls

Now this is simple yet critically important distinction as one type is still useful and the other is essentially an instant detection from any remotely competent AV / EDR solution. This all comes down to the specific address space that the syscall instruction is called from. Operating normally within Windows OS it is expected that a sycall should only ever be called from within the address space of the legitimate system module NTDLL.dll. However when making a direct syscall the instruction is being called from within the address space of the executable making the syscall. A simple elegant solution to this issue is a simple jmp instruction. By calling a jmp instruction to a syscall instruction within the address space of NTDLL.dll the call now looks legitimate. Now legitimate in this context is a bit of a spectrum as if the call stack is analysed then it will appear very suspicious, however the call stack will not always be examined.

Closing thoughts

Within this post I have emphasised how userland hooking techniques have become irrelevant however since the crowdstrike incident Windows has expressed a desire to move security away from the kernal. Potentially this may result in some older userland evasion techniques becoming relevant once again. Another interesting thought regarding this is even if AVs and EDRs are moved out of the kernel there is nothing stopping malicious actors from continuing to use current kernal techniques. This inherent asymmetry in capability will pose some interesting challenges for the security industry moving forward and I look forward to learning about it once more information is available.

Having gone over the basics in part 2 I will go through the libraries, resources and specific techniques and attempt to compare how they work and detections on virustotal between each technique.

Work in progress - [1] Would like to add in some diagrams for indirect vs direct syscalls - [2] Need to check my spelling and phrasing