Compatibility with HLL Compilers (C++, C#) and Operating Systems

The integration of assembler code with applications written in high-level languages brings benefits in particular scenarios, such as implementing complex mathematical algorithms and real-time tasks that require efficient, compact code. No one uses an assembler to implement a graphical user interface (GUI) anymore, as there is no reason to do so. Modern desktop operating systems are designed to provide a rich user experience, supporting languages such as C#, C++, and Python for implementing user interfaces (UIs) through libraries. While those UI generation functions can be executed from the assembler level, there is virtually no reason to do it. A more effective approach is to have the main application is written in a high-level language and execute assembly code as needed to perform backend operations efficiently.

In the case of multi-tier web applications, assembly code is usually hidden in the backend, oriented towards efficient computation, and it is wrapped in a high-level API library. e.g. ASP.NET Core Web API or many other REST libraries.

It is possible to merge assembler code with high-level languages either as:

• static, where assembler code is compiled as a library object file and merged with the code during linking (figure 1), or
• dynamic, where the assembler code library is loaded during runtime (figure ##REF:dynamiclinking##).

Dynamic loading of code is considered an advantage because the original application does not contain the assembler binary executable; it is kept in a separate file and loaded on demand, so it can be compiled and exchanged independently. On the other hand, it raises a number of challenges, such as versioning, compatibility, and the time required to load the library from the file system before the first call to the contents.

Windows OS has historically supported unmanaged code written primarily in C++. This kind of code runs directly on the CPU, but divergence in hardware platforms, such as the introduction of ARM-core-based platforms running Windows, causes incompatibility issues. Since the introduction of the .NET framework, Windows has provided developers with a safer way to execute their code, called “managed code”. The difference is that managed code, typically written in C#, is executed by a .NET framework interpreter rather than being compiled into machine code, as unmanaged code is. The use of managed code brings multiple advantages for developers, including automated memory management and code isolation from the operating system. This, however, raises several challenges when integrating managed code and assembly code. In any case, the integration model is common: the assembler implements functions (usually stateless) that are later called from the high-level language and return data to it (figure 3).

There are significant differences between x86 (32-bit) and x64 (64-bit) code, mostly in the scope of integration methods. As we're at a very low level of programming, there are no shorts, and all program flow from higher-level code to assembler code and the opposite must follow strict calling conventions. Details are presented in chapter Procedures, Functions and Calls in Windows and Linux.

Using dynamic memory management at the level of the assembler code is troublesome: allocating and releasing memory require calls to the hosting operating system. It is possible, but complex. Moreover, there is no dynamic, automated memory management, as in .NET, Java, and Python, so the developer is on their own, similar to programming in C++. For this reason, it is common to allocate adequate memory resources on the high-level code, e.g., the GUI front-end and pass them to the assembler code as pointers. Note, however, that for some higher-level languages, such as C#, it is necessary to follow a strict pattern to ensure correct and persistent memory allocation, as described in the following sections.

It is possible to write an application for Windows solely in assembler. While the reason to do it is doubtful, some hints presented below, such as calling system functions, may be helpful. Calls to the Windows system functions is possible via classical call, and require explicit declaration of the functions as external, and linking kernel32.lib and user32.lib. Use of legacy_stdio_definitions.lib and legacy_stdio_wide_specifiers.lib may be helpful when using advanced stdio functions and enumerations.
A common approach to development is to start with a stub command-line C++ application and manually convert it to assembler requirements. Visual Studio Community (https://visualstudio.microsoft.com/vs/community/) is a free version and the first choice for developing apps written in pure assembler, for Windows OSes. It requires several configuration steps to compile an executable from assembly code. Still, once done, it has a great feature: it enables debugging of assembly code, even in high-level integration scenarios, where the application is written in mixed languages.

A template of the typical pure assembler, command-line application for Windows is as follows:

...
.code 
hello_world_asm PROC
push rbp ; save frame pointer
mov rbp, rsp ; fix stack pointer
sub rsp, 8 * (4 + 2)
 
.... ; here comes your code
 
 
mov rsp, rbp
pop rbp
ret
hello_world_asm ENDP
END

The name hello_world_asm must be specified to the compiler as the so-called entry point.

Calling system functions, such as the system message box, requires understanding the arguments passed to them. As there is no direct assembler help, documentation of the Windows system API for C++ is helpful. Code below presents the necessary components of the assembler app to call system functions (library includes are configured on the project level):

.data
STD_INPUT_HANDLE = -10
STD_OUTPUT_HANDLE = -11
STD_ERROR_HANDLE = -12
 
handler dq 0
hello_msg db "Hello world", 0
info_msg  db "Info", 0
...
includelib          kernel32.lib 
includelib          user32.lib 
EXTERN MessageBoxA: PROC
...
 
 WINUSERAPI int WINAPI MessageBoxA(
;  RCX =>  _In_opt_ HWND hWnd,
;  RDX =>  _In_opt_ LPCSTR lpText,
;  R8  =>  _In_opt_ LPCSTR lpCaption,
;  R9  =>  _In_ UINT uType);
mov rcx, handler
mov rdx, offset hello_msg
mov r8,  offset info_msg
mov r9,  0 ; 0 is MB_OK
and rsp, not 8 
call MessageBoxA
...

The majority of standard library functions accept ASCII strings and must be terminated with a 0 byte (0 is a value), so they do not require passing the string length. The and rsp, not 8 instruction causes stack alignment to be required before application leave or before following system function calls.

A common scenario is to wrap assembler code with stateless functions and encapsulate it in one or more DLL files. All arguments are passed from the calling code, usually written in C++

Programming for applications written in unmanaged code

In the case of the unmanaged code, integration is straightforward. Assembler code is usually encapsulated in the DLL library (or multiple libraries). Below is a sample dummy assembler function that returns an integer (no parameters), along with relevant C++ code that dynamically loads a DLL, including the full library-loading lifecycle.

Assembler code (source for DLL):

AssemblerDll.asm

.code
 
MyAsmProc proc
mov RAX, 2026
ret
MyAsmProc endp
end

Relevant definition file:

AssemblerDll.def

LIBRARY AssemblerDll
EXPORTS MyAsmProc

C++ application, dynamically loading a DLL and dynamically obtaining the assembler function's handler (address):

WindowsCmdX64.cpp

#include <Windows.h>
#include <iostream>
 
typedef int(_stdcall* MyProc)();
HINSTANCE dllHandle = NULL;
int main()
{
	dllHandle = LoadLibrary(TEXT("AssemblerDll.dll"));
	if (!dllHandle)
	{
		std::cerr << "Failed to load DLL library\n";
		return 1;
	}
	MyProc myAsmProcedure = (MyProc)GetProcAddress(dllHandle, "MyAsmProc");
	if (!myAsmProcedure)
	{
		std::cerr << "Failed to find assembler procedure\n";
		FreeLibrary(dllHandle);
		return 2;
	}
	std::cout << myAsmProcedure();
	FreeLibrary(dllHandle);
	return 0;
}

Programming for applications written in managed code

In the case of managed code, things get more complex. The .NET framework features automated memory management that releases unused memory (e.g., objects for which there are no more references) and optimises variable locations to improve performance. It is known as a .NET Garbage Collector (GC). GC instantly traces references and, in the event of an object relocation in memory, updates all references accordingly. It also releases memory (objects) that are no longer referenced. This automated mechanism, however, applies only across managed code apps. The problem arises when developers integrate a front-end application written in managed code with assembler libraries written in unmanaged code. All pointers and references passed to the assembler code are not automatically traced by the GC. Using dynamically allocated variables on the .NET side and accessing them from the assembler code is a very common scenario. GC cannot “see” any reference to the object (variable, memory) made in unmanaged code; thus, it may release or relocate memory without updating the reference address on the assembler side. It causes very hard-to-debug errors that occur randomly and are very serious (e.g. null pointer exception). Possible reference cases are presented visually in the:

• figure 4 - proper reference, initially OK untill GC runs,
• figure 5 - incorrect reference - GC relocated object in the memory,
• figure 6 - incorrect reference - GC removed the object from the memory.

Luckily, there is a strict set of rules to follow when integrating managed and unmanaged code, to avoid situations presented in 5 and 6 regarding arguments passed to the assembler code:

1. It is safe to call a function that does not have any arguments and returns no value or returns a simple type (by value, stored in a register).
2. It is safe to pass simple type values as arguments (e.g. int), because they are passed to the calling function “by value”, not “by reference”.

   using System;
   using System.Runtime.InteropServices;
    
   namespace ConsoleAsmTestTypes
   {
       class Program
       {
           [DllImport("DLLAsm.dll")]
           private static extern int ProcAsm1(int a, int b);
           static void Main(string[] args)
           {
               int a = 10;
               int b = 20;
               unsafe
               {
                   int c = ProcAsm1(a, b);
               }
           }
       }
   }

3. To ensure seamless use of complex types referenced by address (pointer) between .NET and assembler code, all variables must be declared using fixed{} before calling the assembler function.
4. Code should also be marked as unsafe in any case, regardless of the type of the arguments.

    [DllImport("DLLAsm.dll")]
   private static unsafe extern int ProcAsm2(int* a, int pos);
           ...
   int[] n1Array = { 1, 2, 3, 4, 5, 6 };
           ...
   unsafe 
    { 
      fixed(int* aAddress = &n1Array[0])
      {
         c = ProcAsm2(aAddress, 4); 
      }
    }

Principles for composing assembler code and high-level language into a single application on Linux OSes are similar to those on Windows; dynamic loading is more complex. Thus, we consider only static linking of the code. The most common use of C++ is as a high-level application. Still other options are possible, such as Python.

Linux provides more parameters passed via registers in its x64 standard calls (up to 6) than Windows (only up to 4). Refer to the chapter Procedures, Functions and Calls in Windows and Linux for details.

A common scenario is to use the g++ compiler to compile high-level applications and nasm to compile assembler code. It is also common to help compose a heterogeneous project using makefiles, as presented below.

The sample project is composed of the main.cpp file (main file with high-level, C++ application), asmfunc.asm containing the assembler source code and the aforementioned Makefile.

The Makefile contains definitions of the compilation and linking of the main application and also a definition of the cleanup commands (“clean” section):

Makefile

all: main
 
main: main.o asmfunc.o
	g++ -o main main.o asmfunc.o
 
main.o: main.cpp
	g++ -c -g -F dwarf main.cpp
 
asmfunc.o: asmfunc.asm
	nasm -g -f elf64 -F dwarf asmfunc.asm -l asmfunc.lst
 
clean:
	rm -f ./main || true 
	rm -f ./main.o || true
	rm -f ./asmfunc.o || true
	rm -f ./asmfunc.lst || true

Assembler code exposes functions to the linker using the global directive. Without it, assembler functions remain “private” and cannot be called, so linking won't succeed if there is a reference to the function from the high-level language part of the code. The following code presents a dummy function that performs integer addition of two arguments. Directives “section” are optional in this example.

asmfunc.asm

section .data
section .bss
section .text
 
global addInAsm
 
addInAsm:
	nop
	mov rax, rsi
	add rax, rdi
	ret

Finally, the calling side (C++ application) uses the extern directive to inform the linker about the external function, written in assembler.

main.cpp

#include <iostream>
 
extern "C" {long long int addInAsm(long long, long long);}
 
long long a=10;
long long b=7;
long long returnValue;
 
int main() {
    std::cout << "Hello, Assembler!" << std::endl;
    returnValue = addInAsm(a,b);
    std::cout << "Sum of " << a << " and " << b << " is " << returnValue << std::endl;
    return 0;
}