Chapters
Chapters

Chapters

Hide chapters

Advanced Apple Debugging & Reverse Engineering

Fourth Edition · iOS 16, macOS 13.3 · Swift 5.8, Python 3 · Xcode 14

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning LLDB Commands

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Understanding Assembly

Section 2: 3 chapters
Show chapters Hide chapters

Section III: Low Level

Section 3: 6 chapters
Show chapters Hide chapters

Section IV: Custom LLDB Commands

Section 4: 8 chapters
Show chapters Hide chapters

Section V: DTrace

Section 5: 3 chapters
Show chapters Hide chapters

Appendices

Section 6: 3 chapters
Show chapters Hide chapters

15. Shared Libraries
Written by Walter Tyree

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Shared libraries are essential for any program to run. This chapter focuses on the compilation and linking process, highlighting how to write code that uses public and private APIs.

A shared library is a bundle of code loaded into a program at runtime instead of being included at compile time. Shared libraries can’t run by themselves — they need to be loaded in by an executable. Examples of shared libraries in iOS include UIKit and the Foundation frameworks. These are first-party shared libraries provided by Apple that you can link against. You can also create your own frameworks and package them inside your app bundle.

Creating your own shared libraries is an attractive development option. They provide encapsulation of code that can be shared between different projects. It can also lead to a more rigorous testing strategy, where you can test the shared library in isolation and be sure it works as intended.

Shared Libraries 101

Several types of shared libraries can be loaded in at runtime: dynamic libraries, frameworks and plugins.

A dynamic library, or dylib, is a shared executable that only contains executable code.

On the other hand, a framework is more of a directory that can contain executable code — as well as other content, like images and JSON files — almost anything! In fact, a framework doesn’t even need to contain any code for it to be classified as a framework. A framework ends in a *.framework extension and contains the directory for the encapsulated resources.

Finally, there are plugins. These are a bit like frameworks, except they should be self-contained as much as possible. That is, if someone gave you a framework, you’d expect to call those APIs implemented in the framework. A plugin should do as much as possible on its own without having another entity have to call on its APIs.

This chapter’s focus is primarily on dynamic libraries because it’s the simplest option to showcase a shared library and an executable calling code from it.

To appreciate how dynamic libraries, linking and loading work, you’ll build a dynamic library and an executable that references it. You’ll compile all of this using clang without Xcode to appreciate what’s happening. For this example, you’ll create a C executable that calls code from a Swift dynamic library. You’re using Swift with C on purpose — instead of Swift with Swift — as it emphasizes the concept of resolving symbol names. You’ll learn how to import Swift code from a Swift dylib later when you learn about TBDs and module maps.

Building a Swift Dynamic Library

In the following section, you’re encouraged to write the code yourself, but the source is available in the starter directory for copy/pasters.

@_cdecl("swift_function")
public func swift_function() {
   print("hello from \(#function)")
}

public func mangled_function() {
   print("!!!!!! \(#function)")
}

$ swiftc SwiftSharedLibrary.swift -emit-library -o libSwiftSharedLibrary.dylib

$ nm -U libSwiftSharedLibrary.dylib | grep function
0000000000003954 T _$s18SwiftSharedLibrary14swift_functionyyF
0000000000003c3c T _$s18SwiftSharedLibrary16mangled_functionyyF
0000000000003940 T _swift_function

$ nm -U libSwiftSharedLibrary.dylib | grep function | swift demangle
0000000000003a48 T SwiftSharedLibrary.swift_function() -> ()
0000000000003cf4 T SwiftSharedLibrary.mangled_function() -> ()
0000000000003a34 T _swift_function

Building a C Executable

You’ll now create the executable to reference the unmangled swift_function implemented in libSwiftSharedLibrary.dylib. Add the following code to /tmp/exe.c:

extern void swift_function(void);

int main() {
  swift_function();
  return 0;
}

$ clang exe.c -o exe libSwiftSharedLibrary.dylib

$ clang exe.c -o exe -lSwiftSharedLibrary -L./

$ ./exe

hello from swift_function()

Symbols and Dependencies

When linking dynamic frameworks, it’s insightful to be able to inspect dependencies from a particular module. Use otool -L on the exe executable to display the shared libraries exe needs in order to run:

$ otool -L exe

exe:
   libSwiftSharedLibrary.dylib (compatibility version 0.0.0, current version 0.0.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

$ otool -L libSwiftSharedLibrary.dylib
libSwiftSharedLibrary.dylib:
  libSwiftSharedLibrary.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
  /usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 5.8.0)


$ nm -mu ./exe
  (undefined) external _swift_function (from libSwiftSharedLibrary)

$ sed  's/swift_function/bad_function/g'  exe.c > bad_exe.c
$ clang exe.c /tmp/libSwiftSharedLibrary.dylib
UUndefined symbols for architecture arm64:
  "_bad_function", referenced from:
      _main in bad_exe-d1e251.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Linking Tricks

The @_cdecl() attribute is a common trick to read Swift symbols in non-Swift code because it’s typically not possible to reference mangled Swift symbols whose names begin with a dollar sign, like $s18SwiftSharedLibrary16mangled_functionyyF.

Compile Time Linking

One way to call the mangled Swift function is with an option from ld to make an alias for the desired function.

$ clang exe.c libSwiftSharedLibrary.dylib -Xlinker -alias -Xlinker '_$s18SwiftSharedLibrary16mangled_functionyyF' -Xlinker _swift_function  -o ./exe

$ ./exe
!!!!!!! hello from mangled_function()

Runtime Linking

Another alternative is to use runtime linking via the dlsym API. You’ll explore this function and dlopen in depth in the next chapter. But for now, you’ll just take a quick look.

#include <dlfcn.h>
#include <stdlib.h>

int main() {
  void (*mangled_function)(void) = NULL;
  mangled_function = dlsym(
    RTLD_NEXT, "$s18SwiftSharedLibrary16mangled_functionyyF");

  if (mangled_function) {
    mangled_function();
  }
  return 0;
}

clang exe2.c libSwiftSharedLibrary.dylib -o exe2

$ ./exe2
!!!!!!! hello from mangled_function()

Linking Symbols? Meh!!! Symbols

If you know that a dynamic library would never change (i.e., would never be updated, never recompiled), you can hardcode offsets to that library based on the module’s load address.

$ nm ./libSwiftSharedLibrary.dylib | grep mangled
0000000000003cf4 T _$s18SwiftSharedLibrary16mangled_functionyyF

#include <mach-o/dyld.h> // _dyld.* APIs
#include <string.h>      // strcmp
#include <libgen.h>      // basename

int main() {
  uintptr_t base = 0;
  // iterate over loaded images
  for (int i = 0; i < _dyld_image_count(); i++) {
    if (strcmp(basename((char*)_dyld_get_image_name(i)), "libSwiftSharedLibrary.dylib") == 0) {
      // we found the load address for libSwiftSharedLibrary.dylib
      base = (uintptr_t)_dyld_get_image_header(i);
      break;
    }
  }

  // execute mangled_function
  if (base) {
    void (*mangled_function)(void) = (void*)(base + 0x00000000003cf4);
    mangled_function();
  }

  return 0;
}

clang dontdothis.c libSwiftSharedLibrary.dylib -o ./dontdothis && ./dontdothis
!!!!!!! hello from mangled_function()

Defensive Linking

One final trick is to use a weak attribute for a symbol. A weak attribute won’t crash the program if the symbol can’t be resolved.

#include <stdio.h>

__attribute__((weak))
extern void swift_function(void);

__attribute__((weak))
extern void bad_function(void);

int main() {
  swift_function ? swift_function() : printf("swift_function not found!\n");
  bad_function ? bad_function() : printf("bad_function not found!\n");
  return 0;
}

$ clang exe3.c -o exe3 libSwiftSharedLibrary.dylib -undefined dynamic_lookup && ./exe3
hello from swift_function()
bad_function not found!

$ clang exe3.c -o exe3 -undefined dynamic_lookup && ./exe3
swift_function not found!
bad_function not found!

Static Libraries

Discussing dynamic libraries wouldn’t be complete without mentioning their counterpart — static libraries!

$ mv libSwiftSharedLibrary.dylib libMovedSharedLibrary.dylib
$ ./exe
dyld: Library not loaded: libSwiftSharedLibrary.dylib
  Referenced from: /private/tmp/exe
  Reason: image not found
[1]    6074 abort      exe

$ mv libMovedSharedLibrary.dylib libSwiftSharedLibrary.dylib

$ swiftc SwiftSharedLibrary.swift -static -emit-library -o SwiftSharedLibrary.a

$ clang SwiftSharedLibrary.a exe.c -o exe

ld: warning: Could not find or use auto-linked library 'swiftSwiftOnoneSupport'
ld: warning: Could not find or use auto-linked library 'swiftCore'
Undefined symbols for architecture arm64:
  "Swift.String.init(stringInterpolation: Swift.DefaultStringInterpolation) -> Swift.String", referenced from:
      SwiftSharedLibrary.swift_function() -> () in SwiftSharedLibrary.a(SwiftSharedLibrary-c49738.o)
      SwiftSharedLibrary.mangled_function() -> () in SwiftSharedLibrary.a(SwiftSharedLibrary-c49738.o)
... snip ...

$ clang SwiftSharedLibrary.a exe.c -L/usr/lib/swift -lswiftCore -o exe

$ ./exe

hello from swift_function()

$ otool -L exe

exe:
   /usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 1205.0.24)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

$ nm -mU exe | grep function
0000000100003908 (__TEXT,__text) external _$s14SwiftSharedLibrary14swift_functionyyF
0000000100003bf0 (__TEXT,__text) external _$s14SwiftSharedLibrary16mangled_functionyyF
00000001000038f4 (__TEXT,__text) external _swift_function

Text-Based Dynamic Library Files

In the example above, you created a main executable and a dynamic library. When compiling, the linker had to look into libSwiftSharedLibrary.dylib, parse its symbol table, and ensure that the appropriate symbol was there for the linking to succeed.

$ find /Applications/Xcode.app -name "*.tbd"

$ cat /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/PrivateFrameworks/CoreSymbolication.framework/CoreSymbolication.tbd

TBD Format and TAPI

The text-based dynamic library stubs need to have an agreed-upon format for ld to know how to utilize the TBD file. An open-source implementation called Text-based Application Programming Interface, or TAPI, can generate TBD files from headers or compiled code. tapi has source code found in the LLVM repo and is also part of opensource.apple.com. You can also check out additional documentation of the TBD parameters.

$ clang -shared -o /tmp/PrivateFramework.dylib Private.m -fmodules -arch arm64e -arch arm64 -arch x86_64

$ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/tapi stubify /tmp/PrivateFramework.dylib -o /tmp/libPrivateFramework.tbd

--- !tapi-tbd
tbd-version:     4
targets:         [ x86_64-macos, arm64-macos, arm64e-macos ]
flags:           [ not_app_extension_safe ]
install-name:    '/tmp/PrivateFramework.dylib'
current-version: 0
compatibility-version: 0
exports:
  - targets:         [ x86_64-macos, arm64-macos, arm64e-macos ]
    symbols:         [ _SomeCode, _SomeStringConstant ]
    objc-classes:    [ PrivateObjcClass ]
...

$ clang tbdpoc.m -I. -L/tmp/ -lPrivateFramework -fmodules -o /tmp/tbdpoc
$ /tmp/tbdpoc
2023-04-06 14:42:56.710 tbdpoc[9757:587074] much wow, stuff of doing!
2023-04-06 14:42:56.710 tbdpoc[9757:587074] SomeStringConstant is: com.kodeco.tbd.example

$ clang tbdpoc.m -I. libPrivateFramework.tbd -fmodules -o /tmp/tbdpoc
ld: warning: text-based stub file libPrivateFramework.tbd and library file libPrivateFramework.tbd are out of sync. Falling back to library file for linking.

Modules and Module Maps

You’ve played with Swift code mangling names and importing them into C, and you’ve created a TBD file importing Objective-C/C code into an Objective-C/C executable. Now, it’s time to take an Objective-C dynamic library and import it into Swift. This is the final piece of the linking puzzle, as Swift requires one additional component to properly import symbols from an external library.

$ cp Private.h /tmp/

module YayModule {
  header "Private.h"
  export *
}

import YayModule

print("calling external: \(SomeStringConstant)")
SomeCode();
let c = PrivateObjcClass()
c.doStuff()

$ swiftc mmpoc.swift -I. -L/tmp -lPrivateFramework -o /tmp/mmpoc && /tmp/mmpoc
calling external: com.kodeco.tbd.example
2023-04-06 15:22:45.109 mmpoc[10164:605856] SomeStringConstant is: com.kodeco.tbd.example
2023-04-06 15:22:45.110 mmpoc[10164:605856] much wow, stuff of doing!

Xcode Equivalent

You jumped down to the command line to do all this work. It’s worth going back up to Xcode to see how to achieve the same thing.

dyld Shared Cache

If you were to inspect any executable’s linked frameworks, you’d notice that libSystem.B.dylib is included in pretty much everything.

$ touch /tmp/anexample.swift && swiftc /tmp/anexample.swift -o /tmp/anexample && otool -L /tmp/anexample
/tmp/anexample:
   /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

$ file /usr/lib/libSystem.B.dylib
/usr/lib/libSystem.B.dylib: cannot open '/usr/lib/libSystem.B.dylib' (No such file or directory)

module YayModule {
  header "Private.h"
  header "dyld_priv.h"
  export *
}

import YayModule  

let cache_uuid =
  UnsafeMutablePointer<UInt8>.allocate(capacity: 16)
let manager = FileManager.default

if _dyld_get_shared_cache_uuid(cache_uuid) {
    let cachePath = String(
      cString: dyld_shared_cache_file_path())
    print("Inspecting dyld cache at \"\(cachePath)\"")

    dyld_shared_cache_iterate_text(cache_uuid) { info in
        if let module = info?.pointee {
            let uuid = UUID(uuid: module.dylibUuid).uuidString
            let path = String(cString: module.path)
            let exists = manager.fileExists(atPath: path)

            print("\(exists ? "*" : " ") \(uuid) - \(path)")
        }
    }
}

$ swiftc -I. -o /tmp/dyldlist dyldlist.swift

$ cat $(xcrun --show-sdk-platform-path)/Developer/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd | grep _dyld

$ /tmp/dyldlist  | wc -l
  2500

$ /tmp/dyldlist  | grep -E "^\*" | wc -l
4
$ /tmp/dyldlist  | grep -E "^\*"
* AED7DD2C-0325-3172-83E7-3BE31F6D4069 - /usr/lib/dyld
* 222F8841-2BFD-3804-AA0C-F6D80A73FBDF - /usr/lib/system/libsystem_kernel.dylib
* 7AF7B500-9A6E-3121-A66A-397C209B5C83 - /usr/lib/system/libsystem_pthread.dylib
* 61D6CE46-BF8C-34EA-B81A-879743AD4063 - /usr/lib/system/libsystem_platform.dylib

Key Points

  • Shared libraries are code external to your code that are loaded at runtime or compile time.
  • Unless you’re planning to actually share the code with multiple clients, making a library is often not worth the overhead.
  • The nm and otool commands let you inspect shared libraries to find function names.
  • The swift demangle command converts mangled function names into the form you can use in your code to call them.
  • The linker will resolve dependencies for dynamic frameworks automatically. You need to do the resolution yourself for static libraries.
  • The linker has a weak attribute you can use when linking so that a program won’t crash if a symbol can’t be resolved.
  • A text-based dynamic library, or TBD, file with the .tbd extension can stand in for a shared library when compiling code.
  • A module map file serves as a bridge between Swift and Objective-C code.

Where to Go From Here?

Does your head hurt? In this chapter, you learned more than you ever wanted to know about dynamic frameworks. You explored the compiling and linking process while realizing it’s not about the language, but more about the linker needing to resolve symbols. You’ve learned how to generate \*.tbd files to link to a library you don’t physically have on your computer’s disk. You’ve learned how to use clang‘s modules to import private code to use in Swift. Finally, you’ve learned about the dyld shared cache and how to list its libraries in Swift.

14. System Calls & Ptrace
16. Hooking & Executing Code With dlopen & dlsym
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now