showxu / objc4

A latest buildable and debuggable Objective-C runtime (objc4-818.2) project.

objc4 runtime

简体中文

This project is a buildable and debuggable version of latest Objective-C runtime (objc4-818.2) on Apple Open Source

• Which version to use
• Installation
• Usage
• objc4 tarballs
• Build Phases
• Build Setting
• License

Which Version To Use

macOS	macOS Version	Xcode Version	objc4 tarball version
			objc4-818.2
			objc4-787.1
			objc4-781
			objc4-723

Installation

Manually

Download zip or clone this repo, select objc scheme and build.

Usage

After building the objc scheme, manually integrate generated libobjc.A.dylib into your project manually. Or you can use objc-inspect scheme which is a preset inspector for debugging objc4 runtime.

objc4 tarballs

• objc4-818.2
• xnu-7195.50.7.100.1
• Libc-1439.40.11
• dyld-832.7.1
• libauto-187.tar.gz
• libclosure-74
• libdispatch-1173.40.5
• libplatform-220
• libpthread-416.40.3

Build Phases

Private Header

objc header	#include	tarball
objc-os.h	#include <sys/reason.h>	/xnu-6153.41.3/bsd/sys/reason.h
objc-os.h	#include <mach-o/dyld_priv.h>	/dyld-733.6/include/mach-o/dyld_priv.h
objc-os.h	#include <os/lock_private.h>	/libplatform-220/private/os/lock_private.h
lock_private.h	#include <os/base_private.h>	/libplatform-220/private/os/base_private.h
objc-os.h	#include <System/pthread_machdep.h>	removed in latest Libc tarball (Libc-1353.41.1), this header should be commented-out
objc-class.mm	#include <os/linker_set.h>	/xnu-7195.50.7.100.1/os/linker_set.h
pthread_machdep.h	#include <System/machine/cpu_capabilities.h>	/xnu-6153.41.3/osfmk/machine/cpu_capabilities.h
objc-os.h	#include <pthread/workqueue_private.h>	/libpthread-416.40.3/private/workqueue_private.h
objc-os.h	#include <objc-shared-cache.h>	/dyld-733.6/include/objc-shared-cache.h
objc-errors.mm	#include <_simple.h>	/libplatform-220/private/_simple.h
objc-block-trampolines.mm	#include <Block_private.h>	/libclosure-74/Block_private.h
objc-os.h	#include <crt_externs.h>	/Libc-1353.41.1/include/crt_externs.h
objc-runtime-new.mm	#include <mach/shared_region.h>	/xnu-6153.41.3/osfmk/mach/shared_region.h
objc-cache.mm	#include <kern/restartable.h>	/xnu-6153.41.3/osfmk/mach/restartable.defs, build from xnu kernel
objc-os.h	#include_next <CrashReporterClient.h> => #include <CrashReporterClient.h>	/Libc-825.24/include/CrashReporterClient.h
NSObject.mm	#include <os/reason_private.h>	/xnu-7195.50.7.100.1/os/reason_private
NSObject.mm	#include <os/variant_private.h>	/xnu-7195.50.7.100.1/os/reason_private
objc-exception.mm	#include <objc/objc-abi.h>	removed
objc-gdb.h	#include <objc/maptable.h>	removed

Private Header Included Header

private header	#include	tarball
tsd_private.h	#include <os/tsd.h>	/xnu-6153.41.3/libsyscall/os/tsd.h
tsd_private.h	#include <pthread/spinlock_private.h>	/libpthread-416.40.3/private/spinlock_private.h
lock_private.h	#include <pthread/tsd_private.h>	/libpthread-416.40.3/private/tsd_private.h
workqueue_private.h	#include <pthread/qos_private.h>	/llibpthread-416.40.3/private/qos_private.h
qos_private.h	#include <sys/qos_private.h>	/libpthread-416.40.3/sys/qos_private.h

Bridge OS

In public macosx sdk (latest Xcode 12.3), bridgeos (e.g. __has_feature(attribute_availability_bridgeos)) is unavailable, bridgeos availability should be removed or commented-out.

dyld

In latest dyld-733.6 (dyld-421.2 later), apple use this ruby script to expand versions, platfrom versions from a versionSets which defined in a YAML file, code generated by this script will be inserted after @MAC_VERSION_DEFS@, @IOS_VERSION_DEFS@, @WATCHOS_VERSION_DEFS@, @TVOS_VERSION_DEFS@ and @BRIDGEOS_VERSION_DEFS@ in dyld_priv.h. For more detail please refer to dyld.

XNU

<kern/restartable.h> is generated form restartable.defs in xnu tarball during building xun kernel, which is a little different from the one that shipped with public sdk that located in /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/kern/restartable.h.

Cambria framework

Have no idea about this framework so far, maybe related with -loah for lib oah. Related Code should be commented out for now.

// File objc-cache.mm
// 
// Line 87 ~ 88
#include <Cambria/Traps.h>
#include <Cambria/Cambria.h>
// 
// ···
//
// Line 1121 ～ 1128 
if (oah_is_current_process_translated()) {
    kern_return_t ret = objc_thread_get_rip(threads[count], (uint64_t*)&pc);
    if (ret != KERN_SUCCESS) {
        pc = PC_SENTINEL;
    }
} else {
    pc = _get_pc_for_thread (threads[count]);
}

#include <os/feature_private.h>

Have no idea about this haeder so far. Related Code should be commented out for now.

// File objc-runtime.mm
// 
// ···
//
// Line 36
#include <os/feature_private.h> // os_feature_enabled_simple()
//
// ···
//
// Line 444 ～ 446
if (!os_feature_enabled_simple(objc4, preoptimizedCaches, true)) {
    DisablePreoptCaches = true;
}
// 

dyld_fall_2020_os_versions, dyld_fall_2018_os_versions

Found releated source in xnu-7195.50.7.100.1/bsd/kern/kern_mman.c.

static uint32_t
proc_2020_fall_os_sdk(void)

Maybe these function is releated to dyld build-scripts. Comment out for now.

dyld_platform_version_macOS_, dyld_platform_version_iOS_, dyld_platform_version_tvOS_, dyld_platform_version_watchOS_, dyld_platform_version_bridgeOS_

This Marco is generated by dyld build-scripts, comment out for now.

STATIC_ASSERT

objc4/runtime/objc-runtime-new.mm:176:1: '_static_assert' declared as an array with a negative size

Comment out for now.

Build Setting

Declaration	Value
HEADER_SEARCH_PATHS	$(SRCROOT)/../macosx.internal/System/Library/Frameworks/System.framework/PrivateHeaders, also append $(inherited) to target objc
GCC_PREPROCESSOR_DEFINITIONS	LIBC_NO_LIBCRASHREPORTERCLIENT, also append $(inherited) to target objc
ORDER_FILE	$(SRCROOT)/libobjc.order
OTHER_LDFLAGS[sdk=macosx*]	-lc++abi -Xlinker -sectalign -Xlinker __DATA -Xlinker __objc_data -Xlinker 0x1000 -Xlinker -interposable_list -Xlinker interposable.txt, remove build setting in target objc
OTHER_LDFLAGS[sdk=iphoneos*][arch=*]	-lc++abi -Wl,-segalign,0x4000 -Xlinker -sectalign -Xlinker __DATA -Xlinker __objc_data -Xlinker 0x1000 -Xlinker -interposable_list -Xlinker interposable.txt, remove build setting in target objc
OTHER_LDFLAGS[sdk=iphonesimulator*][arch=*]	-lc++abi -Xlinker -interposable_list -Xlinker interposable.txt, remove build setting in target objc

Run Script

Evidently public macosx sdk is our only choice, we need to update value of parameter -sdk from macosx.internal to macosx in run script of objc target.

License

This project is released under the MIT License. The objc4 project is released under the APPLE PUBLIC SOURCE LICENSE Version 2.0.

objc4's Issues

Mac OS 10.15.7 报这个错怎么解决

Static_assert failed due to requirement 'bucketsMask >= ((unsigned long long)140737454800896ULL)' "Bucket field doesn't have enough bits for arbitrary pointers."

Add macOS 11 Big Sur support

How does libobjc.A.dylib integrate into iOS projects?

iOS scheme can not compile pass, and i can not get libobjc.A.dylib product for iOS.
so, How does libobjc.A.dylib integrate into iOS projects to debug?

Run debug error on mojave

dyld: Symbol not found: _objc_debug_taggedpointer_obfuscator
Referenced from: /usr/lib/system/libxpc.dylib
Expected in: /Users/xxxx/Library/Developer/Xcode/DerivedData/objc-ccgnoycdykfcdtfetssxtbpuvizg/Build/Products/Debug/libobjc.A.dylib
in /usr/lib/system/libxpc.dylib

can we run objc in macOS 12.1

can we run objc in macOS 12.1 and xcode: 13.2.1
?

build crash

hi coder
a.can this demo be build ?
b.i just clone this project and add a new target and config dependencies(objc) and comple sources(libobjc.A.dylib).
c. run
this is come to an crash here is the crash message
Assertion failed: (sel_registerName(sel_getName(meth.name())) == meth.name()), function methodizeClass, file objc-runtime-new.mm, line 1536.
is there any soluation?
thanks

crash at cache_init();

dyld: lazy symbol binding failed: Symbol not found: _task_restartable_ranges_register
Referenced from: /Users/hsz/Library/Developer/Xcode/DerivedData/objc-bcmgrkittkwfkggudysyarbtscjy/Build/Products/Debug/libobjc.A.dylib (which was built for Mac OS X 10.15)
Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _task_restartable_ranges_register
Referenced from: /Users/hsz/Library/Developer/Xcode/DerivedData/objc-bcmgrkittkwfkggudysyarbtscjy/Build/Products/Debug/libobjc.A.dylib (which was built for Mac OS X 10.15)
Expected in: /usr/lib/libSystem.B.dylib

Add target IOS_PLATFORM debugging support

Currently, both objc-inspect and objc target only support macOS Catalina, support debugging libobjc.A.dylib on iPhone OS and iOS simulator will be helpful for iOS development.

Thread 1: hit program assert

objc-os.h

line:72

---

compile error~

OS X 13.4.1
Xcode 14.3.1

Track of macOS 14.4 + Xcode 15.2 build issue

1. We need the following patch to make it compile on the latest Xcode.

// objc-os.h
+ #include <utility>

2. Once we run objc-inspect, we'll get a crash on _objc_init -> _dyld_objc_notify_register.

The abort message is "_dyld_objc_notify_register is unsupported"

Checking the latest dyld's source code, the API is still there and suggesting we move to use "_dyld_objc_register_callbacks".

https://github.com/apple-oss-distributions/dyld/blob/d1a0f6869ece370913a3f749617e457f3b4cd7c4/dyld/DyldAPIs.cpp#L876-L877
// FIXME: Remove this once libobjc moves to _dyld_objc_register_callbacks()

The latest SDK's dyld has changed it and make it abort here.

Checking the latest objc code, they did change to use _dyld_objc_register_callbacks here.

https://github.com/apple-oss-distributions/objc4/blob/196363c165b175ed925ef6b9b99f558717923c47/runtime/objc-os.mm#L934

So the final answer is we need to update the objc source to debug it on the latest OS due to the API change in dylb.

Objc4-838 has open source

https://github.com/apple-oss-distributions/objc4

Don't comment them out

I figured out how <os/feature_private.h> was, and those dyld magic values.

https://github.com/Torrekie/apple_internal_sdk

I started working on this while compiling latest objc4 for porting it to older iOS. Commenting out featureflag and OS detection codes was really a bad idea since it ACTUALLY affecting how libobjc works. The reason I am doing this is, I can't find any useful information except bunch copies of articles telling me to comment these undefined values out, no, don't do that.

These undefined values and undeclared functions can be easily figured out by simple searching or reversing. For that non-open feature_private.h, I have done that here.

And for those dyld version check constants, they should originally be in <mach-o/dyld_priv.h> but stripped out by Apple. They are simple structs containing two uint32_t values referring to platforms and versions.

typedef uint32_t dyld_platform_t;

typedef struct {
    dyld_platform_t platform;
    uint32_t        version;
} dyld_build_version_t;

These are actual values of those dyld_fall_20*_os_versions stuff

// Sat Mar 26 18:48:53 CST 2022
// reversed from /usr/lib/libobjc.A.dylib and /usr/lib/system/libcommonCrypto.dylib
// Also tested
#define dyld_fall_2017_os_versions		(dyld_build_version_t) { 0xFFFFFFFF, 0x07E10901 }
#define dyld_fall_2018_os_versions		(dyld_build_version_t) { 0xFFFFFFFF, 0x07E20901 }
#define dyld_fall_2019_os_versions		(dyld_build_version_t) { 0xFFFFFFFF, 0x07E30901 }
#define dyld_fall_2020_os_versions		(dyld_build_version_t) { 0xFFFFFFFF, 0x07E40901 }
#define dyld_fall_2021_os_versions		(dyld_build_version_t) { 0xFFFFFFFF, 0x07E50901 }

#define dyld_platform_version_macOS_10_11       (dyld_build_version_t) { PLATFORM_MACOS, 0x000A0B00 }
#define dyld_platform_version_macOS_10_12       (dyld_build_version_t) { PLATFORM_MACOS, 0x000A0C00 }
#define dyld_platform_version_macOS_10_13	(dyld_build_version_t) { PLATFORM_MACOS, 0x000A0D00 }
#define dyld_platform_version_macOS_10_14	(dyld_build_version_t) { PLATFORM_MACOS, 0x000A0E00 }
#define dyld_platform_version_macOS_10_15	(dyld_build_version_t) { PLATFORM_MACOS, 0x000A0F00 }
#define dyld_platform_version_macOS_10_16	(dyld_build_version_t) { PLATFORM_MACOS, 0x000A1000 }
#define dyld_platform_version_macOS_12_00	(dyld_build_version_t) { PLATFORM_MACOS, 0x000C0000 }

#define dyld_platform_version_tvOS_10_0         (dyld_build_version_t) { PLATFORM_TVOS, 0x000A0000 }
#define dyld_platform_version_tvOS_11_0         (dyld_build_version_t) { PLATFORM_TVOS, 0x000B0000 }
#define dyld_platform_version_tvOS_12_0		(dyld_build_version_t) { PLATFORM_TVOS, 0x000C0000 }
#define dyld_platform_version_tvOS_13_0		(dyld_build_version_t) { PLATFORM_TVOS, 0x000D0000 }
#define dyld_platform_version_tvOS_14_0		(dyld_build_version_t) { PLATFORM_TVOS, 0x000E0000 }
#define dyld_platform_version_tvOS_15_0		(dyld_build_version_t) { PLATFORM_TVOS, 0x000F0000 }

#define dyld_platform_version_iOS_10_0          (dyld_build_version_t) { PLATFORM_IOS, 0x000A0000 }
#define dyld_platform_version_iOS_11_0          (dyld_build_version_t) { PLATFORM_IOS, 0x000B0000 }
#define dyld_platform_version_iOS_12_0		(dyld_build_version_t) { PLATFORM_IOS, 0x000C0000 }
#define dyld_platform_version_iOS_13_0		(dyld_build_version_t) { PLATFORM_IOS, 0x000D0000 }
#define dyld_platform_version_iOS_14_0		(dyld_build_version_t) { PLATFORM_IOS, 0x000E0000 }
#define dyld_platform_version_iOS_15_0		(dyld_build_version_t) { PLATFORM_IOS, 0x000F0000 }

#define dyld_platform_version_watchOS_3_0       (dyld_build_version_t) { PLATFORM_WATCHOS, 0x00030000 }
#define dyld_platform_version_watchOS_4_0       (dyld_build_version_t) { PLATFORM_WATCHOS, 0x00040000 }
#define dyld_platform_version_watchOS_5_0	(dyld_build_version_t) { PLATFORM_WATCHOS, 0x00050000 }
#define dyld_platform_version_watchOS_6_0	(dyld_build_version_t) { PLATFORM_WATCHOS, 0x00060000 }
#define dyld_platform_version_watchOS_7_0	(dyld_build_version_t) { PLATFORM_WATCHOS, 0x00070000 }
#define dyld_platform_version_watchOS_8_0	(dyld_build_version_t) { PLATFORM_WATCHOS, 0x00080000 }

#define dyld_platform_version_bridgeOS_2_0      (dyld_build_version_t) { PLATFORM_BRIDGEOS, 0x00020000 }
#define dyld_platform_version_bridgeOS_3_0      (dyld_build_version_t) { PLATFORM_BRIDGEOS, 0x00030000 }
#define dyld_platform_version_bridgeOS_4_0      (dyld_build_version_t) { PLATFORM_BRIDGEOS, 0x00040000 }
#define dyld_platform_version_bridgeOS_5_0      (dyld_build_version_t) { PLATFORM_BRIDGEOS, 0x00050000 }
#define dyld_platform_version_bridgeOS_6_0      (dyld_build_version_t) { PLATFORM_BRIDGEOS, 0x00060000 }

Using that CrashReporterClient.h stub which appears in older Libc was not quite a good idea, since linking libCrashReporterClient would insert a load command section to Mach-O files (that also indicates libCrashReporterClient was a static library)

Section
  sectname __crash_info
   segname __DATA
      addr 0x00000001d97f8ad8
      size 0x0000000000000040
    offset 246296
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0

We can find CrashReporterClient implementation under WTF/wtf/spi/cocoa/CrashReporterClientSPI.h and WTF/wtf/cocoa/CrashReporter.cpp, these codes also appears in many other Apple OSSes.

Now we have <CrashReporterClient.h> and libCrashReporterClient.a

For those Cambria APIs, we can see objc-cache.mm was calling two of them

#if TARGET_OS_OSX
        if (oah_is_current_process_translated()) {
            kern_return_t ret = objc_thread_get_rip(threads[count], (uint64_t*)&pc);
            if (ret != KERN_SUCCESS) {
                pc = PC_SENTINEL;
            }
        } else {
            pc = _get_pc_for_thread (threads[count]);
        }
#else
        pc = _get_pc_for_thread (threads[count]);
#endif

It was really easy to figure out that oah_is_current_process_translated() was returning bool, with no arguments, then we have its prototype

bool oah_is_current_process_translated(void);

objc_thread_get_rip() was kinda tricky but still easy to find out how to correctly call it, the definition of thread was
thread_act_port_array_t threads;, and the actual type of thread_act_port_array_t was

// <arm/_types.h> or <i386/_types.h>
typedef unsigned int            __darwin_natural_t;
// <sys/_types.h>
typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
typedef __darwin_mach_port_name_t __darwin_mach_port_t; /* Used by mach */
// <sys/_types/_mach_port_t.h>
typedef __darwin_mach_port_t mach_port_t;
// <mach/mach-types.h>
typedef mach_port_t             thread_act_t;
typedef thread_act_t            *thread_act_array_t;
typedef thread_act_array_t      thread_act_port_array_t;

We can see the 1st arg of objc_thread_get_rip() was a member of thread_act_port_array_t, then the type is thread_act_t but not thread_act_port_array_t, then we get prototype

kern_return_t objc_thread_get_rip(thread_act_t thread, uint64_t** buf);

These codes cannot successfully compile without linking liboah, which was not presenting in MacOSX.sdk, we can extract dyld shared cache, get /usr/lib/liboah.dylib, generate liboah.tbd from it. I have done Cambria headers also