Invalid offset calculation for some ivars

[xavierjurado]

Runtime is not able to calculate the correct ivar offsets under some specific circumstances. Take a look at the following example:

import Foundation
import Runtime

class IncorrectLayout: NSObject {
  let ivar1 = AttributedString("ivar1") // Value type
  let ivar2 = NSDate()                  // Reference type
}

Inspecting this type through Runtime gives the following offsets:

PropertyInfo(name: "ivar1", type: Foundation.AttributedString, isVar: false, offset: 0, ownerType: ...)
PropertyInfo(name: "ivar2", type: NSDate, isVar: false, offset: 8, ownerType: ...)

But that's not correct as far as I understand. According to ivar_getOffset the offsets are off by 8 bytes. Because of that, using get(from:) will return an instance pointing to an invalid memory address, resulting in a crash when invoking any method on it. Eg:

let instance = IncorrectLayout()
let info = try typeInfo(of: IncorrectLayout.self)

try info.properties.forEach { property in
  if property.name == "ivar2" {
    let ivar = try property.get(from: instance) as NSDate
    print(ivar.timeIntervalSinceNow) // Unrecognized selector -[_TtCV10Foundation16AttributedString4Guts timeIntervalSinceNow]!
  }
}

Additional notes

Attempting to change almost anything from IncorrectLayout will likely fix the issue. Eg:

• Removing the NSObject superclass
• Changing the type of ivar1 to a reference type
• Removing ivar1

[xavierjurado]

Context

I partially know what's going on. First, here's a simpler example that illustrates the problem:

// This is the API Swit.Mirror uses under the hood: https://github.com/swiftlang/swift/blob/ff790c6d01edab1695c744d22dad0d8b32f2d749/stdlib/public/core/ReflectionMirror.swift#L42-L46
@_silgen_name("swift_reflectionMirror_recursiveChildOffset")
internal func _getChildOffset(
  _: Any.Type,
  index: Int
) -> Int

class IncorrectLayout: NSObject {
  let ivar1 = AttributedString("ivar1") // Value type
}

let info = try typeInfo(of: IncorrectLayout.self)

let mirrorOffset = _getChildOffset(IncorrectLayout.self, index: 0)
let runtimeOffset = info.properties[0].offset

XCTAssert(mirrorOffset == runtimeOffset)

The assertion fails, with Swift API correctly returning 8 bytes but Runtime returning 0. With this example, attempting to try info.properties[0].get(from: instance) will crash the process.

Problem

As far as I can tell the underlying problem here is not Runtime's implementation, but Swift's metadata representation which just happens to be incorrect for some classes. Apple gets away from this bug/limitation by using Objective-C's ivar_getOffset API as per:

https://github.com/swiftlang/swift/blob/ff790c6d01edab1695c744d22dad0d8b32f2d749/stdlib/public/runtime/ReflectionMirror.cpp#L731-L747

    // FIXME: If the class has ObjC heritage, get the field offset using the ObjC
    // metadata, because we don't update the field offsets in the face of
    // resilient base classes.
    uintptr_t fieldOffset;
    if (usesNativeSwiftReferenceCounting(Clazz)) {
      fieldOffset = Clazz->getFieldOffsets()[i];
    } else {
  #if SWIFT_OBJC_INTEROP
      Ivar *ivars = class_copyIvarList(
          reinterpret_cast<Class>(const_cast<ClassMetadata *>(Clazz)), nullptr);
      fieldOffset = ivar_getOffset(ivars[i]);
      free(ivars);
  #else
      swift::crash("Object appears to be Objective-C, but no runtime.");
  #endif
    }

By using the debugger to force usesNativeSwiftReferenceCounting to always return true I've reproduced the same issue in Apple's Swift.Mirror API. The FIXME is 7 years old so this seems like a longtime issue.

As for what type of layout triggers this error I'm not sure. Looking at the different values of ClassTypeDescriptor I've found that for cases where the offset is invalid, var flags: ContextDescriptorFlags has the bit 16 (MetadataInitialization) set to 1 (SingletonMetadataInitialization).

According to Apple's source code this means there's something unusual about how the metadata is initialized, but I don't know if/how that could affect the ivar offset calculation. It does seem to affect the number of trailing objects.

Next steps

The easiest workaround could be doing what Apple does and rely on ivar_getOffset. A bit more risky approach would be to figure out how the offset should be corrected, but with upstream having the same issue it may be difficult. I'll try to at least report the issue to Apple's Swift repository.