add failover to naive location assumptions

The current hacked together location API we added to drgn works extremely
poorly with modern C++ compilers. It largely works with the clang-12 compiler
on CircleCI, but works very poorly with clang 15/16/17/18 in Nix or when
updating the CircleCI compiler to clang-15.

This change adds a backup mechanism for locating arguments when drgn has
failed. The mechanism is extremely naive and makes several assumptions which
are often not correct. Currently, however, it drastically reduces the number of
tests that must be skipped in the Nix CI.

Test plan:
- CI

Author: JakeHillion

.github/workflows/tests_failing_under_nix.txt

@@ -1,235 +1,42 @@
 AddChildrenTest.InheritancePolymorphic
-ClangTypeParserTest.MemberAlignment
-ClangTypeParserTest.SimpleStruct
 DrgnParserTest.ClassTemplateInt
+DrgnParserTest.ClassTemplateTwo
+DrgnParserTest.ClassTemplateValue
 DrgnParserTest.Container
 DrgnParserTest.TemplateEnumValue
 DrgnParserTest.TemplateEnumValueGaps
 DrgnParserTest.TemplateEnumValueNegative
-OidIntegration.alignment_wrapper_member_alignment
-OidIntegration.alignment_wrapper_member_lower
-OidIntegration.alignment_wrapper_member_override
-OidIntegration.alignment_wrapper_struct
-OidIntegration.alignment_wrapper_two_members
-OidIntegration.alignment_wrapper_union_member
-OidIntegration.anonymous_anon_struct
-OidIntegration.anonymous_anon_typedef
-OidIntegration.anonymous_anon_union
-OidIntegration.anonymous_nested_anon_struct
-OidIntegration.anonymous_regular_struct
-OidIntegration.arrays_member_int0
-OidIntegration.arrays_member_int10
-OidIntegration.arrays_multidim
-OidIntegration.arrays_multidim_legacy
-OidIntegration.bitfields_enum
-OidIntegration.bitfields_mixed
-OidIntegration.bitfields_single
-OidIntegration.bitfields_straddle_bytes
-OidIntegration.bitfields_within_bytes
-OidIntegration.bitfields_zero_bits
-OidIntegration.cycles_raw_ptr
-OidIntegration.cycles_raw_ptr_wrapped
-OidIntegration.cycles_shared_ptr
-OidIntegration.cycles_unique_ptr
-OidIntegration.enums_params_scoped_enum_val
-OidIntegration.enums_params_scoped_enum_val_cast
-OidIntegration.enums_params_scoped_enum_val_gaps
-OidIntegration.enums_params_scoped_enum_val_negative
-OidIntegration.enums_params_unscoped_enum_val_cast
-OidIntegration.fbstring_empty
-OidIntegration.fbstring_heap_allocated
-OidIntegration.fbstring_inline
-OidIntegration.fbstring_string_pooled_unique
-OidIntegration.folly_f14_fast_map_a
-OidIntegration.folly_f14_fast_set_a
-OidIntegration.folly_f14_node_map_a
-OidIntegration.folly_f14_node_set_a
-OidIntegration.folly_f14_value_map_a
-OidIntegration.folly_f14_value_set_a
-OidIntegration.folly_f14_vector_map_a
-OidIntegration.folly_f14_vector_set_a
-OidIntegration.folly_small_vector_int_always_heap
-OidIntegration.folly_small_vector_int_default_empty
-OidIntegration.folly_small_vector_int_default_inlined
-OidIntegration.folly_small_vector_int_default_overflow
-OidIntegration.folly_small_vector_vector_3_empty
-OidIntegration.folly_small_vector_vector_3_inlined
-OidIntegration.folly_small_vector_vector_3_overflow
-OidIntegration.folly_sorted_vector_map_int_int_empty
-OidIntegration.folly_sorted_vector_map_int_int_reserve
-OidIntegration.folly_sorted_vector_map_int_int_some
-OidIntegration.ignored_member
-OidIntegration.ignored_roottype
-OidIntegration.ignored_subtype
-OidIntegration.inheritance_access_private
-OidIntegration.inheritance_access_protected
-OidIntegration.inheritance_access_public
-OidIntegration.inheritance_access_public_as_base
-OidIntegration.inheritance_multiple_a
-OidIntegration.inheritance_polymorphic_a_as_a
+DrgnParserTest.Typedef
+DrgnParserTest.Using
 OidIntegration.inheritance_polymorphic_b_as_a
-OidIntegration.inheritance_polymorphic_b_as_b
 OidIntegration.inheritance_polymorphic_c_as_a
 OidIntegration.inheritance_polymorphic_c_as_b
-OidIntegration.inheritance_polymorphic_c_as_c
-OidIntegration.inheritance_polymorphic_diamond_child_as_child
 OidIntegration.inheritance_polymorphic_diamond_child_as_middle1
 OidIntegration.inheritance_polymorphic_diamond_child_as_middle1_root
 OidIntegration.inheritance_polymorphic_diamond_child_as_middle2
 OidIntegration.inheritance_polymorphic_diamond_child_as_middle2_root
-OidIntegration.inheritance_polymorphic_diamond_middle1_as_middle1
 OidIntegration.inheritance_polymorphic_diamond_middle1_as_root
-OidIntegration.inheritance_polymorphic_diamond_middle2_as_middle2
 OidIntegration.inheritance_polymorphic_diamond_middle2_as_root
-OidIntegration.inheritance_polymorphic_diamond_root_as_root
-OidIntegration.inheritance_polymorphic_non_dynamic_base_a_as_a
-OidIntegration.inheritance_polymorphic_non_dynamic_base_a_no_polymorphic
-OidIntegration.inheritance_polymorphic_non_dynamic_base_b_as_a
 OidIntegration.inheritance_polymorphic_non_dynamic_base_b_as_b
 OidIntegration.inheritance_polymorphic_non_dynamic_base_b_no_polymorphic
-OidIntegration.inheritance_polymorphic_non_dynamic_base_c_as_a
 OidIntegration.inheritance_polymorphic_non_dynamic_base_c_as_b
 OidIntegration.inheritance_polymorphic_non_dynamic_base_c_as_c
 OidIntegration.inheritance_polymorphic_non_dynamic_base_c_no_polymorphic
-OidIntegration.multi_arg_tb_all_fail_crashes
-OidIntegration.multi_arg_tb_fail_first_arg
-OidIntegration.namespaces_queue
-OidIntegration.namespaces_stack
-OidIntegration.packed_a
-OidIntegration.padding_bool_padding
-OidIntegration.padding_nested_padding
-OidIntegration.pointers_feature_config
-OidIntegration.pointers_feature_flag_disabled
-OidIntegration.pointers_incomplete_containing_struct
-OidIntegration.pointers_incomplete_containing_struct_no_follow
-OidIntegration.pointers_incomplete_shared_ptr
-OidIntegration.pointers_incomplete_shared_ptr_null
-OidIntegration.pointers_incomplete_unique_ptr
-OidIntegration.pointers_incomplete_unique_ptr_null
-OidIntegration.pointers_struct_primitive_ptrs
-OidIntegration.pointers_struct_primitive_ptrs_no_follow
-OidIntegration.pointers_struct_primitive_ptrs_null
-OidIntegration.pointers_struct_vector_ptr
-OidIntegration.pointers_struct_vector_ptr_no_follow
-OidIntegration.pointers_struct_vector_ptr_null
-OidIntegration.pointers_vector_of_pointers
-OidIntegration.primitives_long_double
-OidIntegration.simple_class
-OidIntegration.simple_struct
-OidIntegration.simple_union
-OidIntegration.sorted_vector_set_no_ints
-OidIntegration.sorted_vector_set_some_ints
-OidIntegration.std_array_uint64_length_0
-OidIntegration.std_array_uint64_length_1
-OidIntegration.std_array_uint64_length_8
-OidIntegration.std_array_vector_length_1
-OidIntegration.std_array_vector_length_2
-OidIntegration.std_conditional_a
-OidIntegration.std_deque_del_allocator_a
-OidIntegration.std_deque_deque_int_empty
-OidIntegration.std_deque_deque_int_some
-OidIntegration.std_deque_int_empty
-OidIntegration.std_deque_int_some
-OidIntegration.std_list_del_allocator_a
-OidIntegration.std_list_int_empty
-OidIntegration.std_list_int_some
-OidIntegration.std_list_list_int_empty
-OidIntegration.std_list_list_int_some
-OidIntegration.std_list_struct_some
-OidIntegration.std_map_custom_comparator_a
-OidIntegration.std_multimap_custom_comparator_a
-OidIntegration.std_multiset_custom_comparator_a
-OidIntegration.std_optional_uint64_empty
-OidIntegration.std_optional_uint64_present
-OidIntegration.std_optional_vector_empty
-OidIntegration.std_optional_vector_present
-OidIntegration.std_pair_uint64_uint32
-OidIntegration.std_pair_uint64_uint64
-OidIntegration.std_pair_vector_vector
-OidIntegration.std_priority_queue_adapter_deque_empty
-OidIntegration.std_priority_queue_adapter_deque_some
-OidIntegration.std_priority_queue_int_empty
-OidIntegration.std_priority_queue_int_some
-OidIntegration.std_queue_adapter_vector_empty
-OidIntegration.std_queue_adapter_vector_some
-OidIntegration.std_queue_int_empty
-OidIntegration.std_queue_int_some
-OidIntegration.std_queue_queue_int_empty
-OidIntegration.std_queue_queue_int_some
-OidIntegration.std_reference_wrapper_int
-OidIntegration.std_reference_wrapper_vector
-OidIntegration.std_set_custom_comparator_a
-OidIntegration.std_smart_ptr_shared_ptr_const_uint64_empty
-OidIntegration.std_smart_ptr_shared_ptr_const_vector_empty
-OidIntegration.std_smart_ptr_shared_ptr_uint64_empty
-OidIntegration.std_smart_ptr_shared_ptr_uint64_present
-OidIntegration.std_smart_ptr_shared_ptr_vector_empty
-OidIntegration.std_smart_ptr_shared_ptr_vector_present
-OidIntegration.std_smart_ptr_shared_ptr_void_empty
-OidIntegration.std_smart_ptr_shared_ptr_void_present
-OidIntegration.std_smart_ptr_unique_ptr_const_uint64_empty
-OidIntegration.std_smart_ptr_unique_ptr_const_vector_empty
-OidIntegration.std_smart_ptr_unique_ptr_uint64_empty
-OidIntegration.std_smart_ptr_unique_ptr_uint64_present
-OidIntegration.std_smart_ptr_unique_ptr_vector_empty
-OidIntegration.std_smart_ptr_unique_ptr_vector_present
-OidIntegration.std_smart_ptr_unique_ptr_void_empty
-OidIntegration.std_smart_ptr_unique_ptr_void_present
-OidIntegration.std_smart_ptr_weak_ptr_int64_empty
-OidIntegration.std_smart_ptr_weak_ptr_int64_expired
-OidIntegration.std_smart_ptr_weak_ptr_int64_expired_chase
-OidIntegration.std_smart_ptr_weak_ptr_int64_present
-OidIntegration.std_smart_ptr_weak_ptr_int64_present_chase
-OidIntegration.std_smart_ptr_weak_ptr_int64_void_empty
-OidIntegration.std_stack_adapter_vector_empty
-OidIntegration.std_stack_adapter_vector_some
-OidIntegration.std_stack_int_empty
-OidIntegration.std_stack_int_some
-OidIntegration.std_stack_stack_int_empty
-OidIntegration.std_stack_stack_int_some
-OidIntegration.std_string_empty
-OidIntegration.std_string_heap_allocated
-OidIntegration.std_string_sso
-OidIntegration.std_tuple_uint64_uint64
-OidIntegration.std_unordered_map_custom_operator_a
-OidIntegration.std_unordered_multimap_custom_operator_a
-OidIntegration.std_unordered_multiset_custom_operator_a
-OidIntegration.std_unordered_set_custom_operator_a
-OidIntegration.std_variant_256_params_256
-OidIntegration.std_variant_256_params_empty
-OidIntegration.std_variant_char_int64_1
-OidIntegration.std_variant_char_int64_2
-OidIntegration.std_variant_empty
-OidIntegration.std_variant_optional
-OidIntegration.std_variant_vector_int_1
-OidIntegration.std_variant_vector_int_2
 OidIntegration.std_vector_del_allocator_a
-OidIntegration.std_vector_int_empty
-OidIntegration.std_vector_int_some
-OidIntegration.std_vector_reserve
-OidIntegration.std_vector_struct_some
-OidIntegration.std_vector_vector_int_empty
-OidIntegration.std_vector_vector_int_some
 OidIntegration.templates_int
 OidIntegration.templates_two
 OidIntegration.templates_value
 OidIntegration.templates_vector
 OidIntegration.typedefed_parent_multilevel_typedef_parent
 OidIntegration.typedefed_parent_simple_typedef_parent
 OidIntegration.typedefs_anonymous
+OidIntegration.typedefs_c_style
 OidIntegration.typedefs_container
+OidIntegration.typedefs_using
 OidIntegration.unions_alignment
 OidIntegration.unions_int
 OidIntegration.unions_tagged_int
 OidIntegration.unions_tagged_unordered_map
 OidIntegration.unions_tagged_vector
 OidIntegration.unions_unordered_map
 OidIntegration.unions_vector
-OilIntegration.folly_f14_fast_map_a
-OilIntegration.folly_f14_fast_set_a
-OilIntegration.folly_f14_node_map_a
-OilIntegration.folly_f14_node_set_a
-OilIntegration.folly_f14_value_map_a
-OilIntegration.folly_f14_value_set_a
-OilIntegration.folly_f14_vector_map_a
-OilIntegration.folly_f14_vector_set_a

flake.nix

@@ -34,7 +34,7 @@
         mkOidPackage =
           llvmPackages:
           with pkgs;
-          pkgs.llvmPackages.stdenv.mkDerivation rec {
+          llvmPackages.stdenv.mkDerivation rec {
             name = "oid";
 
             src = self;
@@ -60,6 +60,7 @@
             buildInputs = [
               llvmPackages.libclang
               llvmPackages.llvm
+              llvmPackages.openmp
 
               boost
               bzip2
@@ -85,8 +86,6 @@
               sqlite
               tomlplusplus
               zstd
-
-              llvmPackages.openmp # should match the stdenv clang version, see: https://github.com/NixOS/nixpkgs/issues/79818
             ];
 
             cmakeFlags = [

oi/Descs.cpp

@@ -50,10 +50,13 @@ std::optional<uintptr_t> FuncDesc::Arg::findAddress(
   if (auto* err = drgn_object_locate(&locator, &modifiedRegs, &object)) {
     LOG(ERROR) << "Error while finding address of argument: " << err->message;
     drgn_error_destroy(err);
-    return std::nullopt;
+  } else {
+    return object.address;
   }
 
-  return object.address;
+  LOG(WARNING) << "failed to locate argument with drgn! failing over to naive "
+                  "argument location";
+  return oi::detail::arch::naiveReadArgument(*regs, index);
 }
 
 std::optional<uint8_t> FuncDesc::getArgumentIndex(const std::string& arg,

oi/Descs.h

@@ -110,7 +110,8 @@ struct FuncDesc {
   };
 
   struct Arg final : virtual TargetObject {
-    struct drgn_object_locator locator;
+    uint8_t index;
+    drgn_object_locator locator;
 
     ~Arg() final {
       drgn_object_locator_deinit(&locator);

oi/Serialize.h

@@ -40,7 +40,7 @@
 DEFINE_TYPE_VERSION(PaddingInfo, 120, 3)
 DEFINE_TYPE_VERSION(struct drgn_location_description, 32, 2)
 DEFINE_TYPE_VERSION(struct drgn_object_locator, 72, 2)
-DEFINE_TYPE_VERSION(FuncDesc::Arg, 128, 2)
+DEFINE_TYPE_VERSION(FuncDesc::Arg, 136, 3)
 DEFINE_TYPE_VERSION(FuncDesc::Retval, 56, 2)
 DEFINE_TYPE_VERSION(FuncDesc::Range, 16, 2)
 DEFINE_TYPE_VERSION(FuncDesc, 104, 4)

oi/arch/Arch.h

@@ -26,4 +26,7 @@ void setProgramCounter(user_regs_struct& regs, uintptr_t pc);
 
 std::optional<uintptr_t> getReturnValueAddress(const user_regs_struct&);
 
+std::optional<uintptr_t> naiveReadArgument(const user_regs_struct&,
+                                           uint8_t idx);
+
 }  // namespace oi::detail::arch

oi/arch/aarch64.cpp

@@ -31,6 +31,35 @@ void setProgramCounter(user_regs_struct& regs, uintptr_t pc) {
   regs.pc = pc;
 }
 
+std::optional<uintptr_t> naiveReadArgument(const user_regs_struct&,
+                                           uint8_t idx) {
+  /*
+   * The ARM64 argument passing practices are surprisingly well documented. The
+   * PDF I am using here is
+   * https://github.com/ARM-software/abi-aa/releases/download/2023Q3/aapcs32.pdf
+   * from https://github.com/ARM-software/abi-aa/releases/tag/2023Q3. Relevant
+   * information appears to be in §6.8.2.
+   *
+   * This is an extremely naïve estimation of register placement. It is expected
+   * to work when all preceding arguments (this, arg0, arg.., argIdx) are:
+   * - Pointers. Pointers are all placed in general purpose registers
+   * incrementing as expected.
+   * - >16 byte by-value structures. These are defined to be placed on the stack
+   * and have a pointer placed in a general purpose registers, so increment in
+   * the same way.
+   * - <=8 byte integers. Also placed in general purpose registers.
+   *
+   * Any other types, including floats, will mess up our indexing. Looking at
+   * the types of all the preceding arguments could get us a lot closer. For
+   * now, we rely on OID correctly restoring the process if we get this wrong,
+   * and might produce garbage data.
+   */
+  if (idx < 8)
+    return regs.regs[idx];
+
+  return std::nullopt;
+}
+
 }  // namespace oi::detail::arch
 
 #endif

oi/arch/x86_64.cpp

@@ -31,6 +31,33 @@ void setProgramCounter(user_regs_struct& regs, uintptr_t pc) {
   regs.rip = pc;
 }
 
+std::optional<uintptr_t> naiveReadArgument(const user_regs_struct& regs,
+                                           uint8_t idx) {
+  /*
+   * This function is based on the information available at
+   * http://6.s081.scripts.mit.edu/sp18/x86-64-architecture-guide.html. I have
+   * no idea under which conditions these registers are selected. We rely on the
+   * fact that OID will safely exit if incorrect, potentially producing some
+   * incorrect data but otherwise leaving the process unharmed.
+   */
+  switch (idx) {
+    case 0:
+      return regs.rdi;
+    case 1:
+      return regs.rsi;
+    case 2:
+      return regs.rdx;
+    case 3:
+      return regs.rcx;
+    case 4:
+      return regs.r8;
+    case 5:
+      return regs.r9;
+    default:
+      return std::nullopt;
+  }
+}
+
 }  // namespace oi::detail::arch
 
 #endif