compile error on main branch: TensorImpl

[tiger-of-shawn]

runtime/core/portable_type/tensor_impl.cpp:27:29: error: use of undeclared identifier 'TensorImpl'; did you mean 'torch::executor::TensorImpl'?
ssize_t compute_numel(const TensorImpl::SizesType* sizes, ssize_t dim) {
                            ^~~~~~~~~~


/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:27:29: error: use of undeclared identifier 'TensorImpl'; did you mean 'torch::executor::TensorImpl'?
ssize_t compute_numel(const TensorImpl::SizesType* sizes, ssize_t dim) {
                            ^~~~~~~~~~
                            torch::executor::TensorImpl
/home/qwen/executorch_latest/../executorch/runtime/core/portable_type/tensor_impl.h:54:7: note: 'torch::executor::TensorImpl' declared here
class TensorImpl {
      ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:43:1: error: use of undeclared identifier 'TensorImpl'; did you mean 'torch::executor::TensorImpl'?
TensorImpl::TensorImpl(
^~~~~~~~~~
torch::executor::TensorImpl
/home/qwen/executorch_latest/../executorch/runtime/core/portable_type/tensor_impl.h:54:7: note: 'torch::executor::TensorImpl' declared here
class TensorImpl {
      ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:43:13: error: cannot define or redeclare 'TensorImpl' here because namespace 'etensor' does not enclose namespace 'TensorImpl'
TensorImpl::TensorImpl(
~~~~~~~~~~~~^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:51:14: error: use of undeclared identifier 'sizes'; did you mean 'sizeof'?
    : sizes_(sizes),
             ^~~~~
             sizeof
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:52:18: error: use of undeclared identifier 'dim_order'
      dim_order_(dim_order),
                 ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:53:16: error: use of undeclared identifier 'strides'
      strides_(strides),
               ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:54:13: error: use of undeclared identifier 'data'; did you mean 'std::data'?
      data_(data),
            ^~~~
            std::data
/home/qwen/toolchain/android-ndk-r26d/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/__iterator/data.h:27:6: note: 'std::data' declared here
auto data(_Cont& __c)
     ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:55:12: error: use of undeclared identifier 'dim'
      dim_(dim),
           ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:56:28: error: use of undeclared identifier 'sizes'; did you mean 'sizeof'?
      numel_(compute_numel(sizes, dim)),
                           ^~~~~
                           sizeof
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:56:33: error: expected expression
      numel_(compute_numel(sizes, dim)),
                                ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:56:35: error: use of undeclared identifier 'dim'
      numel_(compute_numel(sizes, dim)),
                                  ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:57:20: error: use of undeclared identifier 'numel_'
      numel_bound_(numel_),
                   ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:58:13: error: use of undeclared identifier 'type'
      type_(type),
            ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:59:23: error: use of undeclared identifier 'dynamism'
      shape_dynamism_(dynamism) {
                      ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:61:15: error: use of undeclared identifier 'type_'
      isValid(type_), "Invalid type %" PRId8, static_cast<int8_t>(type_));
              ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:61:67: error: use of undeclared identifier 'type_'
      isValid(type_), "Invalid type %" PRId8, static_cast<int8_t>(type_));
                                                                  ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:62:16: error: use of undeclared identifier 'dim_'
  ET_CHECK_MSG(dim_ >= 0, "Dimension must be non-negative, got %zd", dim_);
               ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:62:3: error: redefinition of '_log_level'
  ET_CHECK_MSG(dim_ >= 0, "Dimension must be non-negative, got %zd", dim_);
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:39:7: note: expanded from macro 'ET_CHECK_MSG'
      ET_ASSERT_MESSAGE_EMIT(" (%s): " _format, #_cond, ##__VA_ARGS__); \
      ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:22:3: note: expanded from macro 'ET_ASSERT_MESSAGE_EMIT'
  ET_LOG(                                        \
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/log.h:159:16: note: expanded from macro 'ET_LOG'
    const auto _log_level = ::executorch::runtime::LogLevel::_level; \
               ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:60:3: note: previous definition is here
  ET_CHECK_MSG(
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:39:7: note: expanded from macro 'ET_CHECK_MSG'
      ET_ASSERT_MESSAGE_EMIT(" (%s): " _format, #_cond, ##__VA_ARGS__); \
      ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:22:3: note: expanded from macro 'ET_ASSERT_MESSAGE_EMIT'
  ET_LOG(                                        \
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/log.h:159:16: note: expanded from macro 'ET_LOG'
    const auto _log_level = ::executorch::runtime::LogLevel::_level; \
               ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:62:3: error: redefinition of '_timestamp'
  ET_CHECK_MSG(dim_ >= 0, "Dimension must be non-negative, got %zd", dim_);
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:39:7: note: expanded from macro 'ET_CHECK_MSG'
      ET_ASSERT_MESSAGE_EMIT(" (%s): " _format, #_cond, ##__VA_ARGS__); \
      ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:22:3: note: expanded from macro 'ET_ASSERT_MESSAGE_EMIT'
  ET_LOG(                                        \
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/log.h:163:18: note: expanded from macro 'ET_LOG'
      const auto _timestamp =                                        \
                 ^
/home/qwen/executorch_latest/runtime/core/portable_type/tensor_impl.cpp:60:3: note: previous definition is here
  ET_CHECK_MSG(
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:39:7: note: expanded from macro 'ET_CHECK_MSG'
      ET_ASSERT_MESSAGE_EMIT(" (%s): " _format, #_cond, ##__VA_ARGS__); \
      ^
/home/qwen/executorch_latest/../executorch/runtime/platform/assert.h:22:3: note: expanded from macro 'ET_ASSERT_MESSAGE_EMIT'
  ET_LOG(                                        \
  ^
/home/qwen/executorch_latest/../executorch/runtime/platform/log.h:163:18: note: expanded from macro 'ET_LOG'
      const auto _timestamp =                                        \
                 ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
gmake[2]: *** [CMakeFiles/executorch_core.dir/build.make:118: CMakeFiles/executorch_core.dir/runtime/core/portable_type/tensor_impl.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
/home/qwen/executorch_latest/runtime/executor/method.cpp:575:13: error: no type named 'EventTracerProfileMethodScope' in namespace 'executorch::runtime::internal'; did you mean 'EventTracerProfileScope'?
  internal::EventTracerProfileMethodScope event_tracer_profile_scope =
  ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            EventTracerProfileScope
/home/qwen/executorch_latest/../executorch/runtime/core/event_tracer_hooks.h:41:7: note: 'EventTracerProfileScope' declared here
class EventTracerProfileScope final {
      ^
/home/qwen/executorch_latest/runtime/executor/method.cpp:576:17: error: no member named 'EventTracerProfileMethodScope' in namespace 'executorch::runtime::internal'
      internal::EventTracerProfileMethodScope(event_tracer_, "Method::init");
      ~~~~~~~~~~^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1025:17: error: no type named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'; did you mean 'EventTracerProfileScope'?
      internal::EventTracerProfileOpScope event_tracer_op_scope =
      ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
                EventTracerProfileScope
/home/qwen/executorch_latest/../executorch/runtime/core/event_tracer_hooks.h:41:7: note: 'EventTracerProfileScope' declared here
class EventTracerProfileScope final {
      ^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1026:21: error: no member named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'
          internal::EventTracerProfileOpScope(event_tracer_, "OPERATOR_CALL");
          ~~~~~~~~~~^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1060:17: error: no type named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'; did you mean 'EventTracerProfileScope'?
      internal::EventTracerProfileOpScope event_tracer_op_scope =
      ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
                EventTracerProfileScope
/home/qwen/executorch_latest/../executorch/runtime/core/event_tracer_hooks.h:41:7: note: 'EventTracerProfileScope' declared here
class EventTracerProfileScope final {
      ^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1061:21: error: no member named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'
          internal::EventTracerProfileOpScope(event_tracer_, "DELEGATE_CALL");
          ~~~~~~~~~~^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1104:17: error: no type named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'; did you mean 'EventTracerProfileScope'?
      internal::EventTracerProfileOpScope event_tracer_op_scope =
      ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
                EventTracerProfileScope
/home/qwen/executorch_latest/../executorch/runtime/core/event_tracer_hooks.h:41:7: note: 'EventTracerProfileScope' declared here
class EventTracerProfileScope final {
      ^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1105:21: error: no member named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'
          internal::EventTracerProfileOpScope(event_tracer_, "JF_CALL");
          ~~~~~~~~~~^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1123:17: error: no type named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'; did you mean 'EventTracerProfileScope'?
      internal::EventTracerProfileOpScope event_tracer_op_scope =
      ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
                EventTracerProfileScope
/home/qwen/executorch_latest/../executorch/runtime/core/event_tracer_hooks.h:41:7: note: 'EventTracerProfileScope' declared here
class EventTracerProfileScope final {
      ^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1124:21: error: no member named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'
          internal::EventTracerProfileOpScope(event_tracer_, "MOVE_CALL");
          ~~~~~~~~~~^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1132:17: error: no type named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'; did you mean 'EventTracerProfileScope'?
      internal::EventTracerProfileOpScope event_tracer_op_scope =
      ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
                EventTracerProfileScope
/home/qwen/executorch_latest/../executorch/runtime/core/event_tracer_hooks.h:41:7: note: 'EventTracerProfileScope' declared here
class EventTracerProfileScope final {
      ^
/home/qwen/executorch_latest/runtime/executor/method.cpp:1133:21: error: no member named 'EventTracerProfileOpScope' in namespace 'executorch::runtime::internal'
          internal::EventTracerProfileOpScope(event_tracer_, "FREE_CALL");
          ~~~~~~~~~~^
12 errors generated.
gmake[2]: *** [CMakeFiles/executorch_core.dir/build.make:132: CMakeFiles/executorch_core.dir/runtime/executor/method.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:271: CMakeFiles/executorch_core.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

[Gasoonjia]

hi @tiger-of-shawn can you share more details about the issue? like which branch/commit are you working on? What's the command to reproduce, etc?

[tiger-of-shawn]

hi @tiger-of-shawn can you share more details about the issue? like which branch/commit are you working on? What's the command to reproduce, etc?

branch: main

commit id: 46ea1a4

build step:

refs to: https://github.com/pytorch/executorch/blob/main/examples/demo-apps/android/LlamaDemo/docs/delegates/mediatek_README.md

git clone https://github.com/pytorch/executorch.git
cd executorch
git submodule sync
git submodule update --init

./install_requirements.sh

export BUCK2=path_to_buck/buck2 # Download BUCK2 and create BUCK2 executable
export ANDROID_NDK=path_to_android_ndk
export NEURON_BUFFER_ALLOCATOR_LIB=path_to_buffer_allocator/libneuron_buffer_allocator.so

cd examples/mediatek
./mtk_build_examples.sh

[dbort]

This is probably related to the namespace changes I made for v0.4.0, though we know that runtime/core/portable_type/tensor_impl.cpp can build successfully in most situations.

I'll take a look, but it's possible that this is related to the way that mtk_build_examples.sh works, or possibly with the local build environment.

cc: @neuropilot-captain @cccclai @cmodi-meta

[dbort]

Based on the error message:

/home/qwen/executorch_latest/../executorch/runtime/core/portable_type/tensor_impl.h

I believe what's happening is that there are two copies of executorch cloned into the same parent directory:

• /home/qwen/executorch/
• /home/qwen/executorch_latest/

This build is happening from inside executorch_latest. However, to make our include paths work, we require that the repo lives in a directory named executorch, and we pass -I.. to set the include path.

This means that, when building the system under executorch_latest, it's actually including the files that live under executorch_latest/../executorch, which appears to be an older clone from before the namespace changes.

You should be able to solve this problem by deleting or renaming the current (old) executorch directory, then renaming executorch_latest to executorch.

I recognize that this is a pain and a subtle gotcha. I'll create an issue and link it back here. This is something we've wanted to fix for a while.

[dbort]

@tiger-of-shawn I am going to close this issue in favor of #6475, but please re-open if renaming the directory doesn't fix your problem.