Update packaging in AOTI path

[angelayi]

Added a aoti_package path, dependent on pytorch/pytorch#129895. Follow up will be to delete the --output-dso-path.

To export, use the --output-aoti-package-path to specify a file with a .pt2 extension. This will generate an artifact containing all the AOTI generated files

python3 torchchat.py export stories15M --output-aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --device cpu

If we look into the contents of the package, we can see the following:

unzip -l exportedModels/stories15M_artifacts_cpu.pt2

Archive:  exportedModels/stories15M_artifacts_cpu.pt2
  Length      Date    Time    Name
---------  ---------- -----   ----
        1  09-13-2024 08:36   version
        3  09-13-2024 08:36   archive_format
   299452  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.cpp
       26  09-13-2024 08:35   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_metadata.json
   644608  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.o
     1727  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_compile_flags.json
     1738  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_linker_flags.json
173405576  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.so
172941232  09-13-2024 08:36   data/aotinductor/model/cghfy3rmc5pj6omsgyzll5b7btki66najqkyd3544pmitg26ihvj.o
---------                     -------
347294363                     9 files

To run with Python:

python3 torchchat.py generate stories15M --aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --prompt "Hello my name is"

To run with the C++ runner, first build the runner binary:

torchchat/utils/scripts/build_native.sh aoti

To run:

cmake-out/aoti_run exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"

AOTI changes in PyTorch also now allow users to not need to specify --device when generating, as we will save this information during export time as metadata and use it during runtime to determine what device to use. A followup can be to extend the metadata to save model information so that users do not need to specify tokenizer version to use during runtime.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/896

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 32c5807 with merge base f20f5e7 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

Weird that the pin bump in #1144 doesn't have the same errors as we see here....

Side question: Is it possible for us to push -l 2 into the dso as metadata as well or does that require looping through pytorch/pytorch? It's an arg when running the binary, but we know AOT what the arg should be

[byjlw]

This command fails out of the box. I had to create the directory exportedModels for this to run
the command should create the directories in the specified path if they don't exist already.
python3 torchchat.py export stories15M --output-aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --device cpu

[byjlw]

command to build the runner fails
torchchat/utils/scripts/build_native.sh aoti

Building aoti native runner...
Defaulting TORCHCHAT_ROOT to /home/warden/source/torchchat/torchchat/utils/scripts/../../.. since it is unset.
~/source/torchchat ~/source/torchchat
Synchronizing submodule url for 'tokenizer/third-party/abseil-cpp'
Synchronizing submodule url for 'tokenizer/third-party/re2'
Synchronizing submodule url for 'tokenizer/third-party/sentencepiece'
~/source/torchchat
-- VERSION: 0.2.1
-- Not Found TCMalloc: TCMALLOC_LIB-NOTFOUND
-- Using ET BUILD DIR: --[et-build]--
-- TORCHCHAT_ROOT="/home/warden/source/torchchat"
-- Looking for excutorch in /home/warden/source/torchchat/et-build/install
-- Could NOT find executorch (missing: executorch_DIR)
-- Caffe2: CUDA detected: 12.0
-- Caffe2: CUDA nvcc is: /usr/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr
-- Caffe2: Header version is: 12.0
-- Could NOT find nvtx3 (missing: nvtx3_dir) 
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- USE_CUDSS is set to 0. Compiling without cuDSS support
-- USE_CUFILE is set to 0. Compiling without cuFile support
-- Autodetected CUDA architecture(s):  8.9 8.6
-- Added CUDA NVCC flags for: -gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_86,code=sm_86
-- Configuring done (0.3s)
-- Generating done (0.1s)
-- Build files have been written to: /home/warden/source/torchchat/cmake-out
[1/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_strings_internal.a
[2/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_cordz_functions.a
[3/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/win32_waiter.cc.o
[4/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_strings.a
[5/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/debugging/libabsl_symbolize.a
[6/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_str_format_internal.a
[7/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/crc/libabsl_crc32c.a
[8/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/crc/libabsl_crc_cord_state.a
[9/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_cord_internal.a
[10/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/types/CMakeFiles/bad_optional_access.dir/bad_optional_access.cc.o
[11/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_posix.cc.o
[12/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/types/CMakeFiles/bad_variant_access.dir/bad_variant_access.cc.o
[13/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/types/libabsl_bad_optional_access.a
[14/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_marshalling.a
[15/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/types/libabsl_bad_variant_access.a
[16/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_commandlineflag.a
[17/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/hash/libabsl_hash.a
[18/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_private_handle_accessor.a
[19/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o
[20/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/civil_time.dir/internal/cctz/src/civil_time_detail.cc.o
[21/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_fixed.cc.o
[22/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/zone_info_source.cc.o
[23/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/time/libabsl_civil_time.a
[24/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/perl_groups.cc.o
[25/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_libc.cc.o
[26/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_lookup.cc.o
[27/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/graphcycles_internal.dir/internal/graphcycles.cc.o
[28/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_if.cc.o
[29/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/synchronization/libabsl_graphcycles_internal.a
[30/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_format.cc.o
[31/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/sem_waiter.cc.o
[32/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/pthread_waiter.cc.o
[33/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/kernel_timeout_internal.dir/internal/kernel_timeout.cc.o
[34/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/blocking_counter.cc.o
[35/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/clock.cc.o
[36/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/format.cc.o
[37/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/barrier.cc.o
[38/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/per_thread_sem.cc.o
[39/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/unicode_casefold.cc.o
[40/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/futex_waiter.cc.o
[41/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/notification.cc.o
[42/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/waiter_base.cc.o
[43/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/util/rune.cc.o
[44/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/create_thread_identity.cc.o
[45/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/unicode_groups.cc.o
[46/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/stdcpp_waiter.cc.o
[47/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_impl.cc.o
[48/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o
[49/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/io_win32.cc.o
[50/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/time.cc.o
[51/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/civil_time.cc.o
[52/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/util/strutil.cc.o
[53/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o
[54/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/duration.cc.o
[55/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/mutex.cc.o
[56/127] Building CXX object tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o
FAILED: tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o 
/usr/bin/c++  -I/home/warden/source/torchchat/tokenizer -I/home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src -I/home/warden/source/torchchat/tokenizer/third-party/re2 -I/home/warden/source/torchchat/tokenizer/third-party/abseil-cpp -D_GLIBCXX_USE_CXX11_ABI=0 -MD -MT tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o -MF tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o.d -o tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o -c /home/warden/source/torchchat/tokenizer/tiktoken.cpp
In file included from /home/warden/source/torchchat/tokenizer/tiktoken.cpp:18:
/home/warden/source/torchchat/tokenizer/base64.h:37:11: error: ‘uint32_t’ does not name a type
   37 | constexpr uint32_t DECODE_TABLE[] = {
      |           ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:29:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
   28 | #include <string>
  +++ |+#include <cstdint>
   29 | #include <string_view>
/home/warden/source/torchchat/tokenizer/base64.h:57:13: error: variable or field ‘validate’ declared void
   57 | inline void validate(uint32_t v) {
      |             ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:57:22: error: ‘uint32_t’ was not declared in this scope
   57 | inline void validate(uint32_t v) {
      |                      ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:57:22: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:70:3: error: ‘uint32_t’ was not declared in this scope
   70 |   uint32_t val = 0;
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:70:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:72:3: error: ‘uint8_t’ was not declared in this scope
   72 |   uint8_t c = input[0];
      |   ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:72:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:73:12: error: ‘DECODE_TABLE’ was not declared in this scope
   73 |   auto v = DECODE_TABLE[c];
      |            ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:73:25: error: ‘c’ was not declared in this scope
   73 |   auto v = DECODE_TABLE[c];
      |                         ^
/home/warden/source/torchchat/tokenizer/base64.h:74:3: error: ‘validate’ was not declared in this scope
   74 |   validate(v);
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:75:3: error: ‘val’ was not declared in this scope
   75 |   val = v;
      |   ^~~
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode_1_padding(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:105:3: error: ‘uint32_t’ was not declared in this scope
  105 |   uint32_t val = 0;
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:105:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:107:3: error: ‘uint8_t’ was not declared in this scope
  107 |   uint8_t c = input[0];
      |   ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:107:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:108:12: error: ‘DECODE_TABLE’ was not declared in this scope
  108 |   auto v = DECODE_TABLE[c];
      |            ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:108:25: error: ‘c’ was not declared in this scope
  108 |   auto v = DECODE_TABLE[c];
      |                         ^
/home/warden/source/torchchat/tokenizer/base64.h:109:3: error: ‘validate’ was not declared in this scope
  109 |   validate(v);
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:110:3: error: ‘val’ was not declared in this scope
  110 |   val = v;
      |   ^~~
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode_2_padding(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:131:3: error: ‘uint32_t’ was not declared in this scope
  131 |   uint32_t val = 0;
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:131:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:133:3: error: ‘uint8_t’ was not declared in this scope
  133 |   uint8_t c = input[0];
      |   ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:133:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:134:12: error: ‘DECODE_TABLE’ was not declared in this scope
  134 |   auto v = DECODE_TABLE[c];
      |            ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:134:25: error: ‘c’ was not declared in this scope
  134 |   auto v = DECODE_TABLE[c];
      |                         ^
/home/warden/source/torchchat/tokenizer/base64.h:135:3: error: ‘validate’ was not declared in this scope
  135 |   validate(v);
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:136:3: error: ‘val’ was not declared in this scope
  136 |   val = v;
      |   ^~~
[57/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_info.cc.o
[58/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/parse.cc.o
[59/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/bytestream.cc.o
[60/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/simplify.cc.o
[61/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/statusor.cc.o
[62/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_enum_util.cc.o
[63/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/common.cc.o
[64/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prefilter.cc.o
[65/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/onepass.cc.o
[66/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/status.cc.o
[67/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/tostring.cc.o
[68/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/set.cc.o
[69/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/nfa.cc.o
[70/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arena.cc.o
[71/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/int128.cc.o
[72/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringpiece.cc.o
[73/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/implicit_weak_message.cc.o
[74/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o
[75/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringprintf.cc.o
[76/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arenastring.cc.o
[77/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/coded_stream.cc.o
[78/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/re2.cc.o
[79/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prog.cc.o
[80/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/compile.cc.o
[81/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/parse_context.cc.o
[82/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/message_lite.cc.o
In file included from /usr/include/string.h:548,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/port.h:39,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/macros.h:34,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/common.h:46,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/message_lite.h:45,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/third_party/protobuf-lite/message_lite.cc:36:
In function ‘void* memcpy(void*, const void*, size_t)’,
    inlined from ‘google::protobuf::uint8* google::protobuf::io::EpsCopyOutputStream::WriteRaw(const void*, int, google::protobuf::uint8*)’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/io/coded_stream.h:699:16,
    inlined from ‘virtual google::protobuf::uint8* google::protobuf::internal::ImplicitWeakMessage::_InternalSerialize(google::protobuf::uint8*, google::protobuf::io::EpsCopyOutputStream*) const’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/implicit_weak_message.h:85:28,
    inlined from ‘bool google::protobuf::MessageLite::SerializePartialToZeroCopyStream(google::protobuf::io::ZeroCopyOutputStream*) const’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/third_party/protobuf-lite/message_lite.cc:419:30:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:33: warning: ‘void* __builtin___memcpy_chk(void*, const void*, long unsigned int, long unsigned int)’ specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
   29 |   return __builtin___memcpy_chk (__dest, __src, __len,
      |          ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
   30 |                                  __glibc_objsize0 (__dest));
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~
[83/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prefilter_tree.cc.o
[84/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/dfa.cc.o
[85/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/regexp.cc.o
[86/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_util.cc.o
[87/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_table_driven_lite.cc.o
[88/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/repeated_field.cc.o
[89/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/extension_set.cc.o
ninja: build stopped: subcommand failed.

[angelayi]

the command should create the directories in the specified path if they don't exist already.

That should be fixed with pytorch/pytorch#138919

[larryliu0820]

the command should create the directories in the specified path if they don't exist already.

That should be fixed with pytorch/pytorch#138919

Seems like we need to bump the pin to pytorch as well?

[Jack-Khuu]

https://github.com/pytorch/torchchat/pull/1319/files

There's a bump here that's not actively being worked on, seems like it's not a free bump (+ the infra issue we're seeing here)

[byjlw]

currently running into this issue when trying to run using the runner.

(.venv) warden@Vikander:~/source/torchchat$ cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"
2024-11-01:14:45:53,463 INFO     [sdpa_with_kv_cache.py:29] Loading custom ops library: /home/warden/source/torchchat/.venv/lib/python3.11/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.so
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to initialize zip archive: file open failed
Aborted (core dumped)

[byjlw]

currently running into this issue when trying to run using the runner.

(.venv) warden@Vikander:~/source/torchchat$ cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"
2024-11-01:14:45:53,463 INFO     [sdpa_with_kv_cache.py:29] Loading custom ops library: /home/warden/source/torchchat/.venv/lib/python3.11/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.so
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to initialize zip archive: file open failed
Aborted (core dumped)

path issue. Just need to update the instructions in the first comment so it's

cmake-out/aoti_run exportedModels/stories15M_artifacts_cpu.pt2 -z python3 torchchat.py where stories15M /tokenizer.model -l 2 -i "Once upon a time"
and not
cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z python3 torchchat.py where stories15M /tokenizer.model -l 2 -i "Once upon a time"