wget with DCE?

[mas2tg]

• ns3 version (cat $NS3FOLDER/VERSION or the commit hash): 3-dev
• DCE version (git log|head -n1): 94de508
• Operating system/version (lsb_release -a): Ubuntu 16.04.7 LTS
• Libc version (ldd -v): 2.23-0ubuntu11.2
• compiler version (gcc -v/g++ -v/clang -v/clang++ -v): gcc 5.4.0

Description of the problem

I was trying to run wget with DCE. To do that, I have downloaded the source and added the necessary flags. I have also modified the dce-emu-ping script to use instead wget. My problem is when I run that script, it complains about a missing 0002thread.so.2, which I suspect was created during waf --configure ... being renamed from libpthread.so.2. I happen to fix this issue by copying 0002thread.so.2 from ./elf-cache/0/ to ./build/lib/. This was also the case for 0009rypto.so.1.0.0.

Now, the ./waf --run dce-emu-wget fails at ./model/elf-ldd.cc:231, because the program is trying to access m_depth, but it segfaults, probably because SharedLibrary is null. Any ideas as to why this is happening/pointers to solve this problem?

Here are the values of a few variables I thought relevant:

Thread 3 "dce-emu-wget" hit Breakpoint 2, ns3::SharedLibrary::CalcDepth (this=0x7fffe000d660, max=3, mapScanned=...) at ../model/elf-ldd.cc:247
247	      if (!mapScanned[(*i).second]->CalcDepth (max - 1, mapScanned))
(gdb) p mapScanned[(*i).second]
$3 = (std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ns3::SharedLibrary*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, ns3::SharedLibrary*> > >::mapped_type &) @0x7fffe00142a0: 0x0
(gdb) p (*i).second
$4 = {static npos = 18446744073709551615, 
  _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    _M_p = 0x7fffe000d8b0 "/home/wahoo/Documents/research/ns-3-dce/build/lib/libcrypto.so.1.0.0"}, _M_string_length = 68, {
    _M_local_buf = "D\000\000\000\000\000\000\000L\214N\360\377\177\000", _M_allocated_capacity = 68}}
(gdb) c
Continuing.

Thread 3 "dce-emu-wget" hit Breakpoint 1, ns3::SharedLibrary::CalcDepth (this=0x0, max=2, mapScanned=...) at ../model/elf-ldd.cc:231
231	  if (m_depth >= 0)
(gdb) p this
$5 = (ns3::SharedLibrary * const) 0x0

Dependencies of ./build/lib/libcrypto.so.1.0.0

$ ldd build/lib/libcrypto.so.1.0.0 
	linux-vdso.so.1 =>  (0x00007fffe55a0000)
	0005l.so.2 => not found
	0001.so.6 => /usr/local/lib/0001.so.6 (0x00007f14836f3000)

Stack trace

#0  ns3::SharedLibrary::CalcDepth (this=0x0, max=2, mapScanned=...) at ../model/elf-ldd.cc:231
#1  0x00007ffff790dac9 in ns3::SharedLibrary::CalcDepth (this=0x7fffe000d660, max=3, mapScanned=...) at ../model/elf-ldd.cc:247
#2  0x00007ffff790f2b8 in ns3::ElfLdd::Loop (this=0x7ffff7f7a5a0, s=..., f=...) at ../model/elf-ldd.cc:437
#3  0x00007ffff790fbdf in ns3::ElfLdd::ElfLdd (this=0x7ffff7f7a5a0, s=..., f=...) at ../model/elf-ldd.cc:479
#4  0x00007ffff78ae204 in ns3::ElfDependencies::NewGather (this=0x7ffff7f7a890, sName=..., fullname=...) at ../model/elf-dependencies.cc:59
#5  0x00007ffff78ae002 in ns3::ElfDependencies::ElfDependencies (this=0x7ffff7f7a890, filename=..., failsafe=false)
    at ../model/elf-dependencies.cc:47
#6  0x00007ffff78bb0f3 in ns3::CoojaLoader::LoadModule (this=0x636670, filename=..., flag=256, failsafe=false)
    at ../model/cooja-loader-factory.cc:222
#7  0x00007ffff78bacc0 in ns3::CoojaLoader::Load (this=0x636670, filename=..., flag=256, failsafe=false) at ../model/cooja-loader-factory.cc:173
#8  0x00007ffff780a733 in ns3::DceManager::LoadMain (ld=0x636670, filename=..., proc=0x6b2a80, err=@0x7ffff7f7ab74: 0)
    at ../model/dce-manager.cc:1296
#9  0x00007ffff7802c4e in ns3::DceManager::PrepareDoStartProcess (current=0x6a9210) at ../model/dce-manager.cc:267
#10 0x00007ffff78031fc in ns3::DceManager::DoStartProcess (context=0x6a9210) at ../model/dce-manager.cc:292
#11 0x00007ffff78a6148 in ns3::TaskManager::Trampoline (context=0x6a93b0) at ../model/task-manager.cc:275
#12 0x00007ffff78a2398 in ns3::PthreadFiberManager::Run (arg=0x6b3030) at ../model/pthread-fiber-manager.cc:402
#13 0x00007fffefc926ba in start_thread (arg=0x7ffff7f7b700) at pthread_create.c:333
#14 0x00007fffef9c84dd in clone () from /home/wahoo/Documents/research/ns-3-dce/build/lib/libc.so.6

Output of ./waf configure

Setting top to                           : /home/wahoo/Documents/research/ns-3-dce 
Setting out to                           : /home/wahoo/Documents/research/ns-3-dce/build 
Checking for 'gcc' (C compiler)          : /usr/bin/gcc 
Checking for cc version                  : 5.4.0 
Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 
Checking for program 'pkg-config'        : /usr/bin/pkg-config 
Checking for pkg-config version >= '0.0.0' : yes 
Checking for -Wl,--soname=foo              : yes 
pcfiles: ['/home/wahoo/Documents/research/ns-3-allinone/ns-3-dev/build/src/core/libns3-dev-core-debug.pc']
len(pcfiles): 1
Checking for libns3-dev-core-debug (mandatory) : yes 
Checking for libns3-dev-network-debug (mandatory) : yes 
Checking for libns3-dev-internet-debug (mandatory) : yes 
Checking for libns3-dev-point-to-point-debug (optional) : yes 
Checking for libns3-dev-tap-bridge-debug (optional)     : yes 
Checking for libns3-dev-netanim-debug (optional)        : yes 
Checking for libns3-dev-wifi-debug (optional)           : yes 
Checking for libns3-dev-csma-debug (optional)           : yes 
Checking for libns3-dev-mobility-debug (optional)       : yes 
Checking for libns3-dev-point-to-point-layout-debug (optional) : yes 
Checking for libns3-dev-mpi-debug (optional)                   : not found 
Checking for libns3-dev-lte-debug (optional)                   : yes 
Checking for libns3-dev-visualizer-debug (optional)            : not found 
Checking for libns3-dev-applications-debug (optional)          : yes 
Checking for libns3-dev-fd-net-device-debug (optional)         : yes 
Checking for header stdint.h                                   : yes 
Checking for header inttypes.h                                 : yes 
Checking for header sys/inttypes.h                             : not found 
Checking for header sys/types.h                                : yes 
Checking for header sys/stat.h                                 : yes 
Checking for header dirent.h                                   : yes 
Checking for library dl                                        : yes 
Checking for glibc get_cpu_features                            : yes 
Checking for glibc secure_getenv                               : yes 
Checking for glibc explicit_bzero                              : no 
Checking for header valgrind/valgrind.h                        : not found 
Checking for header valgrind/memcheck.h                        : not found 
Checking for header sim.h                                      : yes 
Checking for 'gcc' (C compiler)                                : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)                              : /usr/bin/g++ 
Checking for 'gcc' (C compiler)                                : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)                              : /usr/bin/g++ 
Checking for libns3-dev-flow-monitor-debug (mandatory)         : yes 
Checking for header hook-manager.h                             : not found 
Checking for program 'valgrind'                                : not found 
Checking for program 'doxygen'                                 : not found 
Checking for header netinet/sctp.h                             : not found 
Checking for 'gcc' (C compiler)                                : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)                              : /usr/bin/g++ 
Checking for program 'python'                                  : /usr/bin/python3 
Checking for python version >= 2.3                             : 3.5.2 
python-config                                                  : not found 
Checking for library python3.5m in LIBDIR                      : not found 
Checking for library python3.5m in python_LIBPL                : yes 
Checking for header Python.h                                   : Distutils not installed? Broken python installation? Get python-config now! 
---- Summary of optional NS-3 features:
Static build                  : not enabled (option --enable-static not selected)
Logging                       : enabled
Assert checks                 : enabled
Code coverage                 : not enabled (option --enable-gcov not selected)
Example programs              : enabled
Test programs                 : enabled
Debug Symbols                 : enabled
ELF magic loader              : not enabled (elf-loader not found)
Aspect-based tracing          : not enabled (libaspect not found)
sctp-tools-dev                : not enabled (sctp-tools (netinet/sctp.h) not found)
Python Bindings               : not enabled (Python library or headers missing)

Steps to reproduce

I can provide more information if necessary (e.g., which wget I am using, my steps installing it, etc).

Other questions

1. What is the purpose of renaming so libraries (e.g from libc-ns3.so to 0001.so.6)?
2. After running ./waf configure && ./waf build , ./waf run fails because it complains about missing so dependencies libraries. By calling ldd ./build/lib/libc-ns3.so (for example) I notice that it wasn't statically linked. To fix this problem, I have to copy a statically linked version from ./elf-cache/0/ to ./build/lib/ before trying to run anything. Does anyone have any idea why this is?

I really appreciate all the help and any pointers you can provide!

[mas2tg]

I actually got past the problem with the stack trace above. I just set up the project within Visual Studio and the program doesn't crash at CalcDepth anymore (not exactly sure why).
Dce manages to start wget, but the problem now is that it crashes when calling clock_gettime() which is defined in libc-ns3.h. I know this might be a bit of a shot in the dark, but has anyone seen this problem before? Any help is appreciated.

Failing statement

Thread 3 "dce-emu-wget" received signal SIGSEGV, Segmentation fault.
__mktime_internal (tp=0x1, convert=0x7fffef97c9a0 <__localtime_r>, offset=0x7fffefc87a38 <localtime_offset>) at mktime.c:369

Corresponding stack trace

libc.so.6!__mktime_internal(struct tm * tp, struct tm *(*)(const time_t *, struct tm *) convert, time_t * offset) (/build/glibc-e6zv40/glibc-2.23/time/mktime.c:369)
libc-ns3.so!clock_gettime(clockid_t a0, timespec * a1) (/home/wahoo/Documents/research/ns-3-dce/model/libc-ns3.h:398)
wget!posix_measure(ptimer_system_time * pst) (/home/wahoo/Documents/research/wget-1.21/src/ptimer.c:169)
wget!ptimer_reset(struct ptimer * pt) (/home/wahoo/Documents/research/wget-1.21/src/ptimer.c:346)
wget!ptimer_new() (/home/wahoo/Documents/research/wget-1.21/src/ptimer.c:325)
wget!main(int argc, char ** argv) (/home/wahoo/Documents/research/wget-1.21/src/main.c:1372)
libns3-dce.so!ns3::DceManager::DoStartProcess(void * context) (/home/wahoo/Documents/research/ns-3-dce/model/dce-manager.cc:299)
libns3-dce.so!ns3::TaskManager::Trampoline(void * context) (/home/wahoo/Documents/research/ns-3-dce/model/task-manager.cc:275)
libns3-dce.so!ns3::PthreadFiberManager::Run(void * arg) (/home/wahoo/Documents/research/ns-3-dce/model/pthread-fiber-manager.cc:402)
libpthread.so.0!start_thread(void * arg) (/build/glibc-e6zv40/glibc-2.23/nptl/pthread_create.c:333)
libc.so.6!clone() (/build/glibc-e6zv40/glibc-2.23/sysdeps/unix/sysv/linux/x86_64/clone.S:109)

A quick google search for glibc-2.23 source code shows the following snippet in time/mktime.c

time_t
__mktime_internal (struct tm *tp,
		   struct tm *(*convert) (const time_t *, struct tm *),
		   time_t *offset)
{
  time_t t, gt, t0, t1, t2;
  struct tm tm;

  /* ... */
  int sec = tp->tm_sec;
  int min = tp->tm_min;
  int hour = tp->tm_hour;
  int mday = tp->tm_mday; 
  int mon = tp->tm_mon; // <= this is line 369; seg fault takes place here 
  int year_requested = tp->tm_year;
  int isdst = tp->tm_isdst;

[mas2tg]

It turns out my libns3-dce.so at /usr/local/lib/ was an older version, and for some reason it was calling mktime as opposed to dce's overriden clock_gettime (see stack trace above). I have copied that library from ./build/lib/ to /usr/local/lib/, solving the problem.

My problem now is that I don't understand why the build is generating a dynamically linked libc-ns3.so, which I need to copy to /usr/local/lib/ (since that file now contains a symbol to a missing system call, which I have added to libc-ns3.h). This is an issue because that library being dynamically linked breaks the execution of the simulation.

[tomhenderson]

I avoid putting any DCE libraries into system paths. They install to $BAKE_HOME_DIR/build/lib, so I put that on my LD_LIBRARY_PATH; from bake top-level-directory, I do:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:`pwd`/build/lib

In your case, if you also have potential conflicts in /usr/local/lib, you may want to switch the order.

[tomhenderson]

When I use DCE, I set these variables:

export PATH=$PATH:`pwd`/build/bin
export DCE_PATH=`pwd`/build/bin_dce:`pwd`/build/sbin
export PYTHONPATH=$PYTHONPATH:`pwd`/build/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:`pwd`/build/lib

[mas2tg]

Thank you for the help!
Yeah, my DCE wasn't happy with my LD_LIBRARY_PATH, but I added the correct the path for the generated libraries, and DCE stopped looking at system paths.

My main problem was that the libc-ns3 shared library was being dynamically linked, requiring libdl.so.2 and libc.so.6. When DCE attempted to import the dependencies for that library, it would crash at ElfCache::Add(), since it wasn't expecting real shared libraries.
I don't understand why libc-ns3.so was only statically linked the first time I built the project. All subsequent builds linked it dynamically, resulting in the crash. I solved the problem by adding /usr/lib/x86_64-linux-gnu/libdl.a as a compiler argument in the main wscript.

And after adding a few missing system calls to libc-ns3.h, I successfully ran wget with DCE!
P.S.: I am using waf as opposed to bake when building my project.

[mas2tg]

Let me know if you would like me to do a pull request for the changes in libc-ns3.h.

Here you can find all changes. Give me a heads up, if you see something there that will make the project better for waf.