extend sysimg:

1. more of the 'slowest' imports in MyTB (the ones with ~0.4 s. The remaining all have <0.02 s or so).
2. JLD2 precompile of jldsave and load of a tempfile.

both helped, but just a little (load cached sim went from 13s to 10s. MyTB import a lick faster)

Author: tfiers

Manifest.toml

@@ -566,9 +566,9 @@ version = "1.0.0"
 
 [[deps.JLD2]]
 deps = ["FileIO", "MacroTools", "Mmap", "OrderedCollections", "Pkg", "Printf", "Reexport", "TranscodingStreams", "UUIDs"]
-git-tree-sha1 = "81b9477b49402b47fbe7f7ae0b252077f53e4a08"
+git-tree-sha1 = "6c38bbe47948f74d63434abed68bdfc8d2c46b99"
 uuid = "033835bb-8acc-5ee8-8aae-3f567f8a3819"
-version = "0.4.22"
+version = "0.4.23"
 
 [[deps.JLLWrappers]]
 deps = ["Preferences"]

Project.toml

@@ -4,16 +4,19 @@ ComponentArrays = "b0b7db55-cfe3-40fc-9ded-d10e2dbeff66"
 Cthulhu = "f68482b8-f384-11e8-15f7-abe071a5a75f"
 DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
 DataFramesMeta = "1313f7d8-7da2-5740-9ea0-a2ca25f37964"
+DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
 FilePathsBase = "48062228-2e41-5def-b9a4-89aafe57970f"
 IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a"
+JLD2 = "033835bb-8acc-5ee8-8aae-3f567f8a3819"
 LsqFit = "2fda8390-95c7-5789-9bda-21331edee243"
 MyToolbox = "54cd1024-cafd-4d62-948d-ced4874502bf"
 PackageCompiler = "9b87118b-4619-50d2-8e1e-99f35a4d4d9d"
+Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
 ProfileView = "c46f51b8-102a-5cf2-8d2c-8597cb0e0da7"
-PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
 PyPlot = "d330b81b-6aea-500a-939a-2ce795aea3ee"
 Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
 Sciplotlib = "61be95e5-9550-4d5f-a203-92a5acbc3116"
+StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
 Suppressor = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
 Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
 TerminalLoggers = "5d786b92-1e48-4d6f-9151-6b4477ca9bed"

nb/2022-09-15__Two-pass_conntest_ptp-then-cor.ipynb

@@ -30,8 +30,6 @@ using VoltoMapSim
 
 # ## Params
 
-using ProfileView
-
 p = get_params(
     duration = 10minutes,
     p_conn = 0.04,
@@ -48,14 +46,14 @@ p = get_params(
 
 @time s = cached(sim, [p.sim]);
 
-@profview s = augment(s, p);
+s = augment(s, p);
 
 using PyPlot
 using VoltoMapSim.Plot
 
-@profview plotSTA(1=>1,s,p);
-plotSTA(2=>2,s,p);
-plotSTA(801=>801,s,p);
-plotSTA(802=>802,s,p);
+@time plotSTA(1=>1,s,p);
+# plotSTA(2=>2,s,p);
+# plotSTA(801=>801,s,p);
+# plotSTA(802=>802,s,p);
 
 

nb/2022-09-15__Two-pass_conntest_ptp-then-cor.jl

@@ -5,23 +5,33 @@ The goal is to have to wait less long for package imports and first function cal
 fresh Julia session.
 
 
-### Build
-To build the image, run, in the repo root:
+## Build
+
+To build the image, run, in this directory:
 ```
-julia sysimg/build.jl
+julia build.jl
 ```
 This takes a few minutes.
 
 
-### Use
+## Use
 
 To use the generated system image:
-- On the command line:
+
+- On the command line, in the repo root:
   ```
   julia --sysimg=sysimg/mysis.dll
   ```
-- As a 'kernel' in Jupyter: [relevant IJulia docs](
-    https://julialang.github.io/IJulia.jl/stable/manual/installation/#Installing-additional-Julia-kernels)
-    - Make sure to also add the flag `"--project=@."`.
-    - For me, the generated kernel definition file is located at:
-      `C:\Users\tfiers\AppData\Roaming\jupyter\kernels\julia-preloaded-1.7\kernel.json`
+
+- As a 'kernel' in Jupyter: see the
+  [relevant IJulia docs](
+    https://julialang.github.io/IJulia.jl/stable/manual/installation/#Installing-additional-Julia-kernels).  
+  Make sure to also add the flag `--project=@.`.
+
+
+## Notes
+
+- If on Linux or MacOS, you can replace the `.dll` extension inside these scripts with `.so` or `.dylib` respectively (though leaving it as is will also just work).
+
+- For me, the generated Jupyter kernel definition file is located at:  
+  `C:\Users\tfiers\AppData\Roaming\jupyter\kernels\julia-preloaded-1.7\kernel.json`

sysimg/ReadMe.md

@@ -1,15 +1,20 @@
-using PackageCompiler
+using Pkg
+Pkg.activate("..")
 
+using PackageCompiler
 create_sysimage(
     [
         :Revise,
-        :PyCall,
         :IJulia,
         :PyPlot,
         :DataFrames,
         :ComponentArrays,
+        :DataStructures,
+        :Parameters,
+        :StatsBase,
+        :JLD2,
     ];
-    sysimage_path               = "sysimg/mysys.dll",
-    precompile_execution_file   = "sysimg/to_precompile.jl",
-    script                      = "sysimg/pyplot_delay_init.jl",
+    sysimage_path               = "mysys.dll",
+    precompile_execution_file   = "to_precompile.jl",
+    script                      = "pyplot_delay_init.jl",
 )

sysimg/build.jl

@@ -1,7 +1,19 @@
+using DataStructures, Parameters, StatsBase
+
 using PyPlot, DataFrames, ComponentArrays
 
 df = DataFrame([(a=1, b=2), (a=2, b=2)])
 cv = ComponentVector(a=3, b=8)
 
 fig, ax = plt.subplots()
 ax.plot(df.a, collect(cv))
+
+
+using JLD2
+
+fn = tempname()*".jld2"
+struct MyType
+    x::String
+end
+jldsave(fn; somedata=(; a=2, b=""), p=MyType("yo"))
+load(fn)