-
|
Hello, I have used Jint to implement some data import and data query work in OrchardCore. But in the process of processing data, I found that the memory occupied by the script has not been released after each execution. In OrchardCore it is used as follows public IScriptingScope CreateScope(IEnumerable<GlobalMethod> methods, IServiceProvider serviceProvider, IFileProvider fileProvider, string basePath)
{
var engine = new Engine();
foreach (var method in methods)
{
engine.SetValue(method.Name, method.Method(serviceProvider));
}
return new JavaScriptScope(engine, serviceProvider);
}
public object Evaluate(IScriptingScope scope, string script)
{
static void ThrowInvalidScopeTypeException()
{
throw new ArgumentException($"Expected a scope of type {nameof(JavaScriptScope)}", nameof(scope));
}
if (scope is not JavaScriptScope jsScope)
{
ThrowInvalidScopeTypeException();
}
var parsedAst = _memoryCache.GetOrCreate(script, static entry => Engine.PrepareScript((string) entry.Key));
var result = jsScope.Engine.Evaluate(parsedAst).ToObject();
return result;
}It's called like this in my code var script = $@"var parameters={JsonConvert.SerializeObject(parameters)};
{extDbQuery.Scripts}";
var engine = _scriptingManager.GetScriptingEngine("js");
var scope = (JavaScriptScope)engine.CreateScope(_scriptingManager.GlobalMethodProviders.SelectMany(x => x.GetMethods()), _serviceProvider, null, null);
try
{
dynamic jsValue =scope.Engine.Evaluate(script);
//dynamic jsValue = _scriptingManager.Evaluate("js:"+script,null,null,null);
I've seen some issues about it and there doesn't seem to be any mention of how to completely free an instance of an engine that is no longer in use In my case, it seems possible to work around memory limitations by limiting the number of data rows, such as limiting users to importing only 10,000 data at a time. Every time I create a new engine instance, but I don't know how to fully release them. |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 15 replies
-
|
When I execute the following code,that the memory is immediately reclaimed var arr = Array(10_000_000).fill(new Array(10_000_000));
return arr.length
but in the other script, no collection is performed
this is my testing code : var insertList = [];
var total = 0, start = 0;
while (start < 1_000_000) {
insertList.push({
"CnTy": "edfgwegweg",
"CreationTime": "234123t'rtr",
"CreatorUserId": 1,
"IsDeleted": 0,
"Soldto": "2342341",
"MarkFor": null,
"SalesOrg": "123",
"Material": "123124",
"Remark": "234234",
"ValidFrom": "124123123",
"ValidTo": "223423412300Z",
"ListPriceType": null,
"Currency": "CNY",
"ScaleQty": 1,
"PriceFrom": 2,
"CustomerGroup": null,
"SOA": "123124",
"UoM": "234234123",
"Unit": 23,
"Amount": 123123.124
})
start++
}
var output = {
total: insertList.length
}
insertList = null
return output |
Beta Was this translation helpful? Give feedback.
-
|
Running code below as using Jint;
const string script = """
var insertList = [];
var total = 0, start = 0;
while (start < 1_000_000) {
insertList.push({
"CnTy": "edfgwegweg",
"CreationTime": "234123t'rtr",
"CreatorUserId": 1,
"IsDeleted": 0,
"Soldto": "2342341",
"MarkFor": null,
"SalesOrg": "123",
"Material": "123124",
"Remark": "234234",
"ValidFrom": "124123123",
"ValidTo": "223423412300Z",
"ListPriceType": null,
"Currency": "CNY",
"ScaleQty": 1,
"PriceFrom": 2,
"CustomerGroup": null,
"SOA": "123124",
"UoM": "234234123",
"Unit": 23,
"Amount": 123123.124
})
start++
}
var output = {
total: insertList.length
}
insertList = null
return output
""";
var engine = new Engine();
for (var i = 0; i < 1_000_000_000; ++i)
{
engine.Execute(script);
} |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, I tried to modify your code, this code does free some memory, but after the execution is completed, there is still 2GB of memory will not be freed, is there any way to reclaim the 2GB of memory?
public static void Main(string[] args)
{
TestExcute();
while (true)
{
Thread.Sleep(1000 * 2);
}
Console.ReadKey();
}
public static void TestExcute()
{
MyExecuter myExecuter = new MyExecuter();
for (var i = 0; i < 2; ++i)
{
myExecuter.Execute(script);
}
GC.Collect();
Thread.Sleep(1000 * 2);
}
public class MyExecuter
{
static Engine engine = new Engine();
public void Execute(string script)
{
engine.Execute(script);
}
} |
Beta Was this translation helpful? Give feedback.
-
|
The "Diagnostic Tools" "Process Memory" graph shows the Private Bytes of the entire process, if you hover your mouse on "Private Bytes" it will display a tooltip:
I tested a modified version of your script in "a"+"b"+3+new Array(1).fill(1).map(a=>a+1).push({a:1}) // warmup
System.GC.Collect()
// take snapshot
var insertList = []; var total = 0, start = 0; while (start < 10000) { insertList.push({ "CnTy": "edfgwegweg", "CreationTime": "234123t'rtr", "CreatorUserId": 1, "IsDeleted": 0, "Soldto": "2342341", "MarkFor": null, "SalesOrg": "123", "Material": "123124", "Remark": "234234", "ValidFrom": "124123123", "ValidTo": "223423412300Z", "ListPriceType": null, "Currency": "CNY", "ScaleQty": 1, "PriceFrom": 2, "CustomerGroup": null, "SOA": "123124", "UoM": "234234123", "Unit": 23, "Amount": 123123.124 }); start++; }; var output = { total: insertList.length }; return output
// take snapshot
System.GC.Collect()
// take snapshot
insertList = null;
// take snapshot
System.GC.Collect()
// take snapshot
From the snapshots we can see
As for why "Private bytes" didn't get released back to the OS, I don't know enough about memory management on Windows/.NET to comment. |
Beta Was this translation helpful? Give feedback.
-
|
I have faced the same problem. And If I understand it correctly, there is no memory leak but bizarre behavior of GC. Result of Result of gcroot showed that there are no objects that hold Engine. I called GC manually, and it cleaned everything after some time. It looks like a problem with GC and some data structures |
Beta Was this translation helpful? Give feedback.
-
|
You can try to tune the GC parameters: https://learn.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector Generally if your computer has memory available GC might not be in too much hurry to free things. |
Beta Was this translation helpful? Give feedback.
-
|
The description is poor for people who are not so deep in GC. However, I tried the most obvious options, like memory limits, and didn't notice any changes. |
Beta Was this translation helpful? Give feedback.
-
|
You should also try to disable object identity tracking under interop: Line 248 in 4839fd1 |
Beta Was this translation helpful? Give feedback.
-
|
When I disable Tracking and remove Constraints, the problem is gone. If any of these are present - my app eats gigabytes of memory. public object EvaluateScript(string script)
{
// if (_engine == null)
{
var options = new Options { Strict = true };
options.Interop.TrackObjectWrapperIdentity = false;
// options.LimitMemory(4_000_000);
// options.TimeoutInterval(TimeSpan.FromSeconds(30));
_engine = new Engine(options);
}
_engine.SetValue("scope", Scope);
return _engine.Evaluate(script).ToObject();
} |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, it worked fine in my test code.
|
Beta Was this translation helpful? Give feedback.
-
|
More testing, it seems, is the only way memory can be freed TestExcute();
while (true)
{
GC.Collect();
Task.Delay(1000).GetAwaiter().GetResult();
}
|
Beta Was this translation helpful? Give feedback.
-
|
I'm starting today lean towards having |
Beta Was this translation helpful? Give feedback.
-
|
There's now a new release 3.0.0-beta-2046 which contains manual clearing of object identity tracking in Engine's |
Beta Was this translation helpful? Give feedback.
-
|
Seems to be a different thing than the identity tracking as you still have |
Beta Was this translation helpful? Give feedback.
-
|
Yes I tried to remove it and it did go down to 1.8GB |
Beta Was this translation helpful? Give feedback.
-
|
I think we can call this a feature until someone in the community has energy to dig deeper and possibly submit some patches. |
Beta Was this translation helpful? Give feedback.
-
|
I can confirm. There are no changes in memory usage. |
Beta Was this translation helpful? Give feedback.
-
|
Memory is not immediately released |
Beta Was this translation helpful? Give feedback.
-
|
In latest release the identity tracking has been disabled by default. |
Beta Was this translation helpful? Give feedback.
You should also try to disable object identity tracking under interop:
jint/Jint/Options.cs
Line 248 in 4839fd1