You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<pclass="description">We are a community of developers building free open source tools to secure the Node.js & JavaScript ecosystem. Our area of expertise is <b>SCA</b> (Software Composition Analysis).</p>
34
+
<divclass="header-buttons">
35
+
<ahref="https://github.com/NodeSecure" target="_blank" title="NodeSecure on GitHub" class="view-on-github">
<imgwidth="64" height="64" src="https://img.icons8.com/glyph-neue/64/circled-left-2.png" style="width:28px;height:28px;filter:invert(1) brightness(2);" alt="Back to blog"/>
47
+
<span>Back to blog</span>
48
+
</a>
49
+
</div>
50
+
</div>
51
+
</header>
52
+
<main>
53
+
54
+
<article>
55
+
<divclass="article-content">
56
+
<h1class="article-title">JS-X-Ray 1.0</h1>
57
+
<p>Hi,</p>
58
+
<p>While I was working on the next release (0.6.0) of <ahref="https://github.com/ES-Community/nsecure">Node-Secure</a> I thought that the AST analysis was getting bigger and bigger (and much more complicated too).</p>
59
+
<p>That's why I decided to separate all the analysis from the Node-secure project to allow easier maintenance and future enhancement. This also allows other projects to use my package if they need to!</p>
<p>This is how <strong>JS-X-RAY</strong> was born. I have chosen the word x-ray because in games this is often a feature that allow to see through the walls, I like to imagine my analysis as being able to see through the most common techniques (obfuscation, etc.).</p>
62
+
<h1>The goal</h1>
63
+
<p>One of the primary goals of this package is to be able to find any required Node.js dependencies in a given code. If the analysis is not able to follow a require statement then an <strong>unsafe-import</strong> warning will be throw.</p>
64
+
<p>The more time goes and the more I think to make my code generic to also detect patterns specific to the front.</p>
65
+
<p>So I think the code will evolve in this direction :) </p>
66
+
<h1>Example</h1>
67
+
<hr>
68
+
<h2><strong>Purescript</strong></h2>
69
+
<p>Take the <ahref="https://badjs.org/posts/purescript-installer/">purescript-installer incident</a> and specially the corrupted <ahref="https://badjs.org/posts/purescript-installer/#heading-compromised-version-of-rate-map">rate-map code</a>.</p>
70
+
<blockquote>
71
+
<p>One of the objectives of node-secure is to be able to quickly identify code with warnings and give a bunch of very useful informations to the developer.</p>
72
+
</blockquote>
73
+
<p>In this case node-secure was able to detect the following dependencies:
<p>My AST analysis has detected a Buffer.from and as converted the value to <code>dl-tar</code> itself. In this case an <strong>unsafe-import</strong> will be throw with the file name and the Source Location.</p>
80
+
<hr>
81
+
<h2><strong>Event-stream</strong></h2>
82
+
<p>Take the <ahref="https://badjs.org/posts/event-stream/">Payload A</a> in the event-stream incident.</p>
83
+
<p>So what's going on here?</p>
84
+
<ul>
85
+
<li><ol>
86
+
<li>assign of process and require into new variables.</li>
87
+
</ol>
88
+
</li>
89
+
<li><olstart="2">
90
+
<li>hexa value.</li>
91
+
</ol>
92
+
</li>
93
+
<li><olstart="3">
94
+
<li>code obfuscated (all identifiers have a length of 1).</li>
95
+
</ol>
96
+
</li>
97
+
</ul>
98
+
<p>I'm working on a bench of experimental analysis and warnings to be able to detect similar cases to event-stream incident.</p>
99
+
<pre><codeclass="language-json">[
100
+
{
101
+
"kind": "unsafe-assign",
102
+
"start": { "line": 3, "column": 12 },
103
+
"end": { "line": 3, "column": 23 },
104
+
"value": "require"
105
+
},
106
+
{
107
+
"kind": "unsafe-assign",
108
+
"start": { "line": 4, "column": 12 },
109
+
"end": { "line": 4, "column": 23 },
110
+
"value": "process"
111
+
},
112
+
{
113
+
"kind": "hexa-value",
114
+
"start": { "line": 9, "column": 20 },
115
+
"end": { "line": 9, "column": 44 },
116
+
"value": "./test/data"
117
+
},
118
+
{
119
+
"kind": "short-ids",
120
+
"start": { "line": 0, "column": 0 },
121
+
"end": { "line": 0,"column": 0 },
122
+
"value": 1
123
+
}
124
+
]
125
+
</code></pre>
126
+
<p>However, A lot of packages may be detected as false positives (even if it's always better than nothing 😅). It will surely take time to discover and improve these parameters.</p>
127
+
<h1>Conclusion</h1>
128
+
<p>Still a LOT of work has to be done to be able to achieve an accurate analysis. Right now the analysis is capable of gathering a whole of very useful information (unsafe-regex, unused and missing dependencies etc.).</p>
129
+
<p>I am always very excited to experience new warnings because they can detect patterns and errors that are often (un)common. Step by step they also lead me to a better understanding of the most dangerous patterns of the ecosystem.</p>
130
+
<blockquote>
131
+
<p>For example <strong>90%+ of the false positive</strong> are always generated because of files that was not mean to be published on the npm registry (tests, coverage files, etc.).</p>
<pclass="description">We are a community of developers building free open source tools to secure the Node.js & JavaScript ecosystem. Our area of expertise is <b>SCA</b> (Software Composition Analysis).</p>
34
+
<divclass="header-buttons">
35
+
<ahref="https://github.com/NodeSecure" target="_blank" title="NodeSecure on GitHub" class="view-on-github">
<imgwidth="64" height="64" src="https://img.icons8.com/glyph-neue/64/circled-left-2.png" style="width:28px;height:28px;filter:invert(1) brightness(2);" alt="Back to blog"/>
47
+
<span>Back to blog</span>
48
+
</a>
49
+
</div>
50
+
</div>
51
+
</header>
52
+
<main>
53
+
54
+
<article>
55
+
<divclass="article-content">
56
+
<h1class="article-title">NodeSecure v0.7.0 and v0.8.0@next</h1>
57
+
<p>Hello,</p>
58
+
<p>It's been a few weeks now that I've been working on a new major release for JS-X-Ray. This new version brings a lot of important changes including:</p>
59
+
<ul>
60
+
<li>New warnings names (I've taken the time to think of consistent names).</li>
61
+
<li>New features to detect an obfuscated code (Still experimental though).</li>
62
+
<li>New format for the SourceLocation (an array instead of the ESTree SourceLocation Object).</li>
63
+
<li>Complete documentation for warnings (With explanations on technical implementation when necessary).</li>
64
+
<li>Improvement of the code as a whole (it is much more maintainable).</li>
65
+
<li>Improvement of unit tests.</li>
66
+
</ul>
67
+
<p>The project is completely open-source and accessible on github: <ahref="https://github.com/fraxken/js-x-ray">https://github.com/fraxken/js-x-ray</a> (Remember to star 💖).</p>
68
+
<h1>What is JS-X-Ray?</h1>
69
+
<p>I'll make a summary for the latecomers. (Also feel free to read the other articles in the series to better understand.)</p>
70
+
<p>JS-X-Ray is a free and open-source JavaScript/Node.js SAST scanner. It was mainly built to meet the needs of the <ahref="https://github.com/ES-Community/nsecure">Node-secure</a> project but gradually became independent.</p>
71
+
<p>The project as a whole analyzes JavaScript SourceCode on format AST (Abstract Syntax Tree) and provides a set of information on it including "security" warnings.</p>
72
+
<p>The goal is to quickly identify dangerous patterns (in the given code) for Developers and Security researchers. </p>
73
+
<h1>For who ?</h1>
74
+
<p>As previously mentioned, the project is currently being used as a dependency of other security projects (Like Node-secure).</p>
75
+
<p>This tool is not magic and still requires basic security knowledge to tell the difference between a real problem and a false positive..</p>
76
+
<p>The target of the project is mainly security researchers as well as developers interested in the development of security tools.</p>
77
+
<h1>An example?</h1>
78
+
<p>Let's take a look at one of the previous incidents in the ecosystem (npm). For example the event-stream incident where malicious codes are still accessible <ahref="https://badjs.org/posts/event-stream/">here on badjs</a>.</p>
79
+
<p>We're going to run an analysis on the <ahref="https://badjs.org/posts/event-stream/#heading-payload-c">Payload C</a>.</p>
<p>That's what JS-X-Ray return. We find the dependencies that were required within the script and some warnings:</p>
119
+
<ul>
120
+
<li>Two encoded literals.</li>
121
+
<li>A warning telling us that identifiers in the code are too short (below an average of 1.5).</li>
122
+
</ul>
123
+
<p>What might give us a clue here is the nature of the warnings and the used dependencies...Of course tools such as Node-secure will give you a much better view when the need is to analyse a complete project.</p>
<p>All warnings are explained on the README of the github. Advanced documentation on how they work and how they are implemented can be found <ahref="https://github.com/fraxken/js-x-ray/blob/master/WARNINGS.md">here</a>.</p>
127
+
<table>
128
+
<thead>
129
+
<tr>
130
+
<th>name</th>
131
+
<th>description</th>
132
+
</tr>
133
+
</thead>
134
+
<tbody><tr>
135
+
<td>parsing-error</td>
136
+
<td>An error occured when parsing the JavaScript code with meriyah. It mean that the conversion from string to AST as failed. If you encounter such an error, <strong>please open an issue</strong>.</td>
137
+
</tr>
138
+
<tr>
139
+
<td>unsafe-import</td>
140
+
<td>Unable to follow an import (require, require.resolve) statement/expr.</td>
141
+
</tr>
142
+
<tr>
143
+
<td>unsafe-regex</td>
144
+
<td>A RegEx as been detected as unsafe and may be used for a ReDoS Attack. Under the hood we use the package <strong>safe-regex</strong>.</td>
145
+
</tr>
146
+
<tr>
147
+
<td>unsafe-stmt</td>
148
+
<td>Usage of dangerous statement like <code>eval()</code> or <code>Function("")</code>.</td>
149
+
</tr>
150
+
<tr>
151
+
<td>unsafe-assign</td>
152
+
<td>Assignment of a protected global like <code>process</code> or <code>require</code>.</td>
153
+
</tr>
154
+
<tr>
155
+
<td>encoded-literal</td>
156
+
<td>An encoded literal has been detected (it can be an hexa value, unicode sequence, base64 string etc)</td>
157
+
</tr>
158
+
<tr>
159
+
<td>short-identifiers</td>
160
+
<td>This mean that all identifiers has an average length below 1.5. Only possible if the file contains more than 5 identifiers.</td>
161
+
</tr>
162
+
<tr>
163
+
<td>suspicious-literal</td>
164
+
<td>This mean that the sum of suspicious score of all Literals is bigger than 3.</td>
<td>There's a very high probability that the code is obfuscated...</td>
169
+
</tr>
170
+
</tbody></table>
171
+
<h2>unsafe-import</h2>
172
+
<p>What do we mean when it is impossible to follow an expression or statement? Let's take the following example:</p>
173
+
<pre><codeclass="language-js">function boo() {
174
+
// something is going on here!
175
+
}
176
+
177
+
require(boo());
178
+
</code></pre>
179
+
<p>Here the analysis is not able to follow because it would be too painful and time consuming to know what the function really returns.</p>
180
+
<h2>unsafe-assign</h2>
181
+
<p>A fairly common pattern among hackers is to assign global variables to new variables to hide the use of a require or eval. JS-X-Ray is able to trace the use of these variables and will consider this pattern as dangerous.</p>
182
+
<p>Example:</p>
183
+
<pre><codeclass="language-js">const g = global.process;
184
+
const r = g.mainModule;
185
+
const c = r.require;
186
+
c("http");
187
+
r.require("fs");
188
+
</code></pre>
189
+
<h2>obfuscated-code</h2>
190
+
<p>He's the new kid. However the results are not yet perfect and a lot of work will be necessary in the coming months to allow the detection of more obfuscated codes.</p>
191
+
<ul>
192
+
<li><ahref="https://twitter.com/fraxken/status/1290850085442670593/photo/1">One of my recent tweet on this feature</a>. </li>
193
+
<li><ahref="https://docs.google.com/document/d/11ZrfW0bDQ-kd7Gr_Ixqyk8p3TGvxckmhFH3Z8dFoPhY/edit?usp=sharing">The Google Drive document on JavaScript obfuscated patterns</a>.</li>
194
+
</ul>
195
+
<h1>On the future</h1>
196
+
<p>I wish I could iterate over the entire npm registry. I think that this project could provide us valuable insight on packages and maybe even prevent a lot of malicious code to reach npm users.</p>
197
+
<p>This is already what I do personally with Node-secure which allows me to secure and improve the <ahref="https://github.com/SlimIO">SlimIO</a> solution.</p>
198
+
<p>Beyond the security aspect, this project allows to detect and understand the use of a set of bad patterns/practices. We could also eventually guide and prevent these practices to improve the ecosystem as a whole.</p>
199
+
<p>At the moment I'm investing my free time to work on this project... But I would obviously like to invest myself professionally in it!</p>
200
+
<h1>Conclusion</h1>
201
+
<p>There's still a lot of work to be done. One of the blocking points I'm encountering at the moment is the analysis of common patterns in identifiers (which can be diverse and varied depending on the generation method).</p>
202
+
<p>The current version is not yet implemented on Node-secure and it might take a few weeks (I'm a bit too busy at the moment).</p>
203
+
<p>Hope you enjoy this article to keep you up to date with the developments and progress I have made!</p>
204
+
<p>Thank you for reading this series and see you soon for an article on Node-secure :)</p>
0 commit comments