Update analysis library for Rizin v0.4.x #335

haeter525 · 2022-04-23T06:06:35Z

Description

Close #334.
This PR updates the Rizin-based analysis library to work with the current development version of Rizin v0.4.x (commit: de8a5).

It includes the following changes -

Update parser for command axtj
Replace command axffj with pdfj and is.j since Rizin v0.4.x removed it.
Rewrite _get_method_by_address by using is.j to precisely find the method via a given address.
Add support to parse methods with variable-length arguments.
Refactor to use the same code to parse the output of isj and is.j.

Code Changes

quark/core/rzapkinfo.py
quark/utils/tools.py

Test Plans
All tests passed.

codecov-commenter · 2022-04-23T08:17:15Z

Codecov Report

Merging #335 (f7420d2) into master (155e76c) will increase coverage by 0.08%.
The diff coverage is 90.69%.

@@            Coverage Diff             @@
##           master     #335      +/-   ##
==========================================
+ Coverage   78.77%   78.86%   +0.08%     
==========================================
  Files          50       50              
  Lines        3704     3710       +6     
==========================================
+ Hits         2918     2926       +8     
+ Misses        786      784       -2

Flag	Coverage Δ
unittests	`78.86% <90.69%> (+0.08%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
quark/core/rzapkinfo.py	`76.95% <90.24%> (+1.22%)`	⬆️
quark/utils/tools.py	`100.00% <100.00%> (ø)`
tests/core/test_apkinfo.py	`95.61% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 155e76c...f7420d2. Read the comment docs.

pulorsok · 2022-04-24T04:10:39Z

Hi @haeter525 ,
As I mention in PR#336,

However, since our project wants to be more contributor friendly,
I suggest we edit documents for the functions in the codebase.
The documents give a short description of the function and how to use it.
And I think it also will help us to maintain the codebase.

I suggest we add documents for the functions in the codebase.

haeter525 · 2022-04-24T10:25:10Z

@pulorsok. Thanks for the suggestion. I have updated the documents for each function in the files.

sidra-asa · 2022-11-09T02:13:32Z

I performed some tests, but the results are needed to verify.

Result on Ahmyth.apk between core lib with androguard and rizin.

$ diff Ahmyth_androguard Ahmyth_rizin 
8c8
< | 00189.json | Get the content of a SMS message                                                | 100%       | 1     | 1.0    |
---
> | 00189.json | Get the content of a SMS message                                                | 80%        | 1     | 0.5    |
29c29
< | 00193.json | Send a SMS message                                                              | 100%       | 1     | 1.0    |
---
> | 00193.json | Send a SMS message                                                              | 80%        | 1     | 0.5    |
39c39
< | 00002.json | Open the camera and take picture                                                | 100%       | 1     | 1.0    |
---
> | 00002.json | Open the camera and take picture                                                | 80%        | 1     | 0.5    |
53c53
< | 00188.json | Get the address of a SMS message                                                | 100%       | 1     | 1.0    |
---
> | 00188.json | Get the address of a SMS message                                                | 80%        | 1     | 0.5    |
60c60
< | 00005.json | Get absolute path of file and put it to JSON object                             | 100%       | 1     | 1.0    |
---
> | 00005.json | Get absolute path of file and put it to JSON object                             | 80%        | 1     | 0.5    |
72c72
< | 00009.json | Put data in cursor to JSON object                                               | 100%       | 1     | 1.0    |
---
> | 00009.json | Put data in cursor to JSON object                                               | 80%        | 1     | 0.5    |
93c93
< | 00008.json | Check if successfully sending out SMS                                           | 100%       | 1     | 1.0    |
---
> | 00008.json | Check if successfully sending out SMS                                           | 80%        | 1     | 0.5    |
104c104
< | 00004.json | Get filename and put it to JSON object                                          | 100%       | 1     | 1.0    |
---
> | 00004.json | Get filename and put it to JSON object                                          | 80%        | 1     | 0.5    |
123c123
< | 00011.json | Query data from URI (SMS, CALLLOGS)                                             | 100%       | 1     | 1.0    |
---
> | 00011.json | Query data from URI (SMS, CALLLOGS)                                             | 80%        | 1     | 0.5    |
139c139
< | 00026.json | Method reflection                                                               | 100%       | 1     | 1.0    |
---
> | 00026.json | Method reflection                                                               | 60%        | 1     | 0.25   |
151c151
< | 00010.json | Read sensitive data(SMS, CALLLOG) and put it into JSON object                   | 100%       | 1     | 1.0    |
---
> | 00010.json | Read sensitive data(SMS, CALLLOG) and put it into JSON object                   | 80%        | 1     | 0.5    |
178c178
< | 00200.json | Query data from the contact list                                                | 100%       | 1     | 1.0    |
---
> | 00200.json | Query data from the contact list                                                | 80%        | 1     | 0.5    |
180c180
< | 00187.json | Query a URI and check the result                                                | 100%       | 1     | 1.0    |
---
> | 00187.json | Query a URI and check the result                                                | 80%        | 1     | 0.5    |
185c185
< | 00001.json | Initialize bitmap object and compress data (e.g. JPEG) into bitmap object       | 100%       | 1     | 1.0    |
---
> | 00001.json | Initialize bitmap object and compress data (e.g. JPEG) into bitmap object       | 80%        | 1     | 0.5    |
200c200
< | 00201.json | Query data from the call log                                                    | 100%       | 1     | 1.0    |
---
> | 00201.json | Query data from the call log                                                    | 80%        | 1     | 0.5    |
203c203
< | 00190.json | Query a URI and append the result into a string                                 | 80%        | 1     | 0.5    |
---
> | 00190.json | Query a URI and append the result into a string                                 | 60%        | 1     | 0.25   |
206c206
< | 00020.json | Get absolute path of the file and store in string                               | 80%        | 1     | 0.5    |
---
> | 00020.json | Get absolute path of the file and store in string                               | 60%        | 1     | 0.25   |
208c208
< | 00077.json | Read sensitive data(SMS, CALLLOG, etc)                                          | 100%       | 1     | 1.0    |
---
> | 00077.json | Read sensitive data(SMS, CALLLOG, etc)                                          | 80%        | 1     | 0.5    |

Result on 13667fe3b0ad496a0cd157f34b7e0c991d72a4db.apk

Traceback (most recent call last):
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/bin/quark", line 8, in <module>
    sys.exit(entry_point())
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/sasaki/program/python/quark-engine/quark/cli.py", line 315, in entry_point
    data.run(rule_checker)
  File "/Users/sasaki/program/python/quark-engine/quark/core/quark.py", line 469, in run
    if self.check_parameter(
  File "/Users/sasaki/program/python/quark-engine/quark/core/quark.py", line 227, in check_parameter
    pyeval.eval[instruction[0]](instruction)
  File "/Users/sasaki/program/python/quark-engine/quark/evaluator/pyeval.py", line 34, in warp
    func(*args, **kwargs)
  File "/Users/sasaki/program/python/quark-engine/quark/evaluator/pyeval.py", line 276, in INVOKE_VIRTUAL_RANGE
    self._invoke(instruction, look_up=True)
  File "/Users/sasaki/program/python/quark-engine/quark/evaluator/pyeval.py", line 205, in _invoke
    self.ret_type = executed_fuc[executed_fuc.index(")") + 1:]
ValueError: substring not found

Result on 14d9f1a92dd984d6040cc41ed06e273e.apk

Traceback (most recent call last):
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/bin/quark", line 8, in <module>
    sys.exit(entry_point())
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/sasaki/.local/share/virtualenvs/quark-engine-KzHyODU-/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/sasaki/program/python/quark-engine/quark/cli.py", line 315, in entry_point
    data.run(rule_checker)
  File "/Users/sasaki/program/python/quark-engine/quark/core/quark.py", line 469, in run
    if self.check_parameter(
  File "/Users/sasaki/program/python/quark-engine/quark/core/quark.py", line 227, in check_parameter
    pyeval.eval[instruction[0]](instruction)
  File "/Users/sasaki/program/python/quark-engine/quark/evaluator/pyeval.py", line 34, in warp
    func(*args, **kwargs)
  File "/Users/sasaki/program/python/quark-engine/quark/evaluator/pyeval.py", line 276, in INVOKE_VIRTUAL_RANGE
    self._invoke(instruction, look_up=True)
  File "/Users/sasaki/program/python/quark-engine/quark/evaluator/pyeval.py", line 205, in _invoke
    self.ret_type = executed_fuc[executed_fuc.index(")") + 1:]
ValueError: substring not found

haeter525 · 2023-01-17T10:55:29Z

Close this PR as we are developing the core library for the next major release of Rizin (#457).

haeter525 force-pushed the update_parser_for_rizin_0.4 branch from cf176d0 to df3cd1a Compare April 23, 2022 08:08

haeter525 added 3 commits April 23, 2022 16:08

Update parser for Rizin 0.4.x

c5afc4e

Update tests for Rizin 0.4.x

d61e6d4

Update pytest

559b88f

haeter525 force-pushed the update_parser_for_rizin_0.4 branch from df3cd1a to 559b88f Compare April 23, 2022 08:09

haeter525 requested a review from pulorsok April 23, 2022 10:22

haeter525 added the pr-processing-state-05 label Apr 23, 2022

haeter525 added 2 commits April 24, 2022 18:03

Update docstrings for rzapkinfo.py

8c5e925

Update docstring for tools.py

f7420d2

haeter525 mentioned this pull request Apr 24, 2022

Update analysis library to use Multidex support of Rizin #336

Open

haeter525 assigned sidra-asa Nov 9, 2022

sidra-asa mentioned this pull request Jan 17, 2023

Update analysis library for Rizin v0.5.x #457

Closed

haeter525 closed this Jan 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update analysis library for Rizin v0.4.x #335

Update analysis library for Rizin v0.4.x #335

Uh oh!

haeter525 commented Apr 23, 2022 •

edited

Loading

Uh oh!

codecov-commenter commented Apr 23, 2022 •

edited

Loading

Uh oh!

pulorsok commented Apr 24, 2022 •

edited

Loading

Uh oh!

haeter525 commented Apr 24, 2022

Uh oh!

sidra-asa commented Nov 9, 2022

Uh oh!

haeter525 commented Jan 17, 2023

Uh oh!

Uh oh!

Update analysis library for Rizin v0.4.x #335

Update analysis library for Rizin v0.4.x #335

Uh oh!

Conversation

haeter525 commented Apr 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Apr 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pulorsok commented Apr 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haeter525 commented Apr 24, 2022

Uh oh!

sidra-asa commented Nov 9, 2022

Uh oh!

haeter525 commented Jan 17, 2023

Uh oh!

Uh oh!

haeter525 commented Apr 23, 2022 •

edited

Loading

codecov-commenter commented Apr 23, 2022 •

edited

Loading

pulorsok commented Apr 24, 2022 •

edited

Loading