Scanning with 0.4.2 fails on Azure Pipelines due to dependency failure

[vpuonti]

Started seeing this failure in Azure Pipelines after latest 0.4.2 version.

pip3 install --upgrade mobsfscan

mobsfscan --exit-warning --sarif --output mobsf-report-android.sarif .

/usr/lib/python3/dist-packages/requests/__init__.py:87: RequestsDependencyWarning: urllib3 (2.2.3) or chardet (4.0.0) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

##[error]Bash exited with code '1'.

Fixed by downgrading to 0.4.1

pip3 install --force-reinstall -v "mobsfscan==0.4.1"

[ajinabraham]

This is probably an environmental issue. Are you doing pip3 install --upgrade mobsfscan on an environment where other Python dependencies are already installed by something in your pipeline as system wide packages? You might want to create a virtualenv before installing mobsfscan to not pollute other dependencies.

[vpuonti]

Hi, seems like I've managed to go from one error to another.

I followed your suggestion of installing the packages in a venv, to no success.

I tried updating to 0.4.3, which ended up in some semgrep related issues.

I manually installed the required semgrep version, but then I received failures from the actual scan. Those failures were new, as the builds ran just fine before.

I mirrored the CI-pipe's environment locally and noticed that indeed, there were new warnings. I see is warnings on every single line where the code triggers android_kotlin_hardcoded, but all of those lines have been ignored using // mobsf-ignore: android_kotlin_hardcoded.

I tried making a simple repro project with a single file containing a match for this rule, but with the ignore-rule in place. It did not raise a warning. But when I run the scan on our main project, the ignored warnings are listed in the output.

I ran the scan one-by-one on the files which trigger the incorrect warnings -> no warnings!

In essence:

• Single file, ignores work fine.
• Whole project, ignores are ignored.

I'm stumped, it is definitely something with 0.4.2, since the main project's ignored lines do not raise warnings on 0.4.1, but with later versions 0.4.2, 0.4.3 and 0.4.4 I can repro this behaviour.

I have not been able to produce a reliable repro for this though.

[ajinabraham]

I tried updating to 0.4.3, which ended up in some semgrep related issues.

This was a bug, but should be fixed in 0.4.4

Can you share a test sample that reproduce the // mobsf-ignore: android_kotlin_hardcoded issue?

[vpuonti]

Managed to get a reliable repro.
Has something to do with scanning folders.
Repro here:
https://github.com/vpuonti/mobsfscan-issue

[ajinabraham]

Thanks for that. I will take a look and get back.