feat: Allow including binary files when auto build is enabled

[osdemah]

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Description

• Arbitrary ignoring of asset files causes build failures since they might be used in test or even directly imported in code (e.g. png/ico files in TS/JS). We should only drop files matching with gitignore rules and some other generally known folders (e.g. node_modules, .git) if auto build is enabled.
• Jar files are not really code files and should not be included for agents that are solely textual.
• Refactor the file ignore heuristic logic to improve readability.
• Add the 1MB size limit per file to match the logic with vscode plugin

Checklist

• My code follows the code style of this project
• I have added tests to cover my changes
• A short description of the change has been added to the CHANGELOG if the change is customer-facing in the IDE.
• I have added metrics for my changes (if required)

License

I confirm that my contribution is made under the terms of the Apache 2.0 license.