Add to wiki: How to get the tumblr oauth tokens

[ItsIgnacioPortal]

The tumblr oauth docs are insanely complicated. I've found a far easier way of getting an Oauth Token and the Oauth Token Secret

Install requests-oauthlib

python -m pip install requests_oauthlib

Run the following in a python console:

# Credentials from the application page
key = '<the app key>'
secret = '<the app secret>'

# OAuth URLs given on the application page
request_token_url = 'http://www.tumblr.com/oauth/request_token'
authorization_base_url = 'http://www.tumblr.com/oauth/authorize'
access_token_url = 'http://www.tumblr.com/oauth/access_token'

# Fetch a request token
from requests_oauthlib import OAuth1Session
tumblr = OAuth1Session(key, client_secret=secret, callback_uri='http://www.tumblr.com/dashboard')
tumblr.fetch_request_token(request_token_url)

A response like the following should appear:

{'oauth_token': 'REDACTED', 'oauth_token_secret': 'REDACTED', 'oauth_callback_confirmed': 'true'}

[matamorphosis]

@PinkDev1 yep this is something I can put into the next release of Scrummage (3.8) if it makes life easier. I'd actually encourage you as someone who seems to know Python to have a crack at implementing it yourself, as we recognise there are sometimes better ways of integrating Scrummage with third-parties than what we provide in the freely available, open-source tools. In fact this entire project is built due to the founders being dissatisfied with the way OSINT is being handled by other platforms. If you do this your name will pop up as a contributor in the project.

The plugin you would need to update is: https://github.com/matamorphosis/Scrummage/blob/main/app/plugins/Tumblr_Search.py

Additionally, if you go down that road, please add the library to https://github.com/matamorphosis/Scrummage/blob/main/installation/support_files/python_requirements.txt

It may seem a little hard at first due to Scrummage being heavily frameworked, but I can assure you it's not too difficult, especially if you follow our plugin development guide https://github.com/matamorphosis/Scrummage/wiki/Plugin-Development-Guide

Lastly, due to this not actually being a bug, I am transferring this to a discussion.

[ItsIgnacioPortal]

I've finished configuring most of the API Keys.

Creating tasks is awkward with such small input fields, and there aren't any predefined task groups (like, search username in all platforms)

Mutiple tasks can't be deployed simultaneously using the GUI, and sometimes they don't even create any results.

How is scrummage supposed to be used? Is the project still in alpha?

[matamorphosis]

That's good to hear, it's quite an effort setting up all of those API keys.

I appreciate the feedback around input fields, it's something we can widen quite easily. Also apologies for the following essay, but it is difficult to answer your questions without the provided level of detail.

The purpose of the platform is to have more manual control over each plugin. Hence why it's not like Spiderfoot, where you give it some details and it runs a whole series of plugins on your behalf.

Plugins don't always return results, and it can be for one of a few reasons:

1. The were no results for the given query
2. The given query is not in the correct format. Some plugins are a little confusing in terms of whether a domain or URL is supplied. It hasn't yet been designed, but under 3.8 I'm planning on integrating plugin-specific errors for the query that advise if you have entered something in the wrong format.
3. The results have already been generated and have been cached. A cache clearing page is coming under 3.8.
4. The plugin runs into an error, I'd keep an eye on your CLI for this for specific errors.
5. A limitation with the specific API key you have. Some APIs limit the amount of information shared.

The project is 3 years old, but like a lot of security products is built in the free time of the devs. We recently obtained our first sponsorship, but prior to that that project has had no funding. The path of Scrummage is very similar to that of other cybersecurity products.

If you need further convincing of this read the following:

It's quite important to note that the way open-source security products are built differs from traditional software development. So the project is not in Alpha, but the project is a rolling project, so some things to note:

1. In software development there are three key areas of focus (Design (UX/UI), Functionality, and Security). For most software the priority goes Functionality > Design > Security. Which is why software gets hacked left, right, and centre.
2. In secure software development the priority goes Security > Functionality > Design. This is so that our software doesn't end up on some CVE list, as the reputation of the product is critical to protect.
   So it really is a rolling project that gets better with time. Managing the plethora of plugins also makes the tool impossible to make stable 100% of the time. APIs change all the time and ensuring our plugins stay up to date is something we fix when it is either identified by us or reported to us. But we do our best to test plugins regularly to ensure they are up-to-date and working.

This is why some of the early versions of software look and feel pretty average (Scrummage included). One of the biggest OSINT tools out there is Maltego, supported by offensive security. The product is very clunky and ugly, but provides immense security value. The main thing Maltego provides, which is also in the works for 3.8, is an excellent way of integrating with several APIs, and mapping the results together.

Scrummage's Unique Value

The value that Scrummage provides that I doubt you'll find elsewhere for a free product is the breadth of the plugins available to you at no cost. There are over 80 plugins, which allow you to configure over 100 types of tasks.

• Maltego allows you to configure ~77 plugins (https://www.maltego.com/transform-hub/). However, you will not find any common social media plugins in that list (Twitter, Tumblr, Pinterest, etc.).
• Spiderfoot is much more of a competitor, but good luck gaining access to their full suite of plugins for no cost. Their list of modules can be found at (https://www.spiderfoot.net/modules/)
• The OSINT framework is just a list of sites you have to perform OSINT manually on.
• Scumblr provided a foundation to replicate for the product, but has an incredibly limited set of plugins, and has been unmaintained for over 3 years. (https://github.com/Netflix-Skunkworks/Scumblr)

Some other unique features:

• Plugins:
  • A domain name fuzzer that has several options for identifying domain spoofs.
  • OSINT for vehicle registrations
• Other:
  • Scrummage is powerful at evidencing your findings, providing both local copies of result HTML pages, the raw JSON response from the API, as well as screenshot functionality straight out of the application.

Use cases

1. I want to conduct a perimeter assessment against an organisation. I set up a series of domain-related tasks to search for potential issues with that domain. (Cert Transparency, Dark Web Search, DNS security check, Domain Fuzzing, Shodan search, etc.)
2. I want to identify a new hire's behaviour on social media, so I set up a series of social media tasks to look for that user (Username Search, Twitter Search, Tumblr Search, etc.)
3. I'm investigating a fraudulent business. I need to search for business records (Business Search for the relevant country, A search of doing business in the relevant country to identify what is required to operate a business there - Doing Business Search)
4. Maybe I want to do a social media search for a russian company or a Korean company. Search engines and social media sites of the non english-speaking world are provided (Naver, Yandex, Vkontakte, OK)

Ultimately you work out your own use case before using the product :)

Future Roadmap

Again I understand that using a manual OSINT tool might be annoying to get everything set up, but it's helpful for you to know which plugins you are executing as each OSINT scenario differs. At some point I might create task groups, where you provide a domain and the tool generates and runs a series of tasks on your behalf. But the more streamlined this becomes, the more maintenance. At some point we are looking to introduce a premium version of the product which would automate a lot more, and be organisation-centric. Meaning you put in some company details, and that's used to define the task queries and other settings. But this actually reduces a lot of the flexibility the product currently gives you, which is why as an individual you may feel lost, because you pick the tasks based on what you want to do.

Summary

This tool is really huge, building platforms is a massive task that evolves with time, Elasticsearch is a good example of how an open-source tool increases in value with time on the SIEM side of cybersecurity. So many tools at some point in time have been unstable, clunky and ugly. I think anyone getting into cybersecurity in the last couple of years doesn't understand/appreciate this. Not saying this is you, but I hope this provides context.

[ItsIgnacioPortal]

I see. I wish you good luck with this project then. Make of my suggestions what you will.

I don't wish to colaborate on a project that is planning to be sold for profit.

I'll stick to using multiple, small but polished tools.

[matamorphosis]

@PinkDev1 you're welcome to use whichever tools you prefer. However, I think you may have misunderstood. The goal of Scrummage has never been to make money. The project has been free-of-charge, and running for > 3 years. If I wanted to produce a proprietary product I would have done so from the get go, and the project would be further along but not free. There will always be a free edition of the product which will receive the same level of attention and maintenance as what is currently provided.

What I was referring to with the premium edition, is a version of the product targeted at organisations, not individuals. The needs of organisations are always greater, and many organisations using the product would have various clients they are performing OSINT for. The automation and organisational focus components of OSINT previously described were never part of the intention of Scrummage. As previously mentioned Scummage is a manual OSINT tool, that centralises OSINT capabilities; therefore, this effort would be additional to the scope of the project.

This is more of an idea at this point anyway, that I haven't even begun developing. But it would not compromise the availability of the project.

Lastly, if there is still a chance you would consider collaborating on this project I can assure you that it would go be freely available as you are contributing to the open-source project. Especially plugins, as at no point would I charge for plugins.