This repository was archived by the owner on May 29, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 5
How to pass static configuration through url list file #86
Copy link
Copy link
Open
Labels
bugSomething isn't workingSomething isn't working
Description
@petrospaps I'm trying to override the dynamic=true parameter in the localconfig.properties file by using the value on the end of the url list file.
Following the instructions in the README has not resulted in the static scraper being used. Below are the snippets of the url list file that I tried.
https://bgee.org/sitemap_main.xml,static
https://bgee.org/?page=gene&gene_id=ENSG00000274928,static
https://bgee.org/?page=gene&gene_id=ENSG00000274928, static
None of these overrode the dynamic setting in the local config file.
Even removing the setting from the local config file did not result in the static scraper being used.
I'm basing this on the following exert of the log file
11:26:16.619 [INFO] hwu.elixir.scrape.scraper.examples.FileScraper - Attempting to scrape: https://bgee.org/?page=gene&gene_id=ENSG00000274928
11:26:16.619 [INFO] hwu.elixir.scrape.scraper.ScraperFilteredCore - dynamic scraping setting
11:26:27.713 [ERROR] hwu.elixir.scrape.scraper.ScraperCore - URL timed out: https://bgee.org/?page=gene&gene_id=ENSG00000274928. Trying JSoup.
11:26:28.295 [DEBUG] hwu.elixir.scrape.scraper.ScraperFilteredCore - Number of JSONLD sections: 0
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working