Skip to content
This repository was archived by the owner on May 29, 2025. It is now read-only.

How to pass static configuration through url list file #86

@AlasdairGray

Description

@AlasdairGray

@petrospaps I'm trying to override the dynamic=true parameter in the localconfig.properties file by using the value on the end of the url list file.

Following the instructions in the README has not resulted in the static scraper being used. Below are the snippets of the url list file that I tried.

https://bgee.org/sitemap_main.xml,static
https://bgee.org/?page=gene&gene_id=ENSG00000274928,static
https://bgee.org/?page=gene&gene_id=ENSG00000274928, static

None of these overrode the dynamic setting in the local config file.

Even removing the setting from the local config file did not result in the static scraper being used.

I'm basing this on the following exert of the log file

11:26:16.619 [INFO] hwu.elixir.scrape.scraper.examples.FileScraper - Attempting to scrape: https://bgee.org/?page=gene&gene_id=ENSG00000274928
11:26:16.619 [INFO] hwu.elixir.scrape.scraper.ScraperFilteredCore - dynamic scraping setting
11:26:27.713 [ERROR] hwu.elixir.scrape.scraper.ScraperCore - URL timed out: https://bgee.org/?page=gene&gene_id=ENSG00000274928. Trying JSoup.
11:26:28.295 [DEBUG] hwu.elixir.scrape.scraper.ScraperFilteredCore - Number of JSONLD sections: 0

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions