Added Psychological Science scraper#40
Added Psychological Science scraper#40tarrow merged 177 commits intoContentMine:masterfrom chartgerink:master
Conversation
Initial work on science direct/elsevier scraper.
Create science_direct.json
…ks and ran test sometime before breaking everything
|
I rewrote this into #44 with a rebase because it could be merged to the master (it needed a rebase) before merging the 176 commits |
|
I did some additional checking of the scrapers. I removed Also checked and updated wiley, sage, springer, elsevier scrapers. Elsevier contains almost no metadata so the scraper only uses html and pdf extraction. I also incorporated some changes, but they eliminated a lot of metadata scraping and did renaming of elements. Are we still adhering to the scraperJSON standard or did that become a thing of the past? Sorry for the extent of commits, I forgot about this. I can also create a new fork to make things easier and do a new PR. Let me know. |
|
We're still adhering to the scraperJSON standard; but not least because given that QS and thresher are the reference implementations basically if it works it's scraperJSON :). If you could create a new branch from the current origin/master and cherrypick over these changes you've just made that would be awesome! Otherwise I can do that and make another PR. Let me know if you have problems. |
|
Great! I'll merge now! :) |
Hi,
I attempted to write my first scraper, according to your scraperJSON template, and succeeded for the most part. I have also included test links. I tried to scrape as much information as possible, and include some of my problems below, FYI.
Kind regards,
Chris