docs/introduction/setting-up

[giscus[bot]]

docs/introduction/setting-up

Your first steps into the world of scraping with Crawlee

https://crawlee.dev/docs/introduction/setting-up

[codrant]

"You will see log messages in the terminal as Crawlee boots up and after a second a Chromium browser window will open. In the window, you'll see quickly changing pages and back in the terminal, you will see the printed titles (contents of the <title> HTML tags) of the pages.":
Chromium work in "headless"!
"The printed titles (contents of the <title> HTML tags) of the pages" is not!

[mnmkng]

Thanks for the reminder. You're right and we will update the tutorial. You can use headful Chromium like this:

const crawler = new PlaywrightCrawler({
    headless: false,
    // other configuration
})

[mnmkng]

This has been updated and fixed 🎉

[simpai]

Hello. When I create and npm start,

npm start
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.

> my-crawler@0.0.1 start
> npm run start:dev

npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.

> my-crawler@0.0.1 start:dev
> ts-node-esm -T src/main.ts

The filename, directory name, or volume label syntax is incorrect.

What am I missing?

[mnmkng]

It seems that you have some non-standard NPM config. I've never seen those warnings or error message before. Try a clean install somewhere with plain Node and NPM and if it doesn't help, please create an issue with more details.

[B4nan]

Sounds like you just have a misconfigured $PATH variable, most probably not related to the library, this is about your environment.

https://stackoverflow.com/questions/24782826/the-filename-directory-name-or-volume-label-syntax-is-incorrect-inside-batch

[ArcticCascade]

"npx crawlee create my-crawler" fails behind corporate proxy.
I get the same result on Ubuntu and MacOS - npm will use the defined proxy settings to download the initial packages, but as soon as that's done, it will timeout.
The template selector ignores both npm and system environment variables for proxy.

[metalwarrior665]

Yeah, this is an annoying problem. For now, I recommend just copy/pasting the template from https://github.com/apify/actor-templates/tree/master/templates and install manually instead of using crawlee CLI

[mnmkng]

Crawlee templates are here https://github.com/apify/crawlee/tree/master/packages/templates/templates

[harraz]

Hi guys, I've been trying to run this tutorial for a while but I keep getting this error. I use Mac M1. I tried this on multiple docker images but I still keep getting this error

[pid=32093][err] [0222/060051.183687:ERROR:stack_trace_posix.cc(773)] Failed to parse the contents of /proc/self/maps
[pid=32093][err] [0222/060052.902439:ERROR:stack_trace_posix.cc(773)] Failed to parse the contents of /proc/self/maps
[pid=32093][err] [0222/060052.915868:ERROR:stack_trace_posix.cc(773)] Failed to parse the contents of /proc/self/maps
[pid=32093][err] [0222/060052.953786:FATAL:zygote_main_linux.cc(162)] Check failed: sandbox::ThreadHelpers::IsSingleThreaded(). 
[pid=32093][err] #0 0x004007c6bb22 base::debug::CollectStackTrace()
[pid=32093][err] #1 0x004007bcdc43 base::debug::StackTrace::StackTrace()
[pid=32093][err] #2 0x004007bcabcf logging::LogMessage::~LogMessage()
[pid=32093][err] #3 0x004007bcb76e logging::LogMessage::~LogMessage()
[pid=32093][err] #4 0x00400715030f content:

I use these versions

root@57cbdfedc686:~/my-crawler# node -v
v18.14.2
root@57cbdfedc686:~/my-crawler# npm -v
9.5.0

Any help is appreciated

[LeMoussel]

It seems to be a web browser error. Which template did you use?
To test, can you communicate the url of the site?

[mnmkng]

Try this configuration option.