My whole project crashed because I listened to the 'start with Python' advice

Last Thursday, I was building a simple web scraper for a local weather site. I followed the common beginner tip to use Python with BeautifulSoup, but the site's JavaScript broke my script completely. After two days of hitting errors, I switched to JavaScript with Puppeteer and got it working in three hours. Maybe telling every new coder to pick Python first isn't always right. Has anyone else found a different starting language worked better for a specific project?

3 comments

3 Comments

valc911mo ago

Two days is a long time to fight a tool that doesn't fit the job. The real lesson is picking the right tool, not blaming the first one you tried.

the_terry1mo ago

Wait, you spent two whole days trying to scrape a JavaScript site with Python? That's brutal. BeautifulSoup just sits there staring at empty divs when the content loads later. Puppeteer was definitely the right call for that mess. Sometimes the "easy" language just makes the actual problem harder.

mianelson1mo ago

Just realized you tried to scrape a live site with BeautifulSoup for two days, that's honestly wild to me. I mean, @valc91 is right about the tool thing, but I would've given up after like two hours of empty tags. Maybe it's just me but starting with Python for that specific job sounds like a nightmare.