[sarp]

Hi! Sarp here, author of the Build your own BitTorrent challenge on Codecrafters.

Some back story: After being laid off from my FAANG job, I found myself very unmotivated to go back. I started looking for interesting programming projects to revive my interest in coding. While nomading, I discovered Codecrafters on Nomadlist and really liked the push code to git and pass different stages interaction. The gamification helped me focus and projects allowed me to go deeper on software I used (SQLite, Git, Redis etc.). I even picked up a new language (Go) to do the challenges with. After completing all the challenges on the site, I ran out of things to do. This is when I decided to build a BitTorrent client which was one of the highly voted ideas on the site.

I learned many new things by building a BitTorrent client: the BitTorrent protocol, how torrent files are structured, encoding issues, pipelining network requests, url encoding binary values, using Channels in Go etc.

I’d love any feedback on the challenge. Also happy to answer any questions!

[anjel]

Its not uncommon for invite-only trackers to be very prescriptive about which torrent clients can and can't be used with their private tracker. Any ideas on how to overcome this obstacle to wider adoption?

[vincentkriek]

Is this something you expect to use as your main torrent client?

[CamperBob2]

How do they tell one client from another?

[thedaly]

User agents, fingerprinting, etc. There are certainly ways to mask your client, but these would be considered cheating by most private trackers and would be grounds for a ban.

[paledot]

There's a whole ton of bluster in the torrent community. Bittorrent is a simple protocol, to the point of being naive (and therefore a fun toy network app project). Clients identify themselves to the tracker by user agent; there's really nothing else to fingerprint against. Claims to the contrary are almost certainly bullshit to scare people out of editing their user agent.

Clients also self-report the amount of data transferred. That's not great in a community that fetishizes share ratios. I've heard an op say "there's no excuse for having a ratio less than 1", which makes as much mathematical sense as the parent who told my (math) teacher friend "this is [private school], no student should be below average".

You can theoretically verify upload/download numbers because the total amount uploaded in a swarm should equal the total amount downloaded, but there are all kinds of reasons why the numbers wouldn't match. Maybe a client lost connection and couldn't send its final announce. Maybe one client is sending bad data (I'm not sure how that is reported, might be implementation specific). And clients only send transfer total updates when they connect to the tracker to change status or request more peers, so every client will have a different degree of staleness.

Even if you can tell that someone in a swarm is lying, who is your culprit? As long as they're not being egregious, there's no way to tell.

[Bu9818]

There's at least some differences, such as HTTP/2 usage, or maybe algorithm usage/bugs in newer versions. Whether or not most tracker staff actually bother to attempt fingerprinting, IDK.

[mu5tach3]

> "there's no excuse for having a ratio less than 1"

Maybe the context of the quote was in regard to a private tracker?

The "ratio" in terms of private trackers isn't always the real ratio of GBs uploaded or downloaded.

There are for example some private trackers that grant additional credit for longer seed-time or they declare specific torrents "freeleech" so they don't "cost" ratio.

In the end they are just some of the measures private trackers take to strengthen their network, but they lead to a confusing definition of "ratio".

[paledot]

Yes, but this tracker did not offer any such things. The only reason the mean ratio of active members was > 1 was because of the steady stream of users being banned for low ratios.

[andnand]

How did you learn to build a BitTorrent client? I love the idea of codecrafters and books that walk you through building something but I always struggle if I don't have something to get me started.

[sarp]

My starting point was searching for tutorials and asking ChatGPT to implement a torrent parser :) There are great blog posts [0] for building a BitTorrent client. Along the way, I referenced open-source implementations and the BitTorrent Protocol Specification as well [1].

[0] https://blog.jse.li/posts/torrent/

[1] https://www.bittorrent.org/beps/bep_0003.html

[friend_and_foe]

I've actually just begun work on a project involving bittorrent, I don't think I will be implementing a client but maybe, I'm certainly going to have to implement generation of .torrent files and magnet links, so I've been learning a little bit about the protocol. Interesting to see this pop up and maybe I can use it to help me make headway.

[rrishi]

> After completing all the challenges on the site, I ran out of things to do. This is when I decided to build a BitTorrent client which was one of the highly voted ideas on the site.

Are you employed by them now?

[sarp]

I'm not an employee of Codecrafters. I worked with them for the BitTorrent challenge as an independent contractor.

[ferfumarma]

This is such a cool result. I would love to see the next step of creating a tool like Resilio sync that builds upon this work!

[ollien]

I don't know if this is feedback on the challenge per-se, but I was a little saddened that I couldn't add dependencies to my `Cargo.toml`; I wanted to solve part of the bencode challenge using nom (perhaps overkill, but it was for fun), but I can't.

If this is a concern of load/execution times on a remote builder, it would be cool if I had some way to run the testcases locally to avoid this concern

[rohitpaulk]

We just shipped support for this last week! You should be able to edit the file now. We’ll remove the comments saying those files can’t be edited soon.

[ollien]

Oh! Awesome! I'll have to give this a shot, then :)

[sarupbanskota]

Are we gonna get extensions to the challenge?

[blurbleblurble]

Better yet, extensions to implement BitTorrent extensions? Such as the mutable torrent BEP and others? It'd be great if this spawned hundreds of top quality BitTorrent clients

[sarupbanskota]

Build your own Extension

[sarp]

To limit the scope of the challenge, I had to leave out a lot of cool features you see in modern clients: magnet links, UDP trackers, DHT

I'm planning to add them over time starting with magnet links which is the highest voted extension idea right now