Mico's Blog

The ever growing amount of traffic

Published (updated: ) in Hetzner, Linux, Self-hosted, Servers.

Phew, It has been almost a year since I started hosting my first public service to the internet. I gotta say that it has been quite a year and I have learned a lot.

The first service that I started hosting to the masses was SearX, pre-installed from Yunohost. I probably didn’t know enough back then to what I was dipping my toes in. Even though SearX isn’t my most popular service, I use it for 95% of all my searches. The 5% of searches go to Google, just because searching for businesses or some school work with SearX doesn’t always yield the best results.

My services have grown so much that I’ve had the need to buy a dedicated server, I’ve only bought it 2 months ago but man has it made things much faster. Not only the websites and services but also just general system administration and service maintenance are just so much faster than with a tiny VPS. That’s to be expected though, I used a VPS from Contabo for most of my services but as more and more people started using my services, the need for a bigger server was obvious.

Of course, I do not have much money to spare on a server so the cheapest dedicated server from Hetzner will do. Hetzner is pretty much the cheapest and best hosting provider in Finland for individuals that don’t have large company money to spend on hosting. I haven’t had problems myself, but I’ve read on Nitter’s GitHub about horror stories from Hetzner’s Legal team. Even though all the services that I host are perfectly legal in the EU and also the US, people still make abuse complaints about stuff that isn’t necessarily even hosted on the server.

UPDATE on this section: Having carefully read the Hetzner Terms of Service, I did notice something:

8.2. The Customer is obligated not to publish any content that

infringes on the rights of third parties or otherwise violates

applicable law. This includes in particular, but is not limited

to, pornographic or obscene material, extremist content or

content that offends common decency, gambling, material

that could seriously endanger the morals of children or young

people or violate the rights of third parties (copyrights, name

rights, trademark rights and data protection rights). This also

includes the publication of defamatory content, insults or

disparagement of persons or groups of persons.

https://www.hetzner.com/assets/Uploads/downloads/AGB-en.pdf

So yes, it is against Hetzner’s Terms of Service to host services like Nitter or Libreddit because they contain porn and other stuff like that. However, Hetzner doesn’t enforce this rule except when they get abuse complaints from them.

I’ve resorted to using a small VPS for these problematic services from a company (BuyVM), that understands these things better than Hetzner. It only serves as a reverse proxy to my server, so if someone we’re to complain, it goes to BuyVM instead of Hetzner. Of course the best case scenario would be to own my own IP-addresses so all the complaints would directly come to me to handle, but that is very expensive nowadays, especially with IPv4 shortages.

I could register my own ASN, rent some cheap or sometimes free IPv6 addresses and then proxy the traffic through Cloudflare to get a IPv4 address, this way if someone we’re to send an abuse complaint to Cloudflare, Cloudflare would forward it to the abuse E-mail address of the IP it is proxying. I do not know enough of networking to really feel confident in doing this but I did read a little bit on how much it would cost, and it came to around 15€ a year + 100€ of one-time fees from tunnelbroker.ch, but I haven’t gotten enough complaints so far to make it worth it.

Oh yeah, traffic, the title of this post.

To my surprise, my traffic levels has risen by about 400% after switching to a bigger server. Even my biggest service, Nitter, has had it’s traffic doubled. The biggest reason for this growth is Piped, videos are large, so they take a fair bit of traffic. Luckily though I’m not even close to using my full port speed and my server has unlimited traffic, so it won’t be a problem for a while.

Here is the traffic statistics for the month of May

Note that on 05-03 and 05-19 I was doing transfers, they do not account for the traffic of my services.

Pretty much everyday my network traffic has grown little by little, I know this isn’t much compared to larger hosters, but it is a lot to me. This doesn’t include Invidious though, the server hosting that and only that uses around 80-90gb everyday, probably because video proxying is disabled by default. Reason I haven’t moved that yet is, I had problems moving the database to my server so I’ve left it as is for now. I will move it the near future, I promise.

Anyways, thanks for reading my longest blog post so far, I wrote this at 1 AM because I was bored :DD so apologies for any grammar issues, I don’t use any grammar checking software or even something that checks if I wrote a word wrong.