TIL that Lemmy.ml is actively blocking user-agent string kbin

One of the main instances of Lemmy (lemmy.ml) is blocking 'kbinbot' user agent requests on their instance. Basically, they are blocking kbin instances. This is not a bug, but on purpose.

I couldn't find anything related in their source-code, so this is block is only present on lemmy.ml (or maybe some other Lemmy instances).

This is causing quite a lot of federation issues with Kbin instances. Resulting in various failed messages (from the messenger handler).

$ curl -I --user-agent "kbinbot" https://lemmy.ml
HTTP/2 403 
server: nginx
date: Wed, 28 Jun 2023 18:35:27 GMT
content-type: text/html
content-length: 146
vary: Accept-Encoding

$ curl -I --user-agent "notkbin" https://lemmy.ml
HTTP/2 200 
server: nginx
date: Wed, 28 Jun 2023 18:35:42 GMT
content-type: text/html; charset=utf-8
content-length: 163388
vary: Accept-Encoding
x-powered-by: Express
content-security-policy: default-src 'self'; manifest-src *; connect-src *; img-src * data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; form-action 'self'; base-uri 'self'; frame-src *; media-src *
etag: W/"27e3c-LCqXylkqSkDOy3K+3w2TijtMn14"
strict-transport-security: max-age=63072000
referrer-policy: same-origin
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
distantorigin ,

The silence from Lemmy developers on this is damning. If this was an accident (i.e. lumping "kbinbot" in with a blanket block of other user agents), it would have been a two second fix. Even more damning is that common agents that are being used for bot attacks, as discussed in the Lemmy matrix, are not blocked. For example:

curl -i --user-agent "python-requests/1.2.3" https://lemmy.ml/

Works fine.

wahming ,

The linked discussion sounds like nobody knows for sure since the admins are keeping quiet. Which is weird.

CosmicBlend ,

Conversation about this on lemmy.ml: https://lemmy.ml/post/1563840

pineapplelover ,

Something something compatibility issues. Hope they get things fixed because it initially sounded like censorship

melroy OP Admin ,
melroy avatar

No it's NOT a compatibility issue.. They just ban the 'kbinbot' user agent string. I have no words for it.

Sal ,
@Sal@mander.xyz avatar

Yeah, it looks to me like an nginx redirect of the kbinbot user-agent. For example, I’ve added the following block to my instance’s nginx config:

<pre style="background-color:#ffffff;">
<span style="color:#323232;">if ($http_user_agent ~* (testblock|megablock)) {
</span><span style="color:#323232;">        return 403;
</span><span style="color:#323232;">    }
</span>

You can now get the same response using:

curl -I --user-agent “testblock” https://mander.xyz

I wouldn’t want to jump ahead and assume some malicious or sneaky reason. The user agent could have been accidentally lumped into a block list during a DDoS mitigation strategy, or they could have run into some unexpected issues when upgrading to 0.18.0 and they thought this might be a quick temporary fix while patching - but patching is taking them longer than expected. I don’t know, but it is a curious observation.

snaf ,

discussion on lemmy.ml: lemmy.ml/post/1563840

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • drbboard
  • tech
  • updates
  • til
  • testing
  • bitcoincash
  • programming
  • Sacramento
  • All magazines
    •