Update: for LW, this behaviour stopped around about Friday 12th April. Not sure what changed, but at least the biggest instance isn’t doing it anymore.

I’ve been coerced into reporting it as bug in Lemmy itself - perhaps you could add your own observations here so I seem like less of a crank. Thanks.

I’ve since relented, and filed a bug

okay

I’m only running one process, I’d assume the problem isn’t happening for Feddit.dk.

Perhaps. The lemmy.ca post has a comment in from the mander.xyz admin who’s only running one, and there’s a new comment in this thread saying mander.xyx is one of the instances they see the most duplicates from.

Yeah, that’s the conclusion I came away with from the lemmy.ca and endlesstalk.org chats. That’s it due to multiple docker containers. In the LW Matrix room though, an admin said he saw one container send the same activity out 3 times. Also, LW were presumably running multiple containers with 0.18.5, when it didn’t happen, so it maybe that multiple containers is only part of the problem.

When I’ve mentioned this issue to admins at lemmy.ca and endlesstalk.org (relevant posts here and here), they’ve suggested it’s a misconfiguration. When I said the same to lemmy.world admins (relevant comment here), they also suggested it was misconfig. I mentioned it again recently on the LW channel, and it was only then was Lemmy itself proposed as a problem. It happens on plenty of servers, but not all of them, so I don’t know where the fault lies.

A bug report for software I don’t run, and so can’t reproduce would be closed anyway. I think ‘steps to reproduce’ is pretty much the first line in a bug report.

If I ran a server that used someone else’s software to allow users to download a file, and someone told me that every 2nd byte needed to be discarded, I like to think I’d investigate and contact the software vendors if required. I wouldn’t tell the user that it’s something they should be doing. I feel like I’m the user in this scenario.

We were typing at the same time, it seems. I’ve included more info in a comment above, showing that they were POST requests.

Also, the green terminal is outputting part of the body of for each request, to demonstrate. If they weren’t POST requests to /inbox, my server wouldn’t have even picked up them.

EDIT: by ‘server’ I mean the back-end one, the one nginx is reverse-proxying to.

They’ll all POST requests. I trimmed it out of the log for space, but the first 6 requests on the video looked like (nginx shows the data amount for GET, but not POST):

ip.address - - [07/Apr/2024:23:18:44 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
ip.address- - [07/Apr/2024:23:18:44 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
ip.address - - [07/Apr/2024:23:19:14 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
ip.address - - [07/Apr/2024:23:19:14 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
ip.address - - [07/Apr/2024:23:19:44 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"
ip.address - - [07/Apr/2024:23:19:44 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.world"

If I was running Lemmy, every second line would say 400, from it rejecting it as a duplicate. In terms of bandwidth, every line represents a full JSON, so I guess it’s about 2K minimum for the standard cruft, plus however much for the actual contents of comment (the comment replying to this would’ve been 8K)

My server just took the requests and dumped the bodies out to a file, and then a script was outputting the object.id, object.type and object.actor into /tmp/demo.txt (which is another confirmation that they were POST requests, of course)

I can’t re-produce anything, because I don’t run Lemmy on my server. It’s possible to infer that’s it’s related to the software (because LW didn’t do this when it was on 0.18.5). However, it’s not something that, for example, lemmy.ml does. An admin on LW matrix chat suggested that it’s likely a combination of instance configuration and software changes, but a bug report from me (who has no idea how LW is set up) wouldn’t be much use.

I’d gently suggest that, if LW admins think it’s a configuration problem, they should talk to other Lemmy admins, and if they think Lemmy itself plays a role, they should talk to the devs. I could be wrong, but this has been happening for a while now, and I don’t get the sense that anyone is talking to anyone about it.

Oh, right. The chat on GitHub is over my head, but I would have thought that solving the problem of instances sending every activity 2 or 3 times would help with that, since even rejecting something as a duplicate must eat up some time.

Yeah. It seems like the kind of community he’d post in. I think there was some drama between him and startrek.website, resulting in him or the admins deleting the account.

(same person on different account, just so I can preview what the table will look like)

Hmmm. This might not help you much. That community was launched on 2024-01-22, with a page full of posts, so by the time the crawler picked up on it, it was already at 501 subs, 23 posts.
It lost 15 posts on 2024-02-12 (Stamets?).
There’s data missing from 2024-02-26 - 2024-03-01 (this data is from the bot at [email protected]; I think those days are missing because I changed it from measuring Active Users Month (AUM) to Active Users Week (AUW)).
In terms of active users, the jump on 2024-03-04 is due to lemmy.world ‘upgrading’ to 0.19.3.

[data missing]

I know for sure that Lemmy won’t, it’s likely the same for Mastodon.

I was wrong about not being able to WebFinger your account - I still had the @ at the beginning when I trying. Doing it properly:

curl --header 'accept: application/json' https://mostr.pub/.well-known/webfinger?resource=acct:910af9070dfd6beee63f0d4aaac354b5da164d6bb23c9c876cdf524c7204e66d@mostr.pub | jq .

gets the right response.

However, I’m logged into lemmy.world and it still couldn’t get your account. At a guess, it’s because there’s a 20 character limit on usernames.

Lemmy instances won’t search outside of their own databases if you’re not logged in.

But if you are, what it does can be recreated on a command-line by doing:
curl --header 'accept: application/json' https://nerdica.net/.well-known/webfinger?resource=acct:nate0@nerdica.net | jq .

This shows that your profile is at https://nerdica.net/profile/nate0. Lemmy puts all users at a /u/, but using webfinger means that other fediverse accounts don’t have to follow the same structure. For lemmy.world, you’re at https://lemmy.world/u/[email protected] in the same way that a mastodon user is at e.g https://lemmy.world/u/[email protected].

edit: However if you webfinger your mostr.pub account, you get: {"error":"Invalid host"} so any ActivityPub instances will only ever be able to find you if you’ve interacted with them in some way to get a database entry. Edit: also, I tried to do this again, thought I’d try the npub1 account as well, but got Gateway Timeouts, so there’s a bit of jankiness going on too.

lemmynsfw has implemented (or intends to) an interesting compromise, in that you can only downvote posts on that instance’s communities that you’re already subscribed to. Ideally, this means that downvotes are for the quality of the individual post, rather than as a reaction to the type of content.