I tried to share a permalink like https://codefloe.com/robink/ctaphid-rs/src/commit/0d078c1b4b8184b98c1d05964e21bd1c9e824ad0/Cargo.toml#L8 but noticed that it returns 403 Forbidden if the visitor is not logged in. Is this behavior intentional? Maybe it is related to rate limiting but it also occurs without previous requests to codefloe.com.
Yes, that is blocked currently by our current HAProxy rules to gate automated scraping. We’ve just updated the 403 error page with some useful information. In your case, the following patterns would work without a session token:
/owner/repo/commit/HASHshows the same commit as a diff → https://codefloe.com/robink/ctaphid-rs/commit/0d078c1b4b8184b98c1d05964e21bd1c9e824ad0/owner/repo/src/branch/main/path/to/fileshows the current file on a branch → https://codefloe.com/robink/ctaphid-rs/src/branch/main/Cargo.toml
We are aware this is not super great for users but we have to do this for a reason, otherwise we get overrun by automated bots. Would the above workarounds work for you?
Thanks for the explanation! I’m very used to permalinks when referencing a specific code location and the mentioned workarounds unfortunately don’t really work for that. The commit does not necessarily include the relevant lines, and the file on the current branch may have changed when the link is accessed. But of course I understand that you have to make some restrictions.
Still I think it would be good to show a warning when using the „Copy permalink“ option in the file view as long as this restriction is in place. Most likely the link will be stored or sent somewhere and not used directly and it is very unexpected if it does not work.
Permalinks are an important feature, we are aware of that.
We’d like to bring it back - we just need to find a good pattern to prevent the abusive scraping requests and allow “normal” users. The tricky part is that these requests are spread across many (we’re talking about 1000+) servers and the overall count per IP is low (and sometimes even just one) for unique IPs. So the scrapers are actively trying to get around rate limiting here.
Besides the scraping, such hash commits are super expensive on the Git side, i.e. they take substantially more resources for the lookup than non-hash URLs.
We’ll let you know once we have found a better solution!
We have now implemented changes that generally allow permalinks again and send a little challenge for these paths which, hopefully, automated processed won’t be able or don’t want to solve. These shouldn’t be a problem for normal users though. It’s a similar attempt as what Anubis is doing, just more lightweight.
Let us know how it goes for you and if you encounter any additional issues whatsoever! Thanks for reaching out again!
Great news! That works for me.