What if we went on several tangents about nix, mid-way through a blog post about atproto?

Things this isn’t

This post is not the post in response to “is Bluesky dying”, I’m still stewing on this thought and it deserves more time. It’s also not an argument for/against islands (ie, running a set of services completely detached from the network at large - even though that’s what we’re doing), and I’m not arguing the merits of doing such a thing.

What are we trying to achieve

It sounds simple in theory: we want to use Bluesky, on our own instance, completely independently, avoiding any Bluesky code and servers.

In practise, this rules out the reference PDS, relay, Postgres AppServer, and us using PLC. Also some of the useful tools, like goat and the relay dashboard.

on our own instance, completely independently

To be truly free of Bluesky services, we can’t touch the main PLC directory¹. We could run our own, but we might want to connect our little island to the rest of the network eventually. Mirroring counts as touching here, and that cuts us off completely², and settles identity to did:web.

There are other did:web accounts on the network though. We could (and may) find those on non-reference PDSes, and connect, but that’s quite the curious combination - we might be lucky to get double digits.

A Software Survey

As luck may have it, there’s a load of independent PDS implementations now. Cocoon³, Pegasus, Tranquil, and many others. For fun, why not deploy all of them!⁴ I know there are people running Cocoon⁵, Pegasus⁶, and Tranquil⁷ in production as their main accounts. There’s also rsky-pds, but that doesn’t support OAuth, and I'm not aware of anyone using it in production yet.

When it comes to relays, there’s Blacksky’s rsky-relay, which is written in Rust, and used in production, and Nate’s zlay, which is written in Zig, although has been around for less time.

Relays proved to be an interesting endeavour - both zlay and rsky-relay want(ed) to follow an upstream host (i.e., bsky.network). In rsky’s case, this was hard-coded, with no way of being disabled, so I forked. Meanwhile, I opened an issue on zlay, which has since been implemented.

This means we can run both relays with no Bluesky dependencies, and let them discover the PDSes on our network normally.

did:web hosting is simple: yeet them onto various subdomains using Caddy (or any other web server), apart from tranquil which can host its own. I’ve never used a did:web before, so there’s almost certainly going to be something to learn there.

For an AppServer, we’ll be using Parakeet, obviously.

Also, this is all going to be done on NixOS, because I’m one of those people now.

The path we’re going to take may seem counterintuitive, but trust me, there’s reasonable cause to do it this way.

We’re going to start at the middle with Relays, then PDSes, back around to AppServers, and finishing on clients.

“Why not deploy them all” (an aside)

Astute readers may have noticed that I referenced Pegasus in the software survey, but didn’t deploy it:

Damn, OCaml got hands. I spent half of a Saturday trying to package Pegasus, but parked it because there’s only so long that one woman can spend manually packaging dependencies in a day.

From the afternoon I’ve spent with ocaml, it seems to have a similar package management story to python, which does not spark joy.

I’m also choosing not to (at the moment) deploy single user PDSes, or those with special requirements, such as the ones on Cloudflare Workers.

Relays

Relays are an optimisation⁸

This means that, technically, you don’t need them. Whatever is on the other end, be it a feed generator, AppServer, or moderation tool, could just open connections to each PDS on the network. Once you begin thinking through the logistics of this, especially if you run multiple services, or you remember there are over 2000 different PDSes now, it’s very very easy to invent the relay from first principles.

In short, the relay subscribes to the stream from a bunch of PDS instances (usually all of them but partial network relays⁹ exist) and collates them into one big stream that we call the Firehose.

Both Relays ended up being rather interesting to get set up.

rsky, because Cargo workspaces are hard, especially when they contain multiple binaries. I first attempted to package without using Flakes, however that proved to be an exercise in futility, so I gave up and switched to using Flakes and Crane (like Parakeet). This ended up working quite nicely!

zlay, because it’s the first time I’ve ever used Zig, although once I figured out that zlay only works on Linux, it’s worked well.

I quickly wrote the services based on the Bluesky Relay one I wrote for Parakeet, and then threw them up on the server. Of course, this failed because nothing is easy¹⁰. But, with a bit of fixing¹¹, we can connect over websockets.

PDS (Personal Data Server)

Luckily for me, Tranquil has a flake in-repo, and I’d previously packaged up rsky-pds, so getting everything up was fairly fast.

All I had to do is add them in to the system config (after writing one for rsky-pds), take a quick[citation needed] break to configure Garage as an S3 service, and wait for those good old Rust compile times (I went for dinner and let it sit).

Both PDSes want a Postgres server, which is no issue because we already have one for zlay, and need one for Parakeet. I had to do a little fiddling with rsky, as its migrations aren’t packaged in like Tranquil.

They both get configured with the keys and other bits required, and then we’re done. Tranquil spits an invite code out on first boot, and rsky-pds can generate one using the admin password.

Webbing my DID

In Tranquil, this is nice and easy. When we create an account, it’s possible to select a hosted did:web.

Once we’ve done this, we can make a post from Tranquil’s web interface and finally see events!

On rsky, we have to do this the manual way. The first step was, of course, to type “bluesky did:web” into Kagi because this is the first time I’ve done this. Thankfully Bryan wrote a post with how to do this using goat... which we can’t use.

There’s Luke’s did:web tool, although it uses bluesky’s NPM packages, and fails anyway.

A quick try with goat anyway, just to double check...

They say insanity is doing the same thing over and over hoping for a different result, I call it debugging.

And that doesn’t work either. I tried taking the JWT out of the goat request, but that just started spitting handle errors. I need to debug this locally, because I think I may be holding it wrong¹².

So now we’ve made a post from a PDS, on a did:web account, and listened to it from a non-bluesky relay, what now?

AppServers

AppServers (fka: AppViews) are the ‘backend’ of the app. They pull in data from the network, build aggregations, indices, search, and make them available to easily query from your phone or web browser. Bluesky’s main one is api.bsky.app, but there’s also now Blacksky’s (api.blacksky.community).

Non-Bluesky AppViews (ie, one whose primary role is not Bluesky compatible microblogging) also exist, such as Tangled.

We’re going to be running Parakeet, my homegrown AppServer written in Rust. I connected it to zlay, and it immediately picked up the accounts.

If we pop through and edit the profile from PDSls, that should show up too.

Good news!

Blobs

Okay, so we can serve text content perfectly fine, but Parakeet is, by default, configured to use the Bluesky CDN. Luckily, just days before I started making actual progress on this post, Lyna released Porxie. Porxie is a blob proxy, written in Rust - you give it a DID and a blob CID, and it goes off, fetches and validates the content and serves it back.

For our final trick, we’re going to spin up Porxie and imgproxy (for conversion to webp).

Porxie has a flake in the repo, which worked perfectly fine, with no additional config needed (I will need to do a policy service to restrict this to my own PDSes eventually). Imgproxy needed a bit, but not really much, and there’s loads of config examples on the internet. I took the bluesky presets from the Porxie README and they worked first time.

Don’t worry, I’m not leaving a publicly accessible image service on the internet for longer than I have to¹³. In production, you’d want to add something like Cloudflare or Bunny CDN between the user and imgproxy.

Clients

We’re coming to the end now, we want to actually be able to look at some data and make some posts that aren’t from PDSls or the Tranquil Account Manager.

The thing, however, with bluesky clients is that a lot of them are either social-app forks, like Witchsky, and Blacksky.community, or otherwise depend on Bluesky code.

I started off with Catbird (a Bluesky client using SwiftUI), because it supports a custom AppView URL and I already had it on my phone, and what do you know? I can see my account!

On the web, I tried Red Dwarf, a client by Whey that uses Microcosm and direct calls to PDSes:

Alone! on the Timeline

It’d be nice to be able to see a timeline with some other people than just me, so I asked the internet for some help.

Lewis and Nel (warning: NSFW) pointed me to some of their Tranquil PDSes and did:webs, to make the timeline more interesting. Luckily I put an admin endpoint in Parakeet a few weeks back to trigger an index.

Of course, if we interact with them, they won't be able to see it, because we're not connected to Bluesky. They'd have to log in to this Parakeet instance (and Parakeet doesn't have notifications yet)

Conclusions

What did we learn? Don’t do sysadmin for fun I mean, it’s possible to do this. Doing exactly this creates an island, so you have to be okay with that! If that’s what you want, you might even be able to skip the relay and just run a PDS and AppViewLite or Konbini. I’d still recommend using did:web though, just in case you ever do want to connect to the wider network.

The current independent PDS and client implementations work quite well¹⁴! Meanwhile, the relay story is something I’ve been concerned about the whole time I thought about this project – I don’t think I feel too much better, there’s essentially only three (Bluesky, rsky, and zlay) at the moment, so it’d be nice to see another fully featured one.

There’s a couple of AppServers out there, Parakeet, AppViewLite, and Konbini are the main non-reference ones I’m aware of. Parakeet aims for full or large-subsets of the network, whereas the other two are good for smaller deployments amongst friends. I think we're quite strong here too.

It would be remiss of me not to mention Wafrn here too as an option for a combined PDS (using a self-hosted Bluesky PDS internally), AppServer, and client – it also bridges to the Fediverse.

I haven’t mentioned DMs here, and that’s because, as far as I’m aware, there are no other implementations of the Bluesky chat service. That’d probably be a good weekend project, if someone is open for the nerdsnipe. Germ DM takes advantage of ATProto identities, but doesn’t directly replace the chat.bsky lexicon.

Ulterior Motives

I lied a bit, earlier in this post, when I said this was going to be an island with zero Bluesky code. For the sake of this experiment and blog post – to prove the point – it is, but this is part of a bigger project.

I’ve been (badly) teasing this on Bluesky for a bit: I’m opening up a service for ATProto devs and other interested parties to use for testing applications, tools, and integrations across many different pieces of software. I’m opening it up soon, hopefully (I'll need to reset the relays first from my Catbird testing), so if you need to test against a different PDS or Relay but don’t want to spin one up, then drop me a DM (bonus points if you include what you’re working on too, I’m genuinely interested!).

Eventually I’m going to automate the creation of invites, but it’ll be manual for now.

If you’re interested in the Nix setup behind this, see my dotfiles. The server is breloom.