Blog Urbit Development Update June 2024
Jeremy Tunnell at
~2024.5.31 UF Update
May 31st, 2024
OKRs
The following are the OKRs we’ve been operating with that were shared with the board during our meeting last week. We’re expecting to revise them next month upon analysis of the developer experience survey results and discussion with leaders of other ecosystem organizations during Lake Summit.
Make Urbit’s network and runtime fast
- Q2 (DONE): Launch Ares developer alpha
- Q3 (WIP): Increase network throughput from 400kbps to 200mbps in the best case, and 20mbps in the worst case
- Q3 (TODO): Improve baseline Nock performance by 100x
- Q4 (TODO): Ares network release to ≥ 50% of all online ships
Status
We released the Ares developer alpha today. The below release announcement and linked release notes describe the remaining work to be done, which is in service of improving baseline Nock performance and getting to a point that Ares can be rolled out to the live network.
Ares Developer AlphaMay 31st, 2024
We’re pleased to announce the release of the Ares developer alpha.
The release notes can be found here on GitHub, and the instructions for running it can be found here.
Please be aware that this is an alpha. There are many features known to be missing that will be incrementally added in subsequent alpha releases throughout the coming weeks, and many bugs that need fixing as well.
Huge shoutout to ~ritpub-sipsyl , ~finmep-lanteb , ~barter-simsum , ~mastyr-bottec , ~master-morzod, and the many others that have contributed to this project so far — TYFYS!
UF•Announcements
Directed messaging (DM) is moving along well. The team has this running end-to-end with a simple file transfer app that achieves speeds of 40mbps in the worst (and best) case. Here’s a rundown on the lines of work:
- Encryption: Our new encryption algorithm, lock-step streaming, has a bug that prevents it from hitting peak throughput. That bug has likely been resolved as of today, but still needs testing.
- Migration/Driver Unification: Rolling out the new Ames implementation is tricky. We need to ensure that, as new ships become DM-aware, they’re able to communicate with ships that don’t yet speak DM, learn which ships do speak DM, and negotiate an upgrade to DM on a per-ship basis, migrating old Ames state into DM. This is a big change, so we’re designing the migration such that ships can switch back and forth between old ames and new ames (DM). The initial OTA will roll DM out to all ships with DM off by default. Once we’re confident that it’s free of bugs, we’ll roll out a subsequent OTA that turns it on by default for all ships. Driver unification refers to combining the DM work with existing ames so that ships can use both simultaneously. This work is under active development and nearing completion.
- Symmetric routing: the network currently uses asymmetric routing. When Ship A attempts to contact Ship B, A will ask its galaxy for a route. When B responds, it asks its own galaxy for a route to A, rather than using the same route that A used to reach B. The logic for this still needs to be written.
Our stretch goal is to cut a release candidate (including all of the above) by ~2024.6.17, ~2024.6.30 at latest.
Improve Urbit’s fundamental capabilities
- Q1 (DONE): Increase storage limit to 64TB
- Q2 (WIP): Developer alpha of shrubbery released
- Q3 (TODO): 10 independently developed shrub-based PoC apps
- Q4 (TODO): Secure the kernel from userspace
- Q4 (TODO): Secure userspace software from other userspace software
Status
The developer alpha of shrubbery is on track for release at Lake Summit. It’ll be available as a desk that can be installed over the network bundled with:
- Sky, the developer IDE for writing and interacting with shrubs (which is itself written in shrubbery).
- A suite of documented sample applications, like Accel, the P2P networked version of excel.
The only remaining step to complete before Shrubbery can be launched is the jinx hint, which prevents user-defined code (e.g. the kind of code that can be defined within Accel) from triggering an infinite loop.
Lots of discussion about security is going on right now as we’re nearing the completion of Directed Messaging, which frees up many of our most senior core developers. Building security into Gall is known to be really tough, as we’ve cut our teeth on it multiple times throughout last year. Shrubbery presents compelling alternative solutions to userspace permissioning and frontend sandboxing. These designs are being evaluated and explored, and if found viable, may allow us to deliver a complete security solution for shrub-based applications before EOY. Doing the same in Gall by contrast will take far longer, which we know from experience.
Eliminate barriers to developer adoption
- Q1 (DONE): Wasm programs can run on Urbit
- Q2 (WIP): Provide an accessible standard library to userspace
- Q3 (WIP): Wasm programs run performantly (≥75% performance of Hoon programs)
- Q3 (TODO): Ship a real app written in a Wasm-compatible language
- Q4 (TODO): Four apps written in Wasm-compatible languages by external developers
Status
The accessible standard library to userspace is written and awaiting merge, which should happen in the next few weeks. Updating documentation to reference the existence of these new libraries still needs to be done.
The urwasm project continues to make steady progress. This project is the most viable approach to supporting Urbit development with mainstream languages that we have. It’s also the most compelling, because it provides support for many mainstream languages in one fell swoop. It already provides the ability to run Wasm programs natively on Urbit, albeit very slowly. The current line of work is to provide sufficient optimization. A full writeup on the project can be found here. A status report from the developer working on this can be found here.
While not represented by KRs, we recently made some progress on a project that is related to the spirit of this objective: a series of CI tests were written for several parts of the developer documentation to catch broken code samples when Urbit updates are released.
I expect this OKR to have some additions post-Lake Summit. Wasm is still super important, but the developer experience survey shone some light on other things that we should tackle.