The Linux Containers project maintains Long Term Support (LTS) releases for its core projects. Those come with 5 years of support from upstream with the first two years including bugfixes, minor improvements and security fixes and the remaining 3 years getting only security fixes.
This is now the second round of bugfix releases for LXC, LXCFS and Incus 6.0 LTS.
LXC
LXC is the oldest Linux Containers project and the basis for almost every other one of our projects. This low-level container runtime and library was first released in August 2008, led to the creation of projects like Docker and today is still actively used directly or indirectly on millions of systems.
LXCFS is a FUSE filesystem used to workaround some shortcomings of the Linux kernel when it comes to reporting available system resources to processes running in containers. The project started in late 2014 and is still actively used by Incus today as well as by some Docker and Kubernetes users.
Fix building of LXCFS on musl systems (missing include)
Incus
Incus is our most actively developed project. This virtualization platform is just over a year old but has already seen over 3500 commits by over 120 individual contributors. Its first LTS release made it usable in production environments and significantly boosted its user base.
Completion of transition to native OVSDB for OVS/OVN
Baseline CPU defintiion for clustered users
Filesystem support for io.bus and io.cache
CPU flags in server resources
Unified image support in incus-simplestreams
Completion of libovsdb transition
Using a sub-path of a volume as a disk
Per storage pool projects limits
Isolated OVN networks (no uplink)
Per-instance LXCFS
Support for environment file at create/launch time
Instance auto-restart
Column selection in all list commands
QMP command hooks and scriptlet
Live disk resize support in virtual machines
PCI devices hotplug
OVN load-balancer health checks
Promiscuous mode for OVN NICs
Ability to run off IP allocation on OVN NICs
Customizable OIDC scope request
Configurable LVM PV metadata size
Configurable OVS socket path
What’s next?
We’re expecting another LTS bugfix release for the 6.0 branches later this year.
On top of that, Q4 of 2024 will also feature non-LTS feature releases of both LXC and LXCFS as we’re trying to push out new releases of those two projects every 6 months now.
Incus will keep going with its usual monthly feature release cadence.
And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus
Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.
This release builds upon the recently added OCI support from Incus 6.3, making it even easier to run application containers. It also adds a number of useful new features for clustered and larger environments with more control on the virtual CPU used when live migrating VMs and finer grained resource constraints within projects.
The highlights for this release are:
Cluster group configuration
Per-cluster group CPU baseline
Attaching sub-directories of custom storage volumes
Per storage pool project limits
Isolated OVN networks (no uplink)
Per-instance LXCFS
Environment files at create/launch time
The full announcement and changelog can be found here. And for those who prefer videos, here’s the release overview video:
And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus
Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.
This release includes the long awaited OCI/Docker image support! With this, users who previously were either running Docker alongside Incus or Docker inside of an Incus container just to run some pretty simple software that’s only distributed as OCI images can now just do it directly in Incus.
In addition to the OCI container support, this release also comes with:
Baseline CPU definition within clusters
Filesystem support for io.bus and io.cache
Improvements to incus top
CPU flags in server resources
Unified image support in incus-simplestreams
Completion of libovsdb transition
The full announcement and changelog can be found here. And for those who prefer videos, here’s the release overview video:
And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus
Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.
Zabbly is the company I created for my freelance work. Over the year, a number of my personal projects were transferred over to being part of Zabbly, including the operation of my ASN (as399760.net), my datacenter co-location infrastructure and more.
Through Zabbly I offer a mix of by-the-hour consultation with varying prices depending on the urgency of the work (basic consultation, support, emergency support) as well as fixed-cost services, mostly related to Incus (infrastructure review, migration from LXD, remote or on-site trainings, …).
Other than Incus, Zabbly also provides up to date mainline kernel packages for Debian and Ubuntu and associated up to date ZFS packages. This is something that came out as needed for a number of projects I work on, from being able to test Incus on recent Linux kernels to avoiding Ubuntu kernel bugs on my own and NorthSec’s servers.
Zabbly is also the legal entity for donations related to my open source work, currently supporting:
And lastly, Zabbly also runs a Youtube channel covering the various projects I’m involved with. A lot of it is currently about Incus, but there is also the occasional content on NorthSec or other side projects. The channel grew to a bit over 800 subscribers in the past 10 months or so.
Now, how well is all of that doing? Well enough that I could stop relying on my savings just a few months in and turn a profit by the end of 2023. Zabbly currently has around a dozen active customers from 7 countries and across 3 continents with size ranging from individuals to large governmental agencies.
2024 has also been very good so far and while I’m not back to the level of income I had while at Canonical, I also don’t have to go through 4-5 hours of meetings a day and get to actually contribute to open source again, so I’ll gladly take the (likely temporary) pay cut!
Incus
A lot of my time in the past year has been dedicated to Incus.
This wasn’t exactly what I had planned when leaving Canonical. I was expecting LXD to keep on going as a proper Open Source project as part of the Linux Containers community. But Canonical had other plans and so things changed a fair bit over the few months following my departure.
For those not aware, the rough timeline of what happened is:
So rather than contributing to LXD while working on some other new projects, a lot of my time has instead gone into setting up the Incus project for success.
And I think I’ve been pretty successful at that as we’re seeing a monthly user base growth (based on image server interactions) of around 25%. Incus is now natively available in most Linux distributions (Alpine, Arch Linux, Debian, Gentoo, Nix, Ubuntu and Void) with more coming soon (Fedora and EPEL).
Incus has 6 maintainers, most of whom were the original LXD maintainers. We’ve seen over 100 individual contributors since Incus was forked from LXD including around 20 students from the University of Texas in Austin who contributed to Incus as part of their virtualization class.
I’ve been acting as the release manager for Incus, also running all the infrastructure behind the project, mentoring new contributors and reviewing a number of changes while also contributing a number of new features myself, some sponsored by my customers, some just based on my personal interests.
A big milestone for Incus was its 6.0 LTS release as that made it suitable for production users. Today we’re seeing around 40% of our users running the LTS release while the rest run the monthly releases.
As mentioned in my recent post about the 6.0.1 LTS releases, the Linux Containers project tries to do coordinated LTS releases on our core projects. This currently includes LXC, LXCFS and Incus.
I didn’t have to do too much work myself on LXC and LXCFS, thanks to Aleksandr Mikhalitsyn from the Canonical LXD team who’s been dealing with most of the review and issues in both LXC and LXCFS alongside other long time maintainers, Serge Hallyn and Christian Brauner.
NorthSec
NorthSec is a yearly cybersecurity conference, CTF and training provider, usually happening in late May in Montreal, Canada. It’s been operating since 2013 and is now one of the largest on-site CTF events in the world along with having a pretty sizable conference too.
I’m the current VP of Infrastructure for the event and have been involved with it from the beginning, designing and running its infrastructure, first on a bunch of old donated hardware and then slowly modernizing that to the environment we have now with proper production hardware both at our datacenter and on-site during the event.
This year, other than transitioning everything from LXD to Incus, the main focus has been on upgrading the OS on our 6 physical servers and dozens of infrastructure containers and VMs from Ubuntu 20.04 LTS to Ubuntu 24.04 LTS.
At the same time, also significantly reducing the complexity of our infrastructure by operating a single unified Incus cluster, switching to OpenID Connect and OpenFGA for access control and automating even more of our yearly infrastructure with Ansible and Terraform.
Automation is really key with NorthSec as it’s a non-profit organization with a lot of staffing changes every year, around 100 year long contributors and then an additional 50 or so on-site volunteers!
I went over the NorthSec infrastructure in a couple of YouTube videos:
I’ve cut down and focused my conference attendance a fair bit over this past year. Part of it for budgetary reasons, part of it because of having so many things going on that fitting another couple of weeks of cross-country travel was difficult.
I decided to keep attending two main events. The Linux Plumbers Conference where I co-organizer the Containers and Checkpoint-Restore Micro-Conference and FOSDEM where I co-organize both the Containers and the Kernel devrooms.
With one event usually in September/October and the other in February, this provides two good opportunities to catch up with other developers and users, get to chat a bunch and make plans for the year.
I’m looking forward to catching up with folks at the upcoming Linux Plumbers Conference in Vienna, Austria!
What’s next
I’ve got quite a lot going on, so the remaining half of 2024 and first half of 2025 are going to be quite busy and exciting!
On the Incus front, we’ve got some exciting new features coming in, like the native OCI container support, more storage options, more virtual networking features, improved deployment tooling, full coverage of Incus features in Terraform/OpenTofu and even a small immutable OS image!
NorthSec is currently wrapping up a few last items related to its 2024 edition and then it will be time to set up the development infrastructure and get started on organizing 2025!
For conferences, as mentioned above, I’ll be in Vienna, Austria in September for Linux Plumbers and expect to be in Brussels again for FOSDEM in February.
There’s also more that I’m not quite ready to talk about, but expect some great Incus related news to come out in the next few months!
Github
Twitter
LinkedIn
Mastodon
RSS feed