freebsd: looking forward to another 10 years by jordan hubbard
TRANSCRIPT
FreeBSD: Looking forward to another next 10 years
Jordan Hubbard EuroBSDCon 2014
So, let’s look back at where we came from first…
Prediction: I can’t win this either way…
FreeBSD 1.0
From [email protected] Tue Nov 2 14:51:38 1993!Path: sran230!sranhd!sranha!wnoc-tyo-news!nec-tyo!nec-gw!sgiblab!spool.mu.edu!agate!agate!usenet!From: [email protected] (Jordan K. Hubbard)!Newsgroups: comp.os.386bsd.announce!Subject: FreeBSD 1.0 RELEASE now available!Followup-To: poster!Date: 1 Nov 1993 16:12:20 -0800!Organization: Lotus Development Ireland!Lines: 61!Sender: [email protected]!Approved: [email protected]!Message-ID: <[email protected]>!NNTP-Posting-Host: agate.berkeley.edu!!The first "official" release of FreeBSD 1.0 is now available, no more!greek letters - this is the "production" release ...
Happy 20th Anniversary!
FreeBSD is now almost 21 years old (in two months, it can drink legally in the USA)
Our first distribution media
(OK, I’m lying)
FreeBSD 1.0
Our actual distribution media
1.2 MB
FreeBSD 1.0
Some of the consequences...
The 1.0 ports collection
My first FreeBSD dream build machine...
I paid $1600 for my first 1Gb hard drive to do builds...
My first FreeBSD dream laptop
That is <= Warner!
First conference (1999)
Commercial interests 20 years ago
• Internet Service Providers (large and small)
• SOHO web servers / routers (very DIY)
• Major services like Yahoo!, Hotmail, etc.
• Basic developer desktop machines
• Majority of FreeBSD machines were physical PCs that ran off of AC power
Key take-away: FreeBSD’s role was fairly overt
20 years later...
New install media: 64GB USB thumb drive. Holy crap!
20 years later...
RB-Pi: SBC Far more powerful than my first FreeBSD machine. Cost: Under $50 (with storage card)
20 years later...
My dream laptop has evolved (and has BSD included)
20 years later...• FreeBSD release version is up to 10.0
• Over 24000 ports (vs 70!)
• Hundreds of committers, from both academia and commercial backgrounds
• Sources of long-term funding (Foundation, other sponsors)
• … But the commercial interests and overall market have changed significantly
Hey BSD: Let’s see that license again!
The GPLv3: Scaring the crap out of lawyers since 2007
The GPL experience
Commercial interests Today
• FreeBSD is the underlying OS technology for routers, load balancers, security monitors, file servers, etc.
• Basis for “software appliances” like pfSense, FreeNAS and quite a few others
• Even the base OS for a very popular gaming console
• “Embedded” market is morphing and exploding
Key take-away: FreeBSD’s role, and even its name, is now fairly covert
Some hyperbole (which does actually makes a point)
Unix OS deployments today
(waaaa?)
And in the Enterprise…• “BYOD” movement has killed desktop growth
while giving IT departments severe heartburn
• “Cloud” software / storage / computational resource consolidation have all but killed the DIY datacenter
• Much better automation choices (*stack, chef, puppet) gives rise to the “2 guys with their laptops in a coffee shop” social media / enterprise startup
How we see ourselves
OS Designers
How the world sees us
OS Designers
Closer to the truth
OS Designers
The rise of virtualization
• vmware, Xenserver, Hyper-v, even bhyve are all acceptable hypervisors now. Majority of OS deployments (and devices) are no longer physical.
• Automation tools work hardest to erase the notion of machine personality by pushing OS installation and config metadata to external sources
• Being virtual and/or automated means the entire environment is far more dynamic; individual OS instances not installed/configured by mere humans
The rise of mobile
• More Unix machines are running on batteries than are plugged into AC by several orders of magnitude
• Talking to one or more radios has replaced physical cables
• The obvious: Power consumption and dynamic interface / connection management is important
• Less obvious: High-level debugging and telemetry technology become critical to success
What does this all mean?
• We need to be open to fundamentally new approaches and ruthlessly cull what is no longer demonstrably useful to the 99%
• We need to be willing to shamelessly steal^H^H^H^H adopt things that are working elsewhere
• We need to take on some big-picture challenges that will appeal to the next generation of hackers (where’s the next mountain?)
One Project Idea All OS / App configuration data
• We need a lingua franca format yesterday!
• Whether it be XML, YAML or JSON, we also need a single API to read / write / abstract away the details of finding config data.
• Existing tools /services need to convert their legacy formats into this one and use the same API or the tower of babel will persist
Working Example
• All OS and app configuration data in OS X and iOS are XML plist files, even GNU emacs and X11.org’s preferences!
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>SULastCheckTime</key> <date>2014-02-09T02:45:56Z</date> <key>cache_fonts</key> <true/> <key>done_xinit_check</key> <true/> <key>no_auth</key> <false/> <key>nolisten_tcp</key> <true/> </dict> </plist>
One Project Idea A centralized event notification system
• You just can’t deal with:
• Radios powering up and down at odd times
• Network configuration changing rapidly
• A node changing its identity almost entirely on the fly
• Critical system caches needing invalidation
… without a centralized way of being able to know about these sorts of events!
• notify(3) APIs in OS X / iOS and corresponding notifyd daemon
• Send it a SIGUSR1 on OS X and you will see a huge number of notification names and their subscribers in /var/run/notifyd_pid.status - it has become insanely useful!
"com.apple.system.lowdiskspace.system" uid=0 gid=0 333 17 32 port !"com.apple.system.timezone" uid=0 gid=0 333 slot 7 = 1 1018 1 memory 1011 10 port 1011 8 memory 1011 6 port !"com.apple.system.powermanagement.SystemLoadAdvisory" uid=0 gid=0 333 slot 13 = 19 406 44 port 944 12 port 406 35 memory !"com.apple.system.config.network_change" uid=0 gid=0 333 slot 39 = 129 190 17 memory 845 28 memory 845 26 memory !"ids-device-nearby-0FC85E3A-7779-4EAA-AD02-70292707A33C" uid=0 gid=0 333 229 30 port 402 19 port 257 30 port 242 23 port
Working Example
One Project Idea Service startup and wrangling
• /etc/rc.d is quite sophisticated for what it does, but the paint on /etc/rc is obvious
• Too many things need to know explicitly about dependencies (when can I start? What has to start before me?)
• Power wrangling and automation depend on being able to start and stop services easily and at will
Working Example
!
• I’m trying really hard not to suggest launchd here (so I won’t)
• The idea of registering everything up-front with a broker and then letting IPC / timers / HW events start things from there (in cascade fashion) is still the right architecture
• Even the linux die-hards have essentially grasped the necessity of systemd (even though they’re going to hate on it for awhile longer)
One Project Idea Telemetry & Remote Debugging
• We really need a centralized way of being able to collect data from appliances / mobile devices
• The NSA don’t need our help (they already do this upstream) but FreeBSD downstream end-users DO
• Debugging the software stack on mobile devices is hard. It’s time for remote debugging support
One Project Idea Reference phone / tablet ports
• Running on SBCs is excellent for bootstrapping, but there’s no “real world” hardware to support (displays, radios, accelerometers, etc)
• Running on hardware you can actually use and carry around validates power / telemetry work
• Real Hardware forces you to think about the entire software stack
In summary
• We need to become more lego-like (and toss those legos we don’t need out of the box) in our architecture
• The hardware platforms we choose need to be genuinely relevant in terms of mass appeal
• We need to be more willing to learn from those who have gone before us in these emerging markets (BSD has a bit of a “rep” there)
FreeBSD 20th Anniversary Party
FreeBSD: The future is ours if we want it!