Scott Dowdle's blog
I doubt many people actually noticed but the server that runs this site (and about a dozen more) and my email... died sometime between 4:30 and 6:40 this morning. I texted (using Google Voice since I do NOT own/want a cell phone) Warren (since the server is in his house) about 6:38 and he responding back at 7:10 saying the server was dead. It would power on for about a second and then immediately turn off. It was full of dust bunnies so Warren cleaned it out and re-seated everything that might have had a loose connection... but still power-on failure. Warren opened up the power supply and cleaned it out. Inspecting both the motherboard and the power supply, there were no obviously bad components... no bad caps. Still no results.
Without a quick fix we decided to try to get a temporary system going... perhaps using a desktop computer with 4 SATA ports. 4 SATA ports are needed because the dead server has 4 hard drives that are in a Linux software RAID 5 configuration. Luckily Warren had access to a spare Dell OptiPlex 960 Core 2 Duo system with 8GB of RAM and 4 SATA ports. Many tower systems that have 2 hard drive bays come with 4 SATA ports so they can support 2 HDs and 2 optical drives. Getting the cables to go where they needed to go (both data and power) for loose drives seemed to work and for the time being the desktop power supply seems to be beefy enough to run all of the drives. Seeing as we run CentOS 6.x for OpenVZ... I was concerned that the network chipset would be too new... and for a little while that seemed to be the case... but Warren booted from a CentOS 6.7 LiveDVD and the network worked fine... so we knew it was a configuration issue from the previous hardware configuration. Turns out the NIC was detected as eth2 rather than eth0. While that could probably have been resolved by nuking a udev rule file somewhere, Warren just moved and edited the eth0 config file so it was eth2 and we were up and running again.
While I do have backups (a couple of days old) and have previously researched a few cloud services (6sync and fastmail are on my radar) just in case, it is great to be back up after a few hours rather than having to worry about transferring a few hundred gigabytes of data before we are back up.
For the mid to long-term we haven't thought about where to go... and are just happy to be up and running again. Warren had it back up about 2 PM so he put a good 7 hours in today. Turns out Warren starts the Montanan Marathon early tomorrow morning, which he has been training all year for, so he wanted to get some rest today. I hope he does get some rest after saving our hobby server. Thanks Warren! I hope you do well in the run tomorrow buddy.
About a month ago the Blender folks released a new film project named Cosmos Laundromat.
Two days ago the ffmpeg folks released version 2.8. I saw one of the changes was that for webm they are now defaulting to using the vp9 video codec and the opus audio codec. Previous releases defaulted to webm with vp8 and ogg.
I've been following vp9 for a few years now... and every once in a while I'd try the libvpx tools and ffmpeg's support for vp9... but it was never quite there yet for non-experts (me). With this release, I say that vp9 is very, very close to viable. The only problem is that the encoding speed still leaves a lot to be desired. For the best results, two pass encoding is required. Using a number of 1080p video sources in h.264 format (all of the Blender films downloaded from YouTube) and a contemporary Intel i7 CPU, I get a little over 100 Frames Per Second (FPS) on pass 1 and between 7 - 9 FPS on pass 2. With a video that is ~24 FPS (all my source videos) it takes a little over 3x realtime to encode them. Well, more because of the two passes. The more popular codecs in ffmpeg are better than real time including vp8/ogg-based webm. Of course those numbers are all highly variable depending on the source material and the hardware you run it on... but you get the idea.
Ok, so encoding speed isn't so great. I'm sure that will get better over time. I'd expect it to be cut in half sometime over the next couple of years... if we are lucky... that and faster hardware... and perhaps GPU encoding support in newer hardware.
But anyway, enough about the encoding speed, how is the quality? Well, see for yourself. I think it truly lives up to the 1/2 the filesize for the same quality compared to vp8/ogg or h.264. I embed Cosmos Laundromat above then directly link to additional ones below if you want more. vp9/opus webm files should play in contemporary versions of Firefox, Google Chrome, Microsoft IE if you have some plugin installed (haven't tried it) and supposedly coming soon to Microsoft's Edge browser in Windows 10. Safari? Probably never. Play in your browser or right-click download and play with your preferred media player.
Don't blame any download / playback retrieval slowness on vp9... it's probably a lack of bandwidth on the server side... so be patient and pause it until some is buffered. Moving around the playhead should work fairly well unless bandwidth is an issue.
Full screen that if you want. Doesn't that look great? It's 720 x 302 resolution with 400 kbit video and 96 kbit audio. That's why all of the files have 496k in the name. So that's like 62KB / second transfer. Pretty good quality huh?
More vp9 re-encodes with ffmepg 2.8:
2006-Elephants_Dream-496k.webm (38.4MB, 10:53)
2008-Big_Buck_Bunny-496k.webm (31.8 MB, 9:56)
2010-Sintel-496k.webm (52.9 MB, 14:48)
2012-Tears_of_Steel-496k.webm (43.3 MB, 12:14)
2013-Caminandes-Gran_Dillama-496k.webm (8.7 MB, 2:26)
Want to know how it works on Hollywood / live action movies? Here's the trailer for the upcoming film, "The Martian". I don't think I'll get into any trouble for posting a trailer, right? Again, 720 x 405 @ 496 kbit. The black bars were in the original and I didn't remove them.
2015-The_Martian-Trailer-496k.webm (11.5 MB, 3:17)
Want to give vp9 a try? I doubt many distros have packages for ffmpeg 2.8 yet but you can download the static .tar.xz from ffmpeg's site and run it on most Linux distros. That's what I did. I'll leave finding the URL up to the reader because it will certainly change.
So far as encoding goes, I didn't do anything fancy. Just something like:
ffmpeg28 -y -i source.mp4 -f webm -vf scale=720:-1 -b:v 400k -an -pass 1 output.webm.pass1 ;
ffmpeg28 -y -i source.mp4 -f webm -vf scale=720:-1 -b:v 400k -b:a 96k -pass 2 output.webm
I took the static ffmpeg binary and plopped it in ~/bin/ffmpeg28 so I could easily tell it apart from the stock ffmpeg binary. Enjoy!
Everyone knows Red Hat's Dan Walsh as the SELinux guy... and more recently as the guy who pronounces Docker in a Boston accent as "Dockah". Turns out he was the subject of a recent TNT Network's Rizzoli and Isles episode. Enjoy. Oh, and, "All roads lead... to Dan Walsh." (the missing last 3 seconds)
For those with iFrame issues, here's the direct link: dan-walsh-mystery.webm
Just noticed this short video and I thought it might interest some. I already have a "career in Linux" myself. How about you?
I ran across this video today. I know there is a lot of fever over the $9 C.H.I.P. but I found it very interesting listening to one of the C.H.I.P. guy talk about the challenge and the rewards of such a project. Enjoy!
Ever experienced a presentation by Michael Meeks? He reminds me of Monty Python... but I digress. Michael is a well known LibreOffice person and here is a recent talk (published June 26, 2015) from him for, I assume, the Swish Open Source User Group.
I decided today would be a good day to post this because LibreOffice 5 was released. Enjoy!
While it might be a while before this becomes complete and compatible enough to run on your own mobile device, it sure looks good to this Plasma 5 desktop user. Enjoy.
For those who might be iFrame impaired, here is the direct YouTube link: https://www.youtube.com/watch?v=auuQA0Q8qpM
For anyone who hasn't seen this yet who is interested in containers, this is a must see. Watch Red Hat's SELinux guru Dan Walsh explain and demo Super Privileged Containers from the Red Hat Summit 2015. Enjoy!
For those who are iFrame challenged, here's the direct YouTube link: https://www.youtube.com/watch?v=dM2Fc53Dtd4
Oh look... another presentation by Lennart on systemd... this one from the CoreOS Fest 2015. Enjoy!
For those who are iFrame challenged, here's the YouTube link: https://www.youtube.com/watch?v=VIPonFvPlAs
Odin and the OpenVZ Project announced the beta release of a new version of Virtuozzo today. This is also the next version of OpenVZ as the two are merging closer together. See their release announcement.
There will eventually be two distinct versions... a free version and a commercial version. So far as I can tell they currently call it Virtuozzo 7 but in a comparison wiki page they use the column names Virtuozzo 7 OpenVZ (V7O) and Virtuozzo 7 Commercial (V7C). The original OpenVZ, which is still considered the stable OpenVZ release at this time based on the EL6-based OpenVZ kernel, appears to be called OpenVZ Legacy.
Odin had previously released the source code to a number of the Virtuozzo tools (mailing list post) and followed that up with the release of spec-like source files used by Virtuozzo's vztt OS Template build system. The plan is to migrate away from the OpenVZ specific tools (like vzctl, vzlist, vzquota, and vzmigrate) to the Virtuozzo specific tools although there will probably be some overlap for a while.
The release includes source code, binary packages and a bare-metal distro installer DVD iso.
Bare Metal Installer
I got a chance to check out the bare-metal installer today inside of a KVM virtual machine. I must admit that I'm not very familiar with previous Virtuozzo releases but I am a semi-expert when it comes to OpenVZ. Getting used to the new system is taking some effort but will all be for the better.
I didn't make any screenshots yet of the installer... I may do that later... but it is very similar to that of RHEL7 (and clones) because it is built by and based on CloudLinux... which is based on EL7.
What is CloudLinux? CloudLinux is a company that makes a commercial multi-tenant hosting product... that appears to provide container (or container-like) isolation as well as Apache and PHP enhancements specifically for multi-tenant hosting needs. CloudLinux also offers KernelCare-based reboot-less kernel updates. CloudLinux's is definitely independent from Odin and the CloudLinux products are in no way related to Virtuozzo. Odin and CloudLinux are partners however.
Why is the distro based on CloudLinux and does one need a CloudLinux subscription to use it? Well it turns out that Odin really didn't want to put forth all of the effort and time required to produce a completely new EL7-clone. CloudLinux is already an expert at that... so Odin partnered with CloudLinux to produce a EL7-based distro for Virtuozzo 7. While CloudLinux built it and (I think) there are a few underlying CloudLinux packages, everything included is FOSS (Free and Open Source Software). It DOES NOT and WILL NOT require a CloudLinux subscription to use... because it is not related to CloudLinux's product line nor does it contain any of the CloudLinux product features.
The confusion was increased when I did a yum update post-install and if failed with a yum repo error asking me to register with CloudLinux. Turns out that is a bug in this initial release and registration is NOT needed. There is a manual fix of editing a repo file in /etc/yum.repos.ed/) and replacing the incorrect base and updates URLs with a working ones. This and and other bugs that are sure to crop up will be addressed in future iso builds which are currently slated for weekly release... as well as daily package builds and updates available via yum.
More Questions, Some Answers
So this is the first effort to merge Virtuozzo and OpenVZ together... and again... me being very Virtuozzo ignorant... there is a lot to learn. How does the new system differ from OpenVZ? What are the new features coming from Virtuozzo? I don't know if I can answer every conceivable question but I was able to publicly chat with Odin's sergeyb in the #openvz IRC channel on the Freenode IRC network. I also emailed the CloudLinux folks and got a reply back. Here's what I've been able to figure out so far.
Why CloudLinux? - I mentioned that already above, but Odin didn't want to engineer their own EL7 clone so they got CloudLinux to do it for them and it was built specifically for Virtuozzo and not related to any of the CloudLinux products... and you do not need a subscription from Odin nor CloudLinux to use it.
What virtualization does it support? - Previous Virtuozzo products supported not only containers but a proprietary virtual machine hypervisor made by Odin/Parallels. In Virtuozzo 7 (both OpenVZ and Commercial so far as I can tell) the proprietary hypervisor has been replaced with the Linux kernel built-in one... KVM. See: https://openvz.org/QEMU
How about libvirt support? - Anyone familiar with EL7's default libvirtd setup for KVM will be happy to know that it is maintained. libvirtd is running by default and the network interfaces you'd expect to be there, are. virsh and virt-manager should work as expected for KVM.
Odin has been doing some libvirt development and supposedly both virsh and virt-manager should work with VZ7 containers. They are working with upstream. libvirt has supposedly supported OpenVZ for some time but there weren't any client applications that supported OpenVZ. That is changing. See: https://openvz.org/LibVirt
Command line tools? - OpenVZ's vzctl is there as is Virtuozzo's prlctl.
How about GUIs or web-based management tools? - That seems to be unclear at this time. I believe V7C will offer web-based management but I'm not sure about V7O. As mentioned in the previous question, virt-manager... which is a GUI management tool... should be usable for both containers and KVM VMs. virsh / virt-manager VZ7 container support remains to be seen but it is definitely on the roadmap.
Any other new features? - Supposedly VZ7 has a fourth-generation resource management system that I don't know much about yet. Other than the most obvious stuff (EL7-based kernel, KVM, libvirt support, Virtuozzo tools, etc), I haven't had time to absorb much yet so unfortunately I can't speak to many of the new features. I'm sure there are tons.
About OS Templates
I created a CentOS 6 container on the new system... and rather than downloading a pre-created OS Template that is a big .tar.gz file (as with OpenVZ Legacy) it downloaded individual rpm packages. It appears to build OS Templates on demand from current packages on-demand BUT it uses a caching system whereby it will hold on to previously downloaded packages in a cache directory somewhere under /vz/template/. If the desired OS Template doesn't exist already in /vz/template/cache/ the required packages are downloaded, a temporary ploop image made, the packages installed, and then the ploop disk image is compressed and added to /vz/template/cache as a pre-created OS Template. So the end result for my CentOS 6 container created /vz/template/cache/centos-6-x86_64.plain.ploopv2.tar.lz4. I manually downloaded an OpenVZ Legacy OS Template and placed it in /vz/template/cache but it was ignored so at this time, I do not think they are compatible / usable.
The only OS Template available at time of writing was CentOS 6 but I assume they'll eventually have all of the various Linux distros available as in the past... both rpm and deb based ones. We'll just have to wait and see.
As previously mentioned, Odin has already released the source code to vztt (Virtuozzo's OS Template build system) as well as some source files for CentOS, Debian and Ubuntu template creation. They have also admitted that coming from closed source, vztt is a bit over-complicated and not easy-to-use. They plan on changing that ASAP but help from the community would definitely be appreciated.
How about KVM VMs?
I'm currently on vacation and only have access to a laptop running Fedora 22... that I'm typing this from... and didn't want to nuke it... so I installed the bare-metal distro inside of a KVM virtual machine. I didn't really want to try nested KVM. That would definitely not have been a legitimate test of the new system... but I expect libvirtd, virsh, and virt-manager to work and behave as expected.
Despite the lack of perfection in this initial release Virtuozzo 7 shows a lot of promise. While it is a bit jarring coming from OpenVZ Legacy... with all of the changes... the new features... especially KVM... really show promise and I'll be watching all of the updates as they happen. There certainly is a lot of work left to do but this is definitely a good start.
I'd love to hear from other users to find out what experiences they have.
Congrats Odin and OpenVZ! I only wish I had a glass of champagne and could offer up a respectable toast... and that there were others around me to clank glasses with. :)
This article is translated to the Bosnian language by Vlada Catalic.