The Power of Raspberry Pi–MythTV tweaks and techniques

This post continues my previous ramblings about using the Raspberry Pi aboard the boat.  To start this story at the very beginning, go to this post. To go to the previous post in this series, go here.

Screenshot of MythWeb running on Raspberry Pi 2

Now to discuss some lessons we have learned about using our RPI-2 with MythTV aboard Schooner Mahdee to record over-the-air TV broadcasts for later viewing: Why record when so much is available instantly over the Internet?  Four reasons:

  • we are often anchored where Internet access is unavailable, or the speed is too slow to stream video;
  • we pay for high speed (4G) access per gigabyte;
  • broadcast HDTV provides an effectively free source of additional bandwidth and the HDTV programming is often many times much higher resolution than via Internet streaming;
  • those shows can be saved on a hard drive for later viewing anywhere and anytime.

Stunningly beautiful full 1080 HDTV programming is often broadcast and can be saved for later viewing on a big vibrant HD computer monitor–even in places with no Internet at all.  Those very high resolution shows consume around 5GB per hour.  If using our 4G hotspot, at that rate, our entire monthly Internet bandwidth for work and pleasure is equivalent to a couple of HDTV movies.  By comparison, in some locations, we can record HDTV shows around the clock for free–an incredible deal!  In practice, each 24 hours of saved H-264 encoded HDTV recordings exceeds our total Internet bandwidth usage for the entire month. Our only cost is for the hard drives and each $150 hard drive can save thousands of shows and take up less space on the boat than a small paperback novel.  So, even when we could stream videos over the Internet, we rarely do.

Since we obviously travel a lot, our settings for MythTV are not set and forget–in contrast to more stationary users.  We use Antenna Web in each new location to determine where HDTV transmitters are relative to us and how well we can expect to receive different stations.  We have both an omni-directional and a directional gain antenna to choose from.  Spinning around on a single anchor makes the directional antenna unusable, but if the boat is fixed in orientation with multiple anchors or shore-tie, we can use it to increase reception of more distant HDTV stations.  Sometimes we can get great reception with the antenna below decks in a stealth mode.  For more challenging stations, reception may require an antenna be raised on a flag halyard–not so stealthy.

We have an inexpensive account at Schedules Direct which interfaces very well with MythTV.  With an Internet connection, we generally upload new schedule data every day or two.  There is limited (just the next 12 hours) program schedule data provided for free by the stations themselves on the transmitter frequency using EIT, but scheduling works best with longer range data and Schedules Direct provides about two weeks of scheduling. That data also includes lists of actors and other credits as well as descriptive information which we retain with our recordings and in a separate custom Ruby-on-Rails (RoR) web interface and database.

Schedules Direct also allows us to have four different locations in our profile.  Thus, we have one for each major city area we typically visit.  In those profiles, we can remove stations we do not want–such as those without English language audio–and our custom settings will persist until our next visit to that area.  I am constantly amazed at the number of over-the-air broadcast stations we can receive even when we are far from a metropolitan area.

With our large MythTV database of over 10,000 previously recorded shows, nearly 100 channels of available broadcast programs in many locations, and large number of rules and priorities about what we want to record, it can take several minutes after starting the MythTV back-end for it to figure out what to record. During that time, the MythTV front-end and MythWeb browser interface running on the RPI-2 will will show no scheduled recordings. Whenever possible, I use the command-line mythtv-status to see what is in the recording queue rather than running the more resource-intensive front-end. Even mythtv-status can take several minutes to complete and will show that nothing will be recorded until all of the data has been parsed and prioritized by the MythTV back-end.  Loading new schedule data from Schedules Direct can take nearly 20 minutes to process on our–admittedly overloaded–RPI-2. Sort of like sailing tack to tack to get to one’s ultimate destination, patience is required and one can not expect a newly added show to be put into the recording queue right away. I plan for the RPI-2 to take up to five minutes before a new recording is reflected in the queue.

With an Internet connection, MythTV gets program meta data for each recording so that we have images of cover art–which makes the front-end display nicer.  MythTV can also flag commercials so that they can be easily skipped during replay or even removed from the file entirely.  Commercial detection is very resource intensive, so we set MythTV on the RPI-2 to not do any commercial detection on the recordings. With this setup, the RPI-2 is also very useful as a desktop computer for me to browse the Internet using Firefox/Iceweasel, read and write email and other miscellaneous things all while displaying current weather data and monitoring anchor holding.

One glitch that I have encountered is that the MythTV startup script does not always work in Debian Jessie due to incompatibility with SystemD. The symptom is that the startup script completes with no errors, but the back-end is not running–starting and restarting MythTV using has no effect. The problem seems to be related to the init-functions command. Adding the following just before that call in /etc/init.d/mythtv-backend as follows helped:

_SYSTEMCTL_SKIP_REDIRECT=true
. /lib/lsb/init-functions

Then that fix failed so I am now manually starting MythTV using the old InitV command:

/etc/init.d/mythtv-backend start

It is also good to ensure that MythTV is not running before starting it and to make sure there is no file named /var/run/mythtv/mythbackend.pid. First kill hung processes and then check for the pid file and delete it if necessary before staring MythTV using the above command. Since I rarely reboot the RPI-2, manually starting MythTV works fine and the InitV command is trouble free.

Another glitch is that the HD HomeRun recordings sometimes fail leaving MythTV thinking the recording is still happening. After upgrading the HomeRun to the latest firmware, this situation happens less often. Still, every few days I need to stop MythTV and wait for all the threads to be killed and then restart MythTV.

Next, I will cover some system-wide optimizations that we have made that make MythTV more reliable given the highly loaded RPI2 we run it on.

Addendum: this conversation continues on our technology-focused website Windward Ho!

The Power of Raspberry Pi–rethinking and restructuring to do even more with less

To start this story at the very beginning, go to this post. To go to the previous post in this series, go here. As it may be clear from the discussion so far, I have a tendency to increasingly add more requirements and functionality to my Raspberry Pi (and/or the newer version 2 RPI-2) computer until it cries “uncle”–or more often than not, just quietly dies under the extreme load. This post is about adding more functionality, but by being smarter, avoiding the dying part–at least for now.

myth tv running on raspberry pi
Every traditional boatwright needs to watch the Woodwright’s Shop for inspiration.

One of our luxurious activities when paying to be at a marina with shore power is to run my Shuttle small-form-factor (SFF) computer with its Intel i7 CPU and lots of RAM and lots of storage space. When it is running, it records over-the-air–using an antenna–HDTV movies and shows according to various rules we have programmed. Those shows are re-encoded by the Shuttle computer in a very efficient format to make the files smaller–1/3 to 1/5 original size–and then saved to a hard drive for later viewing–like when storm-bound in a remote bay somewhere in Alaska.

Our time in marinas is limited and the Shuttle requires too much power to run just for recording shows (producing non-shore power electricity is very expensive) and the movie recording program is way too demanding for a regular RPI. But now I thought maybe, just maybe, the new RPI-2 could record TV programs–of course I really mean do that in addition to all the other stuff the RPI-2 already does.

The program we use to record over-the-air shows is called MythTV. I have a love-hate relationship with MythTV, but we have been using it for about 10 years and it keeps track of every show ever recorded by us so that it doesn’t re-record something we already have on a hard disk or DVD or CD-ROM somewhere on Mahdee or in storage. The challenge for us is that the database for MythTV has a huge number of recordings (way over 10 thousand) to keep track of and the scheduler needs to sort through all of our rules (which are numerous and have evolved from our refinements over those same 10 years) and compare rule-matching scheduled showings to already recorded shows and determine which shows to actually record when so as to optimize the recordings. I had my doubts that even the RPI-2 would be capable–let alone doing that task while also doing the really important stuff like keeping track of the weather and how well the anchor is holding–after all, we do have to keep our priorities straight.

To make the RPI-2 able to record TV movies we had to make some changes.

1. We bought a network HDTV tuner (HD HomeRun Extend HDTC-2US) that has a built-in transcoder that re-encodes the movies in a more space-efficient format (H-264) on-the-fly. This removes the requirement for the RPI-2 to re-encode the recordings–which it couldn’t do anyway–and keeps the file sizes reasonable. In addition, H-264 is an open standard whereas the original inefficient proprietary format used in over-the-air transmissions requires one to buy a MPEG license before the RPI-2 can perform hardware decoding for viewing. So not only is the output of the HD HomeRun more space efficient, it also uses an open standard that doesn’t require the purchase of a license. The HD HomeRun even runs on 12V DC which is nice. So we installed a 60W regulated 12V DC power supply on Mahdee to use the HD HomeRun off a 12V DC battery.

2. The other change was to continually power up the Linksys Wifi router. The Linksys is also powered by 12V DC–so it can use the same newly installed regulated power supply as the HD HomeRun. Earlier, we tried to make the RPI be Mahdee’s Wifi access point in order to save the power required to run the Linksys. Running the Linksys offloads the Wifi access point functions but more importantly, also provides a needed Ethernet hub. The hub makes is possible for the HD HomeRun tuner to be available to the RPI-2, as well as to my Shuttle computer, all over fast Ethernet rather than Wifi. Further, we decided that we could also keep running the old RPI and use it via Ethernet to offload some functions from the new RPI-2–such as internet gateway, firewall, GPS server, network time server using GPS, secondary/backup anchor position alarm, Scrabble game server (oh–didn’t I mention that requirement), as well as that coveted contact and calendar schedule web server.

The net result is that, even though we now have two Raspberry Pi’s running (an RPI and an RPI-2) along with a network TV tuner and a Linksys router (the latter two alone adding 24 amp-hours a day to our afloat battery usage), our electric power usage is less than half of what it was at the dock with the Shuttle computer running. We are now getting recordings and a practical, fun to use computer that is available 24-7 even while at anchor. All that while logging weather and boat data and monitoring that important data with alarms to keep us safe. The RPI-2 has been exceptionally reliable with our previous and current up-time exceeding three weeks since the last intentional reboot.

When passage making, we can turn off the network TV tuner, Linksys router and the RPI to reduce power usage without loosing any important functionality–e.g. don’t really need our contact server in the middle of the ocean. And with no TV tuner, we can turn off MythTV and have plenty of CPU capability on the RPI-2 to run the OpenCPN chart plotter which can use either the weather/boat data pseudo serial port on the RPI-2 for GPS data or the GPS data on the RPI if it hasn’t been secured to save power. With this nice flexible and stable setup, I have to keep telling myself not to add anything new. Unfortunately, I know that it is only a matter of time before I come up with new ideas of things/tasks/programs to add to the RPI-2. With any luck, a new more capable RPI-3 will come out before I completely overload the RPI-2. Right now, however, I am really happy with both my RPI and RPI-2.  Next I will cover some tips for running MythTV on the RPI-2 without impacting all the other stuff the RPI-2 needs to do.

Fires and Fire Ants

We recently arrived at one of our favorite, idyllic anchor spots. It’s a spot that is sheltered from wind and has reliable stretches of warm dry weather–perfect for varnishing. Our arrival this year was about a week too soon because the temperatures have been way too hot (upper 90’s F with a couple days over 100F) for either sanding or varnishing. As a result, we have spent time on the computers interspersed with two or three leaps overboard into the very pleasant 78F water to cool off. Night time temperatures cool off nicely, but the ship’s cat has been a real complainer–her thick Norwegian Forrest cat fur being just too much for the local conditions and she has never been one for swimming.

The first sign of real trouble was a huge dragon fly that I had earlier admired on the fore-deck was later covered in and being consumed by ants. I threw the carcass overboard and hosed down the deck and regretted not taking a picture of the big dragon fly before its demise. Then Brenda noticed some ants down below one evening, and more the next evening. Brenda remarked that a visiting friend, Milly, had mentioned ant problems on their boat last year. The pieces were beginning to click together. We wondered where the ants could be coming from. In the light of day, there were no ants on any shore lines. Ants can’t swim, so we wondered if a few were dropping from the over hanging tree limbs or being carried by the wind.  We didn’t yet realize that the ants on Milly’s and our boat were no ordinary ants.

The two mile stretch of water on either side of where we anchor is a posted 5 mile-per-hour (mph) no wake zone to protect the embankments. It is one of just a few areas in the California Sacramento Delta that still has large trees lining the banks and excessive wakes erode the soil around tree roots and leads to fallen trees. During our varnish-fest stay here, we feel we are doing a great public service by giving speed boats another reason to slow down to something near the posted 5 mph limit; the drivers may not be literate because before and after passing us, many are going closer to 50 mph and throwing out huge wakes.  But, where the speed limit signs don’t work, a sense of courteous behavior sometimes does.

The waterway is a little wider where we anchor, but it is important to keep Mahdee near the embankment so that she is out of the way of passing boats. Further, to ensure that wind or wakes of law breaking boaters (without even a sense of what’s courteous) don’t throw Mahdee onto the rocky embankment, we usually have four lines holding us in position; the bow anchor, a stern anchor, a bow shore line and a stern shore line. It takes a little effort to get everything right, but with two of us on board we can get secured pretty fast and we sleep well at night.

Our friend Milly sails solo and so I volunteer to run the shore lines for her boat when she visits and anchors nearby. This year, when we were tying up Mahdee, I recognized the tree trunk from a previous visit, but initially thought it was a tree that I had once tied Mahdee to. The appearance of ants was a clue that perhaps this was the tree trunk that I had tied Milly to last year.

Then Brenda developed a nasty painful blistery welt and I remarked that I hadn’t seen such a welt since we lived in Texas and had fire ants in the yard and eventually the house too–a nightmare. We thought that fire ants couldn’t be this far north in California. I remembered seeing that they had been seen and reported in the LA/San Diego areas, but they couldn’t be this far north near San Francisco and Sacramento–could they?

Some online searching revealed that fire ants love a drought and that the current California drought has enabled the northward spread of these awful creatures.  California has classified fire ants as an invasive species and they are so destructive that some jurisdictions will quarantine an area, but fighting the spread of aggressive fire ants is a losing proposition. It was clear that not only were we once again in fire ant country, we had tied up right next to a big nest of them and they were in our virtual back yard–there goes the neighborhood.

In the video–a hint of things to come: I’m securing the shore line in entirely the wrong spot. The high crotch in a big tree seen towards the end of the video ultimately ended up being our safe-from-all shore anchor spot.

Shore anchor from Schooner People on Vimeo.

It was after dark and the cat’s water bowl was swarming with ants and she was clearly very distressed. I went outside with a flashlight and discovered that the bow shore-line was now twice its normal one inch diameter, thick and brown with an awe-inspiring invading army–now I was distressed too. We went to general quarters and locked down the boat. Our goal was to make it until morning when the ants would go into hiding from the heat and we could see well enough to reposition the shore line away from any fire ant nests.

Brenda prepared a bowl of boric acid and I sprayed and washed the deck and lines to knock off as many ants as possible. We left a dry segment of line where it came aboard the boat and coated that with powdered boric acid. We surrounded the water bowl with more powdered boric acid and put her food bowl in a moat using a baking pan of water. Poor kitty spent a miserable night in the heat and confused about how to flip her dry food out of the moated bowl and onto the floor which she does before eating each morsel. She whined and complained all night so that Brenda could hardly sleep and in the morning the moat around the food bowl was full of soggy uneatable cat food.  I was worried that while sleeping, I would roll over onto a string of these biting ants and be covered in painful welts. No one aboard Mahdee was happy.

First order of business the next morning was to relocate the bow shore line. Fortunately, the wind was blowing from Mahdee’s stern.  I untied Mahdee from the tree trunk and was very careful not to disturb the sleeping ants in the now obvious nest.  This stretch of slough has lots of trees to choose from so I picked the biggest tree and put the line way above ground level.

We are still left with a huge, but diminishing number of ants on the boat which are now cut off from their home nest. As we find them, we are killing and washing them overboard using the deck/anchor wash-down. With relaxed security measures, the ship’s cat is returning to her happy normal self and so are the humans.

In keeping with current events, I am now doing periodic “border patrols” to ensure our shore lines are not breached again and Brenda knows why Milly hasn’t been back to visit this year–she is reluctant to accept my misguided helpful actions of tying her boat up to a fire ant nest. The following morning, during a patrol I found the deck covered in black and white grit–another mysterious first. It turns out that a middle of the night wind-shift carried ash and soot from the devastating wildfires and coated our decks. We realized that the same drought that has enabled the fire ants to move here is also enabling horrific fires–a real double whammy for the area.

Google Analytics Alternative