Tag Archives: featured

Doctor, my wife tells me I stop breathing in my sleep. A lot.

Me, wearing a full face mask, attached by a hose to a Philips BiPap system
Me, wearing a full face mask, attached by a hose to a Philips BiPap system

In early October, I hurt my back, presumably while lifting my son out of his car seat (to be honest, I was lifting a lot of stuff around then, and anything could have done it).  I scheduled an urgent care appointment with my doctor after it didn't clear up for a few days, and was affecting my ability to work, and function. Becky's been bothering me for months, maybe years about my snoring problems, and that I wake up gasping for air most nights. I've always got something more important to do than go to see the doctor, but when I get hurt bad enough to make doing all those things unfeasible, suddenly, I must re-prioritize. So I was given a handful of muscle relaxers and some halfway decent painkillers, and was going to be sent on my way. The PA-C who was looking at me asked if I had any other troubles. I finally decided to say it. – "My wife tells me I stop breathing in my sleep. A lot." She said that she's not supposed to treat non-urgent issues during an urgent care visit, but this was an easy enough referral, so why not?

The next day, I get a call from Troy Sleep Center, letting me know they got the referral and asking me to come in for an appointment. I think the appointment was sometime in November. I stopped in before work and took care of that appointment. The questionnaires I answered had all sorts of benign sounding, but almost certainly a sign of something truly menacing questions (do you wake up with a headache? Have you ever fallen asleep while driving? etc). I've always had sleep problems, ever since I was a newborn, so I could answer a lot of them with "yes". The doctor comes in, asks a few questions, and schedules me for a sleep study on December 5th.

Me, wearing EEG probes and chest strap sensors for a sleep study
Me, wearing EEG probes and chest strap sensors for a sleep study

If you've never been to a sleep study, I'll tell you – it's kind of cool but odd at the same time. I came to an office building at night (parking was really nice, obviously), and was led through interior hallways within the office. They opened a door to an office, and inside was suddenly a completely different world. It looked like a person's bedroom. There was a bed, a nightstand, high pile carpet, draperies, even a TV set. And a night vision camera. And a few subtle wires draped over the headstand. The guy told me to get into my pajamas (honestly, my wife had to buy me something because I don't ordinarily do pajamas, but you don't want to go naked through an office building, so…). They came and had me follow them to a room where they set me up with all sorts of wires. After that, they hooked me to that umbilical of cords draped over the bed, tossed a pulse oximeter on my finger, and told me to have a nice rest.

I don't know if you've ever tried resting at 9pm with glue in your hair, wires attached to random extremities, wearing what feels like two seat belts across your chest… I'm not going to sugar coat this – it's not fun, or easy. I think it's ultimately worth it, but I'm not going to sugar coat it and tell you it was awesome. And for my trouble, the guy just pulls the leads out and sends me home at 5am, telling me they'll be in touch. No feedback, no nothing.

So I have to wait about a week for my followup appointment. The doctor comes in the room, asks me if I know what obstructive sleep apnea is. Of course I do. So she tells me that's what I have (duh), and that I stopped breathing between 40 and 90 times an hour. Not during the test, but PER HOUR. 40 was the overall score, 90 was when I was sleeping on my back (I love sleeping on my back, but I've been prohibited by Becky from sleeping on my back for forever, because I stop breathing in that position more than others).

2013-12-28 22.23.47
So you are having trouble sleeping. Have you tried sleeping with wires glued to your head? No sweat? Let's try it with the wires, a mask and tube, and we'll blow air in your nose!

She gives me a diagnosis of "Severe Obstructive Sleep Apnea" and sends me off for a second sleep study to calibrate the CPAP device. I get my study scheduled for 12/29 (yay, holidays!). I go through the study. it was kind of uncomfortable, but doable. I get woken up and sent on my way.  January 3rd, I come to the office, and my doctor walks into the room. He looks at the sheet, and says something along the lines of "Do you know what central apnea is?" … "Yes. It's where your mind doesn't send signals that you should breathe at all".. "Yes, that's what it is". He hands me a sleep study showing that all my obstructive apnea events basically became central apnea events instead. My body was like 'OH HELL NO' and regularly would not fight the pressure of the machine to exhale.

Damn, and here I thought I did good. I mean sure, it was better than nothing at all, which is scary to think about, but it was clearly not tolerated. My diagnosis was changed to "Severe Complex Sleep Apnea", and sleep study #3 was scheduled for last week. That time, I used a BiPap, and the following Monday, I got a call from my home health care company telling me they were going through insurance company pre-authorization for my BiPap unit (the doctor hadn't called me yet). Today, I was given the results (apparently bilevel pressure of 12 and 9 was perfect, with an AHI score of 0), and picked up my unit (which is considerably more expensive than a standard CPAP device).

So you may be wondering: "Hey, what is a BiPap? I've heard of CPAPs, those just blow air into your nose all the time to keep your airway open, but I've not heard of a BiPap before."

A BiPap works by detecting your breathing cycles. Where a CPAP mask basically leaks any air you're not currently breathing in, and normal people can just breathe against the pressure, I can't. The BiPap detects that I'm no longer breathing in, and switches the pressure to a lower value – enough to keep the airway slightly open, but not so much that I can't breathe out. It then waits for negative pressure in the mask, and switches the pressure up at that point. If I fail to breathe in a certain amount of time, it kicks on the pressure anyway, assuming my airway is blocked, or I'm undergoing a central apnea.

I'm really excited tonight. It may be the first time I've slept well since I was 15, when I first started complaining to doctors about being tired all the time, and the doctors checked every possible thing, but never referred me to a sleep doctor.

How AT&T, Verizon, and Comcast are working together to cheat you by "Discontinuing Landline Service"

(TL;DR summary: AT&T is buying entire legislatures to rewrite the laws to allow them to become a fully unregulated company with no wholesale obligations, creating a de-facto monopoly. They can (and likely will) use it to squash or hurt wireless competitors as well, as they're permitted to favor their own subsidiaries with the network built and created over a hundred plus year monopoly, and Comcast is fully on-board because they'd like to split the market created when all their competitors are dead)

UPDATE: The bill passed with some modifications. Click here for more information.

There is a new bill going through the michigan legislature right now. Referred to as Senate Bill 636, it claims to provide for the discontinuance of landline phone service.

Let me explain what this actually does, and why you care, even if you only use wireless phones and cable.

First, the bill shores up a lot of language in the intercarrier compensation reform that went through last year. No big deal.

The big deal is that there are:

  • Language changes that seem to trend away from the use of tariffs to provide published service types, rates, and service territories. No big deal on it's face, but as part of a larger scheme, why this is being altered makes sense.
  • Language changes that allow a provider to leave a community high and dry, with no telephone service, starting in 2017. That's just over 3 years from now.

So you'll ask – I use cellphones and use Comcast for internet – how does this apply to me?

First – there are thousands of people that will continue to need, and desire landline service. And businesses aren't going to switch away from landline phone service to cell phones. And AT&T has no desire to cede this business to a competitor, so what's the deal?

Part of this dates back to the last "reform" of the Michigan Telecommunications Act. HB 4314 of 2011 removed many regulatory oversights that protected customers and competitors.

Most of interest today are clauses of HB4314 that:

  • Permit AT&T and other companies to sell, lease, or otherwise transfer assets and sell service to an affiliate below cost,
  • Allow companies to discriminate in favor of an affiliated burglar and fire alarm service over a similar service offered by another provider,
  • Allow AT&T and other phone companies to discontinue service in any area provided with anything resembling a two-way telecommunications service including wireless, radio, or Voice Over IP service. Last year's bill does not permit them to leave customers high and dry, there must be something there that people can use as a substitute instead.

(Have you noticed lately that AT&T through their UVerse brand, and Comcast through their Xfinity brand are offering home security and automation? That was sudden, right? Well – there's a reason. AT&T UVerse and Comcast are not required to provide landline service suitable for use by outside alarm company vendors for their services. And quality requirements are eliminated as well, so if your current alarm system doesn't work right, tough. So now AT&T and Comcast can deliberately impair alarm systems, then sell you their own when they don't work instead of fixing the degradation.)

But more concerning is AT&T's trend of wanting to leave the "landline" business. First it's important to understand that what legislators and lawyers consider a landline, and what you consider a landline are TWO COMPLETELY DIFFERENT THINGS.

A layperson looks at a phone coming out of their wall with a wire attached to it, and says "landline!"

What AT&T wants to eliminate is something very specific:

Telephone service that is:

  • Regulated for price and quality,
  • Offered on nondiscriminatory, consistent and identical terms to everyone in their service area,
  • Is delivered on copper pair,
  • Has dialtone even with no equipment attached inside the customer premises

Do you have uverse? Then you don't have "landline" service – even if you have phone service from them.

You'll note that AT&T speaks a lot about how their "landline" service is losing customers hand over fist, and is highly unprofitable. Guess who they are losing much of their lines to? AT&T.

See where I earlier mentioned 2011's HB4314 "Permits AT&T and other companies to sell, lease, or otherwise transfer assets and sell service to an affiliate below cost"? They are allowed to sell their phone service to themselves at below cost. When you switch from AT&T landline to UVerse, you are "disconnecting your landline". For regulatory purposes, AT&T can claim they lost your services to a competitor. A competitor that the more they sell services to, the more unprofitable their "landline" division becomes. Of course, AT&T doesn't really lose money on the deal, they're just taking the profits from your uverse and allocating them all to their "affiliate" rather than the company actually providing the service. They can show that divison as being artificially profitable, and the landline division as a huge drain on their bottom line.

Now they can say their landline division is losing tons of customers, and costing them a fortune. But hey, our uverse division is doing better than ever!

What does this let them do?

Well, the prices they set to sell wholesale services to competitors are based on the costs of providing service. As are the regulated products they tariff and provide to end users. So now, they can go to public commissions and the FCC and show how despite technical advances and the network being a mostly sunk cost, that expenses of providing service are going up, and they must raise rates to competitors and end users to cover for the massive atrophy the network is experiencing (even as it grows exponentially to handle the load from the supposedly nonexistent customers that ride it).

So okay, what's the big deal, right? What will happen in 2017? What's possible here?

First, it's important to go back to the initial Telecommunications act of 1996 and the Triennial Review and Remand Order of 2002.

TA 1996 created the concept of competitive local exchange carriers, and abolished the legal monopoly that the incumbent carriers enjoyed from the 1870s to 1996. The phone companies were required to share their lines (and service) with competitors under the idea that since numerous tax breaks, subsidies, grants and other instruments were used to fund the network at the public's expense, and because the network grew based on the legal monopoly status that was provided to the carriers, the thought is that while the lines legally belonged to the private carrier, they were fully funded by the ratepayers, and the ratepayers had an interest in the facilities, as they had no other options but to pay the monopoly provider for service. Because the incumbent was provided an unequal footing for over a hundred years, the cost of building a competing network overtop of the current one would be financially unfeasible for any market entrant, because the ILEC could simply leverage the fact their network is mostly bought and paid for, and price the competitive entrant out of the market.

So the ILECs, under many (but not all) circumstances, were required to share their networks with competitive entrants based on TELRIC (Total Element Long Run Incremental Cost) pricing. This (to be brief) says that the cost of the network element, from installation to maintenance over the cost of its lifetime, divided by the number of months in its lifetime, PLUS A REASONABLE PROFIT FOR THE ILEC, shall be the monthly cost of that part of the network.

This creates an entrenched position for the ILEC. The competitors are sharing the costs of providing the line, with installation costs and repair risks amortized over time. You'd think this would be good for both sides, right? Problem was, AT&T and others turned out to be incredibly financially inefficient at offering services. Competitive carriers came in, paid for the costs of dialtone, the local wire going to the house, and a DSLAM (the phone company side of a DSL connection) and blew them out of the water! Some carriers were able to cut the cost of a residential line in half, while still providing broadband services over that line. AT&T was obviously quite displeased.

By 2002, AT&T and Verizon were able to argue that the new entrants were no longer in need of certain services, and should have to construct them themselves to continue offering them. (let's ignore here that the entrants were subsidizing the cost of the ILECs existing equipment, so why should they have to go out and buy new equipment that is unnecessary?).

The Triennial Review and Remand Order (TRRO) eliminated the following services:

  • Use of the long haul fiber network of the incumbent, except in certain limited circumstances
  • Use of the incumbent’s excess phone switch capacity to provide dialtone on the line
  • Use of the local fiber loops installed into the customer premises (if you sold someone fiber, get digging, because you now have to overbuild AT&T to provide it, and quickly!)
    • (Does it suddenly make sense why Verizon rolled out FiOS so quickly in their biggest markets, and then stopped? They did just enough to serve a market need, and destroy competitors in that area. In many states, when they install the fiber, they are allowed to remove the copper phone wiring into the building. This permanently eliminates your ability to get broadband services from a competitive phone provider, as they cannot use the fiber into your house, and must pay exorbitant fees to reinstall the copper cabling into your building, IF they can get the ILEC to do it at all. If you previously had copper to the home, and copper is available nearby, there's a way to get the ILEC to provide copper again under the "brownfield" rule in the TRRO, but they hate doing this and often the technician will try to talk you out of it when they arrive to do the work (which is illegal, violating TA 1996 as well as most interconnection agreements, but it definitely happens anyway)).
    • AT&T, when entering new subdivisions, now typically deploys fiber to the home, but typically only offers the same uverse packages over it that they do to copper customers. Why do they roll out fiber and cripple it like this? Because it doesn't require them to upgrade their network heavily, and it invokes the "greenfield" rule in the TRRO – if the area never had copper before, the ILEC is not required to build copper to satisfy a CLEC order. They used their entrenched market position, and economies of scale to ensure that your only choices would be them and the local cable company, if that. When the ILEC does offer faster speeds in fiber areas, they only do so when the cable company comes in with a higher speed, and they actually have to compete to get your business.
  • Services provided by affiliates do not have to be shared with competitors, including DSL services, and uverse/FiOS.

Is it starting to make sense now?

If you were AT&T, what would you do in 2017?

You'd send a letter to all your traditional customers, informing them they have 90 days to convert services to AT&T UVerse or face disconnection, as AT&T is discontinuing landline services. But don't worry, there's some special 3 year promotional deals where you can keep paying what you're paying now, but get more! Worry not!

You'd send a letter to your CLEC competitors, saying 'nice knowing you! Thanks for your subsidy for so many years. See ya!'. Send letters to all their customers offering special deals to convert them.

In 90 days, you shut off all services that are considered "landline phone service", converting a few holdouts to VoIP or cellular based landline services or whatever you have to do in order to make it happen.

You sell your remaining assets to your uverse affiliate for a dollar, or whatever the legal minimum is. You know how you're complaining about how expensive copper is to maintain? Guess nobody noticed that when you enter a new neighborhood with uverse, you often install a new F2 cable into the area that goes directly to the uverse "VRAD", bypassing existing copper cabling and investing new money in, you guessed it! Copper!

Then you evict the CLECs, and the fiber networks they built through your building (that you forced them to build) as the central office access is only for access to unbundled network elements, and since they're not a phone company anymore, they don't have any unbundled network elements, so get out. Upside is, there's now no credible way to compete with you other than buy a nearby building, build fiber to that, and run cables to all your customers from there.

When there's public outcry, or if it makes things easier for you from a regulatory standpoint, pick a few of your favorite CLECs that don't hurt you too much, and offer them special private contracts under secret terms that let them resell uverse wholesale. Don't worry – as you control the pricing for the product, you can make sure they never devalue the product too much, and don't provide any services that require you to really step up your game, as they'll only be able to provide what you can provide. You can sell this to the legislators as a way that you're "preserving competition".

And now that the local government is irrelevant, you don't have to expend any more money supporting every candidate (strangely, all of the AT&T bills passed in the last 10 years have done so nearly unanimously, across party lines. If you know the Michigan legislature, this is no small feat! It doesn't hurt that AT&T contributes equally and heavily to both sides of the aisle, and because of term limits, every person there has their eye on the next elected office they can hold. Get voted out? No problem, there's plenty of lobbying positions and think tanks who could use someone with your wisdom and experience! You'll land on your feet!). Think of the cost savings!

(You're thinking – but there's still a phone switch in Michigan, right? So the call doesn't cross state lines! – hah! AT&T placed their uverse switching systems in Pennsylvania, where they're not a local provider. So all calls, even down the street, are interstate in nature and regulated by the FCC and federal law. And Pennsylvania has no reason to regulate them, as it's just a pile of servers and switches that connect to other states.)

So okay, this is all well and good. I hate government regulations! Why should I care?

Well, it's simple. AT&T just effectively eliminated all competition except the cable company. They also are eliminating wholesale services used by Sprint and T-Mobile to connect their cell towers to the network. See CLECAM13-099 as an example of the changes they're making to eliminate DS1/DS3/OC-3/OC-12/OC-48/OC-192 service around the same time by removing the ability to sign contracts that go past this target date. Who uses these? CLECs do, to some extent, but wireless/cellular companies use them more heavily. If you think that this change is for Sprint and T-Mobile's own good, I have some DS3s I'd like to sell you. Don't worry – you'll still be able to get them – at a private market based contract rate that AT&T can more or less negotiate unilaterally, as there's no effective competition to many areas where towers are at.

When AT&T only has to compete with Comcast, and it's unaffordable for Sprint and T-Mobile to put oodles of bandwidth to their towers (don't worry, AT&T and Verizon each own massive fiber networks (that they'll probably pay $1 to themselves for), and can work out capacity trades to make sure that each has cheap access to their networks outside their ILEC footprints!) – how do you think that will work out for you as the consumer?

There's a reason why Comcast has been speaking to the Senate and House in favor of this "modernization" – AT&T is eliminating Comcast's competitors too!

Even if you and your neighbors don't use competitive services, you benefit strongly by their existence. Even if they're not available in your area, they could be if your existing providers upped your rates enough where it made sense for the competitor to swoop in and try to undercut them. Usually your prices are not based on the cost of providing service, they're based on what the other providers will charge for similar services, and if they keep the prices just low enough to keep other competitors out of the market, you benefit. So you should definitely care.

How to handle a (temporary or permanent) failure of a master server in Xenserver

http://flickr.com/photos/kevincollins/74279815/
http://flickr.com/photos/kevincollins/74279815/

OK. So you found you logged in, and you see one of the servers down, but don't see the virtual machines that were on that host anywhere in the resource pool. It's as if the entire system just freaked out and the server, and the VMs running on it, disappeared without a trace.

You've verified your shared storage is okay, and thus you're stable enough to begin remediation.

Now you have a problem, but it's easy to fix. Begin by connecting to the remaining server as root with ssh.

The following commands will get you back to a known good state. Make sure your master server is good and dead (unreachable and unlikely to come back without intervention of some sort – in my case it was a kernel panic, and a reboot and fsck -y / fixed the issue, this procedure will work fine in this scenario)

Last login: Fri Mar 18 16:42:03 2011 from (hostname redacted)
Type "xsconsole" for access to the management console.
[root@xenserver2 ~]# xe pool-emergency-transition-to-master
Host agent will restart and transition to master in 10 seconds...
[root@xenserver2 ~]# xe pool-recover-slaves
[root@xenserver2 ~]#

You are now at a point where the cluster is now usefully reporting a master server, and any remaining slaves are now pointing at it.

Now for the finishing touch – marking the powered down VMs as dead, so you can restart them on other servers. Note that if you're not correct on this, really awful inconsistent things will happen. You have been warned.

[root@xenserver2 ~]# xe vm-reset-powerstate --force --multiple
operation failed on ae820fb7-8416-68ee-35c8-c37REDACTEDf18: The operation could not be performed because a domain still exists for the specified VM.
vm: ae820fb7-8416-68ee-35c8-c37REDACTEDdf18 (REDACTED)
domid: 51
operation failed on aa2bc87f-9ef4-dd8e-492e-bREDACTED4: The operation could not be performed because a domain still exists for the specified VM.
vm: aa2bc87f-9ef4-dd8e-492e-b4REDACTE14 (REDACTED)
domid: 54
operation failed on e4f76313-2be1-0d4e-1e78-78e1REDACTED75: The operation could not be performed because a domain still exists for the specified VM.
vm: e4f76313-2be1-0d4e-1e78-78e1REDACTED5 (REDACTED)
domid: 2
operation failed on 557c9030-b995-fbaa-ab0b-61REDACTED4df: The operation could not be performed because a domain still exists for the specified VM.
vm: 557c9030-b995-fbaa-ab0b-61e4abb074df (REDACTED)
domid: 21
[root@xenserver2 ~]#

This marked every unreachable VM as "powered off" and thus it can now be restarted. You'll see it mercifully decided not to hurt my VMs that were confirmed to be operational. You may want to be more cautious and use host-id=your-uuid-here rather than –multiple but that's up to you.

More information available here: http://docs.vmd.citrix.com/XenServer/4.0.1/reference/ch02s06.html