adaptiman.com

Author: adaptiman

  • How much is that DOGE in the window?

    How much is that DOGE in the window?

    I was re-reading a really good book this week. A quote stuck out:

    In most governmental services, there is no market to capture. In place of capture of the market, a governmental agency should deliver economically the service prescribed by law or regulation. The aim should be distinction in service. Continual improvement in government service would earn appreciation of the American public and would hold jobs in the service, and help industry to create more jobs.

    –W. Edwards Deming: Out of the Crisis, 1982, MIT Press.

    This seems especially relevant this week as we had the first meeting of the DOGE Subcommittee of the US House Oversight and Government Reform Committee. This is not to be confused with President Trump’s DOGE, headed by Elon Musk. There has been a lot of ink on the nature and relationship between these two DOGEs – enough to perplex and confuse most of the American public. I’m not here to lend an opinion about the relationship or constitutionality of the two organizations. I want to focus on higher questions in light of the above quote. But it seems that we may be losing sight of the bigger picture – that one of the purposes of DOGE is to improve our government by making it more fiscally efficient.

    No Market to Capture

    “No market to capture” means no competition. No competition results in an organizational culture of complacency and mediocrity operating with increasing inefficiency and producing less valuable programs and services unless/until someone/something holds them to account. It’s clear that an organization with no competition is an abnormal condition in a capitalist society. This is the crux of the Marxist argument – that competition should be replaced with socialism and eventually communism. But the end result is centralized control of the means of production, and we have seen what kind of society that leads to.

    Deming makes a key observation. He asserts that since it has a captured market, government has an exceptional duty to deliver economically efficient services in the absence of market forces. Is our government delivering on this promise?

    Distinction in Service

    Distinction in service would seem to indicate that the efficiency and effectiveness of government programs should be exemplary. The way to achieve exemplary services in any sector is to engage in a culture of continual improvement. Since our government services don’t appear to be exemplary in many cases, is this an indication of a lack of focus on continual improvement? How do we change that?

    The first two steps of the ITIL Continual Improvement process are 1) What is the vision? and 2) Where are we now? The vision (or strategy, if you will) comes from our executive branch, i.e., the president. This is the way our government is structured, whether we like it or not. Where are we now? I would point out that the debt-to-GDP ratio of the U.S. over the last 45 years has increase four-fold. In 1980, the ratio was 31%. Today, the ratio is 120%. We can argue about whether or not this fiscal path is sustainable, but that’s not my point. It would seem obvious to anyone that our current state is not efficient and arguably not effective. It is definitely not exemplary. How do we change this?

    Continual Improvement

    Deming points out HOW this is done – by focusing on continual improvement. As an ITSM practitioner and educator, I frequently think about continual improvement and how it affects value. Having worked in the government sector, I have seen how a lack of competition can lead to complacency and mediocrity. But I’ve also seen the results of having the RIGHT people in charge. My observation is that the biggest difference between the right people and the wrong people is a focus on developing a culture of continual improvement within the organization. In the case of our government, these people understand that they have an awesome and sacred responsibility to use their position with honesty and integrity, and in so doing will earn the respect and appreciation of the American people. This is what I believe our government can and should become.

  • Two Points and a Poem

    Two Points and a Poem

    I was talking to a senior IT manager the other day when he lamented that younger managers under his charge didn’t communicate effectively. My colleague, a retired Air Force officer, remarked, “They want to give me a dissertation every time they report. I don’t have time for that. All I need is two points and a poem.”

    I was intrigued. “What does that mean?” He replied, “Be prepared, that is, think about what you’re going to say before you enter the room. Be concise. Speak with executive function. Give me the ten-thousand foot view – I trust you with the details. Summarize your points and be done with it. In essence, move with a purpose.”

    To further process this, I did some searches for the phrase and found two sources. The first was a reference to a traditional expression in homiletics (i.e., three points and a poem) that describes the shape of a sermon. Basically, the minister would present three main points of the message and then conclude with a poem or memorable anecdote to reinforce it. This seemed to me a logical etiology of the phrase, but why would my colleague reduce it to two points? Perhaps this was “military efficiency” at work?

    “A poem is a ‘line’ between any two points in creation.”

    ― Charles Olson

    The second reference was a quote from Charles Olson (1910–1970), an influential American poet; “A poem is a ‘line’ between any two points in creation.” While it was unlikely that this quote was the source of my friend’s order, it gave me an interesting thought. By limiting the report to two points and connecting them figuratively with a poem, don’t we create the most efficient metaphorical figure? To my mind, the figure of speech had become a poem itself with mathematical precision and beauty. So the next time you’re reporting a project status to your boss, give her the mathematical elegance of two points and a poem.

    Prepare, be concise.
    Two points and a poem's grace
    Speak with purpose clear.
    -Dain B.
  • Meditation on a Cold Front

    Meditation on a Cold Front

    Slowly and silently,
    The silvery air slips through the night,
    Borne on the wings of a whispering white dove,
    Who brings me to the presence of the moment,
    And inspires my lips to prayer.

    The cold is forgiveness, after all.

  • Want to Lose Weight?

    Want to Lose Weight?

    The best way to lose weight is to go to confession.
  • Crowdstrike Outage “Not What You Thought”

    Crowdstrike Outage “Not What You Thought”

    It’s been six months since the Crowdstrike outage – enough time to reflect on the incident and take stock. I had lunch with my CISO about a week after the outage. It was the first time we had seen each other in several weeks. “So,” I asked sheepishly, “how have you been since the outage?” “I’ve been fine. But the Service Desk has been swamped. Since my security team wasn’t that busy, we pitched in to help remediate the outage. They touched 15,000 servers and client machines in three days.” I inquired further. His role focused on the management of encryption keys that were necessary to unlock and manually patch the operating systems of the affected machines. “The hard part of the recovery was managing the keys,” he said. As his team was jointly responsible for the security of those keys, that was the extent of his involvement. You see, Crowdstrike pushed a bad patch – one file – but an important one that loads at the kernel level. This caused all of those Windows machines to “blue screen.”

    Something didn’t compute. I thought he was going to be falling asleep at the table, eyes bloodshot, bags under them, a quart jug of coffee in his hand. Instead, he seemed rather chipper. Then it hit me. This wasn’t a security incident. Rather, it’s what we call in ITSM a deployment and release management issue. It’s not that Security Management wasn’t involved, they were. But it was apparent early in the Problem cycle that this wasn’t a cyberattack.

    The response from our university IT was quick and appropriate. Within thirty seconds of the patches being applied, customers began to call and report “blue screens.” This spawned a number of related incidents at the Service Desk. These incidents were quickly correlated into a Problem record, which was upgraded to a major incident (i.e., outage) record in less than an hour, all of this happening around midnight on July 19th. During the early morning hours, an incident response team did a root cause analysis and quickly determined the problem was a vendor patch.

    The vendor response was quick and the patch was available by early morning, although the CEO of Crowdstrike was criticized in subsequent days for not issuing a timely apology. The damage to Crowdstrike’s reputation was done. After all, the outage affected roughly 8.5 million computers. Crowdstrike was quickly seen as the responsible party and IT folks around the world became heroes as the outage response progressed. But Microsoft was also responsible for letting Crowdstrike play in the Windows kernel. Microsoft distanced themselves from responsibility by asserting, “Although this was not a Microsoft incident, given it impacts our ecosystem, we want to provide an update on the steps we’ve taken with CrowdStrike and others to remediate and support our customers.” In this instance, Microsoft was acting as an integrator, more specifically, as a Service Guardian, where they managed both a third-party vendor (Crowdstrike) and provided services (Windows). In this instance, ITIL best-practices dictate that we have a high-level of communication and trust with the integrator, but also acknowledge that our customers will hold us – not our vendors – responsible. After all, who are our customers going to blame – us or our vendor?

    I see a double failure here. Crowdstrike failed by deploying a service with a critical bug in it, which they should’ve uncovered in their acceptance testing. This is not George Kurtz’s first high-visibility failure. In 2010, he was CEO of McAfee when a similar outage occurred. The second failure was Microsoft’s mismanagement of their vendor. One may ask why they allowed a vendor to deploy a file at the kernel level without sufficient testing. You would also expect Microsoft to have caught the error prior to approving the release of the errant file. Was Microsoft’s trust of Crowdstrike so great that they didn’t do acceptance testing and simply passed the updates through? If so, they need to review their Deployment and Release Management practices. Of course, this is pure speculation.

    Meanwhile, back at “the ranch,” the IRT created a Change Request that included testing of the patch on a number of machines. Procedures to apply the patch were documented at both the individual asset level and the more strategic coordination level. On the communication side, customer communication began as soon at the Problem was identified, about an hour into the incident, with a number of communications happening in the early morning hours via IT staff in the colleges and university communications to stakeholders. Communication continued through the next few days as the incidents were remediated and non-reported servers and endpoints patched. An After Action Review was conducted less than a week after the initial incident was reported. Lessons learned were documented. DONE!! YAY!!!

    Since I retired from IT, I’m an “observer” these days and I can tell you that I don’t miss the excitement surrounding outages. Been there, done that, got the t-shirt. But I must say that I’m very proud of the way our university handled this major incident – responsive, professional, by the book. I don’t think our response would’ve been as good five years ago. We’ve come a long way in our journey in understanding ITSM.

    In summary, what ITSM practice areas were involved in this outage?

    1. Service Desk
    2. Incident Management
    3. Problem Management
    4. Continuity Management (via Major Incident/Outage)
    5. Vendor Management
    6. Asset Management
    7. Relationship Management (i.e., communication with stakeholders)
    8. Change Management
    9. Security Management (indirectly)

    This is a pretty impressive slice of the ITIL ITSM Practices for a single issue. I think our IT folks would report that we have varying levels of maturity in each of the Practice areas, but I can tell you from experience that this kind of outage hones our skills to respond better the next time. Iron sharpens iron.

  • The Provenance of a Dram

    The Provenance of a Dram

    Nine and a half today – ugh.
    Stiff legs,
    Cold hexagonal tile,
    Glow belt,
    Running shoes – a little worn,
    Earbuds,
    Running watch,
    Smartphone.

    Queue ’em up:
    Sing the Hours,
    Daily Wire,
    First Up,
    Daily Poem,
    Megyn Kelly,
    Dan Bongino.
    Long enough? Yes.
    Push play.

    Brisk, still, fall air,
    Dogs wet noses,
    Tail of death.
    Start slow.
    Breathe in through the nose
    And out through the mouth.
    Up the pace.

    “Welcome to the Daily Poem…
    Quinquireme…
    Cedar, cinnamon, and sandalwood…”

    Running rhythm becomes poetic pulse.
    Time ceases to hurt my lungs.

    “Emeralds and amethysts…
    Gold moidores and sweet white wine.”

    Sweet white wine? – Blech!
    David, Sean, and Bethany need
    A midwinter taste of Texas with a stamp!

    Sharp fall radishes,
    Brown eggs laid down sparingly,
    Gloriously dead ragweed stalks,
    Whispering post oak snow,
    Family gatherings.

    Love.

    Friends for dinner,
    Pork loin,
    Mashed sweet potatoes with butter,
    Brussels sprouts with bacon,
    Strangely empty rocks glasses.
    Brisk fall sales,
    Sharing the Bread,
    Growing the podcast audience.

    Try a drop of this –
    Kooper’s cooperage,
    Oaky honey,
    Crème Brûlée,
    Apple, cinnamon, lingering citrus –
    Oh My!


  • Bedtime Acclamation

    Bedtime Acclamation

    "I'm getting married on my head,"

    My sweet John Robert to me said.

    "Perhaps my Dear

    Should wait a year.

    But now it's time to go to bed."
  • Lost Improvements: An Analogy to Defects

    Lost Improvements: An Analogy to Defects

    Defects are not free. Somebody makes them, and gets paid for making them.

    W. Edwards Deming

    To summarize Deming’s teaching on defects, they cost an organization thrice. First, the defect is made, which robs the organization of a “working” product or service. Second, the defect must be identified, which also takes time and resources. Lastly, the defect must be resolved, thus taking more resources away from producing non-defective products and services. If this isn’t bad enough, these costs don’t include opportunity costs which could be mitigated with improvements.

    In manufacturing (and IT ;-)), a defect happens because of a quality failure either at the source or somewhere upstream. Once a defect is built into a product, there are two ways to detect it. First, it may be detected prior to shipping. Second, the customer may see the defect, which is significantly worse from a CX perspective. To draw the analogy to lost improvements, if there is no system in place to record improvements, that’s the equivalent of allowing a defect to get to the customer. Lack of improvement causes more technical debt and operational overhead down the line and will be reflected in much of the work that is done by the organization. These defects will be visible to customers, one way or another. How does an organization create a culture of continual improvement?

    First, an organization must embrace a culture of improvement. According to ITIL4, a culture of improvement requires three things; transparency, managing by example, and building trust (CDS, 2.3.4, 2.3.8). I’ll treat these three topics in more detail in a future post, but suffice it to say that my perspective is that the former are dependent on the latter – that is, trust is the “coin of the realm” and other aspects of an improvement culture are dependent on it. For example, organizations that have a high degree of trust manifest a corresponding high level of transparency.

    Trust is the “coin of the realm” and other aspects of an improvement culture are dependent on it.

    Second, an organization must provide mechanisms for conserving, prioritizing, and executing improvement initiatives. Starting with a Continual Improvement Register (CIR) is a good first step. If systems are too proscribed, or improvement processes not defined, team members don’t feel empowered (or able) to record improvement ideas. Without improvement, the organization will continue to produce defects. Making the CIR accessible at all levels of the organization is also recommended. Appointing a small, dedicated improvement person or team responsible for prioritizing and executing on those improvement opportunities closes the loop. Communicating the status of improvement opportunities creates buy-in from the organization and keeps the suggestions rolling in. In my experience, organizations go awry in the second requirement. They may build a culture of trust and improvement, but that culture must be operationalized to realize the true benefits.