An Informal Guide to ISSM Whispering

Part I - Ignorance is Bliss

I did not simply survive painful DoD cybersecurity practices. I was born into it. Molded by it.

Therefore, my understanding of why most DoD policies are simultaneously inane and somewhat sensical did not come to me until later in my career when I was exposed to the seams where modern DevOps rubs up uncomfortably against the heavy, looming requirements of DoD cybersecurity.

I was first introduced to the concepts of DoD cyber from the operations side of things - initially as a client systems technician for the USAF, and then as a SOC analyst for USSTRATCOM. As a client systems technician, my relationship with cybersecurity was scoped to the laundry and dishes of managing an enterprise: security patching, host troubleshooting, and initial triage of data spillage events. I was told at that time that the risk associated with not keeping up with security updates on hosts used by everyone from the squadron commander's assistant down to the services troop folding towels at the gym could lead to a potentially disastrous event, sending the state of information security into an irrecoverable downward spiral. The result of this fear was, and still is, thousands and thousands of man hours dedicated to maintaining the security posture of all enterprise systems to protect this potentially sensitive data. Just as laundry and dishes are important and will cripple the functionality of a household if not kept up with, it is still mundane and monotonous work. I learned most of my basic skills and technical concepts during my time in the USAF so I did not truly know any other way of doing things.

Later, when I was but a fledgling cybersecurity analyst working in the SOC of USSTRATCOM, the power of all those security controls attached to every little component of the enterprise was invaluable to my ability to triage an event. Every little bread crumb was accounted for in the environment, down to the mouse click and keystroke of a user on their terminal. I gained an appreciation for the mission, and an understanding of just why we spend those thousands and thousands of man hours keeping up with the monotony. Compared to my time in the Air Force, the systems that we supported at USSTRATCOM had a far more dire impact if things were to become compromised - the impact was far more real, and far more serious. When I worked with my fellow analysts and interfaced with the policy makers that supported those systems, I believe the emotional connection to that impact we shared helped (not solved, but helped) justify the absolute insanity we put up with in the name of national security - insanity created in part by ourselves.

Like many of my peers in that role, it was my first real brush with cybersecurity principles put into action, and in a USSTRATCOM-sized vacuum, those policies and practices - friction included - all felt completely needed and necessary. With no practical knowledge of the non-DoD world, ignorance truly was bliss.

Part II - Ow, stop, it's chafing 🥲

Soon after my tenure in the SOC I moved upstairs to a newly minted Dev(Sec, sorta)Ops team. Instead of working as an agent of justice upholding the policies defined by my policy makers, I was now on the other side of the fence, pinned into the opposing trenches of ops - and even worse - dev. Insert screaming and discordant piano stabs.

Many in DevOps credit policy and procedure with being the enemy of production. The person that upholds and sometimes creates these policies and procedures in a DoD context is usually the local ISSM/ISSO - the Information Systems Security Manager (or Officer). Unfortunately, the people who uphold policies and create barriers can vary wildly in personality, technical experience, interpersonal skills and problem solving prowess. Something something, box of chocolates.

I quickly became intimate with these barriers through the tenuous relationship between DevOps, and the security principles that act seemingly in direct opposition to progress. Attempting to build a modern development environment inside of one of the most sacred and protected DoD networks was like playing Atari's E.T.: there were pitfalls everywhere and everyone was upset if you accidentally fell into one. It's almost as if the entire enterprise was explicitly designed as a challenge mode to get anything useful created or god forbid - deployed. The role ended up being about 10% engineering and 90% trying to convince the ISSMs and security analysts that, no, I am not in fact attempting to exploit a backdoor C2 node on the internal network, I'm simply trying to troubleshoot nominal connections over known ports to fix my stupid thingy that I am on contract to create. Maddening.

As an engineer, I cannot help but identify flaws in whatever system of controls I am subjected to. Most engineers I have ever worked with do the same, and with enough discord and frowny faces about all of the shenanigans that surround what would conventionally be a simple problem to solve, it makes you and your team of smart engineers start thinking. It makes you start wondering - what is the actual value in any of these security hoops and processes that we have to jump through? Your skepticism of the policies upheld by the local ISSM accumulates and builds through every project until it's so frustrating you start to think and commiserate with your fellow smart engineers about how it should be, and how we would conduct the Sec part of DevSecOps in a far more efficient yet still secure way. You start to think - you know, I think I know better. You start to say:

Well, tough part is, we were not in fact any kind of ISSM, and also would never truly want the responsibilities of one. We still had engineering deadlines, and we still were subject to the process set forth by the policy makers - there would be no feasible path to usurp this reality. Instead of taking the road that I have seen many take as an approach to progress in this space - that of anger and frustration and senseless fist shaking - I realized the most fruitful path was to take a page out of my good ol' buddy Spike's book:

Part III - Like Water

ISSMs and policy creators are humans too. They are trying their best in their role, just like you and I. I have never met an ISSM that deliberately shows up to work maniacally rubbing their hands and petting their white cat as they scheme the downfall of the next new project. Their role is designed to be one of risk aversion and compliance - and for good reason. As I said earlier - our connection to the importance of protecting the mission of USSTRATCOM was not at all in question. I certainly would not want to be in the shoes of the ISSM that would have to report to anyone: "Hey there, yeah, so.. the adversary was able to execute a data exfil using a vulnerability introduced via supply chain attack from a Node package we had not yet vetted". These controls and processes exist for a reason - to make for damn sure that we are not compromising the mission. I do not envy those ISSMs that were and still are ultimately responsible for holding back the gates of hell every day.

Nonetheless, new requirements continue to pop up, and chaos continues to enter the enterprise as we attempt to support the tenuous landscape that is nuclear deterrence. We have no choice, in the short term, but to play ball with the policies that support the mission.

Luckily I had a foot in the door with those same ISSMs, as I was such a good little upholder of justice as a SOC analyst that I gained some intergalactic security credits in my back pocket that aided my day to day struggles with the whole DevOps thing. The interpersonal connection helped me get over some of these hurdles in quick-er-ish fashion than some of my counterparts that the ISSMs may not have known or trusted. Even with that initial trust barrier out of the way, a large amount of time was spent explaining what it is we were doing in meetings out of band of the actual forms we were forced to fill out (that described what exactly was being accomplished). Some people also had issues understanding the technologies we were employing at all, which led to a whole lot of time teaching the ISSMs what the nature of the problem actually was and why our solution would not in fact blow everything up.

Many of my cohorts spent most of their time throwing their hands up in frustration that they had to even explain all these nuances to the ISSMs, which did not really help the issue in any way - often it exacerbated issues and lengthened the time it took to get new approvals over the finish line. There was a distinct leathery bummer aroma in the air whenever security approvals were the object of the day, week or even month at times. This certainly did not encourage any of the ISSMs to want to speed anything up either.

Instead of abrading the relationship with my policy upholders, I took a more fluid approach to trying to get things approved and across the finish line. I had to be more like water, and conform to the container I was put into while still finding the path of least resistance - resistance that often presented itself in a handful of forms:

• ISSMs that fear the unknown
• ISSMs that distrust the submitter of a change
• ISSMs that are ignorant of tech concepts/solutions outside of their experience
• Inflexibility of the approval process due to enterprise constraints

None of these things present in a given ISSM would necessarily mean that they are ineffective or difficult to work with. Many of the phenomena listed above occur naturally as people learn and grow in the role of an ISSM or security policy upholder. The salve to each of these problems is a mix of effective, enthusiastic communication and education, as well as a little bit of charisma to boot.

Part IV - Warm Fuzzies and Sunshine-y Rainbows

A surprising amount of cybersecurity boils down to warm fuzzies. Even outside of the scope of change management, it's all just warm fuzzies. I have seen stuff get approved wicked fast, faster than it really should have due to the right people having warm fuzzies and confidence that the thing they are approving is all good to go, sometimes without even really doing the due diligence on the approval package just because they have seen so many approvals go through correctly by the submitter they now trust. Ahem... that submitter was me, usually.

My one #humblebrag of this article will be that at one point in my career I was lovingly deemed "The ISSM Whisperer", due to my ability to get approval packages across the finish line in half the time or less that it was taking for my cohorts to do the same. I found that it all boiled down to my ability to provide my approvers warm fuzzies and sunshine-y rainbows. If my approver or reviewer felt as if they trusted me, felt as if they knew the entire problem space, knew at a technical level what was going on and the solution aligned with the enterprise constraints,

we'd be zoomin.

Many of my fellow engineers were not beholden to the fact that just like any negotiation, trust was at the core of it.

Many great engineers I worked with, understandably, did not want to sink much of their time into the initial creation of good security artifacts, or diligently explaining what was being changed/added to the network because they did not see it as their job to explain and educate the approvers on what is being changed/added - they should just know, and they should also know all the technologies employed in the solution as well as the engineer. Oftentimes those great engineers would end up frustrated every time that they had to take extra time to explain something they already explained in the initial change management forms because they saw it as a roadblock to their day, or extraneous efforts to their duties. They often did not see that things would actually go faster overall if you just took the time to meet the ISSM where they were at, addressed their hesitations, and put their needs first before your own in order to get something approved. They were not beholden to the absolute sheer power of warm fuzzies.

But I can't just tell you - educate and effectively communicate, and you shall be on your merry way to victory! No - let's talk about a role I had further down the line, and the strategies I employed to gain my ISSM's favor such that the approval process became a small stepping stone, and not the primary sinkhole that many people see it as.

Part V - An Informal Guide to ISSM Whispering

One thing I learned before I left USSTRATCOM was that even in the same role, every ISSM had a different set of things that made them tick. I experienced many ISSMs between the 55th WG and USSTRATCOM, and even in supporting the same mission, the new people that came and went often had a completely different flavor of what was required to gain their trust.

After my time at USSTRATCOM I hopped across the tiny Offutt AFB sized pond to the 2nd largest tenant unit on station: the 557th Weather Wing supporting the TADS contract. Unlike USSTRATCOM, these guys had far less dire risks and threats associated with the environment. If something were to go down, or to become compromised, sure, OPSEC and INFOSEC would be at risk and the weather data used for mission planning across the DoD would be affected but we were not going to accidentally blow up a nation state or cripple our ability to perform global nuclear deterrence. Understandably, the stakes were a bit lower - and that showed in the amount of funding and attention given to the contract as a whole. We spent a ton of time on development, a bunch of time on sustainment, not a lot of time fundamentally correcting why systems would go down or degrade. Sure, we had SLAs and stuff and slapped band-aids on systems where we could but overall people were pretty understanding and chill about stuff in the environment compared to the (understandable) neuroticism at USSTRATCOM, where outages were uncommon, but a HUGE deal, with tons of root cause analysis on the backend of an event.

That attitude permeated into how SecOps were conducted - things were overall more relaxed, more understanding, and as a baseline, more trusting. Change management and approvals were not taking up 90% of engineering time - they took more like - 30% of the time. Still room for improvement. Even with the more relaxed environment, there were still moans and groans about security artifact generation and lengthy approval processes. The back and forth over lining the ducks up correctly wore on my poor, brilliant engineers.

The following is my informal guide to how I lessened this pain, which enabled everyone involved in change management to feel warm and fuzzy and get projects across the finish line faster. This guide, in the right hands, will enable you to become an ISSM whisperer yourself, and quicken those painful processes for everyone.

---

1. Go and meet your ISSM!

   • Walk over to their desk, shake the hands, make the stupid jokes, whatever. Put your face to the name. If an ISSM knows who you actually are, they are going to be more open to trusting what you say in an approval package.

2. Roll high on your Perception skill so you can identify and anticipate what your ISSM may hesitate on.

   • It might not come in the first conversation, but through either informal or formal engagements, try to take note of what technologies they are familiar with and ones they are not. Take note of what parts of the network give them the most anxiety - maybe it's firewall ports, maybe it's storage, maybe it's encryption - it's going to be something.
   • Don't be weird and quiz them - you'll digest this data over time while working with them. You'll see what they'll kick back to you, and that usually will inform what technologies or constraints they may not be as intimately familiar with.

3. Be prepared to answer hesitations from what you have learned about during step 2 in the way your ISSM will probably like.

   • Some ISSMs like to see a lot of words on a page, and not read them. Some ISSMs want to see links to resources and scholarly articles. Some ISSMs want to feel buttered up that what they are approving is going to solve world hunger or something. Whatever it is - figure out what it is that gives them the warm fuzzies, and present the information in that way.
   • Rely on your smart guys - you might not be the smart guy, but you should know who the smart guys are. If your ISSM asks you a question that you don't know the answer to right away, telling them exactly which SME you are going to ask that question to goes a long way.

4. Create a pattern of expectation.

   • Obviously there are expectations in the process itself, but as I said earlier, much of the conversation ends up occurring outside of the actual forms or change management process - it happens ad-hoc and informally during work hours in the form of quick questions and clarifications. I found that pre-loading those conversations up-front either in a quick meeting or by ambushing their desk spontaneously for 10 minutes or so helped the tedious week-long back and forth that would otherwise occur in email, chats, approval workflows, etc. Every time I was about to submit something for approval or for a change, I would just swing by their desk, and as a courtesy tell them - "Hey there! I'm going to submit this thingy for approval today, you'll just need X, Y, Z and G right?". They would consider for a moment and usually go "yep, thats all i'll need, thanks!". Doing this every time kept me in the loop if they would need something new I didn't anticipate, and also, gave the ISSM warm fuzzies that I knew exactly what they needed for the approval. That overall resulted in the approval flying through the expected change management rungs, with minimal time spent. A textbook Win-Win-Win.

     

5. Learn about the problem so that you can educate.

   • I mentioned in step 3 that you may not be the smart guy - but consistently being the smart guy certainly helps your case and believability. for example, many ISSMs I experienced in the DoD were pretty unfamiliar with containers. Whenever I'd go up for approvals for a new container, i'd have to be ready to re-explain some basic containerization concepts to them so they understand just what is being deployed and how it would be managed. Being consistently tactful and enthusiastic with educating them about the problem was a huge boon for getting things through the wickets.
   • Foot-stomp on being tactful here - you must be careful not to make your ISSM feel dumb and that you are a super smarty pants. Obviously you want your ISSM to trust your technical ability but not at the expense of the relationship. I have seen this go badly in the past, which is why I feel the need for an extra foot-stomp on this topic.

6. Be on the same team.

   • Try your best to make sure your relationship with the ISSM is not adversarial, even in your own mind outside of their office and with your fellow engineers. Disparaging an approval process and the people who designed it is easy, and often times justifiable. There will be frustrations. There will be personalities that are hard to swallow. Try your best to be like water, and turn your conversations with your ISSM to reflect the reality that this is all a team effort. They want to see stuff get approved too.
   • Back up your ISSM against your team of smart guy engineers too. If everyone's role is understood and respected, enthusiasm will stay just a little higher overall and people's willingness to communicate and educate will come easier.

7. Maliciously Comply.

   • I know I just said to be homies with your ISSM but also, like, they don't need to know absolutely everything. I'm not saying hide information from them - I'm saying, don't tell them all the things that might scare them. You and your smart guy engineers are going to know mostly all the potential pitfalls that a package or firewall approval could bring. It is not your job to tell them every little risk that exists - that just adds more caterwauling and anxiety to the whole thing. They can uncover that themselves on their own time - don't make it easy for them. You want them to feel just good enough, warm and fuzzy enough, to approve your thingy and move on.
   • The above statement is very nuanced. Obviously you do not want dubious packages or containers being accepted into your network - but we understand that SecOps is a balance. Security exists to protect Ops, but there would be no Security if we didn't have some Ops actually Op-ing. We do not want to tank security purely in the name of Ops just to get something done quickly; if we fly through security considerations too fast and flippantly, we risk taking time on the backend cleaning up the mess. If we spend all day losing our minds on the potential risks of a new change to the enterprise, we will never get anything done. You, the submitter, and your fellow ISSM, act as stewards of this balance. Be on the same team, but don't artificially add extra information to the conversation that may distract everyone from the change at its core.
   • Know exactly what is cared about in your enterprise, and help your ISSM stay on track with those goals. I won't get into all of how people decide what's important or not - that is specific to each org. Either way, every enterprise cares about something a lot, and other things not so much. Identify what those are, and apply the correct amount of energy toward each situation. Some enterprises spend all day fretting over STIGs and patching for their servers because they are told that compliance numbers are everything, while some might care more about what might affect availability of their application and will spend a ton of time researching how each new package might affect their apps - recognize where you can spend less effort on security compliance so you can spend more time on the things that matter to the enterprise you are in.

Closing

Well, there it is. You know everything now. There's nothing more you need to know - every change you will go through with your ISSMs will now be seamless and take about 10% of the time it usually took you.

I wish all who participate in change management a great day. Even the ISSMs and policy creators too.