Great tutorial on Information Security Program Metrics

While reading a blog posting this morning I came across a great set of slides called "Measuring Security."

Slide 15 nails what are the questions security programs should answer on the head...
How secure am I?
Am I better off than this time last year?
Am I spending the right amount of money?
How do I compare to my peers?
What risk transfer options do I have?

Slide 36 has a great quote on "Risk Management"
The essence of risk management lies in maximizing the areas where we have some control over the outcome, while minimizing the areas where we have absolutely no control over the outcomes and the linkage between effect and cause is hidden from us.

The next 300 slides is a ton of background detail...overkill until your really ready to dig in. I would simply recommend for now jumping to slide 402 to get to the punchline; here are some of the recommended metrics:

• Cost of security per transaction
• DoS and other attack downtimes
• Data flow per transaction & per source
• Budget correlation with risk measures
• Comparison with like firms
• Percentage of critical systems under DR plan
• Percentage of systems obeying ______ policy
• MTBF & MTTR for security incidents
• Number of security team consultations
• Latency to obey ______ change orders
• Percentage of job reviews involving security
• Percentage of security workers with training
• Ratio of b.u. security staff to central staff
• New system timely security consultations
• Percentage of programs with budgeted security
• Percentage of SLAs with security standards
• Percentage of tested external-facing applications
• Number of non-employees with access
• Percentage of data secure-by-default
• Percentage of customer data outside data center

Where all this detail is extremely important, the beautiful thing about what Securityworks offers is it has built a method to normalize any/all metrics into a single score. Think of it as your grade point average where you then have the ability to drill-down from the top and see how your doing for each subject, on each test, homework assignment, etc.

Going beyond technical security controls

Anton last week had this great write-up in ComputerWorld, "Five Basic Mistakes of Security Policy," that hits the 5 basics that so many busy executives look past when leading a security organization.

1. Not having a policy
2. Not updating the policy
3. Not tracking compliance with the policy
4. Having a "tech only" policy
5. Having a large, unwieldy policy

One of the biggest we see every day is #4. Most enterprises have some policy in place that they update (typically annually before a pending audit). Their current compliance tracking is provided by one or more software products that unfortunately don't have the full picture.

The reason why comes down to #4. Traditionally, enterprises have thrown either a vulnerability scanner, security event/log manager or another security software application at a list of IP addressable assets...generate a few reports...and hope they have things covered.

The truth be told, this misses so much of the full picture (over 50% per previous blog posts) that even the internal or external auditors don't have enough time to do a comprehensive review. The goal of those auditors is not a "witch hunt," it's suppose to be to protect the company! So what happens is each year, things get more and more detailed (which is good) as findings from the prior year are addressed allowing them to "peel the onion" back another layer.

This is why we are seeing the emergence of the IT GRC market that compliments and extends these products you point at IP addressable assets. These solutions use automation techniques to assess security controls around things that are not IP addressable (e.g., people, processes, facilities). The other need these products are offering is a normalized, unified view of the entire security program. Leveraging scoring from other products, they finally deliver the possibility of 100% visibility into the posture of your entire IT security, risk or compliance program.

Another security breach, but this one is different...

Late last week I saw the news around local JC Penney's hit the wire - "Data of 650,000 customers at risk." Now this situation appears completely different then TJX. The data, and I assume the protection of that data, were outsourced.

So this begs the question - should it be a requirement for vendors providing services to enterprises that would include sensitive data be certified against ISO 27001?

Here is a great write-up, case study I came across of a vendor doing this. Just like we expect vendors to achieve specific Service Level Agreements on availability, performance...shouldn't we be doing the same things around security and risk?

2008 - The Year of IT Risk Management?

I've been busy over the holidays enjoying everyones blogs and articles recapping 2007 and making predictions for 2008. Among other things highlighted in those articles, a common point pertains to Securityworks around "true" IT Risk Management (what I mean by "true" is the message is coming from companies who didn't adjust their marketing to be en vogue - e.g., SIEM products or Vulnerability Assessment products).

Before IT Risk Management was "cool" Securityworks has been out their working away on it (for over 4 years now).

One of my favorites that highlights this prediction for 2008 is over at Rational Survivability.

-snip-

Compliance stops being a dirty word & Risk Management moves beyond buzzword
Today we typically see the role of information security described as blocking and tackling; focused on managing threats and vulnerabilities balanced against the need to be "compliant" to some arbitrary set of internal and external policies. In many people's assessment then, compliance equals security. This is an inaccurate and unfortunate misunderstanding.

In 2008, we'll see many of the functions of security -- administrative, policy and operational -- become much more visible and transparent to the business and we'll see a renewed effort placed on compliance within the scope of managing risk because the former is actually a by-product of a well-executed risk management strategy.

We have compliance as an industry today because we manage technology threats and vulnerabilities and don't manage risk. Compliance is actually nothing more than a way of forcing transparency and plugging a gap between the two. For most, it's the best they've got.

What's traditionally preventing the transition from threat/vulnerability management to risk management is the principal focus on technology with a lack of a good risk assessment framework and thus a lack of understanding of business impact.

The availability of mature risk assessment frameworks (OCTAVE, FAIR, etc.) combined with the maturity of IT and governance frameworks (CoBIT, ITIL) and the readiness of the business and IT/Security cultures to accept risk management as a language and actionset with which they need to be conversant will yield huge benefits this year.

-snip-

Well said (but then again I'm biased)!

Users continue to ignore security policies, while security organizations are overlooking non-technical controls

IT Compliance Institute had an article posted this morning that reinforces the point; "it's not the software/hardware/infrastructure/etc but the people and processes that expose the biggest risks to a company.

The article doesn't reveal who/where the survey was taken but it does highlight some key security items that people usually cut corners on.

• Fifty-six percent said they had accessed office e-mail via a public wireless hotspot
• 52 percent said they had accessed office e-mail via a public computer.
• Eight percent admitted to having lost a mobile device containing corporate information.
• Sixty-three percent admitted to sending corporate documents to their personal e-mail addresses so they could work at home.

There are security technologies out their (e.g., encryption, data leakage) that can help with each item but the challenge is keeping up with other IT technologies being deployed and business demands/challenges the users are trying to productively solve. Bottom line, you can't bypass making sure you have the right policies, procedures and education in place for your users (aka non-technical controls).

After reading this I decided to do some searching around for some type of survey numbers around technical vs. non-technical controls. I didn't see much out there but did come across this ("Is Information Security Under Control') from IEEE Computer Society published in early 2007.

The survey focused in on 80 of the highest quality security controls as determined by a group of experts. From that list of 80 their wasn't a place that specifically counted the number of non-technical vs. technical controls BUT, there were two very interesting graphs.

The first one (figure 2 in the article. - see below) showed the top 10 with the highest level of quality implementation. It revealed that 6 are technical controls and 4 are non-technical controls. Meanwhile, the second graphic (figure 3 in the article - see below) showed the bottom 10 related to quality of implementation. It revealed that 3 are technical while 7 were non-technical.



So just running crude number here shows 11 of those 20 were non-technical controls while 9 were technical controls. The articles goes on to make the statement "...we found that of all 80 practices surveyed, management controls (non-technical controls) had substantially lower implementation ratings then controls in the technical and operational categories... Organizations must realize that a large proportion of information security problems extend far beyond technology and learn to appreciate the role that less technical controls, such as policy development, play in minimizing security breaches' impact on mission-critical operations.

So this begs the question, "when was the last time your security group considered software products that help with managing these non-technical controls instead of just technical controls?" I've talked with numerous enterprises that have installed or are investigating various software products like Vulnerability Assessment, Patch/Configuration Management, Antivirus, SEIM, data leakage, etc. Maybe it's time to do something for your non-technical controls also and consider adding IT-GRC products to that 2008 budget/priority list.

Is there a "silver bullet" to IT Compliance Management?

Is there a "silver bullet" to IT Compliance Management
by: Ryan Shopp

A few times I've found myself getting confused or having trouble explaining the relationships between policies, standards, controls, audits, etc when answering questions about IT Compliance & Risk Management? I came across a great two part thread in my blog reader that help crystallize things for me. It also enabled me to finally layout a logical response to a request I hear often. Is there a "silver bullet" to my IT compliance program? Here are some of those key points (from that posting) to help me answer that better now.

• ...numerous standards organizations have issued leading or “best” practices for control design and implementation; however, neither SOX (Sarbanes-Oxley Section 404) nor the PCAOB (Public Company Accounting Oversight Board) recommends a specific set of controls.


• ...In 2004, (PCAOB) issued a statement that COSO (“Committee of Sponsoring Organizations’ Internal Control—Integrated Framework"), or any other generally accepted control framework could be used. Note: it did not say COSO was the only one.


• But COSO can pose a problem...COSO doesn’t set out details. As its name implies, it is a framework.


• Each organization must still go through the difficult process of setting out its own system of internal control to meet its perception of COSO—which, in broad terms, is more of a philosophy than a set of rules.


• To fill the gap between theories and practice in implementing effective general IT controls, managers have turned to other externally developed standards and frameworks, such as the Information Technology Infrastructure Library (ITIL) from OGC, CobiT from ISACA, or the 20000-series of information security standards from the ISO/IEC

Bottom line, today there is no "silver bullet" for an enterprise. They can't simply flip a switch (or install a software product) and say "we have all the IT controls in place we need to meet x, y or z." It's a process, which must include a starter kit of controls and then review, massage and even extend based on your unique business vs. compliance requirements. To solve this "process" you need to work to automate various portions of the process itself, only then will IT compliance close in on the proverbial "silver bullet."

Special thanks to Xenia Ley Parker posts on IT Compliance Institute for the informative thread.

Auditor Answer: Can Internal Policies Overrule the "Rules?"
Auditor Answer: What are the "Right" Controls?