How Can Global Network Reputation Scores Be Used To Guide Cyber Investment? ActiveCyber Interviews Leading Researcher – Mingyan Liu – To Find Out

Every organization operates a networked ecosystem of suppliers, customers, internal sites, and governance bodies connected via a digital network. How does network reputation provide insight into an organization’s security posture within the ecosystem and provide investment incentives to improve its posture? What parameters are useful in establishing a network reputation metric and how can machine learning be applied to leverage reputation as part of predictive cyber analytics? Learn the answers to these questions and more in this recent ActiveCyber interview with Professor Mingyan Liu of the University of Michigan. 

Early this month I was able to attend a day at the DHS Science and Technology Cybersecurity R&D Showcase where several researchers presented briefings of their efforts related to a variety of cybersecurity topics. I found most of the presentations very interesting and was particularly intrigued by the possibilities identified by Professor Mingyan Liu as her research related reputation to cybersecurity investment incentives and strategy. Following the event I reached out to Mingyan who amicably agreed to this interview. Reputation as a metric has always been of interest to me since my participation in the OASIS WG for Reputation Management Systems. Reputation is often used as an incentive or disincentive to invest the time or other resources in an endeavor. It can also be related to an attacker – is the overall networked ecosystem a high value target to attackers or have a reputation as being easily exploited? Can a poor network reputation be an indicator that my supply chain can be penetrated leading to compromise of my assets? Discover how a network reputation metric is being developed by Professor Liu and can be applied to help in making your cyber investment decisions.

Spotlight on Mingyan Liu, Professor and Researcher, University of Michigan

» Title: Professor, Department of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor
» Email: mingyan@umich.edu
» Website: http://grs.eecs.umich.edu/

Read her bio below.

March 15, 2016

Chris Daly, ActiveCyber: Could you please provide an overview of your research on global network reputation systems and describe the key motivators that led you to focus on this research topic?

Mingyan Liu, University of Michigan: May I point you to our project website http://grs.eecs.umich.edu/
for a good description on this. I think information listed there should be sufficient.

ActiveCyber: The NIST Cybersecurity Framework describes four Implementation Tiers that reflect the levels of maturity of an organization’s cyber defense posture. Have you been able to map the results from your research to the different Implementation Tiers to help an organization identify the investment needed to move to a higher tier or even if it is worthwhile to make the investment to achieve a higher tier?

Liu: We have not tried to map our results to Implementation Tiers. What we have done is to map issues and problems we have identified on networks to specific security controls given in the NIST Risk Management Framework / SP 800-53 so there is explicit reference on what actions to take to fix the problem.

ActiveCyber: One of the outcomes from your research that you mention is the ability to forecast cyber incidents for an organization. Can you describe some of the factors that go into this forecast and how accurate your forecasts have been in predicting cyber incidents?

Liu: We tap into a diverse set of externally observed data on organizations, that captures different aspects of a network’s security posture, ranging from the explicit or behavioral, such as externally observed malicious activities originating from a network (e.g., spam and phishing) to the latent or relational, such as mismanagement and misconfigurations in a network that deviate from known best practices. In one prototype study, we extract 258 features from this set of data and feed them to a Random Forest (RF) classifier. We train and test the classifier on these features and more than 1,000 incident reports taken from the VERIS community database, Hackmageddon, and the Web Hacking Incidents Database that cover events from mid-2013 to 2014. The resulting classifier can be configured over a wide range of operating points including one with 90% True Positive (TP) rate and 10% False Positive (FP) rate.

ActiveCyber: The economics involved in cybersecurity investment strategies seem to result in strategies that are still largely reactive in nature – a wait-and-see approach. How does your reputation system provide incentives for investment and how do these incentives lead to investments that proactively target relevant risks and threats to the organization?

Liu: Reputation generally serves two purposes: internal consumption and external consumption. The internal use is essentially a feedback mechanism for an organization to self-inspect and question where things have gone wrong and how they can be improved. The external use is for others interacting with this organization to make more informed decisions, such as in making procurement or partnership decisions, or as in writing cyber insurance policies. Both aspects but especially the latter may be viewed as introducing incentives; e.g., the desire to secure a business relationship or to obtain a lower premium and better coverage on a policy may lead to improved practice and investment.

ActiveCyber: One of the expected uses you mention for the global reputation system is to facilitate the design of network security policies that enable a hierarchical perspective of resource allocation to threat vectors, thereby attaining greater insight into security posture and resulting in more efficient security practices. Given this benefit, can you discuss how the global reputation system may also complement the roll-out of SD-WAN and SDN technologies to enable adaptive security defenses and more resilient networks?

Liu: In general, as long as the impact of a technology is well understood in terms of what it does to improve an entity’s security posture, then decisions can be made to prioritize the use of technologies targeted at improvement in the most needed areas. Our technology seeks to provide guidance on such prioritization. In the case of SD-WAN and SDN, I will just mention that one use of the reputation scoring is indeed to inform better routing and peering decisions.

ActiveCyber: Global reputation scoring can cut multiple ways – it could provide incentives for organizations to invest and improve their cyber defenses and their reputation, but it could also impair an organization in being able to participate in information-sharing arrangements and act as a member of sensitive supply chains if the reputation score is bad – i.e., their bad reputation shows that they cannot protect their systems, networks, and data adequately. This could result in a catch 22 scenario for investment and improving defenses. What are your views towards this dillemma?

Liu: It would seem to make sense to me that information sharing agreements are built on mutual need for information, and not on one or the other’s current security posture — one may have a bad score but very valuable information for others. So I don’t see these being mutually exclusive. It is true that if an organization is part of a supply chain then it may be dropped if deemed to have a bad posture; I would argue in this case the reputation scores will have served its purpose: being part of a supply chain can have severe security implications for those both upstream and downstream, so this would be an example of the incentive (to improve one’s posture in order to remain on a supply chain) illustrated in response to question (4). By the same token, a company can also use business relationship as leverage to require its vendors or partners to improve their security posture.

ActiveCyber: What types of interesting reputation metrics and insights have your research uncovered and what variations in the level of organizational and network diversity have you studied?

Liu: We look at two types of metrics, the first concerning a network as a standalone entity irrespective of other networks in the same ecosystem, the second concerning a network as one of many inter-connected networks. This second type is crucial due to the interdependence or externality nature of network security, i.e., what one network does affects others. We indeed see very different behaviors from organization to organization.

ActiveCyber: There is always a problem of bootstrapping reputation and comparing apples to apples when it comes to reputation systems – how does your model accommodate these issues?

Liu: Since our risk assessment is entirely based on external observations that do not require an entity’s consent or participation, we don’t have the same bootstrapping issue one faces in establishing reputation in social networks. We also don’t have an immediate reference to compare our reputation metrics to. Most of the work done this space is on the host level and ours is exclusively on the organization level. However, we view our breach prediction outcome as a validation for the metrics we have developed and are using.

ActiveCyber: What are some of the nuances you have uncovered when applying machine learning to entities at an aggregate level? For example, what are some of the challenges in stratifying risks and identifying cyber policies or actions that can be used to remedy risks when applying machine learning at this scale?

Liu: Our level of aggregation is a single entity or organization (e.g., a network, a company, a domain, etc.), so what we identify applies to that entity. There are policies that can only be meaningfully applied at this level, such as peering arrangements and routing decisions, and incentive mechanisms (e.g., cyber insurance) aimed at encouraging better security practices and investment by organizations. Others are applied at a finer level of granularity, e.g., software updates on end hosts and server maintenance. At this level decisions will need to be made based on a combination of high level policies which our technique provides (e.g., more frequent software update and personnel training) and specific local conditions (e.g., training may only be applicable to a subset of the personnel).

ActiveCyber: What are the next steps in your research?

Liu: We continue to actively work on two fronts: (1) Internet measurement and advanced data analytics applied to cyber security and (2) leverage our expertise in measurement and data analysis to build better incentive mechanisms. On the latter front, our recent results have shown that our ability to make predictions, or more broadly, our ability to quantify at a global level the security postures of organizations, may be viewed as creating a form of “public monitoring”, which is crucial in designing mechanisms that rely on inter-temporal incentives to induce socially desirable behavior, from security practice to security investment to information sharing (e.g., breach disclosure).

Thanks Professor Liu for sharing these insights regarding your research with ActiveCyber and our audience of readers. I look forward to following your efforts and maybe doing another interview in the future as your research results come in. Looks to me that my quest for cybersecurity investment decision-making parameters needs to include reputation as well.

And thanks for checking out ActiveCyber.net! Please give us your feedback because we’d love to know some topics you’d like to hear about in the area of active cyber defenses. Also, email marketing@activecyber.net if you’re interested in interviewing or advertising with us at ActiveCyber.