This article by Ryan Heidorn originally appeared in Cyber Defense Magazine.
For most Managed Service Providers (MSPs), using a remote monitoring and management (RMM) platform to centrally manage their clients’ networks is a foregone conclusion – it’s generally assumed that RMM tools are necessary to deliver IT services. However, that tide may be turning as more MSPs wake up to the fact that traditional RMM platforms can introduce an increasingly unacceptable level of risk to their business and their clients.
Despite repeated warnings from the U.S. government and security vendors that attackers are targeting IT service providers as a single point of entry to breach multiple organizations at once, RMM platforms have not evolved to address modern threats, and remain a ubiquitous tool among MSPs. In the wake of the massive SolarWinds attack, Jacob Horne, a Managing Partner at DEFCERT and former NSA intelligence analyst, warns that President Biden’s recent Executive Order on Improving the Nation’s Cybersecurity should serve as a wake-up call for MSPs.
“If SUNBURST had zigged instead of zagged, this order would be locked on to MSPs,” he said. “The compromised Orion DLL also existed in N-central’s probe installer [an RMM component widely used by MSPs]. The MSP community dodged a huge bullet. Although N-central wasn’t directly compromised, it was just a half step away from happening if the attackers wanted it.”
Today’s threat landscape necessitates that MSPs adopt a security-first mindset to managing the privileged access they hold within customer networks. In this article, we explore alternatives for remotely managing customer environments, envision a “zero trust RMM” that incorporates contemporary security best practices, and explain how enterprise IT practices like DevOps can be leveraged by MSPs and MSSPs to build cybersecurity maturity and better protect themselves and their clients from modern threats.
The Elements of a Security-First Approach
Remote monitoring and management concepts and capabilities can be reengineered to enable MSPs to put security first. While MSPs themselves may not be able to make direct changes to the RMM tooling – we need vendors to prioritize security, first – but reevaluating assumptions around remote management, especially where current practices are at odds with security, is an opportunity for MSPs to level up their practices to meet modern customer requirements.
- Envisioning the Zero Trust RMM
“Zero trust” has emerged as contemporary wisdom for securing modern IT infrastructure. In contract to the adage, “trust but verify,” a core concept of Zero Trust Architecture (ZTA) is to “never trust, always verify.” ZTA seeks to move cybersecurity defenses away from network-based perimeters (like firewalls, VPNs, and intrusion detection systems) to user identities and individual resources, explicitly verifying every access request in the context of available data points. This is a particularly useful design principle for MSPs managing customers that increasingly rely on cloud services and whose users, in the post-COVID world, now work from anywhere.
How does the system respond when a correct password is used, but the user account logs in from Boston and then 30 minutes later from Los Angeles? Or when the correct device is logging in, but Secure Boot is disabled, or the device is jailbroken? Systems based on ZTA principles flexibly manage access requests based on an organization’s defined policy.
Debate over the term “zero trust” notwithstanding (critics of the term correctly argue that “zero” is a misnomer, and today’s implementations might be more accurately described as “policy-based adaptive risk” or similar), MSPs should look for opportunities to onboard customers into ZTA concepts and seek to apply zero trust principles like defense in depth, microsegmentation, and just-in-time access to how they manage customer environments.
To enable MSPs to employ these practices when managing client environments, a future RMM, built on zero trust principles, might include features like:
- Zero trust network access to client environments, with a central policy engine authorizing each connection to a client environment as a substitute for today’s unattended remote access
- Conditional access rules to protect key RMM functions like remote access and remote code execution.
Trying to connect to a client environment outside of an MSP’s normal business hours? Prompt for multi-factor authentication before authorizing the connection. Trying to connect from outside the U.S.? Block the connection request.
- An allow listing mechanism that only runs scripts that are cryptographically signed by the MSP
- Segmentation of other MSP assets from the RMM platform.
Do we really want to integrate credential managers with remote access tools?
Zero trust is more than just a marketing buzzphrase; it is a security philosophy that reflects the reality that users routinely access corporate data from outside the traditional corporate network, often including third-party cloud services, and increasingly on personal devices. Future iterations of RMM platforms must build these assumptions (and their attendant security considerations) into the platform.
- The Right Amount of Remote Access
Perhaps the most-used feature of RMM platforms is unattended remote access (screen sharing, file transfer, remote code execution). The ability to seamlessly hop on screen with a customer to troubleshoot an IT issue is considered a fundamental capability for an MSP. Particularly among small businesses, customers “just want things to work” and don’t want to be burdened with security processes or protocol.Today’s security realities warrant pushing back on these assumptions, at least until more secure iterations of RMM platforms are available. In the interim, the following practices for managing remote access may be justified to protect an MSP’s client base, even if there are some trade-offs with convenience.
- Rethinking Unattended Access
Day-to-day desktop support in client environments should not require unattended access. What if user support were instead conducted as attended support, with the end user requesting and authorizing remote access at the time of need? By requiring user consent (that cannot be overridden by checking a box in the RMM) to connect to or execute commands on user desktop environments, the ability of an attacker to leverage an RMM platform to breach many customers at once is greatly hampered. For endpoints that truly require unattended access, MSPs could use privileged access workstations (PAWs) to connect to dedicated “jump boxes” within customer environments. By segmenting and protecting the vectors for remote access into a client environment, an MSP demonstrates their understanding that with great power comes great responsibility.
- Utilizing Just-In-Time Access
Minimizing the number of always-on administrator accounts is a key component of managing privileged identities. As stewards of their customers’ security posture, MSPs should insist on reducing the attack surface of always-on, unattended remote access into customer environments. As Dan Ritch explores in the Thycotic cyber security blog, The Lockdown, Just-In-Time (JIT) access seeks to authorize privileged access only when it is required, protecting against compromised administrator accounts and providing an audit trail for privileged access. A future RMM built on JIT principles should include a mechanism for the customer to review, authorize, and log requests from the MSP before granting privileged access to the environment.
- Managing Single-Tenant Customer Environments
Do MSPs really need consolidated access to customer environments through a single pane of glass, or could they administer customer environments individually without much of a trade-off with efficiency? When it comes to the cloud, MSPs are already doing this. Today’s RMMs do not support meaningful management of cloud-native environments such as Microsoft Azure and Office 365. Emerging tools such as Microsoft 365 Lighthouse aim to bridge the gap, but MSPs may be wise to reconsider the necessity of aggregating all customer environments and seek out different styles of management.
- Modernizing IT Service Operations Through DevOps
It’s not just a problem of tooling. As an industry, MSPs are overdue for an upgrade of their internal processes and practices. Observing enterprise trends in IT operations over the last 5-10 years may prove useful for breaking out of the “locked-in” mindset that RMM ecosystems can perpetuate. Specifically: consider DevOps. Many MSPs are small businesses, but a shift toward thinking of themselves as an enterprise with many business units (i.e., clients) may be a helpful first step toward building operational maturity – an imperative for strong cybersecurity practices. Meeting the diverse operational, security, and compliance requirements of an MSP’s various “business units” does not have to mean sacrificing efficiency. To the contrary: for over a decade, enterprise IT teams have successfully integrated practices like DevOps to manage evolving business requirements at scale.
MSPs may not be developing code or forcing agile development cycles on their helpdesk teams, but they are well acquainted with operational issues ranging from resource constraints and bottlenecks, inconsistent system administration practices, to lack of control or visibility into the execution of customer projects. Incorporating iterative, repeatable processes and paying off technical backlogs (internally and in customer environments) are goals that any MSP can get behind, and DevOps offers a roadmap to achieve them.
The Phoenix Project, a 2013 “novel about IT” by Gene Kim, Kevin Behr, and George Spafford, is an excellent introduction to these concepts. Implementing DevOps begins by tracking and prioritizing work objects, identifying bottlenecks and blockers, and continually resyncing on those work items, problems, or issues. As MSPs develop a DevOps-like operational capability, they will soon find that concepts like infrastructure-as-code and configuration-as-code, widely adopted in the enterprise, have already solved some of the major gaps that exist in RMM platforms, such as how to manage single-tenant customer environments in the cloud.
Future iterations of RMM could support MSPs in this evolution by enabling the management and deployment of configuration-as-code. What would it look like if a company like HashiCorp made an RMM? We imagine that it would provide strong controls around least privilege, separation of duties, JIT access, and programmatic review of privileged activity, and it would all be fully driven by APIs. That’s an RMM that the security-first MSP could confidently adopt.
We Say We Need an Evolution
The RMM platforms used by MSPs today are not up to the task of meeting modern cybersecurity challenges. MSP tooling and practices must evolve to keep pace with the threats facing service providers and their customers. As an industry, MSPs play a critical and privileged role in securing the U.S. economy, especially small businesses. It is time for MSPs to rise to the occasion by adopting “security first” as a core business value, even if it means challenging the status quo in process and tooling.