Several months ago, the Samba developers discovered a security vulnerability that turned out to be a flaw in its protocol. It affects Samba and Windows servers both and administrators of these should apply the patches that were released on 12 April as soon as possible. An analysis of the technical details and the importance of the problem.
Stefan Metzmacher, who discovered the vulnerability later named Badlock, is part of the development team of the free software Windows server alternative Samba. He works for SerNet GmbH in the German city of Göttingen which sells maintenance, consultancy and services for Samba. The company decided to drum up interest for the vulnerability three weeks ahead of the release of the patches to make sure people understood its importance. The company created a logo and registered a domain for it but did not divulge any details. Ahead of the patch release, c't talked to Volker Lendecke, one of the founders of SerNet, who was also the first German member of the Samba development team, about the technical details of Badlock.
An Alarming Discovery
"There are things lurking below the surface. I am pretty sure that people are looking left and right for other things to do with RPC."
Samba developer Volker Lendecke on RPC services in Windows, which are the basis for DCOM, WMI and other technologies
The Samba developers had commissioned a specialised company to do some stress tests on parts of their code – with alarming results. Stefan Metzmacher was assigned to look into it. The tests were targeting Samba's own implementation of the RPC services. Remote Procedure Calls (RPCs) are used to execute commands on other systems over the network. In Windows, these are called MSRPC and were originally derived from the DCE/RPC implementation. Among other things, MSRPC is used if Windows is remotely controlled over the network – as an admin effects changes in the registry, manages disks or changes user accounts. These calls are still used for many important administrator tasks, even in modern Windows versions. Like the SMB protocol, MSRPC is looking back at an eventful history.
It turned out that Samba's implementation of MSRPC services will exhibit several defects when it is confronted with mass amounts of malformed data; a process that is known as "fuzzing". This isn't limited to crashes and excessive CPU load. A close look revealed that a secured and encrypted connection with authentication had turned into an insecure one. Stefan Metzmacher now tried to figure out why. He discovered that current versions of Windows Server exhibit the same behaviour. They answer RPC requests that manipulate or request sensitive data, even if the connection is neither signed nor encrypted. The vulnerability affects the user login interface Local Security Authority (LSA) and the user management component Security Account Manager (SAM).
During typical usage, such insecure connections do not happen. Clients and server try very hard to only send data over secure connections, but Samba and Windows servers both may be tricked to deviate from this behaviour. As far as we know one only needs a Windows client system that is part of an Active Directory domain. When an administrator logs in on such a system, it sends several standard RPC requests to the domain controller. If a man-in-the-middle intercepts such a connection, he can change harmless name resolutions into requests to LSA or SAM(R) and remove the request to encrypt the connection which the client includes when initiating it. Similar attacks are possible when an admin introduces a new domain controller to an Active Directory.
The attack depends on the way that RPCs take to reach the server. They can arrive wrapped in authenticated SMB connections that are secured by SMB's intergrity checks and encryption – those can't be manipulated. But RPCs can also arrive without an SMB layer directly over a TCP connection. Data that arrives in such a way initially isn't checked for integrity and it's also unencrypted. With RPC via TCP, client and server are also required to handle authentication. The very first RPCs that Windows sends when a user logs on to a domain are handled over TCP. The client first asks the RPC Endpoint Mapper which TCP port it should use to contact its target to initiate a connection via the Microsoft Directory Replication Service (DRSUAPI). The client then uses that port to send RPCs, including authentication data.
Possibilities for Mischief
This is where a man-in-the-middle can strike: He changes the target of the request from DRSUAPI to SAMR and uses the authentication data to start an unencrypted connection to the server. The attacker now has full control over the connection. Using SAMR, the attacker can change the passwords of all users in the Active Directory or create his own administrator accounts for the domain. Just imagine how many Bitcoins a large company would pay to get all user passwords of a domain back that were changed to random input. That man-in-the-middle could be a hacked network printer or sleeper software on a client computer in the domain that an admin logs into for maintenance.
To exploit the vulnerability, an attacker needs to penetrate the local network and gain admin privileges on a local system. If he can manage that, he can become uncontested king of the Windows network by making himself domain administrator. This is due to the unprotected initialisation of RPC connections via TCP. The data sent as part of this isn't checked for integrity and the authentication tokens contained within aren't tied to that connection. A man-in-the-middle can act without invalidating the request.
Hard to Patch
The vulnerability is present in Samba and Windows Server – patches have been made available on Microsoft's April Patch Tuesday. At this point, admins should already have applied them. The fix isn't exactly rocket science: By not accepting unencrypted TCP connections, SAMR and LSA RPC services are made safe. With Samba, stricter settings can cause problems with older client software. This might necessitate admins to use other options in the smb.conf on the server.
Asking why such a trivial fix took so long to release is an understandable response. The Samba developers have done some serious spring cleaning in their RPC implementation to fix all the problems that were found during the stress tests. The current Samba version 4.3 accumulated 200 patches in this way – it took more than a few days to develop and test these. According to Lendecke, Microsoft knew since September 2015 and needed a lot of time to run compatibility tests to make absolutely sure that the fix wouldn't mess with legitimate applications.
While an inner circle of developers at Microsoft and the Samba team knew about the problems for months, SerNet's drumming up of media attention did lead to plenty of criticism of the company. This included claims that the developers had build in the vulnerability intentionally just to reveal it years later for publicity's sake. Announcing the vulnerability like this, according to some, would be a recipe for bad guys to find the problem and write malware for it before the fix was made available. These were just some of the accusations. The name Badlock was made up, mostly because the domain was available and searches for it didn't turn up competing results. This meant SerNet fooled a lot of people who were inspecting Samba's locking code for errors – the locking code was written by Stefan Metzmacher.
The Samba teams gave OEM partners the option to fix their products before the patches and accompanying technical details were released. The Samba bug tracker allowed them to see the corresponding entries and they were able to request the patches, which were encrypted via PGP. That this was enough to secure all systems out in the wild is questionable – patches are only available starting with Samba version 4.2. Samba 4.1 and 3.6 don't get patches since they are not maintained any more. If even older versions are vulnerable isn't known; these versions use another MSRPC implementation. Samba 3.6 used an MSRPC implementation that was backported from Samba 4. That means there is little hope for NAS boxes that use older Samba versions.
Samba has an additional problem that Windows isn't suffering from. With Samba, SMR and LSA are always available, even if Samba isn't running as an Active Directory domain controller and is only used as a standard file server. If the vulnerability affects this use case hasn't been tested so far. On 12 April, eight CVEs were released that pertain to Samba. The patches that have been released address all of them. So far, Microsoft haven't documented their vulnerabilities in detail.
RPC is a Relic of Simpler Times
Speaking of Microsoft: If any Windows RPC services can be attacked in a similar way is currently unknown as well. Badlock shows that the protocols used with RPC are hardly up to modern standards for secure computing environments. Because it has to maintain backwards compatibility, Microsoft can't just change this either and build in an integrity check, for example. Stefan Metzmacher has some ideas how such checks could be built in without compromising backwards compatibility and he is planning to discuss his findings with Microsoft.