A new potential use case of deep learning is the use of it to develop a Cryptocurrency Trader Sentiment Detector. I am currently developing a Sentiment Analyzer on News Headlines, Reddit posts, and Twitter posts by utilizing Recursive Tensor Neural Networks (RNTN) to provide insight into the overall trader sentiment. Trader Sentiment is a key factor in being able to determine cryptocurrency price movements. This article will further discuss the benefits of Trader Sentiment Analysis for Cryptocurrencies and the advantages RNTN’s offer for Sentiment Analysis. The long-term vision of this project is to be able to develop an Artificial Intelligence (AI) Cryptocurrency Trading Bot that can not only consider trader sentiment to make trading decisions but also take advantage of other opportunities such as arbitrage which is the purchase and sale of an asset to profit from a difference in the price.
In order to understand the technical components of the application, it is important to understand the current state of cryptocurrencies and the blockchain applications that they are based on. This article includes information on where cryptocurrencies derive value and the key characteristics of cryptocurrencies.
Cryptocurrencies are an emerging currency and digital asset class. Cryptocurrencies act as a medium of exchange. Digital currencies and digital assets are designed on top of blockchains. The cryptographic nature of cryptocurrencies prevents the creation and duplication of cryptocurrency tokens. Cryptocurrencies have a finite supply. Cryptocurrencies derive part of their value because investors believe that the finite supply along with rising demand over time will only lead the price of them to increase.
Figure 1: Cryptocurrency
Blockchains enable us to record transactions permanently within a distributed ledger. Blockchains allow us to record and conduct transactions of all kinds (exchange of currency/data/service) without the need for a centralized authority.
Blockchain platforms are highly secure because transactions are automatically recorded and tracked by nodes (machines) on the network. Transactions are processed and recorded only on the ledger once verified by most of the nodes on the network. Transactions are recorded on the ledger with a hash of the date and time stamp on the ledger when they occur. If an entity wants to manipulate or change the information, it becomes impossible because the records would have to be modified on most of the nodes in the decentralized network. Transactions which occur on the blockchain offer many benefits including speed, lower cost, security, fewer errors, elimination of central points of attack and failure.
Figure 2: Blockchain
The immutable nature of the information and transactions recorded on the blockchain allow Smart Contracts to be programmed. Smart Contracts are computer protocols that facilitate, verify, or carry out a transaction when certain criteria are met. Smart Contracts can interact with other contracts, make decisions, store data, and transfer currency between individuals. Smart Contracts can run exactly as programmed without downtime, censorship, fraud, or any interference from a separate entity. Smart Contracts can be used to improve business processes in every industry, business, and system where a transaction of some sort is occurring between two individuals.
Blockchains are often referred to as the trust protocol. The unique design of blockchain, through the distributed ledger and Smart Contracts, allows us to conduct transactions that require trust. Investors in blockchain believe that traditional transaction platforms in society can be replaced by decentralized platforms.
Figure 3: Smart Contract
There are many innovative cryptocurrencies. Some notable and long-standing ones are Bitcoin, Litecoin, Ethereum, Golem, and Siacoin. Bitcoin and Litecoin are both peer to peer payment protocols. Ethereum is a developer platform for Decentralized Applications (DAPPS) which can run Smart Contracts. Golem is a decentralized supercomputer network, which can be used for AI application testing and other tasks requiring high computational power. Siacoin is a decentralized cloud storage network.
For these decentralized applications, their native cryptocurrencies act as an entry point for utilization of each network. On Ethereum programmed applications cannot be run without offering some Ethereum as a payment to the network. Golem’s supercomputer network cannot be utilized unless the Golem cryptocurrency is offered in exchange. Cloud Storage cannot be implemented on the network unless Siacoin is offered in exchange.
Cryptocurrency tokens are also offered as a reward or bounty to nodes which are running on the network. For example, owners of bitcoin nodes receive bitcoin as a reward for offering computational power to maintaining the network. Litecoin node owners receive Litecoin as rewards for helping power the Litecoin network. The Ethereum Network also relies on nodes to maintain the network. The nodes create, verify, publish, and propagate information for the Ethereum Blockchain. In exchange for offering their computers as nodes to the Ethereum network, the node owners receive Ethereum in exchange. Golem and Siacoin also work the same way in that node owners receive the Golem and Siacoin, respectively, for maintaining the network. The owners of Golem contribute their spare computational power to keep the blockchain powered supercomputer running. Siacoin node owners offer their spare computational storage to maintain the decentralized storage blockchain. The reward system for node owners incentivizes them to continue to maintain the network. The nodes maintaining the network ensure the continued existence of the cryptocurrency and its value. The more node owners there are the more valuable the cryptocurrency.
Figure 4: Blockchain Nodes
Teams wanting to start their own decentralized applications often launch Initial Coin Offerings (ICO’s). During ICO’s teams behind a decentralized application offer a portion of the tokens available for sale to early investors at a low rate. They also publish a whitepaper which is a technical paper that thoroughly discusses all the aspects of the platform and cryptocurrency; to give investors an understanding of the blockchain technology behind the project. The whitepaper lists the benefits the blockchain offers as well as the economic incentives it offers the community of investors and future node owners. Investors, who believe in the team and the project, may buy the tokens. The participants in the ICO generate the initial value of a cryptocurrency by investing. This allows the team behind it to use the funds raised to fund development and all other costs associated with the project.
As a project continues to develop more investors are attracted to invest and purchase tokens through exchanges. People are incentivized to run nodes on a network as they can be rewarded in a cryptocurrency. The greater the number of node owners the stronger the network. If all things go well with the project, the result is a peer to peer transaction platform that is significantly faster, cost-efficient, and better than any other transaction platform out there. The entry point which is the cryptocurrency also at this point will be priced very well compared to inception as the cryptocurrency is now utilized on a million, billion, or possibly even a trillion-dollar blockchain platform. Whether blockchain platforms will grow to that scale, will only be realized over time. The following are growth rates and prices for some of the cryptocurrencies discussed in this article.
Figure 5: Cryptocurrency Growth Rates
These growth rates demonstrate that significant profit can be made. The key question is how we can use current forecasting technologies to predict price movements. There is no standard method to forecast price movements. Many blockchain platforms aim to provide enhanced technological solutions to existing inefficiencies in transaction platforms. A primary way to gauge the prospect of a project is performing a thorough evaluation of the project’s whitepaper. The whitepaper can give great insight into the technical aspects of the blockchain being proposed as well as give an overall scope of the project. Unfortunately, there is no standard methodology to track prices. There are no fundamentals to be observed in comparison to the stock market. There are no quarterly reports to perform any valuation. Technical Analysis can be useful in achieving the best spreads in trades as well as used to take advantage of arbitrage situations. Technical Analysis can also be used to predict price movements but this article will focus on market sentiment as this is where Deep Learning can be applied efficiently.
Market Sentiment is important to detect cryptocurrency price movements. Once a cryptocurrency and its blockchain protocol are past initial development stages; macro events, political events, network upgrades, conferences, partnerships, Segwits, and trader sentiment tend to create cryptocurrency price movements. One way to gauge these cryptocurrency price movements is to utilize Sentiment Analysis which is a subset of Natural Language Processing (NLP). Many of these cryptocurrency price movements could be determined by Herd Instinct. Herd Instinct according to behavioral finance is a mentality characterized by lack of individual decision-making, causing people to think and act in the same way as the majority of those around them. The price movements tend to be based on market sentiment and the opinions of the communities surrounding the cryptocurrency. Based on these reasons, I believe that sentiment analysis of News Headlines, Reddit posts, and Twitter posts should be the best indicator of the direction of cryptocurrency price movements.
A prominent technique for Sentiment Analysis currently is the use of a Recurrent Neural Network (RNN). RNN’s parse a given text and tokenize the words. The frequency of words is identified and a bag of words representation is created. Then the subjectivity of each word is searched from an existing lexicon. A lexicon is a database of emotional values prerecorded by researchers. The overall sentiment is then computed to classify the text based on the lexicon. RNN’s work well for longer pieces of texts but are ineffective at analyzing sentiment in shorter texts such as News Headlines, Reddit Posts, and Twitter Posts. RNN’s fail to consider all the semantics of linguistics by failing to consider compositionality (word order). This leads RNN’s to be ineffective at identifying change in sentiment and understanding the scope of negation.
Figure 6: Recurrent Neural Networks
An RNTN is best suited for this type of project as it can consider the semantic compositionality of text. When dealing with shorter pieces of text such as a tweet it becomes very important to be able to detect the compositionality of it as there is less information to determine sentiment.
RNTN’s, as mentioned, are great at considering syntactical order. RNTN’s are made up of multiple parts including the parent group known as the root, the child groups known as the leaves, and the scores. Leaf groups receive input and the root group uses a classifier to determine the class and score.
When data is given to the sentiment analyzer it is parsed into a binary tree. Specific vector representations are formed of all the words and represented as leaves. From the bottom up the vectors are used as parameters to optimize and as feature inputs to a softmax classifer. The vectors are classified into five classes along with a score.
The next step is where recursion occurs. When similarities are encoded between two words, the 2 vectors move across to the next root. A score and class are outputted. A score represents the positivity or negativity of a parse while the class encodes the structure in current parses. The first leaf group receives the parse and then the second leaf receives the next word. The score of the parse with all three words are outputted and it moves on to the next root group. The recursion process continues until all inputs are used up with every single word included. In practical applications RNTN’s end up being more complex than this. Rather than using the immediate next word in a sentence for the next leaf group; a RNTN would try all the next words and eventually checks vectors that represent entire sub-parses. Performing this at every step of the recursive process the RNTN can analyze every possible score of the syntactic parse.
Figure 7: Labeled Sentiment Statement
The RNTN can use the score value produced by the root group to pick the best substructure at each recursive process. After the final structure is determined, the net backtracks and labels the data to figure out the grammatical structure of the sentence.
RNTN’s are trained using backpropagation. They are trained by comparing the predicted sentence structure with the proper sentence structure which is obtained from a set of labeled training data. After the data is trained there is a higher probability of the RNTN’S ability to parse things like what was seen in training. One key difference between Recursive Neural Networks and RNTN’s is that for an RNTN the same composition function is used as a tensor so there are fewer parameters to learn. Because of this mentioned difference, similar words can have similar compositional behavior just as similar words can have similar vectors.
A resource that will be used is StanfordCoreNLP which contains a large set of NLP tools. The sentiment analyzer also utilizes the Stanford Sentiment Treebank which is a large corpus of data with fully labeled parse trees that allows for a complete analysis of the compositional effects of sentiment in language. During the recursion process the RNTN is referring to this data set to determine the class and score for a given parse. The Stanford Sentiment Treebank includes a total of 215,154 unique phrases from 10,662 sentences from parse trees which were annotated by 3 human researchers. The Treebank dataset is based off movie reviews from rottentomatoes.com.
One possible challenge that may be encountered during the project is that the RNTN model along with the Stanford Sentiment Treebank may not contain enough data to be able to determine sentiment of cryptocurrencies. The Stanford Sentiment Treebank is based off movie review data, it may not be able to recognize sentiment for all the newer terminology associated with cryptocurrencies. This type of problem may occur, for example, if someone were to post “Ethereum is going to the moon this weekend”. This is a common phrase that is used when people think a cryptocurrency is experiencing or about to experience a large price surge. This is an example of an extremely positive phrase but the sentiment analyzer classified it below as neutral as it was unable to understand the positive connotation of “going to the moon” in the context/compositionality of cryptocurrency.
Figure 8: Undetected Sentiment in Sentence
A possible workaround to this problem is to label and annotate a dataset of cryptocurrency tweets and reddit posts myself to teach the RNTN to be able to detect sentiment in cryptocurrency prices. The following is an example of the same sentence labeled in this example. The sentiment scores vary as there are five levels. Extremely positive, positive, neutral, negative, and extremely negative. I assigned “moon” a positive connotation as this is normally associated with it. “To”, “The”, “going” and most of the other words are given a neutral connotation as they do not have any positive or negative meaning when associated with cryptocurrencies; however, the parse “going to the moon” yields a higher result because the Sentiment Treebank associated “going” and “moon” in the same parse. This parse grants a more positive result as indicated by the dark blue because “going” a direction of cryptocurrency price movement is associated with “moon”, a positive price movement direction.
Figure 9: Labeled Sentiment for Blockchain Statement
Cryptocurrency Price Movements are driven by trader sentiment and therefore being able to detect sentiment in social media posts and news headlines can yield valuable insight. A sentiment analyzer can be key to being able to detect price movements. The long-term vision of the project is to develop an AI cryptocurrency trading bot. As blockchains and smart contracts continue to develop, the world will see an automation of many processes as well as an increase in blockchain based transaction platforms; whether that includes the exchange of digital currency, digital assets, data, and services. As this transition occurs over the next few years cryptocurrencies will only continue to appreciate, which is why a sentiment analyzer of social media posts and news headlines can yield valuable insight into cryptocurrencies and their price movements.
Image Sources:
Figure 1: Cryptocurrency
Figure 2: Blockchains
Figure 3: Smart Contracts
Figure 4: Blockchain Nodes
Figure 6: Recurrent Neural Networks
Figure 7: Stanford Sentiment Treebank