Previous Posts

PsychSignal

Lucena Research overview of using Social Media Sentiment Analysis with machine learning for forecasting a confidence score in a stock’s price move.  -Feature Video w/Erez Katz, Lucena Research Co-Founder & CEO

JUL. 23 2015

#PSYCHSIGNAL #LUCENARESEARCH

Our Newest Innovation, The “Trader Mood Index” ($MIX) Allows You to Easily See That Sentiment Does in Fact, Lead the Price

July 17, 2015, by Bjorn Simundson:

There is an age-old question which asks: “does sentiment lead price, or is it the other way around?” 

The question raises a multitude of responses and theories, usually based around what kind of trading strategy one employs, however proving it has traditionally been akin to answering “which came first, the chicken or the egg?”

This question has baffled people for years, and was unattainable simply because there was no way to poll people’s opinions in large quantities fast enough to learn if they were optimistic or pessimistic about a financial security before the trading day happened. This limitation relegated the question somewhat moot, and hence why many people if asked will conclude “the price reflects the public’s sentiment”.

Now, while it’s certainly true in that people invest and hold in things they believe in and drop those they deem overvalued, concluding that the price being the sole determinant of sentiment has its limitations. For one, it’s purely reactionary, placing the onus on the daily market moves, and to some degree assumes that the public’s opinion about a security lives in a purely binary world of Red and Green market closes. 

Another way to look at it, is through the eyes of a pollster. Things like the VIX, NAAIM, AAII and the Gallup Poll organization have been querying the public for decades. Whether it be to predict the outcome of a presidential election, or to seek the public’s mood about the S&P 500, their goal is the same: “what’s the public feeling right now?”  What the investor is thinking is: “ok, great. Yesterday’s polls are in, how will that affect tomorrow’s trading at the opening bell?”  

What we wanted to find out, was if one could anticipate future market moves if one could poll enough people, faster than ever before? What we hoped to learn was: “do today’s public Psychological stance generate any Signals anticipating tomorrow’s market performance?” (That’s where the name “PsychSignal” came from.)

As a former professional Wall Street Trader, James Crane-Baker (our founder and CEO) figured that if people are expressing themselves online every day, certainly some percentage of them would likely be talking about some investments they have to other investor friends of theirs. 

They’d most likely be sharing some kind of opinion about the Company, its products or financial performance, and talking online with their friends about the elation of a win, or the frustration of a loss. With millions of these conversations happening online, he imagined that the solution would be to build an artificial intelligence system which could correctly interpret their meaning and then quantify each person’s opinion about each security being discussed. From there one could watch how that trended over time and then see if tomorrow’s market performance could indeed be anticipated by today’s conversations. 

The overall concept was to create a new system that would work like the VIX, NAAIM, AAII and Gallup Poll all rolled into an automated view into the public’s underlying mood for each security, through the eyes of a professional trader, instead of through the eyes of a data scientist. 

If it worked, it would be able to deliver an automated 3D view of the market for the first time, and let investors know where they stand this very minute in relation to public popular opinion for the securities they own.

For investors, this would be a huge win, in that if one were able to look at precisely quantified public opinion over time, one would surely see patterns start to emerge. It’s well known that the public opinion of large group changes in dynamic cycles like the tide coming in and out of a harbor, and only reverses abruptly in the case of breaking news or an event.

Now while everyone agrees that news event systems are incredibly useful, they’re still limited by definition as being a purely reactionary way to trade. If you’re relying on a news event system to guide your trades, you’re inherently limited because you still have to wait around for an event to take place, and then hope you’re the first one to hear the news and react correctly to which way the rest of the market will also interpret the news event.

Rather than try and be faster than everyone else’s event-detection systems, James was more interested in building a system which would tell you in the absence of news when the public was generally bullish or bearish independent of events. The rationale is pretty simple. As any seasoned investor knows, when everyone is in love with something, it usually means that the hype machine has taken over, and the stock is overvalued in relation to its fundamentals. And when everyone hates somethings guts, it usually means that the stock is undervalued and can be picked up on sale.

Therefore, the conclusion was that if you’re able to know what the public’s mood is at any given point in time, you’d probably have an insightful window into the trading pattern and where it will likely lead next. 

That is exactly what our system has been built to do. It listens to millions of conversations online every day, and then categorizes the conversations by specific security or market sector, followed by scoring the conversation’s tone according to the way professional traders would interpret the language in that conversation. The output is a continuously updated dataset which quantifies the volume of conversations about a particular security, according to 9 degrees of Bullishness or Bearishness intensity for each conversation as dictated by the specific language used.

The way to think about our system is like having all of these direct and indirect public polling methods listening to the public chatter online and interpreting their meaning in the way that a professional trader would be listening to them. Then, this info is distilled out into a neatly quantified dataset for each of the 10,000+ symbols we cover, and available to your trading platform as an easy to integrate API. It’s like giving a computer trading system a view into the public’s opinion for the first time. We are calling it the “Mood Index” or “MIX” for short.

To show what this means for traders, we recently ran the $MIX for a one year look back on the $SPY to see if the conversation tone and intensity would precede moves int he stock price. The blue line is the price from July ‘14 to July ‘15, and the red / green bars indicate our systems’ daily bull/bear sentiment activity. 

What we found was striking.

If you look at the days where the green bullish conversation activity is off the charts, you’ll see that the price is going up at the same time, but is generally starting to taper off. It’s admittedly pretty common sense in that as people get excited and chase the price up, they don’t want to miss out in hopes that it’s price will keep going up. Now if you look at what happens right after those long green spikes, the green starts to subside, and the red bearish conversation activity eventually takes over as the price backs off and the mood turns sour.

Now remember, these red and green bars are being generated every day. So, if you imagine that if you’d been buying when the red bars began to subside, and selling or shorting when the green bars began to max out, your trading strategy suddenly becomes incredibly simple. When the $MIX signal is super negative, it’s probably a good time to buy, and when everyone’s going crazy and dancing on the tables, it’s probably a good time to cover.  That’s it.

Now, what’s cool about this is that for the first time with our $MIX charts, you can actually see that sentiment does in fact anticipate or lead the price moves. You just have to know which way to trade when the public loves or hates something.

Now, was that the chicken, or the egg?

JUL. 17 2015

We Just Landed In the Wall Street Journal!

“Firms Analyze Tweets to Gauge Sentiment” by Daniel Huang and Bradley Hope

 

Link Here: http://t.co/k4UW9qoUSq

 

Every day, iSentium LLC, a little-known Florida-based technology company, analyzes one million tweets from traders, investors and market commentators to try to find out whether sentiment for a particular stock is generally high or low.

The answer is simple: either a +1 or a -1 for each stock. Yet a handful of banks, hedge-fund firms and high-frequency traders have signed up for the daily indicator, at a cost of $15,000 per month per stock symbol.

ISentium and several other “sentiment analysis” startups are trying to tap Wall Street’s growing desire to harness the world’s vast amount of data to make predictions about the movements of stocks and other securities and derivatives.

Traders and certain investors for years have subscribed to products from companies such as Dataminr Inc. that help them scan Twitter and other social media to detect events. But the ability to parse those posts to discern subtle trends is a new frontier.

TheySay Ltd., co-founded by a professor of computational linguistics at Oxford University in the U.K., has started selling its sentiment-analysis products to research analysts, banks and hedge funds, executives at TheySay noted. There are a host of others, including PsychSignal, Guidewave Consulting and Tashtego LLC, a Boston-based asset manager that plans to launch a sentiment-driven fund later this year.

“What we’re telling you is what does the mob or the crowd say today,” said Gautham Sastri, president and chief executive of iSentium. “Twitter is a big pipeline of emotion and we’re providing a snapshot.”

The idea of using Twitter as an investing guide has plenty of skeptics, with some pointing to the failure of $40 million Derwent Capital Markets. The investment firm, which made bets based on Twitter analysis, closed after just a month of trading in 2012.

Executives from iSentium and other firms say Twitter can provide a useful signal to investors but generally shouldn’t be relied on as the sole reason to make an investment.

ISentium said it has 10 clients paying on a monthly basis. They include large quantitative hedge funds, which use computers to find statistical relationships between data that help predict price movements, as well as traditional hedge funds, family offices and high-frequency-trading firms that use the indicators for market-making strategies.

The company’s system breaks down sentences into their key components and analyzes adjectives and actions associated with subjects mentioned in tweets, as well as their location in sentences, among other factors.

It also can identify situations where a person may convey both a negative and a positive in a single utterance.

For instance, on June 24 activist investor Carl Icahn sent the following tweet about Netflix Inc. and Apple Inc.: “Sold last of our $NFLX today. Believe $AAPL currently represents same opportunity we stated NFLX several years ago.”

Instead of giving it a score of zero, a negative canceling out a positive, iSentium’s system created a -65 sentiment score for Netflix and a +57 score for Apple, the company said. After aggregating data from other Twitter users discussing those companies that day, iSentium came up with an overall indicator of -1 for both companies. The positive sentiment of Mr. Icahn wasn’t enough to change the overall signal, according to the company.

Social-media-analytics companies use different techniques to drive market insights. TheySay co-founder Stephen Pulman, a professor at Oxford, said that what sets the U.K.-based company apart is a focus on “compositionality,” a principle that examines not only the meaning of words but their arrangement in relation to each other.

“Words in isolation may have a positive or negative sentiment but once you put them together they can often mean something else,” Mr. Pulman said.

PsychSignal employed several individuals with Ph.D.s in psychology to create an engine that can track 12 different emotions, including anger, sadness and love. Clients receive two scores—bullishness and bearishness—and the company plans on introducing more in the coming months.

In addition to popular websites like Twitter and Stocktwits, PsychSignal’s universe of social media data also covers information shared on private chat rooms frequented by traders, said founder James Crane-Baker, giving their inputs more of a trading focus.

Twitter Inc. itself also sells data directly to a range of businesses, including hedge funds and banks, a person familiar with its sales said. Some of those buyers have data scientists who conduct their own analysis of tweets, including of sentiment around companies, the person said.

Twitter also signed an agreement with International Business Machines Corp.granting the computer company access to the full “fire hose” of tweets sent from around the world every day.

IBM said the applications went beyond predicting moves in the markets. Its analysts were finding ways to help large banks and retailers create better products and services for their customers.

Mutual funds also are exploring such tools.

Franklin Templeton Inc. has hosted two conferences for fund managers, including in March of this year, to meet with companies that provide novel data sets and analytic tools. A session on social media was titled: “Is it Signal or Noise? Determining Sentiment and Identifying Trends.”

Franklin Templeton said it came to a major realization when Russia annexed the Ukrainian territory of Crimea in March 2014. The most granular and fastest news came out of Twitter, rather than traditional news sources, said David Lewis, the firm’s head trader.

“We’re looking at different things to help us improve our decision making,” he said.

-end

JUL. 15 2015

Lucena Research QuantDesk Machine Learning Analysis Forecast Using PsychSignal Data

Lucena Research recently shared with us a study using PsychSignal’s trader mood indices within their popular Tiebreaker and BlackDog premium trading model strategies for Alpha Generation. It was very interesting indeed to watch their team demonstrate for us online how they were integrating the data to their proprietary systems…

http://lucenaresearch.com/wp-content/themes/lucena-theme/predictions/6-29-15.html

 

About Lucena Research

Lucena Research brings elite technology to hedge funds, investment professionals and wealth advisors. Our Artificial Intelligence decision support technology enables investment professionals to find market opportunities and to reduce risk in their portfolio.

We employ Machine Learning technology to help our customers exploit market opportunities with precision and scientifically validate their investment strategies before risking capital.

Learn more at:
www.lucenaresearch.com

JUL. 15 2015

Long Only trading Strategy with NLP derived Social Media Sentiment - by Quantopian

This article posted on Quantopian’s website by Seong Lee is a nice, tidy summary about how they recently backtested a trading strategy using our data to measure the emotional pulse of the markets, and what that would do for a quant on a Long Only trading Strategy.  

The post can be found here: http://t.co/SYtVEzpCOR

(be sure to check out the extra content in their post…  it’s important and insightful)

 

The summary by Quantopian is as follows:

Hey all,

Trader mood indices attempt to measure the emotional pulse of the markets; How bullish are traders on $GOOG? or How bearish are traders on $SPY? and Are people equally bullish and bearish on $AAPL?

In order to see what trader mood indices might look like in a trading strategy, I collaborated with the folks over at PsychSignal and backtested a trading strategy based off their data.

The strategy is pretty simple. The PsychSignal dataset is derived from a natural language processing engine that detects bullish/bearish trader moods from raw messages. I follow the BULLISH indicators within the NASDAQ 100 to create a long-only strategy. The raw messages that the scores are based off of are fromStocktwits - you can check out previous work with them here.

PsychSignal has datasets that date back to 2009 with sources like StockTwits, Twitter, T3Live Chat and other private stock chat rooms.

Here are the strategy notes:

Data Fields Used:

  • Bullish Intensity (-5 - 5) – The PsychSignal bullish sentiment score
  • Bullish Minus Bearish Intensity score (-5 - 5) – The PsychSignal bullish minus bearish sentiment score
  • Number of Message Analyzed – The total number of messages analyzed

Strategy Notes:

  • Look for where total messages over the last 30 days are greater than 50
  • Look for where the average bullish - bearish signal is greater than 0
  • Score each security by it’s bullish_intensity level and the number of bull messages
  • Go long on the top 7 securities scored by our factor ranking

While the backtest outperforms the market, there are still many more roads to explore. Roads that any interested quant is free to travel down. On that note, Dr. Checkley from PsychSignal comments that:

“There are no short positions, which goes hand-in-hand with the beta problem. But clearly, it would be no great challenge to, say, “invert” the analysis above and seek-out strong bear signals for short trades. And there is unlimited scope for changing filters, the buy and sell rules, the universe of stocks, etc. As you get comfortable with using these sentiment metrics, we recommend blending the bull and bear measures with other indicators, such as long and short-term price trend or trade volume. You can also use a more sophisticated model, such as a Neural Network, to create your predictions and trading signals. With hundreds of heavily-tweeted stocks and assets to choose from, and your full arsenal of analytical tools and algorithm-refinements, you can improve on our results in short-order.”

For more information, please visit:

www.quantopian.com

JUL. 15 2015

Lucena Research partners with PsychSignal to deliver predictive analysis from Social Media Sentiment Data


Posted on May 14, 2015
ATLANTA, May 14, 2015 – Lucena Research, a leading provider of big-data analytics and decision support technology, and PsychSignal, a provider of real time financial sentiment data, have announced today a partnership to enable PsychSignal data and insights on Lucena’s flagship analytics product QuantDesk®.

QuantDesk® enables hedge funds, portfolio managers and other investment professionals to quickly validate and subsequently seek alpha from a growing array of orthogonal predictive data sources. The open nature of the QuantDesk® platform enables clients to use validated data sets such as PsychSignal in research and active trading environments. Specifically, PsychSignal’s data can be used in the context of Price Forecasting and Event Signal Analysis. Further, Lucena’s machine learning technology enables customers to visualize and quantify the PsychSignal data, as well as recommend additional factors that enhance the data set’s predictability. This combination of predictive analytics and human insights enables the system to rapidly deliver customized portfolio construction, portfolio optimization and proprietary strategies.

Eric Davidson, VP of Business Development at Lucena, said “There is a huge opportunity and trend around non-traditional data sources. The possibility of untapped alpha in the growing body of unique data sets is intriguing to forward-thinking Portfolio Managers, whether quantitative managers or traditional fundamental managers. Portfolio Managers often ask us what unique data sets we have to present to them. However, these Portfolio Managers often lack the quantitative infrastructure and in-house know how to cost-effectively integrate a new data source. Lucena is well positioned in this workflow, solving a pain point for fund managers by streamlining this workflow and helping the managers seek alpha in new ways for their clients. We look forward to working with PsychSignal.”

“We are excited to work with Lucena Research in providing unique solutions for sophisticated investment managers.” said James Crane-Baker, CEO of PsychSignal. “We believe that Lucena’s customers will benefit greatly with access to Lucena’s industry-leading quantitative data integration, further enhanced with PsychSignal’s exclusive access to public and private online sentiment analysis data processing technology. Our two platforms working hand in hand is a very powerful combination indeed, and we are very bullish on the forthcoming results.”

About Lucena Research
Lucena Research is a leader in decision support technology for investment professionals Lucena delivers quantitative analysis and statistical forecasting based on machine learning technology that enable our customers to exploit market opportunities with precision and to reduce risk in their portfolios.

QuantDesk®, Lucena’s flagship product, provides advanced, yet affordable, subscription-based portfolio optimization tools designed to scientifically validate and augment our clients’ investment strategies. The product includes five portfolio management modules – Price Forecaster, Portfolio Optimizer, Hedge Finder, Event Analyzer and Back ester. Lucena also provides investment strategies in the form of model portfolio from which customers can derive their own implementation.

Headquartered in Atlanta GA, Lucena supports a wide range of investment professionals worldwide.

To learn more about Lucena, please visit http://www.lucenaresearch.com or email us at: [email protected]

About PsychSignal
PsychSignal is a privately-held Delaware Corporation which works with hedge funds, trade execution platforms, chart platforms, investment advisory firms, and traders, to deliver innovative solutions and licensable public opinion data mining data feeds & web applications to the investment community. Please visit us online at: http://www.psychsignal.com, or email us at: [email protected]

This entry was posted in Uncategorized by Lucena Research Inc. and can e found at the following link: http://lucenaresearch.com/lucena-research-partners-with-psychsignal-to-deliver-predictive-analysis-from-social-media-sentiment-data/

MAY. 27 2015

PsychSignal Video Presentation at FinovateSpring 2015 is Now LIVE!

MAY. 22 2015

#PSYCHSIGNAL #FINOVATE #FINOVATESPRING

Watch PsychSignal’s James & Bjorn Unveil Our Newest FinTech Product: SquawkrBox at Finovate Spring 2015!  (w/Video Link)

PsychSignal
Friday, May 22, 2015 @ 18:40EDT
www.psychsignal.com
www.squawkrbox.com
www.finovate.com
(Written by Bjorn Simundson)

FINOVATE Spring 2015 Presentation Recap

Last week, James and I attended the 2015 Finovate Spring Conference held in the San Jose City National Civic in the heart of Silicon Valley. San Jose is not only a hotbed for financial technology and innovative entrepreneurs, but the venue also hosts many leading conventions, trade shows, and technology events each year.

The focus of the Finovate convention series is to showcase the future of banking and financial technology known as “FinTech”, connecting new ideas with motivated industry players from around the world.  There were presenters and attendees from every continent, with nearly 2,000 people in the audience.  

PsychSignal was invited by Finovate to be one of the main-stage presenters during the 2-day event, and was given the floor for a 7-minute demonstration of our products and services.

While onstage, we unveiled our newest product: SquawkrBox

SquawkrBox is a new Financial Technology Innovation which allows our customers to view the data from our API in an easy to use online portal. This allows us to expand our product reach to a wider audience including those who don’t have access to a supercomputer.

The concept of SquawkrBox originated with James a number of years ago after he spent 10 years as a trader on Wall Street as a founding member of Broadway Trading LLC of NYC beginning in the 1990′s, where he used what was known as a “Squawk-Box” on his desk to listen to a paid announcer on the floor of the Chicago S&P Futures Pit, who would call the trades coming in and out of the pit, not unlike an announcer at a horserace, and sounded likethis. The tone of the announcers voices, and the sound of the crowd behind them gave the traders listening to the Squawk-Box an inside view of what was happening on the trading floor, and allowed them to take their positions according to the tone of the room.

As one of the first electronic traders in the world when the internet was in its infancy, James hypothesized that at some point in the future, people would not only be executing their trades online, but that people would be communicating online as well.  He imagined that the two realities would likely merge, and that he could build a computer system that would listen to the tone of the conversations being had online, and how that might impact stock trading.

Over the years, James’ hypotheses came true. Traditional paper trading was replaced by computers, and the trading pits, including the Squawk-Boxes were rendered obsolete as the trading world moved online.  Gone were the days of being able to listen to the sound of the room as trading began to be relegated to large data farms instead of loud boisterous rooms full of traders and telephones.

James created the concept behind #PsychSignal as a solution for finding the financial conversations happening online much in the way that he used to listen to the tone of the financial trading rooms via the Squawk-Box, and then filtering out the noise and selling the clean data via API to our Quant Fund and Hedge Fund clients so that they can integrate financial sentiment data into their quantitative strategies and algorithms.  

SquawkrBox, launched @Finovate is a way to visually represent these financial conversations and keep an eye on the sentiment of your portfolio in an easy to use online dashboard.  The SquawkrBox is powered by the SquawkrBot, a robotic, artificial intelligence neural network that lives on our big-data servers and powers both the SquawkrBox and our API.

The innovation that we demonstrated @FinovateSpring2015 was how one could use the SquawkrBox to monitor their portfolio, and to have the SquawkrBot send you automatic alerts via email or SMS should the financial sentiment of securities in your portfolio accelerate, or suddenly change direction.

The presentation was a smashing success, and we were absolutely INUNDATED with interested parties handing us business cards asking about the product immediately after the presentation once we got back to the booth.

Take a look at the 7-minute video here:  http://finovate.com/videos/finovatespring-2015-psychsignal/ 
and please, spread the word and share the link with your friends!

We’d love to hear from you, please visit our contact page, our Twitter handle or our website to learn more!

MAY. 22 2015

#PSYCHSIGNAL #FINOVATE #SQUAWKRBOX #FINTECH #FINTECHHACK #FINOVATESPRING

The Story of PsychSignal’s AI-Based Trading Alerts System:

An Interview with its Creator Henk Alles: 

Well-known in the worlds of computational linguistics and AI, Dutchman Henk Alles is the lead engineer of PsychSignal’s soon-to-be-launched trading alerts tools. We recently tracked down Henk for a chat about his innovative work.

PsychSignal: Morning Henk! Please can you give us an overview of your work on the new trading alerts?

Henk: Of course…we developed alerts over three time horizons and for three different kinds of events. The time windows over which we predict interesting change are minutes, hours, and a day or more. And the alerts themselves focus on predicting price direction, up or down, and notable increases in price volatility. The third style of alert is for anomalous sentiment behavior i.e. when we see highly untypical types and ranges of sentiment with respect to the market, we report that too.

PsychSignal: Okay, but before we drill down into how it’s all done, can you tell us a bit about your background?

Henk: Well, I am a software engineer who specializes in unstructured data. Building on prior research experience at Xerox Parc and MediaLabs, I’ve spent a lot of effort, since the Nineties, on natural language search, semantic mapping and various tools to better manage documents and databases. But over the last ten years, the focus has really turned much more onto Artificial Intelligence, social media and predictive analytics. Many of my projects now concern using Artificial Neural Networks, coupled with various kinds of web and social media content, to forecast markets; demand, new trends, risk and so forth. The beauty of this kind of project is that it really works! It turns out that this kind of information – especially from social media – has real predictive value. For example, it can save companies, comfortably, tens of thousands of dollars per quarter on wasted product and market development effort and, as you know, it has clear value in financial market trading.

PsychSignal: So how successful were the new alerts from PsychSignal?

Henk: Naturally, I’d say they are superb! However, when we train the neural networks to take live sentiments, price trends, trading volumes and so on, as inputs to a predictive model for price direction and volatility, we find the model works well and is robust. In other words, across all symbols analysed, we get the predictions correct on 60-80% of occasions. For some reason, 80% is a kind of “natural ceiling” for accuracy and it’s hard to beat that. But nevertheless, the team was happy to reach this kind of performance level and to do it consistently over time and assets. Put another way, I am happy to have “skin in the game” and trade my own funds with these alerts!

PsychSignal: What kind of Artificial Neural Network (“ANN”) are you using?

Henk: We tested many kinds of ANNs – with different network topologies and ways of learning – across the universe of assets. It’s really interesting to me that four different types of ANN can cover pretty much every one of the thousands of assets covered by PsychSignal’s sentiments. Of course, that raises the question: why do those four kinds of ANN work for everything? At present I don’t have a good answer to that question but I’m looking into the idea that basic industry dynamics – for example, is the stock price driven most by the economic cycle or fashion or innovation or government spending? - might be a factor in discriminating which ANN works best. And if that hypothesis of “basic industry dynamics drives price in around four different ways” is correct, we can also expect sentiment to interact with price dynamics in four quite different ways over time and assets.

PsychSignal: So where next? What might be the next natural development of the new trading alerts system?

Henk: We’d love to hear from users! Nothing beats feedback from those trading live funds. We think the alerts will prove profitable and popular with retail traders and some funds. Having said that, some natural extensions do come to mind for the next generation of alerts. For example, we can say more about precisely what we are alerting: rather than say, “65% chance of $APPL price decrease in the next 3 hours”, we might refine this into, “68% chance of a 1.2% price increase for $APPL, within the next 3 hours”. That kind of specificity is possible and something we can consider adding. And of course users love to get updated overviews on how past alerts have performed. Just knowing that 60-80% of recent alerts have been big profit opportunities is all the inspiration most of us need. Finally, it’s not dauntingly difficult to add – under various controllable assumptions – how the use of these alerts converts to standard metrics of trading performance over time.

PsychSignal: Well thanks so much Henk! Any final thoughts you’d like to add?

Henk: I’d just urge people to use the alerts, give themselves time to build real success, and please contact us with feedback. We’d love to hear from you.

MAY. 18 2015

Revisiting our epic $EUR/USD call from March http://blog.psychsignal.com/post/114409386650/euro-set-to-bounce-as-sentiment-cycle-turns

MAY. 7 2015

Artificial Neural Networks and Forecasting: A Model of Oil

We recently posted on how well Artificial Neural Networks (“ANNs”) and PsychSignal’s sentiment metrics predict the next day closing price of the S&P500. But that predictive success is arguably tainted by $SPY’s long-term tendency for gently ascending the price graph. That kind of little-varying trend makes prediction suspiciously easy. We therefore wanted to test the same ANN-based forecasting method on a far more feral asset. Oil is our answer; see the price graph below for Oil (strictly speaking, the iPath S&P GSCI Crude Oil TR ETN) over the last 5 years. This is a price falling off a cliff during and after October 2014. In fact, the Oil price fell below USD21 – and out of its long-established price corridor - on 15th October 2014.

How we tested the data

We trained the model on sentiment and daily price data from 25th July 2014 to 26th December 2014. We then predict prices with the trained ANN model for 29th December 2014 to 12th March 2015.

Remember that sentiments are gathered 24 hours a day, where a day is defined via the UTC. Hence, within that 24 hours, there are three interesting periods: before New York markets open; during market hours; and following market close. We parsed daily sentiments into those three crucial periods. A day’s Oil market price direction is therefore predicted with sentiments from yesterday’s market period, from after the close yesterday, and then from sentiments gathered before market-open on the same day to be predicted.

In other words, we are predicting prices – in this case, daily prices – with just sentiment data and basic price dynamics. We added just one price action metric; 5 day moving average. We also included the count of relevant messages and ratios of bull and bear metrics. Of the five 5 options on the PsychSignal API, we selected StockTwits with Twitter and re-Tweets, as the sentiment-source.

We then test our model by measuring what percentage of all our next-day forecasts are correct in the pure binary sense of getting it right about either closing higher or lower than the prior day’s trading. A typical feed forward neural network is employed, with one hidden layer with 10 nodes, 15 inputs and just 1 output for forecast daily closing price. Training cycles totalled 100k.

The findings

We got the direction of daily closing prices correct on 63% of the 49 trading days predicted. The regression graph after 100k training cycles is shown below (R-squared is 55% here). There is ample room for improving R-squared with more training. Furthermore, the big swings in price during the predicted period (from well-under USD10 to over USD12) had little effect on the quality of the model.

The ANN diagram is below. The after-hours sum of bull market sentiment (variable number 5) is particularly material in predicting the next day’s market price direction for Oil.

In sum, we regard this ANN prediction exercise as another measured success. Getting more than 60% correct for daily directional forecasts is encouraging. Moreover, the Oil price has behaved in strange and volatile ways in recent times. Yet the predictive model holds-up during the turmoil.

 

Our research attempts to discover the best use cases for a new breed of real time financial sentiment emerging as a result of an explosion in online conversation surrounding financial markets. We’re obviously biased in using and promoting our proprietary financial sentiment, however the results should be reproducible with any similar sentiment measurement which utilizes a system of two separate continuous scales of bullishness and bearishness. Have a go yourself testing our daily data for free on Quandl.https://www.quandl.com/data/PS1

MAY. 6 2015

A Simple Test of the Predictability of Large Daily Changes in SPY

We know market sentiment metrics – used as a stand-alone signal - can predict market changes over minutes and sometimes longer. But what about the vital issue of really big day-to-day changes? And what if we add the simplest of daily price dynamics to the daily sentiment metric-based predictive model?

Here we present tests of how well daily sentiments predict big market shifts. We take daily S&P500 closing prices from 2nd September 2009 to 5th March 2015. These prices are matched to contemporaneous daily sentiment metrics. We then focus on big changes in daily price; we consider prices moving more than 1.5% up or down compared with the prior day’s close.

Of the 1432 trading days tested, 332 change more than 1% in either direction. When considering the bigger change of 1.5%, the count slims to 154 days; 74 up and 80 down. This constitutes what, for us, is the surprisingly large ratio of 11% of the total days analyzed. Note that, while we focus on the bigger change of 1.5% in daily values, the conclusions presented below are broadly the same whether one considers 1% or 1.5% daily changes.

Before we get started on the analysis, consider one of the simplest explorations imaginable; how do the bull and bear metrics from the day just before the big change compare with the average of the bull and bear metrics from 4, 3 and 2 days prior?

Sentiment Metrics for Daily Changes in $SPY (Sept 2009 to March 2015) of more than 1.5% in Either Direction: A Comparison of the Prior Day’s Metrics with those of 2-4 Days Prior to the Large Market Move.

 

As the descriptive table above suggests, big market increases – of more than 1.5% - are preceded by a prior day’s material fall (compared with the 3 days before that) in the count of bear messages and an increase in the bull/bear ratio. In contrast, big market falls – of more than 1.5% - are preceded by a prior day’s material decrease in strength of bullish sentiment coupled, strangely, with an increase in the count of bullish messages. Also relevant is an increase in bearish sentiment coupled with an increase in bearish messages. More simply, falls are predicted by a fall in the bull/bear ratio.

And when you review simple price dynamics, it’s clear that big price rises often follow big falls from the last five trading days. And, counter-intuitively, big price falls will often follow other big price drops in the last few days. Lightning always strikes (at least) twice.

Note that the descriptive statistics in the table above are robust to varying: the threshold for defining a “big” market change; the number of days prior over which average values are taken; and the dates over which data is tested (within bounds of statistical reliability). In other words, these patterns in the data are persistent.

We now take ideas above and create a simple back-test. To predict big price increases we take these criteria for the day before the big change:

-          The bull/bear ratio is at least 5% greater than the average of the last five days

-          The bear metric is at least 1.64

-          The day prior to the day prior to the big predicted change had growth of less than 2%

-          At least one of the five days prior to the forecast day had growth of -0.6% or lower.

To predict big price falls we take these criteria for the day before the big change:

-          The bull/bear ratio is at least 10% lower than the average of the prior three days

-          The bear sentiment is at least 10% higher than the average of the prior three days

-          The total scanned messages is at least 15% higher than the average of the prior three days

-          At least one of the five days prior to the forecast days had growth of -1.0% or lower.

We will return to our predictive model every 2-3 months to review its performance in the intervening time, i.e. a regular forward-test, of both long and short daily trades, will be posted on this blog.

 

Results

The table below offers a mixed picture of the success of this exercise in finding a simple predictive signal for both significant up and down days on the SPY. First, average daily returns are impressive; the bullish signal achieved 0.18% average daily returns, while the bearish signal made -0.34%; and that in the context of average returns across all days of 0.06%. The bullish signal predicted just over 35% of bullish days, while the bearish signal detected only 5%. However, with more sparse bearish signals, the hit-rate-per-signal is a more respectable 17.39%.

Table of Predictive Ability and Returns of Trading Signal for Highly Bullish and Bearish Days

 

 

Discussion

While clearly not the last word in predicting big market shifts, the tests above have the advantage of simplicity and easy refinement. They focus on simply-testable sentiment moves, coupled with very basic price dynamics, in just the context of the few days prior to big price changes. The size of market move considered – more than 1.5% in either direction – is big enough to be very interesting to traders and yet frequent enough to produce reliable statistics.

Because the daily returns of each type of prediction are impressive, while the “hit rate” is modest, we’d have to say these signals are better at forecasting daily price direction than “big” change per se. Nevertheless, despite their simplicity, we believe a framework of this type can be the basis of a sound trading system.

We look forward to returning to the framework outlined above. We will review its practical trading performance in the coming months.

 

Our research attempts to discover the best use cases for a new breed of real time financial sentiment emerging as a result of an explosion in online conversation surrounding financial markets. We’re obviously biased in using and promoting our proprietary financial sentiment, however the results should be reproducible with any similar sentiment measurement which utilizes a system of two separate continuous scales of bullishness and bearishness. Have a go yourself testing our daily data for free on Quandl.https://www.quandl.com/data/PS1

APR. 27 2015

Trading Research Using PsychSignal’s Sentiment Metrics

We just spotted this research paper http://arxiv.org/pdf/1504.02972.pdf , which was recently published on the moneyscience.com blog, here:

It’s an interesting paper, and not just for the promising portfolio returns reported (see graph below). PsychSignal’s daily sentiment metrics are used, along with daily prices for a diverse basket of stocks. A trading algorithm is trained on data from 2010 to 2013 and then run on 2014. But the really interesting thing about this article is the use of a Genetic Algorithm (“GA”) to optimize the trading strategy.

Performance Outcomes for the Genetic Algorithm-Derived Trading Strategy, Using PsychSignal’s Metrics

If you’re not familiar with the world of the GA, here’s a brief-and-simplified introduction. Consider; there’s a whole class of problem out there that exists in a clear framework but which offers combinatorial difficulty. This is the kind of challenge at which we aim our GA. For example, the paper above defines how a rich class of trading strategies can be defined in terms of just eight variables; whether to go long or short, what threshold of bull or bear to take as the signal to buy or sell,…But what is a good choice for each of the eight variables? Rather than stress about that, we can randomly assign a proposed solution and see how it performs in trading i.e. we arbitrarily fix values for each of the eight variables and then assess performance outcomes. Let’s say we want to maximize the Sharpe Ratio of trading returns.

The “genetic” component is the clever bit. Because, instead of testing just one solution, we could – while waiving our concerns about computational load – test thousands or millions of strategies. We can blend or “mate” the eight variable values for those strategies performing best and thereby create a new generation of even-better trading strategies. This process can continue until performance improvements – measured in this discussion as higher Sharpe Ratio - start to level out. We can guess at that point we have approached an “optimal” solution and use that strategy for our trading.

There are several useful websites and books available on this topic e.g.

http://www.obitko.com/tutorials/genetic-algorithms/introduction.php

The paper above uses a GA-based approach, with good results. The author mentions trading costs are not yet modeled, and we’d favor forecasting markets – and therefore selecting trades and building strategies - with not just sentiments as inputs but also some synergistic data of other types, such as something on price trends. But given the stringent requirements of pure-sentiment-based trading, this is a strong beginning.

 

 

Our research attempts to discover the best use cases for a new breed of real time financial sentiment emerging as a result of an explosion in online conversation surrounding financial markets. We’re obviously biased in using and promoting our proprietary financial sentiment, however the results should be reproducible with any similar sentiment measurement which utilizes a system of two separate continuous scales of bullishness and bearishness. Have a go yourself testing our daily data for free on Quandl.https://www.quandl.com/data/PS1

APR. 20 2015

A Familiar Debate Concerning Social Media Signals

It’s a 2012 vintage now, but the themes of this three year-old Economist article do keep reappearing. The article makes several familiar points: there’s lots of good research evidence out there for social media sentiment metrics having predictive power; there are more and more granular sentiment metrics available than ever; but if this data is so wonderful, why do funds using it go bust? And, in contrast, why would anyone sitting on such a gold mine bother to share it?

The Economist article is here:

http://www.economist.com/blogs/graphicdetail/2012/06/tracking-social-media

We take each of the points in turn but focus on the last issue, which we believe is the most-open to misinterpretation i.e. what motivates providers of sentiment data to share it.

Regarding evidence for sentiment metrics’ value in forecasting financial markets, there are – as blogged two weeks ago – over 50 well-composed research papers in this area, with around 40 showing very positive findings on the predictability of prices. PsychSignal’s own research, which was recently accepted at a peer-reviewed journal, also shows good evidence of sentiments predicting prices.

Regarding the volume and granularity of social media sentiment data, we are constantly increasing the richness of data we offer, with the recent addition of well-known and long-established traders’ chat rooms as another source of raw information. And while we love high quantities of data, quality is arguably the big differentiator in this market. Any expert in semantic search and computational linguistics will tell you; there’s nothing trivial about extracting meaning or mood from millions of web-based messages. In fact, the technical challenges are so vast and subtle, we would – to put it diplomatically – expect to see, across this market, quite a spectrum in terms of data quality. And how do we assess quality? There are many definitions but the simplest is proven predictive ability.

Let’s turn to the Economist’s final contention; if you have wonderful predictive data there’s no point in selling it. This argument is easily extended into a tougher form; if you are selling data it must be worthless. We find this view popping-up not infrequently, so let’s give it more attention. We make two rebuttals.

The first rebuttal is easy; there are, literally, millions of ways to profit from sentiments. By the time you’ve considered different symbols, instruments to trade, holding times and forecast windows, and ways of blending sentiment metrics with other signals, the choice is vast. To put it more simply, you could trade on your own data for years and have hundreds of client users, yet still barely cross paths on the markets. Hence, to not sell data would simply be missing the opportunity of serving potent value-added to customers.

The second rebuttal is a little more subtle, but, we believe, even more compelling. Our best understanding of “good strategy” (see, for example, John Kay’s “Foundations of Corporate Success”, or Richard Rumelt’s “Good Strategy, Bad Strategy”) states that long-term out-performance comes from matching distinctive, defensible and value-adding capabilities to market opportunities. The idea is strategy is grounded on you doing something well – such as looking after certain customers, or building a certain type of quality into a product – that people value. They value it by paying for it. But, because your capability is rare, hard-to-copy, or to substitute with some alternative, that flow of value out from you to customers, and the reciprocal flow of money from customers back to you, is sustainable for the long-term. The art of business strategy then reduces to some simple core questions: what are you distinctively good at, how can you defend and nurture that capability, and where can you best exploit it for profit? Extending your activities beyond your distinctive capabilities is a recipe for diluting your value and burn-out through over-ambition. In competitive markets, you just won’t succeed unless you’ve got some superb secret sauce in the mix.

At PsychSignal we believe we are distinctively good at sourcing, classifying, refining and distributing sentiment metrics. We believe this is a high-value activity, appreciated by growing numbers of traders. We believe we sell a quality product because it has repeatedly-proven predictive ability. But are we better than our hedge fund and trader clients at trading algorithm development and testing, or portfolio management, or fund raising? We suspect not. Put directly, no matter how good our data, our long-term success comes from selling data services, not from running big trading funds. We should work where we are distinctively effective. And that doesn’t include big-time fund management.

In sum, the argument “if you don’t eat your own cooking it must be poison” just doesn’t bear scrutiny. Companies like PsychSignal do what they are best at. And that is selling sentiment metrics to traders and funds.

APR. 15 2015

Delving into the Published Research on Sentiments and Financial Market Prediction

There has been huge expansion in interest in the ability of social media sentiment metrics to predict financial markets. We therefore thought we’d offer a brief review of research findings from peer-reviewed academic journals. As you might expect, research is overwhelmingly positive in terms of sentiments’ ability to be usefully predictive. But, taken in sum, it is also contradictory.

The literature on this topic grows; there are now over 50 relevant published papers in academic research journals. But there is not uniform agreement on the predictive ability of sentiments. Research findings range from sentiment analysis being highly predictive, to having no forecasting value whatsoever, particularly for price direction (see Nassirtoussi et al., 2014, for a near-comprehensive and detailed literature review).

A Brief Review of Research Literature

Responding to critiques of the Efficient Market Hypothesis (c.f. Cutler et al., 1989), recent studies in behavioral finance suggest that emotion has a role in investment decision making (Lowenstein and Lerner, 2003). Whether the diverse and decentralized opinions expressed on Twitter and StockTwits, when aggregated, reflect “the wisdom of crowds” is a central issue (Surowiecki, 2004).

Prior research argues that sentiment metrics are usefully predictived of stock price returns (c.f. Mittermayer, 2004), volatility (c.f. Antweiler and Frank, 2004), and traded volume (c.f. Oliveira et al., 2013). In contrast, several researchers find that social media sentiment metrics proffer no material advantage, particularly for stock returns or price direction (Nassirtoussi et al., 2014 list several such studies).

A seminal work in this area, Bollen et al., (2011), sampled sentiments in six dimensions (Calm, Alert, Sure, Vital, Kind, and Happy) and composed, employing artificial neural networks, a model of improved forecasts of the Dow Jones Industrial Average. Similarly, Sprenger and Welpe (2010), used six months of sentiment data to find improved forecasts of S&P stock returns, also showing that increased tweet volume predicts increased trading volume.

A Critical Comment

There remain reasons for doubt. Some studies use only short periods of sampled data (c.f. Yu et al., 2013). Moreover, most test daily data, with only a handful of papers studying intraday day e.g. granular to the half or quarter hour (see Antweiler and Frank, 2004 for data tests at 15 minute intervals). Few papers address whether the trading returns they find are tied to risk premia.

Hence, along with contradictory research findings on sentiments’ forecasting worth, most prior studies use daily sentiments and daily market closing prices. Yet, assuming intraday sentiment metrics are predictive of market behavior at all, there is reason to believe any valuable data would be used speedily by market participants. In accordance with traders using new information rapidly, the forecasting window would be one hour or less, rather than many hours or days (Chordia et al., 2005). Moreover, many of the prior studies using more thorough-going methods actually pre-date the availability of good volumes of data from either Twitter or StockTwits.

Summary

We regard this field of research – studying trade-able predictive ties from sentiments to market behavior – as, largely, up for grabs. Many of the central research issues remain unresolved. For example, at what times are sentiments most predictive and for what kind of stocks or assets? Does the overall level or trend of bullish or bearishness strongly affect predictability of market behaviour? Do different geographies have different relationships between sentiments and markets? Are crises and major reversals predictable?

While the majority of past published studies are very positive about sentiments – particularly for predicting price volatility and trading volume – they are enough negative findings in a few well-composed papers to suggest some doubt. But given that not all research shares the same rigor of methods, or great volumes and granularity of data, it also would be surprising if the extant literature were uniformly affirmative.

Our own soon-to-be-published research with PsychSignal’s metrics do indeed provide robust evidence of a causal link from sentiments to market behaviors. We look forward to sharing more of our findings with you soon.

APR. 2 2015

http://t.umblr.com/redirect?t=NWNlNjI2ZjZjYjkxNGY5N2NhNDM2MjdhNWRlY2MxMjMyMTNiYTIwZixIdXVKN2Z4bg%3D%3D&u=aHR0cDovL2JlaGF2aW9yYWxxdWFudC5jb20vYS1jb21wYXJpc29uLW9mLWludmVzdG9yLXNlbnRpbWVudC1pbmRpY2F0b3JzLw%3D%3D

MAR. 30 2015

Our data is live on Quandl

Head on over to Quandl where we’ve just launched as another one of their premium datasets.

Our dataset on Quandl is limited to daily data. If you’re looking for intra-day data that’s available exclusively via our own API

MAR. 26 2015

Investor sentiment for the S&P 500

Risto Karjalainen has written another fantastic technical analysis of our data. Summary below… read the full post here.

  • There is a clear asymmetry in bullish vs. bearish investor sentiment. This is evident in the distribution of the stock-level sentiment, in the autocorrelation structure of the aggregate sentiment, and in the degree that the sentiment is influenced by the past price action.
  • Pockets of predictability can be found when the sentiment approaches the extremes. But there’s a difference in how the stock market performs after sentiment peaks and troughs.

In general, the findings suggest that there is something tangible in a concept which is prima facie of such an ethereal and emotional nature. It is quite another matter of how to best take advantage of our ability to track the investor sentiment, and in which instruments and asset classes can the best opportunities be found.

Chart 5. The correlation between the S&P 500 return and changes in the social media sentiment as a function of the time window (the number of days). Sources: Quandl, PsychSignal, own calculations.

MAR. 23 2015

Euro set to bounce as sentiment cycle turns positive.

Dominant Cycle Update from our partner Lars over at When to Trade.

https://www.whentotrade.com/euro-dominant-social-media-sentiment-cycle/

EURO SOCIAL MEDIA SENTIMENT SNAPSHOT AS OF MARCH 20TH

MAR. 23 2015

PsychSignal Gives Hedgies, Traders Insight Into Real-Time Market Sentiment

MAR. 20 2015

Come join us at FinovateSpring 2015

We’re excited to announce that PsychSignal has been selected to demo at FinovateSpring May 12-13. 

About FinovateSpring: FinovateSpring is a demo-based conference for innovative startups and established companies in the fields of banking and financial technology. Held in San Jose, the event offers an insight-packed glimpse into the future of money via a fast-paced, intimate, and unique format. FinovateSpring is organized by The Finovate Group. For more information on the event or to view videos of previous demos, please visit finovate.com.

MAR. 18 2015

PsychSignal’s Dr. Erin Olivo on the Dr. Oz show!

The Dr. Oz Show

How to Stop Being a Control Freak

Originally aired on 3/16/2015

Dr. Erin Olivo explains why controlling everything can actually cause stress for control freaks, not relieve it, in this Truth Tube segment from The Dr. Oz Show.

View Video (Part 1)

View Video (Part 2)

The Plan to Stop Being a Control Freak

Dr. Olivo provides a five step plan on how to keep your controlling behavior from controlling your life.

 

http://www.doctoroz.com/article/plan-stop-being-control-freak

MAR. 18 2015

Artificial Neural Networks and Forecasting: A Model of $SPY

Many of our prior blogs have focused on sentiments’ abilities to predict market movements for the next few minutes. We now consider predictions using daily data. And rather than employ yet more fancy forms of regression for data modelling, we employ a simple artificial neural network (“ANN”).

ANNs will do everything a traditional statistical solution will do, and, oftentimes, better. These days there’s much handy ANN code available, and ANNs are embedded in several trading platforms, stats packages and various online tools. It’s easier than ever to deploy these tools. It will be simple for you to replicate our model below.

Why ANNs work well in financial market forecasting

Regression models work well with stable relationships. And sometimes we can guess the proven causal relationships (see prior blogs) between sentiments and market prices or volatility will, indeed, be stable. But experience suggests markets will test - and probably break – any inflexible predictive model you’ve ever had, be it purely mental, used in trading “by hand”, or fully automated. It’s more reasonable to think relationships between causal variables and market outcomes are, at best, semi-flexible; and oftentimes very non-linear and chaotic. They are prone to shift, morph, and perhaps to disappear, reappear and, occasionally, to fundamentally change.

So the most apt market model we can hope for would be flexible; able to learn and adapt while preserving the best of established knowledge of what works. Arguably, ANNs are the closest we currently have to this ideal. But bear in mind that even the most suitable ANN will struggle to adapt to rapid and fundamental shifts in market behaviour.

Recommendations on types of ANNs

The most common ANN for financial market forecasting is probably a multilayer feedforward network, trained with backpropagation. Backpropagation is the process, used during network training, of feeding errors back through the system from the output layer towards the input layer. Training continues until the errors in the weights are acceptably small.

Other powerful ANNs might involve genetic algorithms, recurrent and modular networks. And there are many ways to define what is an “acceptable error” in the output, how many layers to have in the network, how many nodes in any given layer, the threshold function in each neuron, how weights get updated with learning,…Put alternatively, using ANNs well is, arguably, as much craft and experience as it is hard science and analytics.

How we tested the data

It’s the usual story; take a bunch of data, train the model on the first 90% or so, and test the model on the most recent 10% of data. To train the ANN, we took 29th Sept 2009 to 29th August 2014 of daily $SPY prices, matched to daily bull and bear sentiments. We then predict prices with the trained ANN model for 1stSept 2014 to 6th March 2015.

In other words, we are predicting prices – in this case, daily prices – with just sentiment data and price dynamics. We added potentially-helpful price action metrics, such as 5 and 20 day moving averages, price change over the last two days, and the price level from the day before. We also included the count of relevant tweets and ratios of bull and bear metrics. Of the five 5 options on the PsychSignal API, we selected just StockTwits as the sentiment-source.

We then test our model by measuring what percentage of all our next-day forecasts were correct in the pure binary sense of getting it right about either closing higher or lower than the prior day’s trading. We’d hope for at least 60% being correct.

A typical feed forward neural network is employed, with one hidden layer with 8 nodes, 11 inputs and just 1 output for forecast daily closing price. Training cycles totalled 450k which took close to 1 hour of processing. You can find many such tools on the web, e.g.http://www.visualgenedeveloper.net/Func_ANN.html

 

The findings

We got the direction of daily closing prices correct on 68% of the 134 trading days predicted. The predictions were correct up to seventeen days in a row, while incorrect forecasts appeared on just one solitary day, or two consecutive days, and – on just one occasion – three days in a row.

The regression graph after just 10k training cycles is shown below (R-squared is 94% here). The shape of the regression suggests how the ANN is slightly under-shooting both the highest market levels and lower market levels.

The ANN diagram is below. The four input variables with the highest (negative) correlation with output are numbers 1, 2, 8, and 9. These are, respectively: the bull metric; the bear metric; the prior day’s closing price; and the prior 5 days’ moving average price.

In sum, we regard this ANN prediction exercise as a measured success. Getting close to 70% correct for directional daily forecasts is notable. However, the $SPY price graph has a 5 year history of long and steady upward climbing. Even though many of our successful predictions are “down” rather than “up”, the long-term trend makes forecasting suspiciously easy and hence prediction success is not entirely trustworthy. We will soon replicate the basic method, used above, with more long-term volatile assets.

MAR. 17 2015

SELL $AAPL in March

Our work with sentiment cycles, spearheaded by Lars von Thienen and his “Dominant Dynamic Cycles” research, has produced a second prediction.

This time the prediction is to SELL APPLE as we are nearing a top in the most recent sentiment cycle. The model had a 76.92% accuracy when backtested to April 2013 with 10 winning trades and 3 losing trades.

When should you sell?

Good question! We asked Lars to elaborate…

“You need to trade PRICE not forecast. Therefore the entry has to be fine-tuned. An expected sell date is when the cycle reaches its high plateau area – which has begun at the start of march. So we are in a valid sell period for the month of march – the exact “theoretical” cycle is expected end of march. The cycle never gives the exact turns on the day. So the reading is “one would expect APPL to top around mid-march +/- depending on the technical overall market condition during that time.” said Lars von Thienen.

 

see the below slides expanded here:

 https://www.whentotrade.com/wordpress/wp-content/uploads/2015/03/One_more_thing.pdf

 

MAR. 10 2015

Is Twitter Really Bad for Economic Growth?

We do love the Bank of England for its thorough-going conservatism coupled with deep thinking. The latest salvo of cerebral ordinance labels “short attention spans”  (as per the typical Twitter user, apparently) as a threat to the very foundation of economic success i.e. sustained attention and effort.

The BoE report is discussed here:

http://www.theguardian.com/business/economics-blog/2015/feb/18/is-twitter-bad-for-economic-growth-bank-of-england

The argument depends on mounting evidence of ever-shorter cycles of investment and managerial stewardship; factors on which the entire economy depends. Moreover, this is coupled with strong claims that our brains are increasingly wired for rapid-fire cognition. That would be okay were it not for the fact that extended and reflective thinking is also tied to such cognitive wonders as creativity, innovation, good strategy and wise policy-making. Hence the “Twitter Generation” can be blamed for our many societal and economic foibles.

Here’s our rebuttal, and it comes in three rapid-fire and short-attention-span-suiting points. First, we think Twitter captures rather than creates modes of thinking. Put another way, people were always Tweeting in their heads. The genius of the app itself is to explicate and externalize a long-favored mode of thinking. Twitter doesn’t cause short attention spans any more than drinking pilsner causes a love of Czech culture and history. Second, we can easily point to the growth of very long-term projects in bio-tech, AI, infrastructure development, aviation,….While short-term activities, such as high frequency trading and tweeting, might have grown rapidly, so long-term activities and investments burgeon too. At the level of the individual, post-graduate and doctoral education is a case in point. People now worry we create too many PhDs! Third, we think the BoE makes a common and basic mistake. The argument depends on the future looking like an extrapolation of the past. But will it?

The invariant in the future of markets is, we think, sentiment. That’s why PsychSignal invests heavily (and long term) in understanding how emotion links to patterns of buying and selling and hence to market prices. And that would be our surest long-term prediction; whether attention spans are long or short, diverse and aggregated sentiment will continue to move market prices in predictable ways.

FEB. 27 2015

Trading Social Media Sentiment Cycles by Lars von Thienen [pdf]

FEB. 24 2015

April 2015

The PsychSignal API will be featured at the next FinTech Hack-a-thon in New York City April 18-19, 2015 

We’re looking forward to all the creative ways developers and quants will use our data during the 2 day event.

FEB. 17 2015

FEB. 13 2015

Can Market Forecasts Using Sentiments Really Reduce Forecast Errors?

Recent blog posts make the point that sentiments provably cause market outcomes. Sentiments can predict price direction (or “returns”), price volatility, and trading volume. But do forecasts based on sentiment metrics really reduce the errors in predicting these market behaviours? Below, we test our model for Goldman Sachs stock. So, read on for the answer…

Methods

We conduct a one-step ahead prediction over a three months period based on a baseline model, denoted M0, and an augmented model, M1. Both models are represented as follows:

where Y represents the particular financial indicator (return, trading volume, volatility) and X is the sentiment indicator. We estimate M1 for both the Bullishness index and the Agreement index, respectively (recall that these two measures are easy-to-use ratios from prior blog posts; the first being high if the ratio of bullishness to bearishness is high, and the second being high if the metrics for bullishness and bearishness do not differ substantially).

Note the simple intuition here: a good financial market prediction would be measurably better than just predicting price as the last known price. That very simple model is known as the “base” or “naïve” model. Hence the statistics we explain above are really just a way of formalizing that insight; we want a model using either the Bullishness or Agreement metric to perform better than any base model. We “train” the sentiment models on 21 months of data, and then see how good they really are by letting them loose on the last 3 months. We compare their errors with those of the base model on the most recent three months of market data.

To analyse the forecasting accuracy we calculate both the Mean Absolute Percentage Error (MAPE) and also the Root Mean Squared Error (RMSE). If you like, you can check online for how these two standard measures are calculated e.g. http://en.wikipedia.org/wiki/Mean_squared_error

The data we test goes from the 17th February 2012 to 17th October 2014; and we employ the last three months, i.e. 17 July 2014 to 17 October 2014, as the forecast period. In the regressions, the lag is chosen to be 20, equivalent to 40 minutes, and in all models we include controls for hour, day and month fixed effects. The only exception is for returns, for which a lag 10 is selected instead, due to being a better fit to the model. The table shows the forecasting errors expressed as MAPE and RMSE measures.

Table. MAPE and RMSE Scores for Model Predictions for Goldman Sachs Stock

***estimation from 17/10/2012 to 16/07/2014

***forecasting from 17/07/2014 to 17/10/2014

Underlined data indicates that the sentiment-based model produces smaller errors than the base model (“Model O”).

Discussion

The data show modest reductions in errors upon the introduction of sentiment metrics to the forecasting model for market behaviour. Reductions are evident in 8 out of 16 tests. The Agreement metric performs slightly better than Bullishness, offering more cases of reduced forecast error for the three market behaviors studied.

In summary, sentiments help predict market outcomes. Note that our model is “pure sentiment”, in that we add no other variable-types, such as market trends, momentum, etc. We anticipate that forecast errors be further reduced with more factors considered. Moreover, given the value in predicting price direction, future blog posts will address the ability of sentiment models to predict binary (up or down) price action over some limited time window.

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at [email protected]

FEB. 4 2015

Humans Risk Extinction

In trading and fund management, at least, humans risk dying out. A recent article in the Financial Times makes the simple point that people can’t track the volume and speed of data required to maintain a trading edge. And with more and more evidence that tracker funds outperform (net of fees) nearly all hedge funds in the long term, maintaining that edge is more challenging than ever. Full automation is the way ahead.

The story is here:

http://www.ft.com/intl/cms/s/0/17129fc0-a48c-11e4-8959-00144feab7de.html#axzz3Qi7U00sl

And here is a critical part of the argument; “While human fund managers had the ability to interpret market colour and psychology in a way computer-driven systems could not…the human mind would never be able to keep track of the quantity of information that systematic funds can.

For us, the FT does not go far enough. Perhaps for the first time in history, we are now wholly confident in saying that automated approaches are better than human fund managers in interpreting market colour and psychology. If not, then our analyses (many of which are shown in this blog) demonstrating the predictive and profit-making ability of sentiment-based trading algorithms could not exist.

In other words, computational approaches can now offer material advantages in both the hard area of data processing – high volume, high speed, and highly accurate – and in the soft area of interpreting market mood.

Full automation with sentiment metrics is the way ahead.

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at [email protected]

 

Bio of Dr. Matthew S. Checkley

Matthew holds a PhD in Network Science from Warwick University in the UK. He has worked in academia, industry analysis, and equity research. His published research has examined problems of risk and firm performance in evolving financial networks, alongside studies of cognition, strategic management, and sentiment-based trading.

FEB. 3 2015

A Simple Application and Trading Test of PsychSignal’s Directional Alerts

In this post, we go directly to the trading realities of buying and selling via PsychSignal’s sentiment alerts. We test 28 alerts from January 22nd 2015, on just three well-tweeted stocks. This is just enough to give some statistical weight to the arguments and to further suggest some refinements to the alerts with which you might like to experiment.

For simplicity’s sake, we take only directional alerts (not volatility) and only consider sentiment that is above 60% Bullish or above 60% Bearish. The 60% threshold is chosen because there can be a huge number of 50% (or less) sentiments in a trading day. The 60% threshold gets the number of trades down to a handful per day.

Following prior blog posts, we’ll hold the suggested position (going long after the Bullish signal, short following the Bearish signal) for just 10 minutes. When similar or even contrary signals appear within 10 minutes of each other, we just assume we’d trade on all relevant information. Then we’ll compare the performance achieved with the returns on the underlying assets.

The table below presents the data; time of alert, action taken, and returns acquired after a 10 minute holding period. The data are displayed most-recent first. For this quick demonstration model, we use only mid-prices, ignore transaction costs, slippage and allow fractional numbers of shares.

 

 

 

It’s vital background that 22nd January was a bullish day; Amazon, GS and IBM returned 4.40%, 2.76% and 2.17% respectively. So you’d expect that bullish signals would, on this particular day, yield good results. However, five out of six signals for IBM were bearish, and all but one was profitable when shorted. Even these super-simple alerts based on the 60% sentiment threshold appear to have an uncanny ability to suggest intraday turning points in the price direction.

Overall 13 out of 18 signals for Amazon were successful, 3 out of 4 for GS, and five out of six for IBM. In sum, 21 out of 28 signals (i.e. 75%) were as-predicted. 

To refine this approach we’d suggest following our prior research on sentiments; what might offer even more potential is to consider not just some threshold of strong sentiment in one direction, but rather a high ratio of bull to bear, or the inverse. Our past research suggests the ratio of sentiments is a particularly good predictor of directional price movement.

 

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at [email protected]

 

Bio of Dr. Matthew S. Checkley

Matthew holds a PhD in Network Science from Warwick University in the UK. He has worked in academia, industry analysis, and equity research. His published research has examined problems of risk and firm performance in evolving financial networks, alongside studies of cognition, strategic management, and sentiment-based trading.

JAN. 23 2015

Silver Market Snapshot – Sentiment Cycles Update

On Nov. 15th Lars from “When to Trade"posted a video about a possible low in silver based on the cycleresearch using PsychSignal data.

8 weeks after the call, we have a great result. The public and verifiable forecast was sharp and on point!

See the chart below and read more over on Lar’s blog https://www.whentotrade.com/silver-sentiment-market-cycles/

 

 

 

JAN. 18 2015

Current Dominant Cycle in the Silver Market. Using PsychSignal sentiment data and WhenToTrade toolset to predict and forecast market.

JAN. 15 2015

How much market direction is predictable with PsychSignal’s sentiment data? A study of short-term returns forecasting.

Prior blog posts have discussed how PsychSignal’s metrics predict market volatility. In this post, we address some of our recent research into price direction. We test Amazon share price data below and find evidence for price direction, or “returns”, being predictable when using the PsychSignal sentiments.

Data and Methods

(This section is somewhat technical, similar to recent blog posts, and can be skipped with no loss of understanding of the research outcomes! Jump to “Test Results”.)

We matched PsychSignal’s every-two-minutes sentiments (both bull and bear metrics) to two years (until mid-October 2014) of historical share prices for AMZN.

Two derived measures of sentiment are used, following Antweiler and Frank (2004). The first defines an index of bullishness (Bt) for each time window as the log of the ratio of 1+bull at t, and 1+bear at t.

Here, bull represents an index of positive sentiment tweeted within a particular 2-minute period t for a specific stock, while bear represents an index of negative sentiment in the same period for the same stock (AMZN, in this case).

The second measure, also consistent with Antweiler and Frank (2004), is the index of agreement (At) between positive and negative sentiments. It is given by 1 less the square root of (1 less ((bull-bear)/(bull+bear))^2)

If all tweet messages about a particular company are all either bullish or bearish (but not both), agreement would, in that case, be 1 at time t. If sentiment is equally bullish and bearish, then agreement would be 0.

To calculate stock price returns we took the differences of the logs of subsequent periods’ prices.

Granger Causality Analysis

To determine what relationships might exist between stock returns and tweet sentiment features (Bullishness and Agreement), we use Granger Causality Analysis (Granger, 1969). A variable X is said to “Granger-cause Y” if Y can be better predicted using the histories of both X and Y than by using the history of Y alone. Hence, if when controlling for the information contained in past values of Y, past values of X add significantly to the explanation of current Y, then X is said to “Granger-cause” Y. We give technical details for these tests in prior posts, so we won’t rehearse them here.

We have two relationships to test:

(1) Bullishness Granger causes returns

(2) Agreement Granger causes returns

The “direct Granger method” is used to test for Granger causality between sentiments and stock price behaviour. An advantage of this single-stage method is that the estimates of the Autogression Distributed Lag (“ADL”) model can remain unbiased in the presence of autocorrelated time series data. And given our prior blogged research focusing on causal effects over minutes (rather than hours), we test 10 lags, which corresponds to searching for causal effects over 2 to 20 minute time windows.

Test Results

The table below provides statistical output for coefficients in the regression and for the Granger Causality test.

N.B. Each 2 minute lag is denoted “L” eg “L4.bind” denotes the Bullishness index 8 minutes prior to a returns metric. The table below shows coefficient values in the regression, along with p values and, most importantly, the outcomes of the Granger causation test, at the bottom of the table. Ten lags are employed and “FE” refers to making adjustments for Fixed Effects over longer periods.

 We cannot reject that the Bullishness Index causes Returns (at 10%, because p=6.29% in Model 1).

The Agreement Index is not causally significant in this model, i.e. in Model 2.

We therefore find causal ties from the Bullishness metric to changes in the Amazon share price. The most significant coefficient in the regression is lagged by 4 minutes and is positive i.e. we can generally expect share price increases 4 minutes after an increase in the Bullishness sentiment metric. 

So, our expectation that Bullishness would predict share price changes is supported. There is evidence that sentiments predict returns, over a time window of a few minutes. Of course, do note we have only discussed Amazon here.

Trading opportunities

Our findings are consistent with opportunities in directional trading over cycles of a few minutes. A significant increase in the Bull metric (as we have calculated it, or in other ways that indicate a high ratio of bulls to bears) should lead to price increases. 

We have forthcoming reports building on the tests above, albeit with more stocks tested and with the addition of forecasting accuracy tests of this model versus the naïve model of forecasting returns, volatility or volume as being identical to that of the last period’s data. That analysis will be published later this year. 

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at [email protected]

 

Bio of Dr. Matthew S. Checkley

Matthew holds a PhD in Network Science from Warwick University in the UK. He has worked in academia, industry analysis, and equity research. His published research has examined problems of risk and firm performance in evolving financial networks, alongside studies of cognition, strategic management, and sentiment-based trading.

JAN. 11 2015

Bank of England Announces Use of Social Media Metrics in Market Prediction

The use of social media sentiment metrics is growing rapidly. Hence, we present a brief commentary on some notable, recent news stories. What impressed us is how some of the world’s more conservative and influential organizations within Governments are embracing sentiments to help predict economic and financial change. 

The story is here:

http://news.sky.com/story/1397985/bank-of-england-to-monitor-social-networks

Here’s what Andy Haldane, Chief Economist at the Bank of England, had to offer:

“Official statistics tend to be lagging and tend to be revised. And what this scraping of the web can do is give us a better today read on what’s going on,” he said.

He added that these and other “informal sources” of data “have been somewhat more reliable in picking up the uptick in the fortunes of the economy”.

The BoE is focusing on two of the obvious benefits of sentiments: reliability and timeliness. However, reading on there’s a hint they are looking too for more insight to some of the less obvious interactions amongst economic variables. Interest rates, house prices, job vacancies, financial markets,…., all these factors, and others, interact in dynamic and significant ways. In other words, sentiment metric time series provide predictive insight to both changing economic variables and to swiftly changing co-dependencies between many of those variables:

“For instance, analysis of the frequency of internet job searches, or of prices online, can give one an insight into the prospects for unemployment and inflation.”

This announcement from the BoE follows similar statements from the US Federal Reserve and Treasury in recent times. 

Our view: financial and economic policy depends heavily on forecasting. And methods for predicting financial systems are converging with the sophisticated analytical tools employed in weather forecasting. These are both chaotic systems with many sensitive dependencies. A number of seemingly benign conditions can converge to produce a perfect storm. There’s high value in getting warning on any big or rapid change. Some of the best foresight will be offered by diverse, numerous and frequent extraction of data on the environment. Social media sentiment metrics are one of the best routes to this goal. 

While many private investment companies had this insight a few years ago, it has understandably taken the more conservative public sector a little longer. Without question, the predictive value of social media sentiment metrics is being increasingly recognized and deployed. 

 

 

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at [email protected]

 

Bio of Dr. Matthew S. Checkley

Matthew holds a PhD in Network Science from Warwick University in the UK. He has worked in academia, industry analysis, and equity research. His published research has examined problems of risk and firm performance in evolving financial networks, alongside studies of cognition, strategic management, and sentiment-based trading.

JAN. 8 2015

Building a Simple Volatility Alert: A Study of Apple Sentiments and Price Variance

Here we explore a quick-and-simple approach to building an initial trading alert for volatility. Once the thing is built, refining and adapting it is easy. But, for now, we show how even something super-simple – if built on a solid conceptual foundation – can deliver handy results.

 

The Idea

We recently published a blog post showing how the Apple price becomes more volatile a few minutes following strong bull and bear signals appearing at the same time. This finding is not too surprising given that it has appeared several times in the academic research literature over the last few years. It also has intuitive appeal; if sentiment metrics are both strongly bullish and bearish, this might predict increasing volumes of both buying and selling activity. That would cause a short-term increase in volatility.

 

Data and Methods

We took two weeks of recent Apple price data, sampled every 2 minutes, and matched that to sentiments of the same granularity. Reviewing the frequency of strong and concurrent bull and bear sentiments, it is clear that both metrics being greater than 2.0 is a moderately rare event (recall that the PsychSignal sentiment metrics can range from 0 to 3.0). That particular combination of strong and contrary emotion appears, at most, a few times a day. Hence our initial model uses this threshold. So, our predictive signal for increasing volatility is simply defined and easily identified in the sentiment data.

The test of the quality of the alert is also simple; compare the volatility level in the 10 minutes following the alert (i.e. start the volatility measure 2 minutes following the alert and terminate it 12 minutes following the alert) and compare that with similarly-measured volatility when there is no such alert. Volatility is taken as the standard deviation of returns (ie returns are the percentage change from one period to next one; two minutes later) over that rolling 10 minute window. “Standard volatility” is just the standard deviation of returns over the entire data set. We hope for a ratio of:

 

(Volatility after the alert)/(standard volatility)>1.4

 

In other words, we are looking for post-alert volatility to be 40% higher, or more, than normal, everyday “standard” volatility. Again, you might want a higher or lower marker for “quality” in your signal for volatility. That’s fine, but changing thresholds in the model is easy and we feel this is enough to be of interest in trading.

You might also object that this test could miss some interesting data i.e. there could be high realized volatility that is not preceded by an alert. A good objection! Nevertheless, in the interests of a simple and direct analysis, we test the quality of the alert in terms of its ability to consistently predict subsequent volatility. We leave the more stringent test of what might predict all interesting price variances for a future blog. A final point is that sometimes the volatility alert will be triggered off market hours. There is an opportunity to consider this as a signal of volatility in the following market opening. But again, in the interests of simplicity here, we analyse only sentiment signals during market hours. 

 

What we found

The mean of post-alert volatility is calculated as 0.18%. The standard volatility is 0.09%. In other words, the post-alert standard deviation of returns is double the overall standard deviation. Of the 36 alerts tested over the two week data period, one third predicted significantly increased volatility, while two thirds of alerts where followed by fairly typical price behaviour. Hence, the alerts work extremely well overall - easily exceeding our 40% marker for signal quality - while one third accurately predict volatility within the following ten minute period. 

 

 

Developing your own alerts

There are many simple ways to further explore volatility alerts and to develop your own particular formula. Here are a couple of suggestions. First, you can increase the threshold e.g. instead of alerting when bull and bear metric both exceed 2.0, you could test 2.4 or more. Or you might consider having a higher threshold for the bull metric because prior research suggests very strong bulls predict sharp market declines. But do remember the 2.0 threshold sounded just 36 times in 2 weeks, so increasing it will diminish your frequency of alerts.

Similarly, it is easy to implement a “noise out of silence” criterion. Here you consider that strong sentiments emerging after periods of quiet are potentially more significant. You might play around with keeping the 2.0 threshold value, but only after, say, 20 minutes or more of both bull and bear metrics being less than 0.6. Again, these models are easy to build and easy to test.

 

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at [email protected]

 

Bio of Dr. Matthew S. Checkley

Matthew holds a PhD in Network Science from Warwick University in the UK. He has worked in academia, industry analysis, and equity research. His published research has examined problems of risk and firm performance in evolving financial networks, alongside studies of cognition, strategic management, and sentiment-based trading.

DEC. 23 2014

How much market movement is predictable with PsychSignal’s sentiment data? A study of short-term volatility forecasting.

We know clients commonly blend PsychSignal’s sentiment metrics with other signals to make directional predictions for market prices. But what about taking just the pure sentiment metrics themselves as the signal? And what about forecasting volatility instead of price direction or trend?

In order to explore alternative applications of the sentiment data, we’ll discuss some of our recent research: a case study of short-term volatility prediction. Specifically, we test the idea that relatively strong and divergent sentiments (i.e. the simultaneous appearance of larger-than-usual bull and bear sentiments, regarding an asset) can be employed as a signal of increased volatility in the asset price. 

There’s intuitive appeal to the idea that strong and discordant sentiments lead to bigger price fluctuations. Insofar as sentiments predict buying behaviour, we’d expect the co-occurrence of both aggressive bulls and bears to lead to rapid and relatively high-volume buying and selling. And that should lead to de-stabilized market prices. So, the hypothesis is clear; strong and contrary sentiments lead to robust forecasts of price volatility. Now, we move onto the data and methods.

Data and Methods

(This section is somewhat technical and can be skipped with no loss of understanding of the research outcomes! Jump to “Visualizing the Data”.)

We matched PsychSignal’s every-two-minutes sentiments (both bull and bear metrics) to two years (until mid-October 2014) of historical share prices for Apple Inc. These two data sets present some typical complications e.g. there are many tweets when the market is closed, and, during a few minutes within market hours, there might be no tweets to report. For that reason we took market prices every 2 minutes and matched those to the timestamp of relevant tweets during market hours. In other words, we matched as many time-stamped sentiments as possible to the every-two-minute database of Apple stock prices. After-hours sentiments were aggregated to help predict the coming market opening price. There can be many thousands of Apple stock-relevant tweets – and related moods - in a trading day, so the data is not thin and represents a good test bed. 

Two derived measures of sentiment are used, following Antweiler and Frank (2004). The first defines an index of bullishness (Bt) for each time window as the log of the ratio of bull to bear. Here, bull represents an index of positive sentiment tweeted within a particular 2-minute period t for a specific stock, while bear represents an index of negative sentiment in the same period for the same stock. 

The second measure, also consistent with Antweiler and Frank (2004), is the index of agreement (At) between positive and negative sentiments. It is given by a slightly elaborate ratio of bull and bear signals such that, if all tweet messages about a particular company are all either bullish or bearish (but not both), agreement would, in that case, be 1 at time t. If sentiment is equally bullish and bearish, then agreement would be 0.

The volatility of stock returns at t minutes frequency is obtained as a moving average filter of the stock variance at a time window of t minutes. Out of interest, we also added traded volume of shares to the model. Volume is a simple metric; the number of shares traded in a period and no data conversion is used.

Granger Causality Analysis 

To determine what relationships might exist between stock outcomes (volatility and traded volume) and tweet sentiment features (bullishness and agreement), we use Granger Causality Analysis (Granger, 1969). A variable X is said to “Granger-cause Y” if Y can be better predicted using the histories of both X and Y than by using the history of Y alone. Hence, if when controlling for the information contained in past values of Y, past values of X add significantly to the explanation of current Y, then X is said to “Granger-cause” Y. However, if Granger causality holds this does not guarantee that X causes Y in the commonplace sense of observed physical action. Thus we use Granger causality analysis not to establish causality per se, but rather as a tool to investigate the hypothesized statistical pattern of lagged correlation.

Formally, the possible Granger causal links between stock outcomes (variable “S”, defined in two different ways, as volatility and volume), and sentiments (variable “T”) can be expressed using the parameters of the usual equation (see, for example, the Wikipedia entry on this topic).

 

Therefore, there is Granger causality from T to S if the lagged values of T have a statistically significant correlation with S:

 

We have two relationships to test:

 (1) Bullishness Granger causes volatility

(2) Agreement Granger causes volatility

Because it has an established link to Volatility, we included Traded Volume as another outcome of market behaviour. However, we do not focus on Volume in this paper. 

The “direct Granger method” is used to test for Granger causality between sentiments and stock behaviour. An advantage of this single-stage method is that the estimates of the Autogression Distributed Lag (“ADL”) model can remain unbiased in the presence of autocorrelated time series data. In so far as the number of lags used in the model is enough to account for time series autocorrelation, no pre-whitening is required (Freeman, 1983). However, insufficient lags can yield autocorrelated errors (and therefore misleading test statistics); while too many lags reduces the power of the test.

Visualizing the Data

The chart below shows how the sum of bull and bear sentiments (in blue) varies with the (re-scaled) price change in Apple stock (in yellow). Each metric is calculated every 2 minutes. This randomly-selected time period – of over 90 minutes of data - illustrates the intuition we test with the subsequent statistical model; when the blue graph peaks, it tends to predict a relatively large peak or trough in the yellow graphic. The time lag from sentiment peak to the subsequent stronger price changes is typically 6-12 minutes. Clearly this is not, in itself, a wholly convincing analysis (although we could add more “causal” arrows to even this simple graph), but it does graphically suggest a consistent pattern in the data…

 

 

 

In the statistical tests (the results of which are shown below) the Apple share price variance is calculated over several time windows, ranging from 6 minutes to 10 minutes, and hourly to over several hours. The causal time lags tested are set at various levels; from 2 minutes to over 10 minutes, and over several hours and days (not all test results are shown here because there are many pages of tables…!). 

What we found

We find causal ties from both Agreement and Bullishness to increased volatility in the Apple share price. Time windows of 2 to 10 minutes offer robust evidence of causality. Over this time window, strong and discordant sentiments predict around 25% of the variance in the volatility of the Apple share price. 

So, our hypothesis is supported. There is convincing evidence that sentiments predict price volatility (and, by the way, increased trading volume also), in Apple stock, over a time window of a few minutes. Of course, we have only discussed Apple here. But forthcoming reports will address other stocks and other kinds of prediction. However, our initial work suggests that this effect is general i.e. for well-tweeted stocks such as Goldman Sachs, Google, Amazon, IBM, etc., short-term increases in volatility can be predicted to appear a few minutes following strong and discordant sentiment metrics. 

A Sample of Statistical Output for Tests of the Bullishness Index (“bind”) and Agreement Index (“agr”) Granger-causing Realized Volatility.

N.B. Each 2 minute lag is denoted “L” eg “L4.bind” denotes the Bullishness index 8 minutes prior to a volatility metric. The table below shows coefficient values in the regression, along with p values and, most importantly, the outcomes of the Granger causation test, at the bottom of the table.

 

 

*** denotes p<0.1%.

We cannot reject that the Bullishness Index causes realized Volatility (at 10%).

We cannot reject that the Agreement Index causes realized Volatility (at any level).

Hence, in these tests, the data are consistent with volatility being predicted, within 10 minutes, by both the Bullishness Index and (more significantly) by the Agreement Index. Note that we constructed several models of both volatility and ways of building time lags and they all showed comparable results.

Trading opportunities

Our findings are consistent with opportunities in short-term non-directional trading. Suitable trades could include options strategies and the use of specific volatility markets. We’ll leave this issue for future research reports, but our findings also offer the prospect of the absence of strong and diverse sentiments being predictive of diminishing volatility. “Always-in” trading could take the appearance of a threshold signal for bullishness on volatility (e.g. bull and bear metrics both being higher than 2.0) as the time to buy, say, volatility indices. But the absence of the signal – and the expiry of short-term “long” positions - could be tied to shorting volatility. Again, our initial tests show this to be a rewarding approach.

We have forthcoming reports building on the tests above, albeit with more stocks tested and with the addition of forecasting accuracy tests of this model versus the naïve model of forecasting volatility or volume as being identical to that of the last period’s data. That analysis will be published within a few weeks. 

If you’d like to discuss this report or you have any other questions regarding the PsychSignal data and services, then please contact Dr. Matthew Checkley at [email protected] or James Crane-Baker at[email protected]

Bio of Dr. Matthew S. Checkley

Matthew holds a PhD in Network Science from Warwick University in the UK. He has worked in academia, industry analysis, and equity research. His published research has examined problems of risk and firm performance in evolving financial networks, alongside studies of cognition, strategic management, and sentiment-based trading.

DEC. 23 2014