Home Brokerage How External Data Can Help Hedge Funds Seize the Volatility Opportunity

How External Data Can Help Hedge Funds Seize the Volatility Opportunity

by internationalbanker

By Will Freiberg, CEO, Crux





From black swan events such as the COVID-19 pandemic to soaring inflation and a rising interest-rate regime, we’re witnessing a time of heightened volatility. For hedge funds, this volatility presents both a challenge and an opportunity to find more alpha (excess return).

Data from J.P. Morgan Asset Management1 indicates that hedge funds perform better during periods of higher volatility. Since 1990, hedge funds have experienced their strongest performances when implied volatility (VIX) was between 20 and 25—a level of volatility that is similar to today’s.1

Against this backdrop of market volatility, hedge funds are at a critical juncture. Now is the moment to generate risk-adjusted returns and prove the value of hedge funds in tricky market environments. The ability to find more alpha and rise to the top hinges on access to and integration of the breadth of data available today. By onboarding, transforming and applying the full gamut of traditional and alternative data, hedge funds can take advantage of today’s market environment.

Insights from expanded datasets

Extracting value from data is nothing new. Leading hedge funds and larger financial institutions are centralising data operations and creating centres of excellence to harness the value of data and deliver information to quantitative researchers (quants) efficiently so that they can create the complicated models needed to price and trade securities accurately.

Centralised data repositories enable hedge funds to manipulate internal data and onboard external data more efficiently. When a hedge fund can ingest more datasets, it is more likely to gain unique insights about a market and its drivers. To accommodate their insatiable need for information, hedge funds have stored data in public clouds and created data lakes, warehouses and marts to handle the vast information stores. In the quest for more insights, hedge funds are focusing their attention on external data.

Adding third-party data to the mix

To gain an edge over competitors, hedge funds need massive amounts of data from external sources. For example, hedge-fund quants with access to esoteric information about climate and how it might affect the coffee crop in Brazil are better informed when working on a trade involving future Starbucks earnings.

The challenge for hedge funds will be onboarding the massive amounts of data they need. Three distinct steps are involved in working with third-party data: ingestion, transformation and observability. To ingest data successfully, hedge funds work with sources such as Bloomberg, Morgan Stanley Capital International (MSCI), Morningstar and an array of other data providers.

Transforming that data into a usable state requires analysing the data schema and format and then integrating it into the organisation’s own data structure, whether that resides in a data mart such as Snowflake or a public cloud platform such as Amazon Web Services (AWS), Google or Microsoft Azure. Observability then comes into play because it’s critical for the organisation to understand data health comprehensively—all three steps are needed to use external data successfully

Resources needed to maximise external data value

There’s no ongoing debate about whether external data is a critical asset that hedge funds need to help find alpha. That’s a reality that virtually everyone accepts. The remaining questions are around how hedge funds can access the third-party data they need, which is increasingly voluminous and coming from a multitude of sources, and how they can generate value from the data as quickly as possible.

Today, many organisations are still handling the tasks involved in ingesting and transforming data in the traditional manner, which involves bringing many specialised IT (information technology) personnel in-house to manage data integration and transformation. Finding data engineers with the skills necessary to execute data-wrangling activities, such as analysing schema and standardising formats, is an ongoing challenge.

Maintenance difficulties are growing as hedge-fund information officers confront emerging expectations for data access. Some hedge funds and larger financial institutions are able to onboard thousands of datasets per year, which gives them an edge, while other organisations that still use the traditional approach must engage in weeks of tedious, time-consuming activities to bring a single dataset into the fold.

So, how do the nimbler hedge funds and financial institutions bring information on board so quickly? Many look to partnerships with organisations that specialise in third-party data, taking advantage of those vendors’ data catalogues and existing data pipelines to bring the needed information into their workspaces within hours rather than weeks.

Data as a service

It’s a familiar pattern as technology evolves: Organisations and institutions need to access a new technology or tech-enabled capability, so they acquire the hardware and bring experts on board to deploy and manage it in-house. But as technology evolves and/or use cases become more complex, they realise it makes more sense, from a practical point of view, to outsource the function to a vendor and concentrate on their core competencies.

That pattern is now playing out in the financial sector as organisations such as hedge funds and financial institutions confront the need to ingest, transform and apply data rapidly. This involves not only data wrangling to ensure information from external sources is relevant and in usable formats, but it must also be seamlessly blended with internal data.

Hedge funds and other data users in the financial sector also increasingly recognise the importance of observability, which is the ability to monitor the health of data over time. The more sources that organisations access for data, the more relevant observability is because data providers frequently change file formats, breaking integrations downstream and/or compromising the information in other ways.

Observability makes changes visible even when they happen behind the scenes. Whether accessing data using in-house capabilities or through vendor partnerships that handle data sourcing, integration and transformation as a managed service, hedge funds and financial institutions need to ensure they have ways to fully understand changes in data streams, including alert systems to trigger any necessary remedial actions.

Getting to alpha—and peace of mind

So, what does it look like when hedge funds and financial institutions put all the pieces in place to enable seamless data integration, transformation and observability? One of the primary benefits is greater efficiency, which is also behind the movement to centralise data. When hedge funds create a centralised data repository, they can onboard thousands of datasets per year instead of lesser amounts.

Additional information expands the universe of available insights, and greater efficiency also flows from devoting resources, either in-house or through vendor partnerships, to transforming data so that the team of data engineers in which the hedge fund invested to deliver information to quants can focus on that instead of data wrangling. Observability adds peace of mind since the hedge fund’s data users know they can rely on their information.

This could translate into significant revenue gains, and this opportunity comes at a time when hedge funds could benefit from exceptional market volatility. By using data pipelines to blend internal and external data in real-time, hedge funds get more information than their competitors, and they get it more quickly. In a recent engagement with an asset-management firm, the chief data officer stated that an increase in model accuracy from 1 percent to 3 percent would generate a significant boost to the top line. A 1-percent increase alone would bring an additional $80 million annually.

When billion-dollar portfolios are at stake, every advantage is magnified, and even small differentiators are counted in the millions. That’s why efficient and effective handling of external data can help hedge funds seize the volatility opportunity and increase alpha in the years ahead.



1 J.P. Morgan Asset Management: “Guide to the Markets,” Source: CBOE, HFRI, MSCI, ICE BofA, J.P. Morgan Index Research, FactSet, January 31, 2023.



Will Freiberg is the CEO of Crux. He is a technology executive, entrepreneur and investor in enterprise applications, data infrastructure and AI/ML spaces. Previously, Will worked in the Venture Capital and Technology Practice of Gunderson Dettmer and the M&A Group at Simpson Thacher & Bartlett LLP, starting his career as an Analyst at Bank of New York Mellon.


Related Articles

Leave a Comment

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.