Custom data sources#

This page introduces the concept of custom data sources and explores the ways you can define and use them in the SigTech platform. You will learn how to:

Construct your environment
Query available data sources
Set a data source
Compare data sources

Environment#

First, initialize your research environment:

import sigtech.framework as sig
sig.de_init()
sig.init()

When configuring your research environment, a range of parameters can be used to set and define its features and behavior.

The following command lists the first ten environment configuration parameters:

Input:

sorted(sig.env().config)[:10]

Output:

['ALLOWED_DAY_CA_DATA_GAP',
 'ALLOWED_MISSING_CA_DATA',
 'CASH_OVERRIDES',
 'CUSTOM_DATA_SOURCE_CONFIG',
 'DATAFRAME_SERVICE_ACTIVE',
 'DATAFRAME_SERVICE_FACTORY',
 'DATA_ADAPTERS',
 'DATA_SERVICE_DATA_FILE',
 'DATA_SERVICE_RECORD_TO_FILE',
 'DISABLE_T_COST_NETTING']

Learn more: For more detail on configuration parameters, see Environment setup.

Custom data sources can be set by assigning values to the CUSTOM_DATA_SOURCE_CONFIG parameter:

sig.config.CUSTOM_DATA_SOURCE_CONFIG

No custom data sources are set by default. This can be confirmed by running the following command:

sig.env()[sig.config.CUSTOM_DATA_SOURCE_CONFIG]

View available sources#

While custom data sources can be specified for any type of tradable instrument, the following example uses foreign exchange spot (FX spot) data to demonstrate this functionality.

Use the command below to retrieve an FX pair:

fx = sig.obj.get('EURUSD CURNCY')

To determine the default data source for this instrument:

Input:

fx_data_source

Output:

'ICE_SUPER_DERIVATIVES'

To list the data sources available for an instrument, use data_source_all. The results generated by this command will depend on your current data entitlements. To access additional data, please contact our sales team at sales@sigtech.com.

Input:

fx.data_source_all

Output:

['ICE_SUPER_DERIVATIVES', 'CLS', 'QUANDL.FXCM', 'REFINITIV', 'CBOE']

Change data source#

Before resetting the data source, you will need to de-initialize and re-initialize the environment:

sig.de_init()
sig.init()

The following code block selects objects whose internal id is of the form '[A-Z]{6} CURNCY' and changes their data source to 'REFINITIV':

sig.env()[sig.config.CUSTOM_DATA_SOURCE_CONFIG] = [
        ('[A-Z]{6} CURNCY', 'REFINITIV'),
    ]

To confirm this change, retrieve the FX pair and query its data source:

Input:

fx_refinitiv = sig.obj.get('EURUSD CURNCY')
fx_refinitiv.data_source

Output:

'REFINITIV'

To display the FX fix group which the object belongs to, use the following command:

Input:

fx.refinitiv.group()

Output:

REFINITIV FXFIX GROUP <class 'sigtech.framework.instruments.fixes.FXFixGroup'>[139644524666064]

To tabulate the first five available data points within the instrument’s history, use both the history() and head() functions:

Input:

fx_refinitiv.history().head()

Output:

trading_datetime
2000-01-31 01:00:00+00:00    0.97970
2000-01-31 02:00:00+00:00    0.97875
2000-01-31 03:00:00+00:00    0.97955
2000-01-31 04:00:00+00:00    0.98080
2000-01-31 05:00:00+00:00    0.98050
Name: (LastPrice, EURUSD CURNCY), dtype: float64

FXFixGroup provides consolidated data regarding FX fixes. Run the following command to list the available currencies from different data sources:

Input:

sig.FXFixGroup.get_names()

Output:

['CBOE FXFIX GROUP',
 'CLS FXFIX GROUP',
 'ICE TERMSTRUCTURE FXFIX GROUP',
 'QUANDL FXFIX GROUP',
 'REFINITIV FXFIX GROUP']

To view currencies available from Refinitiv but unavailable from ICE:

Input:

ice_ccys = sig.obj.get('ICE TERMSTRUCTURE FXFIX GROUP').query_instrument_names()
refinitiv_ccys = sig.obj.get('REFINITIV FXFIX GROUP').query_instrument_names()
additional_ccys = [ccy for ccy in refinitiv_ccys if ccy not in ice_ccys]
additional_ccys

Output:

['AUDJPY CURNCY',
 'AUDNZD CURNCY',
 'CADJPY CURNCY',
 'EURAUD CURNCY',
 'EURCAD CURNCY',
 'EURCHF CURNCY',
 'EURDKK CURNCY',
 'EURGBP CURNCY',
 'EURHUF CURNCY',
 'EURJPY CURNCY',
 'EURNOK CURNCY',
 'EURRUB CURNCY',
 'EURSEK CURNCY',
 'GBPAUD CURNCY',
 'GBPCAD CURNCY',
 'GBPCHF CURNCY',
 'GBPJPY CURNCY']

Confirm that AUDJPY is unavailable from ICE by querying the data sources associated with the currency pair:

Input:

sig.obj.get('AUDJPY CURNCY').data_source_all

Output:

['CLS', 'QUANDL.FXCM', 'REFINITIV', 'CBOE']

Run the following command to revert to ICE and render this currency pair unavailable:

sig.de_init()
sig.init()

Without a set source, all available sources for an instance are cycled through. The history series output will depend on your data entitlements:

Input:

fx = sig.obj.get('EURUSD CURNCY')
fx.history().head()

Output:

trading_datetime
2015-08-31 21:05:00+00:00    86.2320
2015-08-31 21:20:00+00:00    86.2170
2015-08-31 21:30:00+00:00    86.2505
2015-08-31 21:35:00+00:00    86.2885
2015-08-31 21:45:00+00:00    86.2586
Name: TWAP, dtype: float64

In this case, ICE_SUPER_DERIVATIVES is your data source:

Input:

fx.data_source

Output:

'ICE_SUPER_DERIVATIVES'

Table of Contents

Custom data sources

Custom data sources#

Environment#

View available sources#

Change data source#