Custom data sources

​
This page introduces the concept of custom data sources and explores the ways in which they can be defined and utilised on the SigTech platform. It shows you how to:
Construct your environment
Query available data sources
Set a data source
Compare data sources
Environment
import sigtech.framework as sig
sig.de_init()
sig.init()
During the configuration of your research environment, a range of parameters can be used to set and define its features and behaviour.
The following command lists the first ten environment configuration parameters:
Input
Output
sorted(sig.env().config)[:10]
['ALLOWED_DAY_CA_DATA_GAP',
 'ALLOWED_MISSING_CA_DATA',
 'CASH_OVERRIDES',
 'CUSTOM_DATA_SOURCE_CONFIG',
 'DATAFRAME_SERVICE_ACTIVE',
 'DATAFRAME_SERVICE_FACTORY',
 'DATA_ADAPTERS',
 'DATA_SERVICE_DATA_FILE',
 'DATA_SERVICE_RECORD_TO_FILE',
 'DISABLE_T_COST_NETTING']
Learn more: on configuration parameters, see Environment setup.
Custom data sources can be set by assigning values to the CUSTOM_DATA_SOURCE_CONFIG parameter.
sig.config.CUSTOM_DATA_SOURCE_CONFIG
No custom data sources are set by default. This can be confirmed by running the following command:
sig.env()[sig.config.CUSTOM_DATA_SOURCE_CONFIG]
View available sources
Whilst custom data sources can be specified for any type of tradable instrument, our example uses FX spot data to demonstrate this functionality.
The following command retrieves an FX pair:
fx = sig.obj.get('EURUSD CURNCY')
To determine the default data source for this instrument:
Input
Output
fx_data_source
'ICE_SUPER_DERIVATIVES'
To list the data sources available for an instrument, use data_source_all. The results generated by this command will depend on your current data entitlements. To access additional data, please contact our sales team at [email protected]
Input
Output
fx.data_source_all
['ICE_SUPER_DERIVATIVES', 'CLS', 'QUANDL.FXCM', 'REFINITIV', 'CBOE']
Change source
Before resetting the data source it is necessary to de-initialize and re-initialize the environment:
sig.de_init()
sig.init()
The following code block selects objects whose internal id is of the form '[A-Z]{6} CURNCY' and changes their data source to 'REFINITIV':
sig.env()[sig.config.CUSTOM_DATA_SOURCE_CONFIG] = [
        ('[A-Z]{6} CURNCY', 'REFINITIV'),
    ]
To confirm this change, retrieve the FX pair and querying its data source:
Input
Output
fx_refinitiv = sig.obj.get('EURUSD CURNCY')
fx_refinitiv.data_source
'REFINITIV'
To display the group to which the object belongs, use the following command:
Input
Output
fx.refinitiv.group()
REFINITIV FXFIX GROUP <class 'sigtech.framework.instruments.fixes.FXFixGroup'>[139644524666064]
To tabulate the first five available data points within the instrument’s history, use both the history() and head() functions:
Input
Output
fx_refinitiv.history().head()
trading_datetime
2000-01-31 01:00:00+00:00    0.97970
2000-01-31 02:00:00+00:00    0.97875
2000-01-31 03:00:00+00:00    0.97955
2000-01-31 04:00:00+00:00    0.98080
2000-01-31 05:00:00+00:00    0.98050
Name: (LastPrice, EURUSD CURNCY), dtype: float64
We can also compare currencies available from different data sources:
Input
Output
sig.FXFixGroup.get_names()
['CBOE FXFIX GROUP',
 'CLS FXFIX GROUP',
 'ICE TERMSTRUCTURE FXFIX GROUP',
 'QUANDL FXFIX GROUP',
 'REFINITIV FXFIX GROUP']
To view currencies available from Refinitiv but unavailable from ICE:
Input
Output
ice_ccys = sig.obj.get('ICE TERMSTRUCTURE FXFIX GROUP').query_instrument_names()
refinitiv_ccys = sig.obj.get('REFINITIV FXFIX GROUP').query_instrument_names()
additional_ccys = [ccy for ccy in refinitiv_ccys if ccy not in ice_ccys]
additional_ccys
['AUDJPY CURNCY',
 'AUDNZD CURNCY',
 'CADJPY CURNCY',
 'EURAUD CURNCY',
 'EURCAD CURNCY',
 'EURCHF CURNCY',
 'EURDKK CURNCY',
 'EURGBP CURNCY',
 'EURHUF CURNCY',
 'EURJPY CURNCY',
 'EURNOK CURNCY',
 'EURRUB CURNCY',
 'EURSEK CURNCY',
 'GBPAUD CURNCY',
 'GBPCAD CURNCY',
 'GBPCHF CURNCY',
 'GBPJPY CURNCY']
Confirm that AUDJPY is unavailable from ICE by querying the data sources associated with the currency pair:
Input
Output
sig.obj.get('AUDJPY CURNCY').data_source_all
['CLS', 'QUANDL.FXCM', 'REFINITIV', 'CBOE']
Reverting to ICE renders this pair unavailable:
sig.de_init()
sig.init()
Absent a set source, all available sources for an instance are cycled through. Production of a history series will depend on entitlements:
Input
Output
fx = sig.obj.get('EURUSD CURNCY')
fx.history().head()
trading_datetime
2015-08-31 21:05:00+00:00    86.2320
2015-08-31 21:20:00+00:00    86.2170
2015-08-31 21:30:00+00:00    86.2505
2015-08-31 21:35:00+00:00    86.2885
2015-08-31 21:45:00+00:00    86.2586
Name: TWAP, dtype: float64
In this case, ICE_SUPER_DERIVATIVES is our data source:
Input
Output
fx.data_source
'ICE_SUPER_DERIVATIVES'
Last modified 8mo ago