Single stocks#

This page introduces Single Stock instruments and shows how you can define an equity universe through the EquityUniverseFilter or SIGMaster classes.

Both classes are powerful tools for filtering an equity universe:

The starting point for EquityUniverseFilter is defined as a stock index, which then can be filtered by a number of different metrics, such as fundamental data.
The starting point for SIGMaster is all securities available at a given point in time.

Learn more: Example notebooks

Environment#

Set up your environment using the following three steps:

Import the relevant internal and external libraries
Configure the environment parameters
Initialize the environment

import sigtech.framework as sig

import datetime as dtm
import pandas as pd
import numpy as np
import seaborn as sns

sns.set(rc={"figure.figsize": (18, 6)})
env = sig.init()

Learn more: Setting up the environment

SIGMaster#

SIGMaster is a point-in-time security master. It records all historic stock events, such as corporate actions and industry changes.

When such changes occur to a security, rather than updating values, SIGMaster creates a new point-in-time copy of the security reflecting the change.

SIGMaster also has a data model which stores information at the company, fungible and tradable levels:

Company level: Represents individual companies
Fungible level: Represents a particular share class for a given company
Tradable level: Represents all listings for a given fungible

SIGMaster can be instantiated from the environment:

sig_master = env.sig_master()
sig_master.print_output = True # print any filtering to console

Filtering#

To view all the available filters that can be applied to the sig_master object and how they operate, run the following method:

sig_master.show_filters()

Some of the built-in available filters include:

.filter_to_last_pit_record()
.filter_primary_tradable()
.filter_like()
.filter_operating_mic()
.filter_tradable()
.filter_tradable_active()
.filter_startswith()
.filter_primary_fungible()
.filter_primary()
.filter_notna()

These filters can be applied as multiple filters in sequence, also known as daisy chaining.

In the following example, SIGMaster is filtered for all current Berkshire Hathaway securities:

Input:

sig_master_filter_brk = sig_master\
    .filter_to_last_pit_record()\
    .filter_like(like='BERKSHIRE HATHAWAY', column='ISSUE_NAME')

Output:

SIGMaster: filter_log_record=applied filter 'filter_to_last_pit_record' : SIG.TRADABLE_INTERNAL_ID ~duplicated keep='last' : 6155979 > 845617 records
SIGMaster: filter_log_record=applied filter 'filter_like' : ISSUE_NAME like BERKSHIRE HATHAWAY : 845617 > 128 records

The console logs show that the security master has been filtered to the 128 currently listed securities for Berkshire Hathaway.

This can be further filtered to retrieve the primary listings for the primary fungible or share class:

Input:

sig_master_filter_brk_primary = sig_master_filter_brk\
    .filter_primary_fungible()\
    .filter_primary_tradable()

Output:

SIGMaster: filter_log_record=applied filter 'filter_primary_fungible' : PRIMARY_FUNGIBLE == Y : 128 > 32 records
SIGMaster: filter_log_record=applied filter 'filter_primary_tradable' : PRIMARY_TRADABLE == Y : 32 > 3 records

This example picks out the 5 BRK securities listed on the NYSE, using the exchange’s operating_mic:

Input:

sig_master_filter_brk_xnys = sig_master_filter_brk\
    .filter_operating_mic(operating_mics=["XNYS"])

Output:

SIGMaster: filter_log_record=applied filter 'filter_operating_mic' : OPERATING_MIC in ['XNYS'] : 128 > 16 records

The available filter values for any sig_master column can be retrieved using show_filter_values:

sig_master.show_filter_values(column="OPERATING_MIC", dropna=True)

The Refinitiv Business Classification (TRBC) filtering #

SIGMaster is filtered according to the five-level TRBC scheme:

Economic sectors
Business sectors
Industry groups
Industries
Activities

levels = [
    "TRBC_ECONOMIC_SECTOR", "TRBC_BUSINESS_SECTOR",
    "TRBC_INDUSTRY_GROUP", "TRBC_INDUSTRY", "TRBC_ACTIVITY"]
all_classifications = {
    c: sig_master.show_filter_values(c).tolist()
                       for c in levels}
all_classifications

sig_master.filter_like(like="Industrials", column="TRBC_ECONOMIC_SECTOR")

Viewing SigMaster data#

There are a number of ways to transform a filtered SIGMaster object into a workable format. A few examples:

.to_pandas_trading_view(): Converts the filtered SIGMaster object into a Pandas DataFrame, providing fields relevant for trading and exchanges.
.to_pandas_data_model_view(): Converts the filtered SIGMaster object into a Pandas DataFrame for viewing the company, fungible, and tradable data models.
.to_pandas_point_in_time_view(): Shows fields relevant for time-travel analysis of financial data.

Integration with `SingleStock` objects #

SIGMaster supports the generation of SingleStock objects. This example filters SIGMaster based on exchange tickers and operating Market Identifier Codes (MICs):

Input:

exchange_tickers = ["AAPL", "BRKA", "BRKB", "MAT", "ELAN",
                    "NVST", "PG", "PFE", "WU", "ESRT", "BKI",
                    "JBGS", "BHF", "ROKU", "SWCH", "MDB"]

sig_master_select_tickers = sig_master\
    .filter_to_last_pit_record()\
    .filter_primary_fungible()\
    .filter_primary_tradable()\
    .filter_operating_mic(operating_mics=["XNAS", "XNYS"]) \
    .filter_exchange_tickers(exchange_tickers=exchange_tickers)

Output:

SIGMaster: filter_log_record=applied filter 'filter_to_last_corporate_event' : SIG.TRADABLE_INTERNAL_ID ~duplicated keep='last' : 2400974 > 471354 records
SIGMaster: filter_log_record=applied filter 'filter_primary_fungible' : PRIMARY_FUNGIBLE == Y : 471354 > 410832 records
SIGMaster: filter_log_record=applied filter 'filter_primary_tradable' : PRIMARY_TRADABLE == Y : 410832 > 112046 records
SIGMaster: filter_log_record=applied filter 'filter_operating_mic' : OPERATING_MIC in ['XNAS', 'XNYS'] : 112046 > 16346 records
SIGMaster: filter_log_record=applied filter 'filter_exchange_tickers' : EXCHANGE_TICKER in ['AAPL', 'BRKA', 'BRKB', 'MAT', 'ELAN', 'NVST', 'PG', 'PFE', 'WU', 'ESRT', 'BKI', 'JBGS', 'BHF', 'ROKU', 'SWCH', 'MDB'] : 16346 > 15 records
>>> as_of = 2021-05-02 00:00:00, 15 companies, 15 fungibles, 15 tradables, 2 operating mics, shape (15, 64), 6 filter(s) applied
Filter history:
- applied filter 'filter_as_of' : MARKET_TIMESTAMP <= 2021-05-02 00:00:00 : 2400974 > 2400974 records
- applied filter 'filter_to_last_corporate_event' : SIG.TRADABLE_INTERNAL_ID ~duplicated keep='last' : 2400974 > 471354 records
- applied filter 'filter_primary_fungible' : PRIMARY_FUNGIBLE == Y : 471354 > 410832 records
- applied filter 'filter_primary_tradable' : PRIMARY_TRADABLE == Y : 410832 > 112046 records
- applied filter 'filter_operating_mic' : OPERATING_MIC in ['XNAS', 'XNYS'] : 112046 > 16346 records
- applied filter 'filter_exchange_tickers' : EXCHANGE_TICKER in ['AAPL', 'BRKA', 'BRKB', 'MAT', 'ELAN', 'NVST', 'PG', 'PFE', 'WU', 'ESRT', 'BKI', 'JBGS', 'BHF', 'ROKU', 'SWCH', 'MDB'] : 16346 > 15 records

To access the resulting SingleStock objects:

Input:

sig_master_select_tickers.to_single_stock()

Output:

{'MAT.XNAS': 1010987.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400818448],
 'WU.XNYS': 1036055.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675401378832],
 'BKI.XNYS': 1123343.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675401361424],
 'ROKU.XNAS': 1122878.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675401337232],
 'SWCH.XNYS': 1123697.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400463184],
 'AAPL.XNAS': 1000045.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400464976],
 'MDB.XNAS': 1124358.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400430096],
 'NVST.XNYS': 1153683.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400430352],
 'ELAN.XNYS': 1145377.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400430672],
 'PFE.XNYS': 1011166.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400430928],
 'PG.XNYS': 1004551.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400431120],
 'BHF.XNAS': 1109859.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400431376],
 'ESRT.XNYS': 1078510.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400431632],
 'JBGS.XNYS': 1109694.SINGLE_STOCK.TRADABLE <class 'sigtech.framework.instruments.equities.SingleStock'>[139675400433296]}

Universe creation#

The get_universe_generator method returns a sig_master universe creation object:

universe_generator = sig_master.get_universe_generator()

Filters can then be applied, similarly to sig_master:

universe_generator = universe_generator.filter_operating_mic(operating_mics=["XNAS", "XNYS"])
universe_generator = universe_generator.filter_exchange_tickers(exchange_tickers=exchange_tickers)

The apply method can be called with a list of dates to evaluate the universe. By default, the apply method will filter each point in time for primary fungibles, primary tradable, and single stock securities. The resulting dictionary can be passed to the EquityUniverseFilter for additional filtering:

some_dates = [dtm.datetime(2021, 1, 4), dtm.datetime(2022, 1, 4)]

sm_universe = universe_generator.apply(some_dates, primary_fungible_only=True,
                                       primary_tradable_only=True, single_stocks_only=True)

equity_filter = sig.EquityUniverseFilter(sm_universe)

EquityUniverseFilter#

EquityUniverseFilter is a narrower, but faster way of defining a universe, starting with a stock index. It can also filter on data fields, such as:

Fundamental
Historic
TRBC

An EquityUniverseFilter object takes a string representing an equity index identifier:

equity_filter = sig.EquityUniverseFilter("SPX INDEX")

The universe_ids() method returns a list of index constituent IDs, which represent the universe at a point in time:

constituents = equity_filter.universe_ids(env.asofdate)

Filtering #

The add() method takes a fundamental or history field, such as:

An operator
A value/threshold
The frequency of the field

Fields, operators and frequencies are case-insensitive. To see the available fundamental and history fields, use the available_fields() method on a stock object.

Input:

equity_filter.add("Market Cap", ">", 1e4, "Quarterly")  # Denominated in millions
equity_filter.add("Price/Earnings Ratio", "<", 50, "annual")
equity_filter.add("Dividends", "TOP", 5, "quarterly")
equity_filter.add("Healthcare", "in", "TRBC_ECONOMIC_SECTOR")
equity_filter.list()

Output:

[[['Market Cap USD'], 10000.0, '>', ['Quarterly'], 'Absolute', None, None],
 [['Price/Earnings Ratio'], 50, '<', ['Annual'], 'Absolute', None, None],
 [['Dividend Yield'], 5, 'top', ['Quarterly'], 'Absolute', None, None],
 [['Healthcare'], 'TRBC_ECONOMIC_SECTOR', 'in', None, 'Absolute', None, None]]

The clean() method removes all filters:

Input:

equity_filter.clean()
equity_filter.list()

Output:

[]

The available filter operators available are listed below. When the ‘Body, 5’ operator is applied to a field, it filters out the top five and bottom five stock IDs.

Input:

equity_filter.OPERATORS

Output:

['<', '<=', '==', '!=', '>=', '>', 'top', 'bottom', 'body']

Method apply_filter() takes a date or a list of dates and applies the filters defined using add() in the following order:

All comparison operators (<, <=, ==, !=, >=, >) are first applied in the order you defined.
All ranking operators (top, bottom, body) are subsequently applied in the order you defined.

Parameter ignore_asofdate controls how the underlying time series is loaded by the filter engine:

If set to True, which is the default, the time series loaded will use the as of date set in the environment.
If set to False, the as of date of the time series coincides with the date in which the filter is applied. When applied to multiple dates, this setting may be more computationally expensive.

Only the time series for the fields specified in the filters are loaded. In the following example, filters are applied to multiple points in time. This returns a list of filtered stock identifiers for each point in time.

Input:

some_dates = pd.date_range(dtm.date(2019, 6, 30), dtm.date(2020, 9, 15), freq="Q")
universe = equity_filter.apply_filter(some_dates)

Output:

2019-06-30    [1024626.SINGLE_STOCK.TRADABLE, 1011622.SINGLE...
2019-09-30    [1010613.SINGLE_STOCK.TRADABLE, 1024626.SINGLE...
2019-12-31    [1005282.SINGLE_STOCK.TRADABLE, 1006152.SINGLE...
2020-03-31    [1020669.SINGLE_STOCK.TRADABLE, 1005282.SINGLE...
2020-06-30                      [1003767.SINGLE_STOCK.TRADABLE]
dtype: object

Additional features #

This section contains examples of additional features that are useful for debugging purposes or to analyze the results of a stock universe filter.

Filtered output#

The method display_fields_data() returns a DataFrame containing time series data defined by a date and a list of stock IDs, fields and frequencies.

The following example selects the first 10 unique stocks that were eligible at any point in time in the investment universe. It then loads any corresponding data that was not loaded during filtering:

ids = universe.explode().unique()[:10]

display_fields = ["Market Cap", "Price/Earnings Ratio"]
display_frequencies = ["Quarterly", "Quarterly"]
equity_filter.display_fields_data(
    date=dtm.datetime(2019, 6, 30),
    ids=ids,
    fields=display_fields,
    frequencies=display_frequencies,
)

Multiple stock tradables may be issued by a listed company. The following code block returns the companies associated with the filtered stock IDs:

company_names = [sig.obj.get(id).company_name for id in ids]
company_names

Generic constructor#

The EquityUniverseFilter constructor also supports a dict, such as date or list of stock IDs, for stock universe customizations.

You can populate a data structure representing a stock universe with dates and lists of stock identifiers available on those dates. In the following example, the sm_universe dictionary generated from SigMaster is passed:

equity_filter = sig.EquityUniverseFilter(sm_universe)

Available frequencies#

Given a fundamental or history field, the available_frequencies() method returns a list of frequencies available for all stocks in the universe:

Input:

available_frequencies = equity_filter.available_frequencies("Total Assets")
available_frequencies

Output:

['Quarterly', 'Restated - Annual', 'Restated - Quarterly', 'Annual']

Table of Contents

Single stocks

Single stocks#

Environment#

SIGMaster#

Filtering#

The Refinitiv Business Classification (TRBC) filtering #

Viewing SigMaster data#

Integration with SingleStock objects #

Universe creation#

EquityUniverseFilter#

Filtering #

Additional features #

Filtered output#

Generic constructor#

Available frequencies#

Integration with `SingleStock` objects #