Skip to content

Create data subscriptions for re-selected user defined members - #9930

Open
Divyansh151005 wants to merge 1 commit into
QuantConnect:masterfrom
Divyansh151005:bug-9900-user-defined-resolution-subscription
Open

Divyansh151005 wants to merge 1 commit into
QuantConnect:masterfrom
Divyansh151005:bug-9900-user-defined-resolution-subscription

Conversation

@Divyansh151005

Copy link
Copy Markdown

Description

Adding a new resolution to a security that is already in the user defined universe (AddEquity("SPY", Resolution.Tick) after AddEquity("SPY", Resolution.Minute)), or removing it and re-adding it at a new resolution in the same time step, records the new configuration on Security.Subscriptions but never creates its data subscription. The algorithm keeps the old bars and gets no ticks.

The selected symbols are the same in both cases, so Universe.PerformSelection returns Unchanged and ApplyUniverseSelection returns before the "find new selections" loop. That loop would skip the symbol anyway because it is already a member. On the remove and re-add path the symbol is still selected, so the removal loop keeps the member and its old feed.

ApplyUniverseSelection now syncs the selected members of a UserDefinedUniverse before the unchanged check:

  • It asks the universe for each member's subscription requests and adds the configurations the data manager does not hold yet. This is the same add path as a new member: security.AddData, the currency feed removal, DataManager.AddSubscription and InternalSubscriptionManager.AddedSubscriptionRequest.
  • In one pass over the data feed subscriptions, it removes this universe's request for the member configurations it no longer requests, which is the minute feed in the remove and re-add case. RemoveSubscription(config, universe) only drops this universe's request, so a feed another universe still uses stays.

Coarse, fundamental and chain universes keep the current path, since their configurations do not change for an existing member. A user defined universe only runs selection when its collection changes or on its trigger times, so the extra work per selection is one GetSubscriptionRequests call per member.

Related Issue

Fixes #9900

Motivation and Context

The issue was reported from a live deployment: switching a held symbol to tick resolution silently did nothing. The cloud backtest in the issue shows SPY and QQQ receiving minute bars and no ticks after the switch, while IWM (new symbol) and AAPL (fundamental universe member) receive ticks.

Requires Documentation Change

No.

How Has This Been Tested?

  • New UniverseSelectionTests.AddingResolutionToExistingUserDefinedSecurityCreatesDataSubscription: SPY at minute, then AddEquity("SPY", Resolution.Tick). It asserts the minute feed stays and the tick feed is created. It fails on master.
  • New UniverseSelectionTests.RemoveAndReaddAtNewResolutionInOneTimeStepReplacesDataSubscription: QQQ at minute, then RemoveSecurity plus AddEquity("QQQ", Resolution.Tick) in one time step. It asserts the minute feed is dropped, the tick feed exists, and QQQ is tradable. It fails on master.
  • New UserDefinedUniverseResolutionChangeRegressionAlgorithm (C#, local AIG and BAC tick and minute data on 2013-10-07). At 15:00 it adds tick resolution to AIG and removes and re-adds BAC at tick. It asserts AIG gets ticks and keeps its minute bars, BAC gets ticks and no minute bars after the switch, and BAC is tradable. On master it throws Expected AIG ticks after adding the tick resolution (1,576 data points processed). With the fix it completes with 307,906 data points.

Local runs (macOS, .NET 10):

Engine.DataFeeds.UniverseSelectionTests, Common.Data.UniverseSelection.UserDefinedUniverseTests,
Engine.DataFeeds.PendingRemovalsManagerTests, Engine.DataFeeds.DataManagerTests,
CSharp/UserDefinedUniverseResolutionChangeRegressionAlgorithm
Total tests: 34
     Passed: 34

QuantConnect.Tests.Engine.DataFeeds.*, QuantConnect.Tests.Common.Data.UniverseSelection.*,
CSharp/UserDefinedUniverseResolutionChangeRegressionAlgorithm
Total tests: 1025
     Passed: 1023
     Failed: 1
    Skipped: 1

The one failure is InternalSubscriptionManagerTests.UniverseSelectionAddAndRemove (TravisExclude), a 15 second wall clock live test. It times out the same way on master locally, so it is not related. The full suite is left to CI.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed. (The tests listed above pass, the full unit test suite is left to CI.)
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

Adding a new resolution to a security already in the user defined
universe, or removing and re-adding it at a new resolution in the same
time step, kept the same selected symbols. Universe.PerformSelection then
returned Unchanged and ApplyUniverseSelection skipped existing members, so
the new configurations never reached the data manager and the old feed
kept running.

ApplyUniverseSelection now syncs the selected members of a user defined
universe before the unchanged check: it adds the requested configurations
the data feed does not have yet and removes this universe's request for
the ones it no longer requests. Other universes keep the current path.

Fixes QuantConnect#9900.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AddEquity with a new resolution on an existing user-defined security (or remove + re-add in one time step) never creates the data subscription

1 participant