Repository navigation
fix: undo the gain normalize() applied to the MDX input, not the input peak #319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Rahul-cs50
wants to merge
1
commit into
nomadkaraoke:main
Choose a base branch
from
Rahul-cs50:fix/318-mdx-input-gain
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| """MDX stems keep the input's level whatever gain normalization applied before demixing (#318).""" | ||
|
|
||
| import logging | ||
| from unittest.mock import Mock | ||
|
|
||
| import numpy as np | ||
| import pytest | ||
|
|
||
| from audio_separator.separator.architectures.mdx_separator import MDXSeparator | ||
|
|
||
| VOCAL_SHARE = 0.25 | ||
|
|
||
|
|
||
| def mdx_double(mix, amplification_threshold): | ||
| separator = Mock(spec=MDXSeparator) | ||
| separator.logger = logging.getLogger(__name__) | ||
| separator.normalization_threshold = 0.9 | ||
| separator.amplification_threshold = amplification_threshold | ||
| separator.output_single_stem = None | ||
| separator.prepare_mix.return_value = mix | ||
| # A linear stand-in for the model: the "vocals" are a fixed share of whatever mix it is given. | ||
| separator.demix.side_effect = lambda wave, is_match_mix=False: VOCAL_SHARE * wave | ||
| separator.primary_source = None | ||
| separator.secondary_source = None | ||
| separator.primary_stem_name = "Vocals" | ||
| separator.secondary_stem_name = "Instrumental" | ||
| separator.compensate = 1.0 | ||
| separator.get_stem_output_path.side_effect = ["input_(Instrumental).wav", "input_(Vocals).wav"] | ||
| return separator | ||
|
|
||
|
|
||
| @pytest.mark.parametrize( | ||
| "peak, amplification_threshold", | ||
| [ | ||
| (0.4, 0.0), # between the thresholds: normalize() leaves the mix alone | ||
| (1.5, 0.0), # above normalization_threshold: scaled down before demixing | ||
| (0.05, 0.1), # below amplification_threshold: scaled up before demixing | ||
| ], | ||
| ) | ||
| @pytest.mark.parametrize("invert_using_spec", [False, True]) | ||
| def test_mdx_stems_keep_input_level(peak, amplification_threshold, invert_using_spec, monkeypatch): | ||
| rng = np.random.default_rng(318) | ||
| mix = rng.uniform(-1, 1, (2, 64)).astype(np.float32) | ||
| mix *= peak / np.abs(mix).max() | ||
| original = mix.copy() | ||
|
|
||
| separator = mdx_double(mix, amplification_threshold) | ||
| separator.invert_using_spec = invert_using_spec | ||
| # Spectrogram inversion is linear for these inputs; subtracting is enough to check the scale it is given. | ||
| monkeypatch.setattr("audio_separator.separator.architectures.mdx_separator.spec_utils.invert_stem", lambda raw_mix, stem: raw_mix.T * (1 / VOCAL_SHARE) - stem) | ||
|
|
||
| MDXSeparator.separate(separator, "input.wav") | ||
|
|
||
| np.testing.assert_allclose(separator.primary_source, VOCAL_SHARE * original.T, rtol=1e-5) | ||
| np.testing.assert_allclose(separator.secondary_source, (1 - VOCAL_SHARE) * original.T, rtol=1e-5) | ||
| np.testing.assert_allclose(separator.primary_source + separator.secondary_source, original.T, rtol=1e-5) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Script executed:
Repository: nomadkaraoke/python-audio-separator
Length of output: 5566
Script executed:
Repository: nomadkaraoke/python-audio-separator
Length of output: 14095
Script executed:
sed -n '100,118p' audio_separator/separator/uvr_lib_v5/spec_utils.pyRepository: nomadkaraoke/python-audio-separator
Length of output: 776
Script executed:
sed -n '116,130p' audio_separator/separator/uvr_lib_v5/spec_utils.pyRepository: nomadkaraoke/python-audio-separator
Length of output: 567
Apply the sum-back requirement to exported stems, not only in-memory stems.
The change at
mdx_separator.py:175restores in-memory stems to the input level. The writers then normalize each stem again.write_audio_pydub()andwrite_audio_soundfile()callspec_utils.normalize()on each stem separately, usingnormalization_thresholdasmax_peakandamplification_thresholdasmin_peak. Each stem receives its own gain, so the saved files can fail to sum to the input:amplification_thresholdabove 0, a quiet input can produce stems that are amplified independently. For example, an input peak of 0.05 with a threshold of 0.1 can produce this result.normalization_threshold=0.9, a stem whose peak exceeds 0.9 is attenuated on export, even when the input is loud.Issue
#318requires stems that sum back to the input. This change corrects only the in-memory levels. The exported files must keep the same levels, or they do not meet that requirement. A fix must change the export path for this case. The shared writers serve every architecture, so the change must not alter the other architectures' export behavior.