Skip to content

Schedule shared Unicode data downloads before recursive makes - #368

Open
hengweiyv wants to merge 1 commit into
JuliaStrings:masterfrom
hengweiyv:fix/parallel-data-downloads
Open

hengweiyv wants to merge 1 commit into
JuliaStrings:masterfrom
hengweiyv:fix/parallel-data-downloads

Conversation

@hengweiyv

Copy link
Copy Markdown

Parallel top-level builds launch separate recursive makes for Lowercase.txt and Uppercase.txt. Both can see DerivedCoreProperties.txt as missing and download it into the same file concurrently.

Declare the shared download as a top-level prerequisite of both casing targets and the data generator, so it completes once before those recursive makes start. Add an offline regression to make check that invokes real make -j8 with stand-ins for the downloader and Julia, verifies one download, and checks that both outputs are generated. The unchanged Makefile fails this test with two downloads.

Fixes #234.

Validation on Linux:

  • make test_parallel_make and make -j8 test_parallel_make pass.
  • make -j8 all builds the static and shared libraries.
  • charwidth, misc, valid, iterate, case, custom, and maxdecomposition tests pass.
  • Full make check was not run locally because Julia is not installed.

@eschnett

eschnett commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator

Your correction is valid.

I think the new test is overkill. This is a new test that confirms that other tests do not waste work. I would omit it.

We could simplify this even more and use a single make target to download all unicode files instead of splitting this across many make targets. I contributed that code, and in hindsight, I may have gone over the top with the makefile bits for this.

I'm waiting for someone else to comment as well.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Makefile is not parallel-safe

2 participants